Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro, mail # user - Joining Avro input files in using Java mapreduce


+
Sripad Sriram 2013-04-24, 00:01
Copy link to this message
-
Re: Joining Avro input files in using Java mapreduce
Martin Kleppmann 2013-04-24, 10:35
Hey Sripad,

Take a look at AvroMultipleInputs.
http://avro.apache.org/docs/1.7.4/api/java/org/apache/avro/mapred/AvroMultipleOutputs.html(mapred
version)
http://avro.apache.org/docs/1.7.4/api/java/org/apache/avro/mapreduce/AvroMultipleOutputs.html(mapreduce
version)

Martin
On 23 April 2013 17:01, Sripad Sriram <[EMAIL PROTECTED]> wrote:

> Hey folks,
>
> Aware that I can use Pig, Hive, etc to join avro files together, but I
> have several use cases where I need to perform a reduce-side join on two
> avro files. MultipleInputs doesn't seem to like AvroInputFormat - any
> thoughts?
>
> thanks!
> Sripad
>
+
Sripad Sriram 2013-04-24, 19:02
+
Martin Kleppmann 2013-04-25, 14:43
+
Sripad Sriram 2013-04-25, 15:26
+
Martin Kleppmann 2013-04-25, 16:05
+
Johannes Schulte 2013-04-26, 20:41
+
Sripad Sriram 2013-04-28, 02:11