Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Joining Avro input files in using Java mapreduce


+
Sripad Sriram 2013-04-24, 00:01
+
Martin Kleppmann 2013-04-24, 10:35
Copy link to this message
-
Re: Joining Avro input files in using Java mapreduce
Hey Martin,

I think those classes refer to outputting to multiple files rather than
reading from multiple files, which is what's needed for a reduce-side join.

thanks,
Sripad
On Wed, Apr 24, 2013 at 3:35 AM, Martin Kleppmann <[EMAIL PROTECTED]>wrote:

> Hey Sripad,
>
> Take a look at AvroMultipleInputs.
>
> http://avro.apache.org/docs/1.7.4/api/java/org/apache/avro/mapred/AvroMultipleOutputs.html(mapred version)
>
> http://avro.apache.org/docs/1.7.4/api/java/org/apache/avro/mapreduce/AvroMultipleOutputs.html(mapreduce version)
>
> Martin
>
>
> On 23 April 2013 17:01, Sripad Sriram <[EMAIL PROTECTED]> wrote:
>
>> Hey folks,
>>
>> Aware that I can use Pig, Hive, etc to join avro files together, but I
>> have several use cases where I need to perform a reduce-side join on two
>> avro files. MultipleInputs doesn't seem to like AvroInputFormat - any
>> thoughts?
>>
>> thanks!
>> Sripad
>>
>
>
+
Martin Kleppmann 2013-04-25, 14:43
+
Sripad Sriram 2013-04-25, 15:26
+
Martin Kleppmann 2013-04-25, 16:05
+
Johannes Schulte 2013-04-26, 20:41
+
Sripad Sriram 2013-04-28, 02:11
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB