-Re: Hadoop Avro Question
Harsh J 2013-05-01, 05:06
Oops, moving for sure this time :)
On Wed, May 1, 2013 at 10:35 AM, Harsh J <[EMAIL PROTECTED]> wrote:
> Moving the question to Apache Avro's user@ lists. Please use the right
> lists for the most relevant answers.
> Avro is a different serialization technique that intends to replace
> the Writable serialization defaults in Hadoop. MR accepts a list of
> serializers it can use for its key/value structures and isn't limited
> to Writable in any way. Look up the property "io.serializations" in
> your Hadoop's core-default.xml for more information.
> The Avro project also offers fast comparator classes that are used for
> comparing the bytes/structures of Avro objects. This is mostly
> auto-set for you when you use the MR framework as described at
> (via AvroJob helper class).
> On Tue, Apr 30, 2013 at 6:51 PM, Rahul Bhattacharjee
> <[EMAIL PROTECTED]> wrote:
>> When dealing with Avro data files in MR jobs ,we use AvroMapper , I noticed
>> that the output of K and V of AvroMapper isnt writable and neither the key
>> is comparable (these are AvroKey and AvroValue). As the general
>> serialization mechanism is writable , how is the K,V pairs in case of avro ,
>> travel across nodes?
> Harsh J