Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Avro schema


+
Lior Schachter 2013-08-01, 15:02
+
Harsh J 2013-08-01, 16:36
+
Lior Schachter 2013-08-01, 17:46
Copy link to this message
-
Re: Avro schema
Yes, we seek to 0 and we read the header then seek back to the split offset.
On Aug 1, 2013 11:16 PM, "Lior Schachter" <[EMAIL PROTECTED]> wrote:

> Hi Harsh,
> So for each split you first read the header of the file directly from HDFS
> ?
>
> Thanks,
> Lior
>
>
>
>
> On Thu, Aug 1, 2013 at 7:36 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> We read it from the top of the file at start (just the schema bytes)
>> and then initialize the reader.
>>
>> On Thu, Aug 1, 2013 at 8:32 PM, Lior Schachter <[EMAIL PROTECTED]> wrote:
>> > Hi all,
>> >
>> > When writing Avro schema to the a data file, what will be the expected
>> > behavior if the file is used as M/R input. How does the second/third/...
>> > splits get the schema (the schema is always written to the first split)
>> ?
>> >
>> > Thanks,
>> > Lior
>> >
>> >
>>
>>
>>
>> --
>> Harsh J
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB