Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> reader in hadoop without reader's schema


Copy link to this message
-
Re: reader in hadoop without reader's schema
What about if I use AvroInputFormat? I tried setting the input schema to
null but that did not work
On Dec 5, 2011 6:50 PM, "Doug Cutting" <[EMAIL PROTECTED]> wrote:

> On 12/01/2011 07:32 PM, Koert Kuipers wrote:
> > I am reading from avro container files in hadoop. I know the container
> > files have a (writers) schema stored in them. My reader specifies it's
> > schema using avro.input.schema job parameter. This way any schema
> > changes are gracefully handled with both schema's present.
> >
> > However, i dont always need all this complexity. Is there a way to read
> > without having to specify a reader's schema, where i basically say "just
> > accept the writer's schema and read the data that way".
>
> That's what's done by default if you, e.g., do something like:
>
> Iterable i = DataFileReader.openReader(file, new GenericDatumReader());
> for (Object o : i) {
>  System.out.println(o);
> }
>
> Doug
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB