Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # dev - AvroStorage pig adapter


Copy link to this message
-
RE: AvroStorage pig adapter
Thiruvalluvan M. G. 2010-05-15, 02:08
> Yes, it helps a lot.  One question remains, how can I construct a
> recursive schema programmatically?
> I have a couple options for the pig Tuple avro schema -- write it in JSON
> and put that in the source code or programmatically construct it.
> I'm currently programmatically constructing a schema specific to the Pig
> schema that is serialized, which is straightforward until I hit the map
> type and recursion.

For my benchmark, I put the schema in jar (or wherever the class files are
located) and then used ClassLoader.getResourceAsStream(). This reduces the
possibility that the schema file is not found during runtime.

Thanks

Thiru