Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> How to Use AvroStorage to read avro data and convert into a new (compatible) schema


Copy link to this message
-
Re: How to Use AvroStorage to read avro data and convert into a new (compatible) schema
Try loading the data in schema one, and storing it/transforming it in
schema two in a new location?

Then try to load it, and see what it is. This may not be feasible if
your data is large, but worth a try to help debug?

Russell Jurney http://datasyndrome.com

On Oct 5, 2012, at 7:48 PM, Runping Qi <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I have the following problem to solve:
>
> I have data in HDFS written out using AvroStorage with schema1.
> I have schema2 which is different than schema1 but they are compatible.
> Now, I want to use AvroStorage to load the data and convert to schema2.
> How can I achieve that?
>
> I tried something like:
>
>    data = load 'my_data_dir' using AvroStorage('schema', SCHEMA2_STRING);
>    describe data;
>
> The schema for the loaded-in data is still in schema1, not in schema2.
>
> Any help is highly appreciated.
>
> Thanks,
>
> Runping Qi
> A9.COM
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB