Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> How to Use AvroStorage to read avro data and convert into a new (compatible) schema


Copy link to this message
-
How to Use AvroStorage to read avro data and convert into a new (compatible) schema
Hi,

I have the following problem to solve:

I have data in HDFS written out using AvroStorage with schema1.
I have schema2 which is different than schema1 but they are compatible.
Now, I want to use AvroStorage to load the data and convert to schema2.
How can I achieve that?

I tried something like:

    data = load 'my_data_dir' using AvroStorage('schema', SCHEMA2_STRING);
    describe data;

The schema for the loaded-in data is still in schema1, not in schema2.

Any help is highly appreciated.

Thanks,

Runping Qi
A9.COM
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB