Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> How to Use AvroStorage to read avro data and convert into a new (compatible) schema


Copy link to this message
-
How to Use AvroStorage to read avro data and convert into a new (compatible) schema
Hi,

I have the following problem to solve:

I have data in HDFS written out using AvroStorage with schema1.
I have schema2 which is different than schema1 but they are compatible.
Now, I want to use AvroStorage to load the data and convert to schema2.
How can I achieve that?

I tried something like:

    data = load 'my_data_dir' using AvroStorage('schema', SCHEMA2_STRING);
    describe data;

The schema for the loaded-in data is still in schema1, not in schema2.

Any help is highly appreciated.

Thanks,

Runping Qi
A9.COM