Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - How to Use AvroStorage to read avro data and convert into a new (compatible) schema


Copy link to this message
-
How to Use AvroStorage to read avro data and convert into a new (compatible) schema
Runping Qi 2012-10-06, 02:48
Hi,

I have the following problem to solve:

I have data in HDFS written out using AvroStorage with schema1.
I have schema2 which is different than schema1 but they are compatible.
Now, I want to use AvroStorage to load the data and convert to schema2.
How can I achieve that?

I tried something like:

    data = load 'my_data_dir' using AvroStorage('schema', SCHEMA2_STRING);
    describe data;

The schema for the loaded-in data is still in schema1, not in schema2.

Any help is highly appreciated.

Thanks,

Runping Qi
A9.COM
+
Russell Jurney 2012-10-06, 03:40