Runping Qi 2012-10-06, 02:48
-Re: How to Use AvroStorage to read avro data and convert into a new (compatible) schema
Russell Jurney 2012-10-06, 03:40
Try loading the data in schema one, and storing it/transforming it in
schema two in a new location?
Then try to load it, and see what it is. This may not be feasible if
your data is large, but worth a try to help debug?
Russell Jurney http://datasyndrome.com
On Oct 5, 2012, at 7:48 PM, Runping Qi <[EMAIL PROTECTED]> wrote:
> I have the following problem to solve:
> I have data in HDFS written out using AvroStorage with schema1.
> I have schema2 which is different than schema1 but they are compatible.
> Now, I want to use AvroStorage to load the data and convert to schema2.
> How can I achieve that?
> I tried something like:
> data = load 'my_data_dir' using AvroStorage('schema', SCHEMA2_STRING);
> describe data;
> The schema for the loaded-in data is still in schema1, not in schema2.
> Any help is highly appreciated.
> Runping Qi