Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> pig union with avro


Copy link to this message
-
Re: pig union with avro
Thanks for reply Cheolsoo

What's the output of describe A and B?

Output of A and B are not identical.

If the schema of A and B are not identical, union will type-cast fields to merge them

http://pig.apache.org/docs/r0.11.1/basic.html#union

> I would try to find which fields pig tries to cast to bytearray while
> merging them. Casting anything to bytearray is invalid resulting this error:

I don't exactly understand the above point.

Is it some thing related to PIG 2493

https://issues.apache.org/jira/browse/PIG-2493

This is resolved in pig 0.10 ??

cannot cast to byte array

Thanks,
Cheolsoo
>
> On Sat, Jun 15, 2013 at 7:48 PM, abhishek dodda
> <[EMAIL PROTECTED]>wrote:
>
>> hello,
>>
>> I am doing this
>>
>> DEFINE AVRO_LOAD org.apache.pig.piggybank.strorage.avro.AvroStorage();
>>
>> A = load '/user/abhi/a.txt' using AVRO_LOAD;
>>
>> B = load '/user/abhi/b.txt' using AVRO_LOAD;
>>
>> C = UNION A , B;
>>
>> here script is failing with the following error
>>
>> *ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1051 : cannot cast to byte
>> array*
>>
>> In the pig logs the error is
>>
>> *ERROR 1056 problem while casting inputs of union*.
>>
>> Script was running fine before, but it is failing now with the above error
>>
>> Regards
>> abhishek
>>
>>
>> On Sat, Jun 15, 2013 at 7:44 PM, abhishek dodda
>> <[EMAIL PROTECTED]>wrote:
>>
>>> hello,
>>>
>>> I am doing this
>>>
>>> DEFINE AVRO_LOAD org.apache.pig.piggybank.strorage.avro.AvroStorage();
>>>
>>> A = load '/user/abhi/a.txt' using AVRO_LOAD;
>>>
>>> B = load '/user/abhi/b.txt' using AVRO_LOAD;
>>>
>>> C = UNION A , B;
>>>
>>> here script is failing with the following error
>>>
>>> ERROR org.apache.pig.tools.grunt.Grunt - ERROR
>>