Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> deserializing nested protobufs


Copy link to this message
-
Re: deserializing nested protobufs
Thanks Dmitriy.  Doesn't look like that class supports extensions.  Am I missing something?

- Ben

On Mar 27, 2012, at 10:01 PM, Dmitriy Ryaboy wrote:

> I think you want ProtobufBytesToTuple
> (https://github.com/kevinweil/elephant-bird/blob/master/src/java/com/twitter/elephantbird/pig/piggybank/ProtobufBytesToTuple.java)
>
> This uses ProtobufToPig to do the actual conversion, and wraps it in a UDF.
>
> D
>
> On Tue, Mar 27, 2012 at 7:21 PM, Benjamin Juhn <[EMAIL PROTECTED]> wrote:
>> Hey Jon,
>>
>> I'm not seeing anything in ProtobuftoPig that is specific to handling protobuf byte fields containing serialized protobuf.
>>
>> Do you have a Pig example?
>>
>> Thanks,
>> Ben
>>
>> On Mar 26, 2012, at 4:14 PM, Jonathan Coveney wrote:
>>
>>> Look up the util class ProtobuftoPig
>>>
>>> 2012/3/26 Benjamin Juhn <[EMAIL PROTECTED]>
>>>
>>>> Thanks Jon,
>>>>
>>>> I'm using elephant-bird, and it allows me to load message A.  I'm not sure
>>>> if it has pig bindings allowing me to deserialize message B from message A
>>>> though.
>>>>
>>>> Thanks,
>>>> Ben
>>>>
>>>> On Mar 26, 2012, at 4:09 PM, Jonathan Coveney wrote:
>>>>
>>>>> Ben,
>>>>>
>>>>> take a look at elephantbird https://github.com/kevinweil/elephant-bird
>>>>>
>>>>> Jon
>>>>>
>>>>> 2012/3/26 Benjamin Juhn <[EMAIL PROTECTED]>
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I'm using pig with protobuf and I have some byte fields containing
>>>>>> serialized protobuf data.  Is it possible to handle this nested
>>>> serialized
>>>>>> data with pig?
>>>>>>
>>>>>> ex.
>>>>>> message A {
>>>>>> required bytes data = 1   // serialized message B
>>>>>> }
>>>>>> message B {
>>>>>> ...
>>>>>> }
>>>>>>
>>>>>> Thanks,
>>>>>> Ben
>>>>
>>>>
>>