Is there a way to read BytesWritable using sequence file loader from
piggybank? If not then how should I go about implementing one?

Try the one in Elephant-Bird.

On Tue, Sep 11, 2012 at 11:22 AM, Mohit Anchlia <[EMAIL PROTECTED]> wrote:
> Is there a way to read BytesWritable using sequence file loader from
> piggybank? If not then how should I go about implementing one?

Is it the code that I checkout and build?

On Tue, Sep 11, 2012 at 3:27 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:

> Try the one in Elephant-Bird.
>
> On Tue, Sep 11, 2012 at 11:22 AM, Mohit Anchlia <[EMAIL PROTECTED]>
> wrote:
> > Is there a way to read BytesWritable using sequence file loader from
> > piggybank? If not then how should I go about implementing one?
>

Yup:
https://github.com/kevinweil/elephant-bird

D

On Tue, Sep 11, 2012 at 4:00 PM, Mohit Anchlia <[EMAIL PROTECTED]> wrote:
> Is it the code that I checkout and build?
>
> On Tue, Sep 11, 2012 at 3:27 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:
>
>> Try the one in Elephant-Bird.
>>
>> On Tue, Sep 11, 2012 at 11:22 AM, Mohit Anchlia <[EMAIL PROTECTED]>
>> wrote:
>> > Is there a way to read BytesWritable using sequence file loader from
>> > piggybank? If not then how should I go about implementing one?
>>

Thanks! I'll try it out.

On Tue, Sep 11, 2012 at 4:21 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:

> Yup:
> https://github.com/kevinweil/elephant-bird
>
> D
>
> On Tue, Sep 11, 2012 at 4:00 PM, Mohit Anchlia <[EMAIL PROTECTED]>
> wrote:
> > Is it the code that I checkout and build?
> >
> > On Tue, Sep 11, 2012 at 3:27 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]>
> wrote:
> >
> >> Try the one in Elephant-Bird.
> >>
> >> On Tue, Sep 11, 2012 at 11:22 AM, Mohit Anchlia <[EMAIL PROTECTED]
> >
> >> wrote:
> >> > Is there a way to read BytesWritable using sequence file loader from
> >> > piggybank? If not then how should I go about implementing one?
> >>
>

I got this error when I ran mvn package

[ERROR] Failed to execute goal
com.github.igor-petruk.protobuf:protobuf-maven-pl
ugin:0.4:run (default) on project elephant-bird-core: Unable to find
'protoc' ->
 [Help 1]
[ERROR]

On Tue, Sep 11, 2012 at 4:24 PM, Mohit Anchlia <[EMAIL PROTECTED]>wrote:

> Thanks! I'll try it out.
>
>
> On Tue, Sep 11, 2012 at 4:21 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]>wrote:
>
>> Yup:
>> https://github.com/kevinweil/elephant-bird
>>
>> D
>>
>> On Tue, Sep 11, 2012 at 4:00 PM, Mohit Anchlia <[EMAIL PROTECTED]>
>> wrote:
>> > Is it the code that I checkout and build?
>> >
>> > On Tue, Sep 11, 2012 at 3:27 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]>
>> wrote:
>> >
>> >> Try the one in Elephant-Bird.
>> >>
>> >> On Tue, Sep 11, 2012 at 11:22 AM, Mohit Anchlia <
>> [EMAIL PROTECTED]>
>> >> wrote:
>> >> > Is there a way to read BytesWritable using sequence file loader from
>> >> > piggybank? If not then how should I go about implementing one?
>> >>
>>
>
>

Install protocol buffers 2.3 and thrift 0.5

>From the readme:

Protocol Buffer and Thrift compiler dependencies
Elephant Bird requires Protocol Buffer compiler version 2.3 at build
time, as generated classes are used internally. Thrift compiler is
required to generate classes used in tests. As these are native-code
tools they must be installed on the build machine (java library
dependencies are pulled from maven repositories during the build).

D

On Wed, Sep 12, 2012 at 9:01 PM, Mohit Anchlia <[EMAIL PROTECTED]> wrote:
> I got this error when I ran mvn package
>
> [ERROR] Failed to execute goal
> com.github.igor-petruk.protobuf:protobuf-maven-pl
> ugin:0.4:run (default) on project elephant-bird-core: Unable to find
> 'protoc' ->
>  [Help 1]
> [ERROR]
>
> On Tue, Sep 11, 2012 at 4:24 PM, Mohit Anchlia <[EMAIL PROTECTED]>wrote:
>
>> Thanks! I'll try it out.
>>
>>
>> On Tue, Sep 11, 2012 at 4:21 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]>wrote:
>>
>>> Yup:
>>> https://github.com/kevinweil/elephant-bird
>>>
>>> D
>>>
>>> On Tue, Sep 11, 2012 at 4:00 PM, Mohit Anchlia <[EMAIL PROTECTED]>
>>> wrote:
>>> > Is it the code that I checkout and build?
>>> >
>>> > On Tue, Sep 11, 2012 at 3:27 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]>
>>> wrote:
>>> >
>>> >> Try the one in Elephant-Bird.
>>> >>
>>> >> On Tue, Sep 11, 2012 at 11:22 AM, Mohit Anchlia <
>>> [EMAIL PROTECTED]>
>>> >> wrote:
>>> >> > Is there a way to read BytesWritable using sequence file loader from
>>> >> > piggybank? If not then how should I go about implementing one?
>>> >>
>>>
>>
>>

For a simpler use case, something similar to following should work-

public class PigSequenceFileLoader extends PigStorage {

@SuppressWarnings("rawtypes")

@Override

public InputFormat getInputFormat() {

 return new SequenceFileInputFormat<ByteWritable, Text>();

}

}

Thanks,

Aniket

On Thu, Sep 13, 2012 at 1:24 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:

> Install protocol buffers 2.3 and thrift 0.5
>
> From the readme:
>
> Protocol Buffer and Thrift compiler dependencies
> Elephant Bird requires Protocol Buffer compiler version 2.3 at build
> time, as generated classes are used internally. Thrift compiler is
> required to generate classes used in tests. As these are native-code
> tools they must be installed on the build machine (java library
> dependencies are pulled from maven repositories during the build).
>
>
>
> D
>
> On Wed, Sep 12, 2012 at 9:01 PM, Mohit Anchlia <[EMAIL PROTECTED]>
> wrote:
> > I got this error when I ran mvn package
> >
> > [ERROR] Failed to execute goal
> > com.github.igor-petruk.protobuf:protobuf-maven-pl
> > ugin:0.4:run (default) on project elephant-bird-core: Unable to find
> > 'protoc' ->
> >  [Help 1]
> > [ERROR]
> >
> > On Tue, Sep 11, 2012 at 4:24 PM, Mohit Anchlia <[EMAIL PROTECTED]
> >wrote:
> >
> >> Thanks! I'll try it out.
> >>
> >>
> >> On Tue, Sep 11, 2012 at 4:21 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]
> >wrote:
> >>
> >>> Yup:
> >>> https://github.com/kevinweil/elephant-bird
> >>>
> >>> D
> >>>
> >>> On Tue, Sep 11, 2012 at 4:00 PM, Mohit Anchlia <[EMAIL PROTECTED]
> >
> >>> wrote:
> >>> > Is it the code that I checkout and build?
> >>> >
> >>> > On Tue, Sep 11, 2012 at 3:27 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]>
> >>> wrote:
> >>> >
> >>> >> Try the one in Elephant-Bird.
> >>> >>
> >>> >> On Tue, Sep 11, 2012 at 11:22 AM, Mohit Anchlia <
> >>> [EMAIL PROTECTED]>
> >>> >> wrote:
> >>> >> > Is there a way to read BytesWritable using sequence file loader
> from
> >>> >> > piggybank? If not then how should I go about implementing one?
> >>> >>
> >>>
> >>
> >>
>

--
"...:::Aniket:::... Quetzalco@tl"

What I did was to:

1) extend FileInputLoadFunc class
2) add new type (BYTES_WRITABLE) to swich-case in methods:
inferPigDataType(Type t) and translateWritableToPigDataType(Writable
w, byte dataType)
3) use it as loader

Probably it is not the best way to solve this issue.

2012/9/15 Aniket Mokashi <[EMAIL PROTECTED]>:
> For a simpler use case, something similar to following should work-
>
> public class PigSequenceFileLoader extends PigStorage {
>
> @SuppressWarnings("rawtypes")
>
> @Override
>
> public InputFormat getInputFormat() {
>
>  return new SequenceFileInputFormat<ByteWritable, Text>();
>
> }
>
> }
>
> Thanks,
>
> Aniket
>
> On Thu, Sep 13, 2012 at 1:24 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:
>
>> Install protocol buffers 2.3 and thrift 0.5
>>
>> From the readme:
>>
>> Protocol Buffer and Thrift compiler dependencies
>> Elephant Bird requires Protocol Buffer compiler version 2.3 at build
>> time, as generated classes are used internally. Thrift compiler is
>> required to generate classes used in tests. As these are native-code
>> tools they must be installed on the build machine (java library
>> dependencies are pulled from maven repositories during the build).
>>
>>
>>
>> D
>>
>> On Wed, Sep 12, 2012 at 9:01 PM, Mohit Anchlia <[EMAIL PROTECTED]>
>> wrote:
>> > I got this error when I ran mvn package
>> >
>> > [ERROR] Failed to execute goal
>> > com.github.igor-petruk.protobuf:protobuf-maven-pl
>> > ugin:0.4:run (default) on project elephant-bird-core: Unable to find
>> > 'protoc' ->
>> >  [Help 1]
>> > [ERROR]
>> >
>> > On Tue, Sep 11, 2012 at 4:24 PM, Mohit Anchlia <[EMAIL PROTECTED]
>> >wrote:
>> >
>> >> Thanks! I'll try it out.
>> >>
>> >>
>> >> On Tue, Sep 11, 2012 at 4:21 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]
>> >wrote:
>> >>
>> >>> Yup:
>> >>> https://github.com/kevinweil/elephant-bird
>> >>>
>> >>> D
>> >>>
>> >>> On Tue, Sep 11, 2012 at 4:00 PM, Mohit Anchlia <[EMAIL PROTECTED]
>> >
>> >>> wrote:
>> >>> > Is it the code that I checkout and build?
>> >>> >
>> >>> > On Tue, Sep 11, 2012 at 3:27 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]>
>> >>> wrote:
>> >>> >
>> >>> >> Try the one in Elephant-Bird.
>> >>> >>
>> >>> >> On Tue, Sep 11, 2012 at 11:22 AM, Mohit Anchlia <
>> >>> [EMAIL PROTECTED]>
>> >>> >> wrote:
>> >>> >> > Is there a way to read BytesWritable using sequence file loader
>> from
>> >>> >> > piggybank? If not then how should I go about implementing one?
>> >>> >>
>> >>>
>> >>
>> >>
>>
>
>
>
> --
> "...:::Aniket:::... Quetzalco@tl"