Dan Young
2012-03-25, 04:35
Stan Rosenberg
2012-03-25, 14:41
Stan Rosenberg
2012-03-26, 15:11
Dan Young
2012-04-03, 16:13
Russell Jurney
2012-04-03, 17:07
Dan Young
2012-04-03, 17:32
Bill Graham
2012-04-03, 18:11
Dan Young
2012-04-03, 18:21
Dan Young
2012-04-03, 17:17
-
Trying to store a bag of tuples using AvroStorage.Dan Young 2012-03-25, 04:35
Hello all,
I'm trying to store a bag of tuples using AvroStorage but am not able to
figure out what I'm doing wrong (or if it' supported). What I have is the
following:
grunt>illustrate c;
....
....
---------------------------------------------------------------------------------------------
| c | franchise_id:int |
cast_and_crew:bag{:tuple(full_name:chararray)}|
---------------------------------------------------------------------------------------------
| | 213939 | {(Wang Junzheng), (Li Ling)}
|
---------------------------------------------------------------------------------------------
When I try to store is via AvroStorage I'm getting an error. I've tried
both with and without specifying a schema I've created a gist and posted
it here with the Pig Stack Trace.....
https://gist.github.com/2191363
If anyone has insight into what I'm doing wrong, and can help, I would
greatly appreciate the guidance.
Regards,
Dan
I'm trying to store a bag of tuples using AvroStorage but am not able to
figure out what I'm doing wrong (or if it' supported). What I have is the
following:
grunt>illustrate c;
....
....
---------------------------------------------------------------------------------------------
| c | franchise_id:int |
cast_and_crew:bag{:tuple(full_name:chararray)}|
---------------------------------------------------------------------------------------------
| | 213939 | {(Wang Junzheng), (Li Ling)}
|
---------------------------------------------------------------------------------------------
When I try to store is via AvroStorage I'm getting an error. I've tried
both with and without specifying a schema I've created a gist and posted
it here with the Pig Stack Trace.....
https://gist.github.com/2191363
If anyone has insight into what I'm doing wrong, and can help, I would
greatly appreciate the guidance.
Regards,
Dan
+
Dan Young 2012-03-25, 04:35
-
Re: Trying to store a bag of tuples using AvroStorage.Stan Rosenberg 2012-03-25, 14:41
Hi Dan,
This looks like an avro bug. I'll have a look later tonight unless someone
else has a more immediate answer.
Best,
stan
On Mar 25, 2012 12:36 AM, "Dan Young" <[EMAIL PROTECTED]> wrote:
> Hello all,
>
> I'm trying to store a bag of tuples using AvroStorage but am not able to
> figure out what I'm doing wrong (or if it' supported). What I have is the
> following:
>
> grunt>illustrate c;
> ....
> ....
>
> ---------------------------------------------------------------------------------------------
> | c | franchise_id:int |
> cast_and_crew:bag{:tuple(full_name:chararray)}|
>
> ---------------------------------------------------------------------------------------------
> | | 213939 | {(Wang Junzheng), (Li Ling)}
> |
>
> ---------------------------------------------------------------------------------------------
>
> When I try to store is via AvroStorage I'm getting an error. I've tried
> both with and without specifying a schema I've created a gist and posted
> it here with the Pig Stack Trace.....
>
> https://gist.github.com/2191363
>
>
> If anyone has insight into what I'm doing wrong, and can help, I would
> greatly appreciate the guidance.
>
>
> Regards,
>
> Dan
>
This looks like an avro bug. I'll have a look later tonight unless someone
else has a more immediate answer.
Best,
stan
On Mar 25, 2012 12:36 AM, "Dan Young" <[EMAIL PROTECTED]> wrote:
> Hello all,
>
> I'm trying to store a bag of tuples using AvroStorage but am not able to
> figure out what I'm doing wrong (or if it' supported). What I have is the
> following:
>
> grunt>illustrate c;
> ....
> ....
>
> ---------------------------------------------------------------------------------------------
> | c | franchise_id:int |
> cast_and_crew:bag{:tuple(full_name:chararray)}|
>
> ---------------------------------------------------------------------------------------------
> | | 213939 | {(Wang Junzheng), (Li Ling)}
> |
>
> ---------------------------------------------------------------------------------------------
>
> When I try to store is via AvroStorage I'm getting an error. I've tried
> both with and without specifying a schema I've created a gist and posted
> it here with the Pig Stack Trace.....
>
> https://gist.github.com/2191363
>
>
> If anyone has insight into what I'm doing wrong, and can help, I would
> greatly appreciate the guidance.
>
>
> Regards,
>
> Dan
>
+
Stan Rosenberg 2012-03-25, 14:41
-
Re: Trying to store a bag of tuples using AvroStorage.Stan Rosenberg 2012-03-26, 15:11
Hi Dan,
Could you attach your script and sample input files for both cases
(with and without the schema). In the case where no schema is
provided, the stack trace shows that
the 'pigSchema' which is passed into AvroStorageUtils.isTupleWrapper
has no name, hence the NPE. I am not sure why that happens.
I will investigate further once I can execute your scripts.
Best,
stan
On Sun, Mar 25, 2012 at 10:41 AM, Stan Rosenberg
<[EMAIL PROTECTED]> wrote:
> Hi Dan,
>
> This looks like an avro bug. I'll have a look later tonight unless someone
> else has a more immediate answer.
>
> Best,
>
> stan
>
> On Mar 25, 2012 12:36 AM, "Dan Young" <[EMAIL PROTECTED]> wrote:
>>
>> Hello all,
>>
>> I'm trying to store a bag of tuples using AvroStorage but am not able to
>> figure out what I'm doing wrong (or if it' supported). What I have is the
>> following:
>>
>> grunt>illustrate c;
>> ....
>> ....
>>
>> ---------------------------------------------------------------------------------------------
>> | c | franchise_id:int |
>> cast_and_crew:bag{:tuple(full_name:chararray)}|
>>
>> ---------------------------------------------------------------------------------------------
>> | | 213939 | {(Wang Junzheng), (Li Ling)}
>> |
>>
>> ---------------------------------------------------------------------------------------------
>>
>> When I try to store is via AvroStorage I'm getting an error. I've tried
>> both with and without specifying a schema I've created a gist and posted
>> it here with the Pig Stack Trace.....
>>
>> https://gist.github.com/2191363
>>
>>
>> If anyone has insight into what I'm doing wrong, and can help, I would
>> greatly appreciate the guidance.
>>
>>
>> Regards,
>>
>> Dan
Could you attach your script and sample input files for both cases
(with and without the schema). In the case where no schema is
provided, the stack trace shows that
the 'pigSchema' which is passed into AvroStorageUtils.isTupleWrapper
has no name, hence the NPE. I am not sure why that happens.
I will investigate further once I can execute your scripts.
Best,
stan
On Sun, Mar 25, 2012 at 10:41 AM, Stan Rosenberg
<[EMAIL PROTECTED]> wrote:
> Hi Dan,
>
> This looks like an avro bug. I'll have a look later tonight unless someone
> else has a more immediate answer.
>
> Best,
>
> stan
>
> On Mar 25, 2012 12:36 AM, "Dan Young" <[EMAIL PROTECTED]> wrote:
>>
>> Hello all,
>>
>> I'm trying to store a bag of tuples using AvroStorage but am not able to
>> figure out what I'm doing wrong (or if it' supported). What I have is the
>> following:
>>
>> grunt>illustrate c;
>> ....
>> ....
>>
>> ---------------------------------------------------------------------------------------------
>> | c | franchise_id:int |
>> cast_and_crew:bag{:tuple(full_name:chararray)}|
>>
>> ---------------------------------------------------------------------------------------------
>> | | 213939 | {(Wang Junzheng), (Li Ling)}
>> |
>>
>> ---------------------------------------------------------------------------------------------
>>
>> When I try to store is via AvroStorage I'm getting an error. I've tried
>> both with and without specifying a schema I've created a gist and posted
>> it here with the Pig Stack Trace.....
>>
>> https://gist.github.com/2191363
>>
>>
>> If anyone has insight into what I'm doing wrong, and can help, I would
>> greatly appreciate the guidance.
>>
>>
>> Regards,
>>
>> Dan
+
Stan Rosenberg 2012-03-26, 15:11
-
Re: Trying to store a bag of tuples using AvroStorage.Dan Young 2012-04-03, 16:13
Hello Stan,
I'm back from Mexico now, and here's my GIST with all the information.
https://gist.github.com/2293226
Any insight into what I'm not doing correctly would be greatly appreciated.
Regards,
Dan
On Mon, Mar 26, 2012 at 9:11 AM, Stan Rosenberg <[EMAIL PROTECTED]>wrote:
> Hi Dan,
>
> Could you attach your script and sample input files for both cases
> (with and without the schema). In the case where no schema is
> provided, the stack trace shows that
> the 'pigSchema' which is passed into AvroStorageUtils.isTupleWrapper
> has no name, hence the NPE. I am not sure why that happens.
> I will investigate further once I can execute your scripts.
>
> Best,
>
> stan
>
> On Sun, Mar 25, 2012 at 10:41 AM, Stan Rosenberg
> <[EMAIL PROTECTED]> wrote:
> > Hi Dan,
> >
> > This looks like an avro bug. I'll have a look later tonight unless
> someone
> > else has a more immediate answer.
> >
> > Best,
> >
> > stan
> >
> > On Mar 25, 2012 12:36 AM, "Dan Young" <[EMAIL PROTECTED]> wrote:
> >>
> >> Hello all,
> >>
> >> I'm trying to store a bag of tuples using AvroStorage but am not able to
> >> figure out what I'm doing wrong (or if it' supported). What I have is
> the
> >> following:
> >>
> >> grunt>illustrate c;
> >> ....
> >> ....
> >>
> >>
> ---------------------------------------------------------------------------------------------
> >> | c | franchise_id:int |
> >> cast_and_crew:bag{:tuple(full_name:chararray)}|
> >>
> >>
> ---------------------------------------------------------------------------------------------
> >> | | 213939 | {(Wang Junzheng), (Li Ling)}
> >> |
> >>
> >>
> ---------------------------------------------------------------------------------------------
> >>
> >> When I try to store is via AvroStorage I'm getting an error. I've tried
> >> both with and without specifying a schema I've created a gist and
> posted
> >> it here with the Pig Stack Trace.....
> >>
> >> https://gist.github.com/2191363
> >>
> >>
> >> If anyone has insight into what I'm doing wrong, and can help, I would
> >> greatly appreciate the guidance.
> >>
> >>
> >> Regards,
> >>
> >> Dan
>
I'm back from Mexico now, and here's my GIST with all the information.
https://gist.github.com/2293226
Any insight into what I'm not doing correctly would be greatly appreciated.
Regards,
Dan
On Mon, Mar 26, 2012 at 9:11 AM, Stan Rosenberg <[EMAIL PROTECTED]>wrote:
> Hi Dan,
>
> Could you attach your script and sample input files for both cases
> (with and without the schema). In the case where no schema is
> provided, the stack trace shows that
> the 'pigSchema' which is passed into AvroStorageUtils.isTupleWrapper
> has no name, hence the NPE. I am not sure why that happens.
> I will investigate further once I can execute your scripts.
>
> Best,
>
> stan
>
> On Sun, Mar 25, 2012 at 10:41 AM, Stan Rosenberg
> <[EMAIL PROTECTED]> wrote:
> > Hi Dan,
> >
> > This looks like an avro bug. I'll have a look later tonight unless
> someone
> > else has a more immediate answer.
> >
> > Best,
> >
> > stan
> >
> > On Mar 25, 2012 12:36 AM, "Dan Young" <[EMAIL PROTECTED]> wrote:
> >>
> >> Hello all,
> >>
> >> I'm trying to store a bag of tuples using AvroStorage but am not able to
> >> figure out what I'm doing wrong (or if it' supported). What I have is
> the
> >> following:
> >>
> >> grunt>illustrate c;
> >> ....
> >> ....
> >>
> >>
> ---------------------------------------------------------------------------------------------
> >> | c | franchise_id:int |
> >> cast_and_crew:bag{:tuple(full_name:chararray)}|
> >>
> >>
> ---------------------------------------------------------------------------------------------
> >> | | 213939 | {(Wang Junzheng), (Li Ling)}
> >> |
> >>
> >>
> ---------------------------------------------------------------------------------------------
> >>
> >> When I try to store is via AvroStorage I'm getting an error. I've tried
> >> both with and without specifying a schema I've created a gist and
> posted
> >> it here with the Pig Stack Trace.....
> >>
> >> https://gist.github.com/2191363
> >>
> >>
> >> If anyone has insight into what I'm doing wrong, and can help, I would
> >> greatly appreciate the guidance.
> >>
> >>
> >> Regards,
> >>
> >> Dan
>
+
Dan Young 2012-04-03, 16:13
-
Re: Trying to store a bag of tuples using AvroStorage.Russell Jurney 2012-04-03, 17:07
This looks like a bug fixed in 0.10. Mind trying it?
Russell Jurney http://datasyndrome.com
On Apr 3, 2012, at 9:13 AM, Dan Young <[EMAIL PROTECTED]> wrote:
> Hello Stan,
>
> I'm back from Mexico now, and here's my GIST with all the information.
>
> https://gist.github.com/2293226
>
> Any insight into what I'm not doing correctly would be greatly appreciated.
>
> Regards,
>
> Dan
>
> On Mon, Mar 26, 2012 at 9:11 AM, Stan Rosenberg <[EMAIL PROTECTED]>wrote:
>
>> Hi Dan,
>>
>> Could you attach your script and sample input files for both cases
>> (with and without the schema). In the case where no schema is
>> provided, the stack trace shows that
>> the 'pigSchema' which is passed into AvroStorageUtils.isTupleWrapper
>> has no name, hence the NPE. I am not sure why that happens.
>> I will investigate further once I can execute your scripts.
>>
>> Best,
>>
>> stan
>>
>> On Sun, Mar 25, 2012 at 10:41 AM, Stan Rosenberg
>> <[EMAIL PROTECTED]> wrote:
>>> Hi Dan,
>>>
>>> This looks like an avro bug. I'll have a look later tonight unless
>> someone
>>> else has a more immediate answer.
>>>
>>> Best,
>>>
>>> stan
>>>
>>> On Mar 25, 2012 12:36 AM, "Dan Young" <[EMAIL PROTECTED]> wrote:
>>>>
>>>> Hello all,
>>>>
>>>> I'm trying to store a bag of tuples using AvroStorage but am not able to
>>>> figure out what I'm doing wrong (or if it' supported). What I have is
>> the
>>>> following:
>>>>
>>>> grunt>illustrate c;
>>>> ....
>>>> ....
>>>>
>>>>
>> ---------------------------------------------------------------------------------------------
>>>> | c | franchise_id:int |
>>>> cast_and_crew:bag{:tuple(full_name:chararray)}|
>>>>
>>>>
>> ---------------------------------------------------------------------------------------------
>>>> | | 213939 | {(Wang Junzheng), (Li Ling)}
>>>> |
>>>>
>>>>
>> ---------------------------------------------------------------------------------------------
>>>>
>>>> When I try to store is via AvroStorage I'm getting an error. I've tried
>>>> both with and without specifying a schema I've created a gist and
>> posted
>>>> it here with the Pig Stack Trace.....
>>>>
>>>> https://gist.github.com/2191363
>>>>
>>>>
>>>> If anyone has insight into what I'm doing wrong, and can help, I would
>>>> greatly appreciate the guidance.
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Dan
>>
Russell Jurney http://datasyndrome.com
On Apr 3, 2012, at 9:13 AM, Dan Young <[EMAIL PROTECTED]> wrote:
> Hello Stan,
>
> I'm back from Mexico now, and here's my GIST with all the information.
>
> https://gist.github.com/2293226
>
> Any insight into what I'm not doing correctly would be greatly appreciated.
>
> Regards,
>
> Dan
>
> On Mon, Mar 26, 2012 at 9:11 AM, Stan Rosenberg <[EMAIL PROTECTED]>wrote:
>
>> Hi Dan,
>>
>> Could you attach your script and sample input files for both cases
>> (with and without the schema). In the case where no schema is
>> provided, the stack trace shows that
>> the 'pigSchema' which is passed into AvroStorageUtils.isTupleWrapper
>> has no name, hence the NPE. I am not sure why that happens.
>> I will investigate further once I can execute your scripts.
>>
>> Best,
>>
>> stan
>>
>> On Sun, Mar 25, 2012 at 10:41 AM, Stan Rosenberg
>> <[EMAIL PROTECTED]> wrote:
>>> Hi Dan,
>>>
>>> This looks like an avro bug. I'll have a look later tonight unless
>> someone
>>> else has a more immediate answer.
>>>
>>> Best,
>>>
>>> stan
>>>
>>> On Mar 25, 2012 12:36 AM, "Dan Young" <[EMAIL PROTECTED]> wrote:
>>>>
>>>> Hello all,
>>>>
>>>> I'm trying to store a bag of tuples using AvroStorage but am not able to
>>>> figure out what I'm doing wrong (or if it' supported). What I have is
>> the
>>>> following:
>>>>
>>>> grunt>illustrate c;
>>>> ....
>>>> ....
>>>>
>>>>
>> ---------------------------------------------------------------------------------------------
>>>> | c | franchise_id:int |
>>>> cast_and_crew:bag{:tuple(full_name:chararray)}|
>>>>
>>>>
>> ---------------------------------------------------------------------------------------------
>>>> | | 213939 | {(Wang Junzheng), (Li Ling)}
>>>> |
>>>>
>>>>
>> ---------------------------------------------------------------------------------------------
>>>>
>>>> When I try to store is via AvroStorage I'm getting an error. I've tried
>>>> both with and without specifying a schema I've created a gist and
>> posted
>>>> it here with the Pig Stack Trace.....
>>>>
>>>> https://gist.github.com/2191363
>>>>
>>>>
>>>> If anyone has insight into what I'm doing wrong, and can help, I would
>>>> greatly appreciate the guidance.
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Dan
>>
+
Russell Jurney 2012-04-03, 17:07
-
Re: Trying to store a bag of tuples using AvroStorage.Dan Young 2012-04-03, 17:32
I just updated my pig from svn repo and now am using the latest from trunk:
pig -i
Apache Pig version 0.11.0-SNAPSHOT (r1309051)
compiled Apr 03 2012, 11:18:53
Here's the gist with stack traces, both with or without specifying schema.
Am using piggybank from trunk.
https://gist.github.com/2293909
Regards,
Dan
On Tue, Apr 3, 2012 at 11:07 AM, Russell Jurney <[EMAIL PROTECTED]>wrote:
> This looks like a bug fixed in 0.10. Mind trying it?
>
> Russell Jurney http://datasyndrome.com
>
> On Apr 3, 2012, at 9:13 AM, Dan Young <[EMAIL PROTECTED]> wrote:
>
> > Hello Stan,
> >
> > I'm back from Mexico now, and here's my GIST with all the information.
> >
> > https://gist.github.com/2293226
> >
> > Any insight into what I'm not doing correctly would be greatly
> appreciated.
> >
> > Regards,
> >
> > Dan
> >
> > On Mon, Mar 26, 2012 at 9:11 AM, Stan Rosenberg <
> [EMAIL PROTECTED]>wrote:
> >
> >> Hi Dan,
> >>
> >> Could you attach your script and sample input files for both cases
> >> (with and without the schema). In the case where no schema is
> >> provided, the stack trace shows that
> >> the 'pigSchema' which is passed into AvroStorageUtils.isTupleWrapper
> >> has no name, hence the NPE. I am not sure why that happens.
> >> I will investigate further once I can execute your scripts.
> >>
> >> Best,
> >>
> >> stan
> >>
> >> On Sun, Mar 25, 2012 at 10:41 AM, Stan Rosenberg
> >> <[EMAIL PROTECTED]> wrote:
> >>> Hi Dan,
> >>>
> >>> This looks like an avro bug. I'll have a look later tonight unless
> >> someone
> >>> else has a more immediate answer.
> >>>
> >>> Best,
> >>>
> >>> stan
> >>>
> >>> On Mar 25, 2012 12:36 AM, "Dan Young" <[EMAIL PROTECTED]> wrote:
> >>>>
> >>>> Hello all,
> >>>>
> >>>> I'm trying to store a bag of tuples using AvroStorage but am not able
> to
> >>>> figure out what I'm doing wrong (or if it' supported). What I have is
> >> the
> >>>> following:
> >>>>
> >>>> grunt>illustrate c;
> >>>> ....
> >>>> ....
> >>>>
> >>>>
> >>
> ---------------------------------------------------------------------------------------------
> >>>> | c | franchise_id:int |
> >>>> cast_and_crew:bag{:tuple(full_name:chararray)}|
> >>>>
> >>>>
> >>
> ---------------------------------------------------------------------------------------------
> >>>> | | 213939 | {(Wang Junzheng), (Li Ling)}
> >>>> |
> >>>>
> >>>>
> >>
> ---------------------------------------------------------------------------------------------
> >>>>
> >>>> When I try to store is via AvroStorage I'm getting an error. I've
> tried
> >>>> both with and without specifying a schema I've created a gist and
> >> posted
> >>>> it here with the Pig Stack Trace.....
> >>>>
> >>>> https://gist.github.com/2191363
> >>>>
> >>>>
> >>>> If anyone has insight into what I'm doing wrong, and can help, I would
> >>>> greatly appreciate the guidance.
> >>>>
> >>>>
> >>>> Regards,
> >>>>
> >>>> Dan
> >>
>
pig -i
Apache Pig version 0.11.0-SNAPSHOT (r1309051)
compiled Apr 03 2012, 11:18:53
Here's the gist with stack traces, both with or without specifying schema.
Am using piggybank from trunk.
https://gist.github.com/2293909
Regards,
Dan
On Tue, Apr 3, 2012 at 11:07 AM, Russell Jurney <[EMAIL PROTECTED]>wrote:
> This looks like a bug fixed in 0.10. Mind trying it?
>
> Russell Jurney http://datasyndrome.com
>
> On Apr 3, 2012, at 9:13 AM, Dan Young <[EMAIL PROTECTED]> wrote:
>
> > Hello Stan,
> >
> > I'm back from Mexico now, and here's my GIST with all the information.
> >
> > https://gist.github.com/2293226
> >
> > Any insight into what I'm not doing correctly would be greatly
> appreciated.
> >
> > Regards,
> >
> > Dan
> >
> > On Mon, Mar 26, 2012 at 9:11 AM, Stan Rosenberg <
> [EMAIL PROTECTED]>wrote:
> >
> >> Hi Dan,
> >>
> >> Could you attach your script and sample input files for both cases
> >> (with and without the schema). In the case where no schema is
> >> provided, the stack trace shows that
> >> the 'pigSchema' which is passed into AvroStorageUtils.isTupleWrapper
> >> has no name, hence the NPE. I am not sure why that happens.
> >> I will investigate further once I can execute your scripts.
> >>
> >> Best,
> >>
> >> stan
> >>
> >> On Sun, Mar 25, 2012 at 10:41 AM, Stan Rosenberg
> >> <[EMAIL PROTECTED]> wrote:
> >>> Hi Dan,
> >>>
> >>> This looks like an avro bug. I'll have a look later tonight unless
> >> someone
> >>> else has a more immediate answer.
> >>>
> >>> Best,
> >>>
> >>> stan
> >>>
> >>> On Mar 25, 2012 12:36 AM, "Dan Young" <[EMAIL PROTECTED]> wrote:
> >>>>
> >>>> Hello all,
> >>>>
> >>>> I'm trying to store a bag of tuples using AvroStorage but am not able
> to
> >>>> figure out what I'm doing wrong (or if it' supported). What I have is
> >> the
> >>>> following:
> >>>>
> >>>> grunt>illustrate c;
> >>>> ....
> >>>> ....
> >>>>
> >>>>
> >>
> ---------------------------------------------------------------------------------------------
> >>>> | c | franchise_id:int |
> >>>> cast_and_crew:bag{:tuple(full_name:chararray)}|
> >>>>
> >>>>
> >>
> ---------------------------------------------------------------------------------------------
> >>>> | | 213939 | {(Wang Junzheng), (Li Ling)}
> >>>> |
> >>>>
> >>>>
> >>
> ---------------------------------------------------------------------------------------------
> >>>>
> >>>> When I try to store is via AvroStorage I'm getting an error. I've
> tried
> >>>> both with and without specifying a schema I've created a gist and
> >> posted
> >>>> it here with the Pig Stack Trace.....
> >>>>
> >>>> https://gist.github.com/2191363
> >>>>
> >>>>
> >>>> If anyone has insight into what I'm doing wrong, and can help, I would
> >>>> greatly appreciate the guidance.
> >>>>
> >>>>
> >>>> Regards,
> >>>>
> >>>> Dan
> >>
>
+
Dan Young 2012-04-03, 17:32
-
Re: Trying to store a bag of tuples using AvroStorage.Bill Graham 2012-04-03, 18:11
In the schema approach the error is that your json is invalid. You're
missing a second '}' before the last ']'.
On Tue, Apr 3, 2012 at 10:32 AM, Dan Young <[EMAIL PROTECTED]> wrote:
> I just updated my pig from svn repo and now am using the latest from trunk:
>
> pig -i
> Apache Pig version 0.11.0-SNAPSHOT (r1309051)
> compiled Apr 03 2012, 11:18:53
>
> Here's the gist with stack traces, both with or without specifying schema.
> Am using piggybank from trunk.
>
> https://gist.github.com/2293909
>
> Regards,
>
> Dan
>
>
> On Tue, Apr 3, 2012 at 11:07 AM, Russell Jurney <[EMAIL PROTECTED]
> >wrote:
>
> > This looks like a bug fixed in 0.10. Mind trying it?
> >
> > Russell Jurney http://datasyndrome.com
> >
> > On Apr 3, 2012, at 9:13 AM, Dan Young <[EMAIL PROTECTED]> wrote:
> >
> > > Hello Stan,
> > >
> > > I'm back from Mexico now, and here's my GIST with all the information.
> > >
> > > https://gist.github.com/2293226
> > >
> > > Any insight into what I'm not doing correctly would be greatly
> > appreciated.
> > >
> > > Regards,
> > >
> > > Dan
> > >
> > > On Mon, Mar 26, 2012 at 9:11 AM, Stan Rosenberg <
> > [EMAIL PROTECTED]>wrote:
> > >
> > >> Hi Dan,
> > >>
> > >> Could you attach your script and sample input files for both cases
> > >> (with and without the schema). In the case where no schema is
> > >> provided, the stack trace shows that
> > >> the 'pigSchema' which is passed into AvroStorageUtils.isTupleWrapper
> > >> has no name, hence the NPE. I am not sure why that happens.
> > >> I will investigate further once I can execute your scripts.
> > >>
> > >> Best,
> > >>
> > >> stan
> > >>
> > >> On Sun, Mar 25, 2012 at 10:41 AM, Stan Rosenberg
> > >> <[EMAIL PROTECTED]> wrote:
> > >>> Hi Dan,
> > >>>
> > >>> This looks like an avro bug. I'll have a look later tonight unless
> > >> someone
> > >>> else has a more immediate answer.
> > >>>
> > >>> Best,
> > >>>
> > >>> stan
> > >>>
> > >>> On Mar 25, 2012 12:36 AM, "Dan Young" <[EMAIL PROTECTED]> wrote:
> > >>>>
> > >>>> Hello all,
> > >>>>
> > >>>> I'm trying to store a bag of tuples using AvroStorage but am not
> able
> > to
> > >>>> figure out what I'm doing wrong (or if it' supported). What I have
> is
> > >> the
> > >>>> following:
> > >>>>
> > >>>> grunt>illustrate c;
> > >>>> ....
> > >>>> ....
> > >>>>
> > >>>>
> > >>
> >
> ---------------------------------------------------------------------------------------------
> > >>>> | c | franchise_id:int |
> > >>>> cast_and_crew:bag{:tuple(full_name:chararray)}|
> > >>>>
> > >>>>
> > >>
> >
> ---------------------------------------------------------------------------------------------
> > >>>> | | 213939 | {(Wang Junzheng), (Li Ling)}
> > >>>> |
> > >>>>
> > >>>>
> > >>
> >
> ---------------------------------------------------------------------------------------------
> > >>>>
> > >>>> When I try to store is via AvroStorage I'm getting an error. I've
> > tried
> > >>>> both with and without specifying a schema I've created a gist and
> > >> posted
> > >>>> it here with the Pig Stack Trace.....
> > >>>>
> > >>>> https://gist.github.com/2191363
> > >>>>
> > >>>>
> > >>>> If anyone has insight into what I'm doing wrong, and can help, I
> would
> > >>>> greatly appreciate the guidance.
> > >>>>
> > >>>>
> > >>>> Regards,
> > >>>>
> > >>>> Dan
> > >>
> >
>
--
*Note that I'm no longer using my Yahoo! email address. Please email me at
[EMAIL PROTECTED] going forward.*
missing a second '}' before the last ']'.
On Tue, Apr 3, 2012 at 10:32 AM, Dan Young <[EMAIL PROTECTED]> wrote:
> I just updated my pig from svn repo and now am using the latest from trunk:
>
> pig -i
> Apache Pig version 0.11.0-SNAPSHOT (r1309051)
> compiled Apr 03 2012, 11:18:53
>
> Here's the gist with stack traces, both with or without specifying schema.
> Am using piggybank from trunk.
>
> https://gist.github.com/2293909
>
> Regards,
>
> Dan
>
>
> On Tue, Apr 3, 2012 at 11:07 AM, Russell Jurney <[EMAIL PROTECTED]
> >wrote:
>
> > This looks like a bug fixed in 0.10. Mind trying it?
> >
> > Russell Jurney http://datasyndrome.com
> >
> > On Apr 3, 2012, at 9:13 AM, Dan Young <[EMAIL PROTECTED]> wrote:
> >
> > > Hello Stan,
> > >
> > > I'm back from Mexico now, and here's my GIST with all the information.
> > >
> > > https://gist.github.com/2293226
> > >
> > > Any insight into what I'm not doing correctly would be greatly
> > appreciated.
> > >
> > > Regards,
> > >
> > > Dan
> > >
> > > On Mon, Mar 26, 2012 at 9:11 AM, Stan Rosenberg <
> > [EMAIL PROTECTED]>wrote:
> > >
> > >> Hi Dan,
> > >>
> > >> Could you attach your script and sample input files for both cases
> > >> (with and without the schema). In the case where no schema is
> > >> provided, the stack trace shows that
> > >> the 'pigSchema' which is passed into AvroStorageUtils.isTupleWrapper
> > >> has no name, hence the NPE. I am not sure why that happens.
> > >> I will investigate further once I can execute your scripts.
> > >>
> > >> Best,
> > >>
> > >> stan
> > >>
> > >> On Sun, Mar 25, 2012 at 10:41 AM, Stan Rosenberg
> > >> <[EMAIL PROTECTED]> wrote:
> > >>> Hi Dan,
> > >>>
> > >>> This looks like an avro bug. I'll have a look later tonight unless
> > >> someone
> > >>> else has a more immediate answer.
> > >>>
> > >>> Best,
> > >>>
> > >>> stan
> > >>>
> > >>> On Mar 25, 2012 12:36 AM, "Dan Young" <[EMAIL PROTECTED]> wrote:
> > >>>>
> > >>>> Hello all,
> > >>>>
> > >>>> I'm trying to store a bag of tuples using AvroStorage but am not
> able
> > to
> > >>>> figure out what I'm doing wrong (or if it' supported). What I have
> is
> > >> the
> > >>>> following:
> > >>>>
> > >>>> grunt>illustrate c;
> > >>>> ....
> > >>>> ....
> > >>>>
> > >>>>
> > >>
> >
> ---------------------------------------------------------------------------------------------
> > >>>> | c | franchise_id:int |
> > >>>> cast_and_crew:bag{:tuple(full_name:chararray)}|
> > >>>>
> > >>>>
> > >>
> >
> ---------------------------------------------------------------------------------------------
> > >>>> | | 213939 | {(Wang Junzheng), (Li Ling)}
> > >>>> |
> > >>>>
> > >>>>
> > >>
> >
> ---------------------------------------------------------------------------------------------
> > >>>>
> > >>>> When I try to store is via AvroStorage I'm getting an error. I've
> > tried
> > >>>> both with and without specifying a schema I've created a gist and
> > >> posted
> > >>>> it here with the Pig Stack Trace.....
> > >>>>
> > >>>> https://gist.github.com/2191363
> > >>>>
> > >>>>
> > >>>> If anyone has insight into what I'm doing wrong, and can help, I
> would
> > >>>> greatly appreciate the guidance.
> > >>>>
> > >>>>
> > >>>> Regards,
> > >>>>
> > >>>> Dan
> > >>
> >
>
--
*Note that I'm no longer using my Yahoo! email address. Please email me at
[EMAIL PROTECTED] going forward.*
+
Bill Graham 2012-04-03, 18:11
-
Re: Trying to store a bag of tuples using AvroStorage.Dan Young 2012-04-03, 18:21
Dooohh....thank you for pointing that out....I thought I ran that thru
jsonlint.....That seemed to fix it........
Regards,
Dano
On Tue, Apr 3, 2012 at 12:11 PM, Bill Graham <[EMAIL PROTECTED]> wrote:
> In the schema approach the error is that your json is invalid. You're
> missing a second '}' before the last ']'.
>
>
>
> On Tue, Apr 3, 2012 at 10:32 AM, Dan Young <[EMAIL PROTECTED]> wrote:
>
> > I just updated my pig from svn repo and now am using the latest from
> trunk:
> >
> > pig -i
> > Apache Pig version 0.11.0-SNAPSHOT (r1309051)
> > compiled Apr 03 2012, 11:18:53
> >
> > Here's the gist with stack traces, both with or without specifying
> schema.
> > Am using piggybank from trunk.
> >
> > https://gist.github.com/2293909
> >
> > Regards,
> >
> > Dan
> >
> >
> > On Tue, Apr 3, 2012 at 11:07 AM, Russell Jurney <
> [EMAIL PROTECTED]
> > >wrote:
> >
> > > This looks like a bug fixed in 0.10. Mind trying it?
> > >
> > > Russell Jurney http://datasyndrome.com
> > >
> > > On Apr 3, 2012, at 9:13 AM, Dan Young <[EMAIL PROTECTED]> wrote:
> > >
> > > > Hello Stan,
> > > >
> > > > I'm back from Mexico now, and here's my GIST with all the
> information.
> > > >
> > > > https://gist.github.com/2293226
> > > >
> > > > Any insight into what I'm not doing correctly would be greatly
> > > appreciated.
> > > >
> > > > Regards,
> > > >
> > > > Dan
> > > >
> > > > On Mon, Mar 26, 2012 at 9:11 AM, Stan Rosenberg <
> > > [EMAIL PROTECTED]>wrote:
> > > >
> > > >> Hi Dan,
> > > >>
> > > >> Could you attach your script and sample input files for both cases
> > > >> (with and without the schema). In the case where no schema is
> > > >> provided, the stack trace shows that
> > > >> the 'pigSchema' which is passed into AvroStorageUtils.isTupleWrapper
> > > >> has no name, hence the NPE. I am not sure why that happens.
> > > >> I will investigate further once I can execute your scripts.
> > > >>
> > > >> Best,
> > > >>
> > > >> stan
> > > >>
> > > >> On Sun, Mar 25, 2012 at 10:41 AM, Stan Rosenberg
> > > >> <[EMAIL PROTECTED]> wrote:
> > > >>> Hi Dan,
> > > >>>
> > > >>> This looks like an avro bug. I'll have a look later tonight unless
> > > >> someone
> > > >>> else has a more immediate answer.
> > > >>>
> > > >>> Best,
> > > >>>
> > > >>> stan
> > > >>>
> > > >>> On Mar 25, 2012 12:36 AM, "Dan Young" <[EMAIL PROTECTED]> wrote:
> > > >>>>
> > > >>>> Hello all,
> > > >>>>
> > > >>>> I'm trying to store a bag of tuples using AvroStorage but am not
> > able
> > > to
> > > >>>> figure out what I'm doing wrong (or if it' supported). What I
> have
> > is
> > > >> the
> > > >>>> following:
> > > >>>>
> > > >>>> grunt>illustrate c;
> > > >>>> ....
> > > >>>> ....
> > > >>>>
> > > >>>>
> > > >>
> > >
> >
> ---------------------------------------------------------------------------------------------
> > > >>>> | c | franchise_id:int |
> > > >>>> cast_and_crew:bag{:tuple(full_name:chararray)}|
> > > >>>>
> > > >>>>
> > > >>
> > >
> >
> ---------------------------------------------------------------------------------------------
> > > >>>> | | 213939 | {(Wang Junzheng), (Li Ling)}
> > > >>>> |
> > > >>>>
> > > >>>>
> > > >>
> > >
> >
> ---------------------------------------------------------------------------------------------
> > > >>>>
> > > >>>> When I try to store is via AvroStorage I'm getting an error. I've
> > > tried
> > > >>>> both with and without specifying a schema I've created a gist and
> > > >> posted
> > > >>>> it here with the Pig Stack Trace.....
> > > >>>>
> > > >>>> https://gist.github.com/2191363
> > > >>>>
> > > >>>>
> > > >>>> If anyone has insight into what I'm doing wrong, and can help, I
> > would
> > > >>>> greatly appreciate the guidance.
> > > >>>>
> > > >>>>
> > > >>>> Regards,
> > > >>>>
> > > >>>> Dan
> > > >>
> > >
> >
>
>
>
> --
> *Note that I'm no longer using my Yahoo! email address. Please email me at
> [EMAIL PROTECTED] going forward.*
>
jsonlint.....That seemed to fix it........
Regards,
Dano
On Tue, Apr 3, 2012 at 12:11 PM, Bill Graham <[EMAIL PROTECTED]> wrote:
> In the schema approach the error is that your json is invalid. You're
> missing a second '}' before the last ']'.
>
>
>
> On Tue, Apr 3, 2012 at 10:32 AM, Dan Young <[EMAIL PROTECTED]> wrote:
>
> > I just updated my pig from svn repo and now am using the latest from
> trunk:
> >
> > pig -i
> > Apache Pig version 0.11.0-SNAPSHOT (r1309051)
> > compiled Apr 03 2012, 11:18:53
> >
> > Here's the gist with stack traces, both with or without specifying
> schema.
> > Am using piggybank from trunk.
> >
> > https://gist.github.com/2293909
> >
> > Regards,
> >
> > Dan
> >
> >
> > On Tue, Apr 3, 2012 at 11:07 AM, Russell Jurney <
> [EMAIL PROTECTED]
> > >wrote:
> >
> > > This looks like a bug fixed in 0.10. Mind trying it?
> > >
> > > Russell Jurney http://datasyndrome.com
> > >
> > > On Apr 3, 2012, at 9:13 AM, Dan Young <[EMAIL PROTECTED]> wrote:
> > >
> > > > Hello Stan,
> > > >
> > > > I'm back from Mexico now, and here's my GIST with all the
> information.
> > > >
> > > > https://gist.github.com/2293226
> > > >
> > > > Any insight into what I'm not doing correctly would be greatly
> > > appreciated.
> > > >
> > > > Regards,
> > > >
> > > > Dan
> > > >
> > > > On Mon, Mar 26, 2012 at 9:11 AM, Stan Rosenberg <
> > > [EMAIL PROTECTED]>wrote:
> > > >
> > > >> Hi Dan,
> > > >>
> > > >> Could you attach your script and sample input files for both cases
> > > >> (with and without the schema). In the case where no schema is
> > > >> provided, the stack trace shows that
> > > >> the 'pigSchema' which is passed into AvroStorageUtils.isTupleWrapper
> > > >> has no name, hence the NPE. I am not sure why that happens.
> > > >> I will investigate further once I can execute your scripts.
> > > >>
> > > >> Best,
> > > >>
> > > >> stan
> > > >>
> > > >> On Sun, Mar 25, 2012 at 10:41 AM, Stan Rosenberg
> > > >> <[EMAIL PROTECTED]> wrote:
> > > >>> Hi Dan,
> > > >>>
> > > >>> This looks like an avro bug. I'll have a look later tonight unless
> > > >> someone
> > > >>> else has a more immediate answer.
> > > >>>
> > > >>> Best,
> > > >>>
> > > >>> stan
> > > >>>
> > > >>> On Mar 25, 2012 12:36 AM, "Dan Young" <[EMAIL PROTECTED]> wrote:
> > > >>>>
> > > >>>> Hello all,
> > > >>>>
> > > >>>> I'm trying to store a bag of tuples using AvroStorage but am not
> > able
> > > to
> > > >>>> figure out what I'm doing wrong (or if it' supported). What I
> have
> > is
> > > >> the
> > > >>>> following:
> > > >>>>
> > > >>>> grunt>illustrate c;
> > > >>>> ....
> > > >>>> ....
> > > >>>>
> > > >>>>
> > > >>
> > >
> >
> ---------------------------------------------------------------------------------------------
> > > >>>> | c | franchise_id:int |
> > > >>>> cast_and_crew:bag{:tuple(full_name:chararray)}|
> > > >>>>
> > > >>>>
> > > >>
> > >
> >
> ---------------------------------------------------------------------------------------------
> > > >>>> | | 213939 | {(Wang Junzheng), (Li Ling)}
> > > >>>> |
> > > >>>>
> > > >>>>
> > > >>
> > >
> >
> ---------------------------------------------------------------------------------------------
> > > >>>>
> > > >>>> When I try to store is via AvroStorage I'm getting an error. I've
> > > tried
> > > >>>> both with and without specifying a schema I've created a gist and
> > > >> posted
> > > >>>> it here with the Pig Stack Trace.....
> > > >>>>
> > > >>>> https://gist.github.com/2191363
> > > >>>>
> > > >>>>
> > > >>>> If anyone has insight into what I'm doing wrong, and can help, I
> > would
> > > >>>> greatly appreciate the guidance.
> > > >>>>
> > > >>>>
> > > >>>> Regards,
> > > >>>>
> > > >>>> Dan
> > > >>
> > >
> >
>
>
>
> --
> *Note that I'm no longer using my Yahoo! email address. Please email me at
> [EMAIL PROTECTED] going forward.*
>
+
Dan Young 2012-04-03, 18:21
-
Re: Trying to store a bag of tuples using AvroStorage.Dan Young 2012-04-03, 17:17
Here's the version of Pig I'm using:
pig -i
Apache Pig version 0.11.0-SNAPSHOT (r1304979)
compiled Mar 24 2012, 21:48:44
The version of Hadoop:
*Version:* 1.0.0, r1214675
Regards,
Dan
On Tue, Apr 3, 2012 at 11:07 AM, Russell Jurney <[EMAIL PROTECTED]>wrote:
> This looks like a bug fixed in 0.10. Mind trying it?
>
> Russell Jurney http://datasyndrome.com
>
> On Apr 3, 2012, at 9:13 AM, Dan Young <[EMAIL PROTECTED]> wrote:
>
> > Hello Stan,
> >
> > I'm back from Mexico now, and here's my GIST with all the information.
> >
> > https://gist.github.com/2293226
> >
> > Any insight into what I'm not doing correctly would be greatly
> appreciated.
> >
> > Regards,
> >
> > Dan
> >
> > On Mon, Mar 26, 2012 at 9:11 AM, Stan Rosenberg <
> [EMAIL PROTECTED]>wrote:
> >
> >> Hi Dan,
> >>
> >> Could you attach your script and sample input files for both cases
> >> (with and without the schema). In the case where no schema is
> >> provided, the stack trace shows that
> >> the 'pigSchema' which is passed into AvroStorageUtils.isTupleWrapper
> >> has no name, hence the NPE. I am not sure why that happens.
> >> I will investigate further once I can execute your scripts.
> >>
> >> Best,
> >>
> >> stan
> >>
> >> On Sun, Mar 25, 2012 at 10:41 AM, Stan Rosenberg
> >> <[EMAIL PROTECTED]> wrote:
> >>> Hi Dan,
> >>>
> >>> This looks like an avro bug. I'll have a look later tonight unless
> >> someone
> >>> else has a more immediate answer.
> >>>
> >>> Best,
> >>>
> >>> stan
> >>>
> >>> On Mar 25, 2012 12:36 AM, "Dan Young" <[EMAIL PROTECTED]> wrote:
> >>>>
> >>>> Hello all,
> >>>>
> >>>> I'm trying to store a bag of tuples using AvroStorage but am not able
> to
> >>>> figure out what I'm doing wrong (or if it' supported). What I have is
> >> the
> >>>> following:
> >>>>
> >>>> grunt>illustrate c;
> >>>> ....
> >>>> ....
> >>>>
> >>>>
> >>
> ---------------------------------------------------------------------------------------------
> >>>> | c | franchise_id:int |
> >>>> cast_and_crew:bag{:tuple(full_name:chararray)}|
> >>>>
> >>>>
> >>
> ---------------------------------------------------------------------------------------------
> >>>> | | 213939 | {(Wang Junzheng), (Li Ling)}
> >>>> |
> >>>>
> >>>>
> >>
> ---------------------------------------------------------------------------------------------
> >>>>
> >>>> When I try to store is via AvroStorage I'm getting an error. I've
> tried
> >>>> both with and without specifying a schema I've created a gist and
> >> posted
> >>>> it here with the Pig Stack Trace.....
> >>>>
> >>>> https://gist.github.com/2191363
> >>>>
> >>>>
> >>>> If anyone has insight into what I'm doing wrong, and can help, I would
> >>>> greatly appreciate the guidance.
> >>>>
> >>>>
> >>>> Regards,
> >>>>
> >>>> Dan
> >>
>
pig -i
Apache Pig version 0.11.0-SNAPSHOT (r1304979)
compiled Mar 24 2012, 21:48:44
The version of Hadoop:
*Version:* 1.0.0, r1214675
Regards,
Dan
On Tue, Apr 3, 2012 at 11:07 AM, Russell Jurney <[EMAIL PROTECTED]>wrote:
> This looks like a bug fixed in 0.10. Mind trying it?
>
> Russell Jurney http://datasyndrome.com
>
> On Apr 3, 2012, at 9:13 AM, Dan Young <[EMAIL PROTECTED]> wrote:
>
> > Hello Stan,
> >
> > I'm back from Mexico now, and here's my GIST with all the information.
> >
> > https://gist.github.com/2293226
> >
> > Any insight into what I'm not doing correctly would be greatly
> appreciated.
> >
> > Regards,
> >
> > Dan
> >
> > On Mon, Mar 26, 2012 at 9:11 AM, Stan Rosenberg <
> [EMAIL PROTECTED]>wrote:
> >
> >> Hi Dan,
> >>
> >> Could you attach your script and sample input files for both cases
> >> (with and without the schema). In the case where no schema is
> >> provided, the stack trace shows that
> >> the 'pigSchema' which is passed into AvroStorageUtils.isTupleWrapper
> >> has no name, hence the NPE. I am not sure why that happens.
> >> I will investigate further once I can execute your scripts.
> >>
> >> Best,
> >>
> >> stan
> >>
> >> On Sun, Mar 25, 2012 at 10:41 AM, Stan Rosenberg
> >> <[EMAIL PROTECTED]> wrote:
> >>> Hi Dan,
> >>>
> >>> This looks like an avro bug. I'll have a look later tonight unless
> >> someone
> >>> else has a more immediate answer.
> >>>
> >>> Best,
> >>>
> >>> stan
> >>>
> >>> On Mar 25, 2012 12:36 AM, "Dan Young" <[EMAIL PROTECTED]> wrote:
> >>>>
> >>>> Hello all,
> >>>>
> >>>> I'm trying to store a bag of tuples using AvroStorage but am not able
> to
> >>>> figure out what I'm doing wrong (or if it' supported). What I have is
> >> the
> >>>> following:
> >>>>
> >>>> grunt>illustrate c;
> >>>> ....
> >>>> ....
> >>>>
> >>>>
> >>
> ---------------------------------------------------------------------------------------------
> >>>> | c | franchise_id:int |
> >>>> cast_and_crew:bag{:tuple(full_name:chararray)}|
> >>>>
> >>>>
> >>
> ---------------------------------------------------------------------------------------------
> >>>> | | 213939 | {(Wang Junzheng), (Li Ling)}
> >>>> |
> >>>>
> >>>>
> >>
> ---------------------------------------------------------------------------------------------
> >>>>
> >>>> When I try to store is via AvroStorage I'm getting an error. I've
> tried
> >>>> both with and without specifying a schema I've created a gist and
> >> posted
> >>>> it here with the Pig Stack Trace.....
> >>>>
> >>>> https://gist.github.com/2191363
> >>>>
> >>>>
> >>>> If anyone has insight into what I'm doing wrong, and can help, I would
> >>>> greatly appreciate the guidance.
> >>>>
> >>>>
> >>>> Regards,
> >>>>
> >>>> Dan
> >>
>
+
Dan Young 2012-04-03, 17:17
