This wasn't a problem in 0.9.2, but in 0.10, when I try to access a key in
a map that has a dollar sign in it, I get hammered with errors that I
haven't defined the variable. Specifically:

   blah = FOREACH meh GENERATE source, json_post_id#'$id' AS post_id;

returns

   Undefined parameter : id

That's fine and makes sense, but when I amend it to:

   blah = FOREACH meh GENERATE source, json_post_id#'\$id' AS post_id;

I get:

   Unexpected character '$'

Ideas?

Thanks!
Eli

Two back slashes?
On Thu, Jan 10, 2013 at 6:01 PM, Eli Finkelshteyn <[EMAIL PROTECTED]>wrote:

> This wasn't a problem in 0.9.2, but in 0.10, when I try to access a key in
> a map that has a dollar sign in it, I get hammered with errors that I
> haven't defined the variable. Specifically:
>
>    blah = FOREACH meh GENERATE source, json_post_id#'$id' AS post_id;
>
> returns
>
>    Undefined parameter : id
>
> That's fine and makes sense, but when I amend it to:
>
>    blah = FOREACH meh GENERATE source, json_post_id#'\$id' AS post_id;
>
> I get:
>
>    Unexpected character '$'
>
> Ideas?
>
> Thanks!
> Eli
>

Thanks Dmitriy, but no dice. It returns the whole thing to me with post_id as blank. I think with two backslashes it ends up looking for a backslash in the key (i.e. '\$id') which doesn't exist. I also tried the unicode for the dollar symbol \\u0024.

Eli

On Jan 10, 2013, at 7:33 PM, Dmitriy Ryaboy wrote:

> Two back slashes?
>
>
> On Thu, Jan 10, 2013 at 6:01 PM, Eli Finkelshteyn <[EMAIL PROTECTED]>wrote:
>
>> This wasn't a problem in 0.9.2, but in 0.10, when I try to access a key in
>> a map that has a dollar sign in it, I get hammered with errors that I
>> haven't defined the variable. Specifically:
>>
>>   blah = FOREACH meh GENERATE source, json_post_id#'$id' AS post_id;
>>
>> returns
>>
>>   Undefined parameter : id
>>
>> That's fine and makes sense, but when I amend it to:
>>
>>   blah = FOREACH meh GENERATE source, json_post_id#'\$id' AS post_id;
>>
>> I get:
>>
>>   Unexpected character '$'
>>
>> Ideas?
>>
>> Thanks!
>> Eli
>>

Any other ideas? Still no solution on this.

Eli

On Jan 10, 2013, at 7:33 PM, Dmitriy Ryaboy wrote:

> Two back slashes?
>
>
> On Thu, Jan 10, 2013 at 6:01 PM, Eli Finkelshteyn <[EMAIL PROTECTED]>wrote:
>
>> This wasn't a problem in 0.9.2, but in 0.10, when I try to access a key in
>> a map that has a dollar sign in it, I get hammered with errors that I
>> haven't defined the variable. Specifically:
>>
>>   blah = FOREACH meh GENERATE source, json_post_id#'$id' AS post_id;
>>
>> returns
>>
>>   Undefined parameter : id
>>
>> That's fine and makes sense, but when I amend it to:
>>
>>   blah = FOREACH meh GENERATE source, json_post_id#'\$id' AS post_id;
>>
>> I get:
>>
>>   Unexpected character '$'
>>
>> Ideas?
>>
>> Thanks!
>> Eli
>>

Hi Eli,

I tried to reproduce your problem. What I found is that the behavior is
different when using Grunt shell and running it as a script.

* Input file *

cheolsoo@localhost:~/workspace/pig-svn $cat 1.txt
$id,a

* Grunt shell *

The $ with no backslash works:

grunt> A = LOAD '1.txt' USING PigStorage(',') AS (k:chararray, v:chararray);
grunt> B = FOREACH A GENERATE TOMAP(k, v) AS M;
grunt> C = FOREACH B GENERATE M#'$id';
grunt> DUMP C;
(a)

* Script *

The $ with a single backslash works:

cheolsoo@localhost:~/workspace/pig-svn $cat test.pig
A = LOAD '1.txt' USING PigStorage(',') AS (k:chararray, v:chararray);
B = FOREACH A GENERATE TOMAP(k, v) AS M;
C = FOREACH B GENERATE M#'\$id';
DUMP C;

cheolsoo@localhost:~/workspace/pig-svn $./bin/pig -x local test.pig
(a)

Please let me know if you see different results.

Thanks,
Cheolsoo
On Mon, Jan 14, 2013 at 7:40 AM, Eli Finkelshteyn <[EMAIL PROTECTED]>wrote:

> Any other ideas? Still no solution on this.
>
> Eli
>
> On Jan 10, 2013, at 7:33 PM, Dmitriy Ryaboy wrote:
>
> > Two back slashes?
> >
> >
> > On Thu, Jan 10, 2013 at 6:01 PM, Eli Finkelshteyn <[EMAIL PROTECTED]
> >wrote:
> >
> >> This wasn't a problem in 0.9.2, but in 0.10, when I try to access a key
> in
> >> a map that has a dollar sign in it, I get hammered with errors that I
> >> haven't defined the variable. Specifically:
> >>
> >>   blah = FOREACH meh GENERATE source, json_post_id#'$id' AS post_id;
> >>
> >> returns
> >>
> >>   Undefined parameter : id
> >>
> >> That's fine and makes sense, but when I amend it to:
> >>
> >>   blah = FOREACH meh GENERATE source, json_post_id#'\$id' AS post_id;
> >>
> >> I get:
> >>
> >>   Unexpected character '$'
> >>
> >> Ideas?
> >>
> >> Thanks!
> >> Eli
> >>
>
>

I've encountered the difference myself, and find that it helps immensely
to run pig -r on the script instead of cat, before cutting and pasting
it, so it'll strip the \ off of the $ when it does its parameter
substitution . (I still find the gotcha a little painful the other way
though, when I decide to start writing a script by copying over a chunk
of my .pig_history).

-Kris

On Tue, Jan 15, 2013 at 03:17:39PM -0800, Cheolsoo Park wrote:
> Hi Eli,
>
> I tried to reproduce your problem. What I found is that the behavior is
> different when using Grunt shell and running it as a script.
>
> * Input file *
>
> cheolsoo@localhost:~/workspace/pig-svn $cat 1.txt
> $id,a
>
> * Grunt shell *
>
> The $ with no backslash works:
>
> grunt> A = LOAD '1.txt' USING PigStorage(',') AS (k:chararray, v:chararray);
> grunt> B = FOREACH A GENERATE TOMAP(k, v) AS M;
> grunt> C = FOREACH B GENERATE M#'$id';
> grunt> DUMP C;
> (a)
>
> * Script *
>
> The $ with a single backslash works:
>
> cheolsoo@localhost:~/workspace/pig-svn $cat test.pig
> A = LOAD '1.txt' USING PigStorage(',') AS (k:chararray, v:chararray);
> B = FOREACH A GENERATE TOMAP(k, v) AS M;
> C = FOREACH B GENERATE M#'\$id';
> DUMP C;
>
> cheolsoo@localhost:~/workspace/pig-svn $./bin/pig -x local test.pig
> (a)
>
> Please let me know if you see different results.
>
> Thanks,
> Cheolsoo
>
>
> On Mon, Jan 14, 2013 at 7:40 AM, Eli Finkelshteyn <[EMAIL PROTECTED]>wrote:
>
> > Any other ideas? Still no solution on this.
> >
> > Eli
> >
> > On Jan 10, 2013, at 7:33 PM, Dmitriy Ryaboy wrote:
> >
> > > Two back slashes?
> > >
> > >
> > > On Thu, Jan 10, 2013 at 6:01 PM, Eli Finkelshteyn <[EMAIL PROTECTED]
> > >wrote:
> > >
> > >> This wasn't a problem in 0.9.2, but in 0.10, when I try to access a key
> > in
> > >> a map that has a dollar sign in it, I get hammered with errors that I
> > >> haven't defined the variable. Specifically:
> > >>
> > >>   blah = FOREACH meh GENERATE source, json_post_id#'$id' AS post_id;
> > >>
> > >> returns
> > >>
> > >>   Undefined parameter : id
> > >>
> > >> That's fine and makes sense, but when I amend it to:
> > >>
> > >>   blah = FOREACH meh GENERATE source, json_post_id#'\$id' AS post_id;
> > >>
> > >> I get:
> > >>
> > >>   Unexpected character '$'
> > >>
> > >> Ideas?
> > >>
> > >> Thanks!
> > >> Eli
> > >>
> >
> >

--
Kris Coward http://unripe.melon.org/
GPG Fingerprint: 2BF3 957D 310A FEEC 4733  830E 21A4 05C7 1FEB 12B3

This is exactly right. I had the regular dollar sign without escapes fail on me when running as a script and decided to trouble shoot in grunt without bothering to retry what I already knew didn't work. Really weird that grunt and scripts behave differently, but your answer solves all the problems. Thanks for the help!

Eli

On Jan 15, 2013, at 3:17 PM, Cheolsoo Park wrote:

> Hi Eli,
>
> I tried to reproduce your problem. What I found is that the behavior is
> different when using Grunt shell and running it as a script.
>
> * Input file *
>
> cheolsoo@localhost:~/workspace/pig-svn $cat 1.txt
> $id,a
>
> * Grunt shell *
>
> The $ with no backslash works:
>
> grunt> A = LOAD '1.txt' USING PigStorage(',') AS (k:chararray, v:chararray);
> grunt> B = FOREACH A GENERATE TOMAP(k, v) AS M;
> grunt> C = FOREACH B GENERATE M#'$id';
> grunt> DUMP C;
> (a)
>
> * Script *
>
> The $ with a single backslash works:
>
> cheolsoo@localhost:~/workspace/pig-svn $cat test.pig
> A = LOAD '1.txt' USING PigStorage(',') AS (k:chararray, v:chararray);
> B = FOREACH A GENERATE TOMAP(k, v) AS M;
> C = FOREACH B GENERATE M#'\$id';
> DUMP C;
>
> cheolsoo@localhost:~/workspace/pig-svn $./bin/pig -x local test.pig
> (a)
>
> Please let me know if you see different results.
>
> Thanks,
> Cheolsoo
>
>
> On Mon, Jan 14, 2013 at 7:40 AM, Eli Finkelshteyn <[EMAIL PROTECTED]>wrote:
>
>> Any other ideas? Still no solution on this.
>>
>> Eli
>>
>> On Jan 10, 2013, at 7:33 PM, Dmitriy Ryaboy wrote:
>>
>>> Two back slashes?
>>>
>>>
>>> On Thu, Jan 10, 2013 at 6:01 PM, Eli Finkelshteyn <[EMAIL PROTECTED]
>>> wrote:
>>>
>>>> This wasn't a problem in 0.9.2, but in 0.10, when I try to access a key
>> in
>>>> a map that has a dollar sign in it, I get hammered with errors that I
>>>> haven't defined the variable. Specifically:
>>>>
>>>>  blah = FOREACH meh GENERATE source, json_post_id#'$id' AS post_id;
>>>>
>>>> returns
>>>>
>>>>  Undefined parameter : id
>>>>
>>>> That's fine and makes sense, but when I amend it to:
>>>>
>>>>  blah = FOREACH meh GENERATE source, json_post_id#'\$id' AS post_id;
>>>>
>>>> I get:
>>>>
>>>>  Unexpected character '$'
>>>>
>>>> Ideas?
>>>>
>>>> Thanks!
>>>> Eli
>>>>
>>
>>