|
|
-
error with FLATTEN and nested generate
Yang 2012-06-21, 03:15
I'm trying to run the following code, ---- I had another piece of code that looks exactly the same , just var name differences, that one runs fine, but this one gave errors:
2012-06-20 20:14:38,285 [main] INFO org.apache.pig.Main - Apache Pig version 0.10.0-SNAPSHOT (rexported) compiled Jun 20 2012, 16:31:38 2012-06-20 20:14:38,285 [main] INFO org.apache.pig.Main - Logging error messages to: /crypt/yyang_home/work/matching/ML-places-matcher/control_flows/match_suspects/pig_1340248478278.log 2012-06-20 20:14:38,644 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: file:/// 2012-06-20 20:14:39,148 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: Pig script failed to parse: <file a.pig, line 11, column 39> Invalid scalar projection: aa : A column needs to be projected from a relation for it to be used as a scalar
where I'm I doing wrong?
Thanks a lot yang
######################################
aa = load 'b' as (x:int , y:int);
b = group aa by x;
c = group b by group ;
-- describe c; -- c: {group: int,b: {(group: int,aa: {(x: int,y: int)})}}
d = foreach c { xx = foreach b generate FLATTEN(aa) as (o:int, p:int); generate group , MIN(xx.x); };
dump d;
-
Re: error with FLATTEN and nested generate
Bill Graham 2012-06-21, 04:09
Try replacing MIN(xx.x); with MIN(xx::x);
The xx.x syntax returns a scalar IF the xx relationship only has one tuple. If it doesn't it fails. The xx::x syntax refers to the x values in the xx relationship.
This is a common cause of confusion that we're looking to correct.
On Wed, Jun 20, 2012 at 8:15 PM, Yang <[EMAIL PROTECTED]> wrote:
> I'm trying to run the following code, ---- I had another piece of code that > looks exactly the same , just var name differences, that one runs fine, > but this one gave errors: > > 2012-06-20 20:14:38,285 [main] INFO org.apache.pig.Main - Apache Pig > version 0.10.0-SNAPSHOT (rexported) compiled Jun 20 2012, 16:31:38 > 2012-06-20 20:14:38,285 [main] INFO org.apache.pig.Main - Logging error > messages to: > > /crypt/yyang_home/work/matching/ML-places-matcher/control_flows/match_suspects/pig_1340248478278.log > 2012-06-20 20:14:38,644 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - > Connecting to hadoop file system at: file:/// > 2012-06-20 20:14:39,148 [main] ERROR org.apache.pig.tools.grunt.Grunt - > ERROR 1200: Pig script failed to parse: > <file a.pig, line 11, column 39> Invalid scalar projection: aa : A column > needs to be projected from a relation for it to be used as a scalar > > > > where I'm I doing wrong? > > Thanks a lot > yang > > ###################################### > > aa = load 'b' as (x:int , y:int); > > b = group aa by x; > > c = group b by group ; > > -- describe c; > -- c: {group: int,b: {(group: int,aa: {(x: int,y: int)})}} > > d = foreach c { > xx = foreach b generate FLATTEN(aa) as (o:int, p:int); > generate group , MIN(xx.x); > }; > > dump d; >
-- *Note that I'm no longer using my Yahoo! email address. Please email me at [EMAIL PROTECTED] going forward.*
-
Re: error with FLATTEN and nested generate
Yang 2012-06-21, 04:15
thanks, but the error msg is for the line before the xx.x
also I removed the xx.x completely and it still gave errors On Wed, Jun 20, 2012 at 9:09 PM, Bill Graham <[EMAIL PROTECTED]> wrote:
> Try replacing MIN(xx.x); with MIN(xx::x); > > The xx.x syntax returns a scalar IF the xx relationship only has one tuple. > If it doesn't it fails. The xx::x syntax refers to the x values in the xx > relationship. > > This is a common cause of confusion that we're looking to correct. > > On Wed, Jun 20, 2012 at 8:15 PM, Yang <[EMAIL PROTECTED]> wrote: > > > I'm trying to run the following code, ---- I had another piece of code > that > > looks exactly the same , just var name differences, that one runs fine, > > but this one gave errors: > > > > 2012-06-20 20:14:38,285 [main] INFO org.apache.pig.Main - Apache Pig > > version 0.10.0-SNAPSHOT (rexported) compiled Jun 20 2012, 16:31:38 > > 2012-06-20 20:14:38,285 [main] INFO org.apache.pig.Main - Logging error > > messages to: > > > > > /crypt/yyang_home/work/matching/ML-places-matcher/control_flows/match_suspects/pig_1340248478278.log > > 2012-06-20 20:14:38,644 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - > > Connecting to hadoop file system at: file:/// > > 2012-06-20 20:14:39,148 [main] ERROR org.apache.pig.tools.grunt.Grunt - > > ERROR 1200: Pig script failed to parse: > > <file a.pig, line 11, column 39> Invalid scalar projection: aa : A column > > needs to be projected from a relation for it to be used as a scalar > > > > > > > > where I'm I doing wrong? > > > > Thanks a lot > > yang > > > > ###################################### > > > > aa = load 'b' as (x:int , y:int); > > > > b = group aa by x; > > > > c = group b by group ; > > > > -- describe c; > > -- c: {group: int,b: {(group: int,aa: {(x: int,y: int)})}} > > > > d = foreach c { > > xx = foreach b generate FLATTEN(aa) as (o:int, p:int); > > generate group , MIN(xx.x); > > }; > > > > dump d; > > > > > > -- > *Note that I'm no longer using my Yahoo! email address. Please email me at > [EMAIL PROTECTED] going forward.* >
-
Re: error with FLATTEN and nested generate
Daniel Dai 2012-06-27, 20:21
Looks like a bug. Change a into $1 works for me.
On Wed, Jun 20, 2012 at 9:15 PM, Yang <[EMAIL PROTECTED]> wrote:
> thanks, but the error msg is for the line before the xx.x > > also I removed the xx.x completely and it still gave errors > > > On Wed, Jun 20, 2012 at 9:09 PM, Bill Graham <[EMAIL PROTECTED]> wrote: > > > Try replacing MIN(xx.x); with MIN(xx::x); > > > > The xx.x syntax returns a scalar IF the xx relationship only has one > tuple. > > If it doesn't it fails. The xx::x syntax refers to the x values in the xx > > relationship. > > > > This is a common cause of confusion that we're looking to correct. > > > > On Wed, Jun 20, 2012 at 8:15 PM, Yang <[EMAIL PROTECTED]> wrote: > > > > > I'm trying to run the following code, ---- I had another piece of code > > that > > > looks exactly the same , just var name differences, that one runs fine, > > > but this one gave errors: > > > > > > 2012-06-20 20:14:38,285 [main] INFO org.apache.pig.Main - Apache Pig > > > version 0.10.0-SNAPSHOT (rexported) compiled Jun 20 2012, 16:31:38 > > > 2012-06-20 20:14:38,285 [main] INFO org.apache.pig.Main - Logging > error > > > messages to: > > > > > > > > > /crypt/yyang_home/work/matching/ML-places-matcher/control_flows/match_suspects/pig_1340248478278.log > > > 2012-06-20 20:14:38,644 [main] INFO > > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - > > > Connecting to hadoop file system at: file:/// > > > 2012-06-20 20:14:39,148 [main] ERROR org.apache.pig.tools.grunt.Grunt - > > > ERROR 1200: Pig script failed to parse: > > > <file a.pig, line 11, column 39> Invalid scalar projection: aa : A > column > > > needs to be projected from a relation for it to be used as a scalar > > > > > > > > > > > > where I'm I doing wrong? > > > > > > Thanks a lot > > > yang > > > > > > ###################################### > > > > > > aa = load 'b' as (x:int , y:int); > > > > > > b = group aa by x; > > > > > > c = group b by group ; > > > > > > -- describe c; > > > -- c: {group: int,b: {(group: int,aa: {(x: int,y: int)})}} > > > > > > d = foreach c { > > > xx = foreach b generate FLATTEN(aa) as (o:int, p:int); > > > generate group , MIN(xx.x); > > > }; > > > > > > dump d; > > > > > > > > > > > -- > > *Note that I'm no longer using my Yahoo! email address. Please email me > at > > [EMAIL PROTECTED] going forward.* > > >
|
|