Hi,

I'm trying to filter on the number of columns in a relation as
suggested in the FAQ, but I get the following error. This is in the
types branch. Has the syntax changed or does this look like a bug?

A = LOAD 'foo' USING PigStorage('\t');
B = FILTER A BY ARITY(*) < 5;
DUMP B;
2009-01-05 21:46:56,355 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc
- Caught error from UDF
org.apache.pig.builtin.ARITY[org.apache.pig.data.DataByteArray cannot
be cast to org.apache.pig.data.Tuple
[org.apache.pig.data.DataByteArray cannot be cast to
org.apache.pig.data.Tuple]]
2009-01-05 21:46:56,356 [main] INFO
org.apache.pig.backend.local.executionengine.LocalPigLauncher - Failed
jobs!!
2009-01-05 21:46:56,356 [main] INFO
org.apache.pig.backend.local.executionengine.LocalPigLauncher - 1 out
of 1 failed!
2009-01-05 21:46:56,357 [main] ERROR
org.apache.pig.tools.grunt.GruntParser - java.io.IOException: Unable
to open iterator for alias: B [Job terminated with anomalous status
FAILED]
at org.apache.pig.PigServer.openIterator(PigServer.java:389)
at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:269)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:178)
at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:94)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
at org.apache.pig.Main.main(Main.java:282)
Caused by: java.io.IOException: Job terminated with anomalous status FAILED
... 6 more

2009-01-05 21:46:56,357 [main] ERROR
org.apache.pig.tools.grunt.GruntParser - Unable to open iterator for
alias: B [Job terminated with anomalous status FAILED]
2009-01-05 21:46:56,357 [main] ERROR
org.apache.pig.tools.grunt.GruntParser - java.io.IOException: Unable
to open iterator for alias: B [Job terminated with anomalous status
FAILED]

Thanks,
Tom

You should be able to use SIZE instead of ARITY, which has been deprecated.
There is a lot of useful information at
http://wiki.apache.org/pig/TrunkToTypesChanges.

Write back if this doesn't help.

Thanks,
Kevin

On Mon, Jan 5, 2009 at 1:52 PM, Tom White <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I'm trying to filter on the number of columns in a relation as
> suggested in the FAQ, but I get the following error. This is in the
> types branch. Has the syntax changed or does this look like a bug?
>
> A = LOAD 'foo' USING PigStorage('\t');
> B = FILTER A BY ARITY(*) < 5;
> DUMP B;
> 2009-01-05 21:46:56,355 [main] ERROR
>
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc
> - Caught error from UDF
> org.apache.pig.builtin.ARITY[org.apache.pig.data.DataByteArray cannot
> be cast to org.apache.pig.data.Tuple
> [org.apache.pig.data.DataByteArray cannot be cast to
> org.apache.pig.data.Tuple]]
> 2009-01-05 21:46:56,356 [main] INFO
> org.apache.pig.backend.local.executionengine.LocalPigLauncher - Failed
> jobs!!
> 2009-01-05 21:46:56,356 [main] INFO
> org.apache.pig.backend.local.executionengine.LocalPigLauncher - 1 out
> of 1 failed!
> 2009-01-05 21:46:56,357 [main] ERROR
> org.apache.pig.tools.grunt.GruntParser - java.io.IOException: Unable
> to open iterator for alias: B [Job terminated with anomalous status
> FAILED]
>        at org.apache.pig.PigServer.openIterator(PigServer.java:389)
>        at
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:269)
>        at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:178)
>        at
> org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:94)
>        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
>        at org.apache.pig.Main.main(Main.java:282)
> Caused by: java.io.IOException: Job terminated with anomalous status FAILED
>        ... 6 more
>
> 2009-01-05 21:46:56,357 [main] ERROR
> org.apache.pig.tools.grunt.GruntParser - Unable to open iterator for
> alias: B [Job terminated with anomalous status FAILED]
> 2009-01-05 21:46:56,357 [main] ERROR
> org.apache.pig.tools.grunt.GruntParser - java.io.IOException: Unable
> to open iterator for alias: B [Job terminated with anomalous status
> FAILED]
>
> Thanks,
> Tom
>

Yes, SIZE it a better function to use. Unfortunately, it will have the
same problem. It is a bug and we are looking into it now and will update
once we know what the solution is.

Olga

> -----Original Message-----
> From: Kevin Weil [mailto:[EMAIL PROTECTED]]
> Sent: Monday, January 05, 2009 2:07 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Number of columns in a relation
>
> You should be able to use SIZE instead of ARITY, which has
> been deprecated.
> There is a lot of useful information at
> http://wiki.apache.org/pig/TrunkToTypesChanges.
>
> Write back if this doesn't help.
>
> Thanks,
> Kevin
>
> On Mon, Jan 5, 2009 at 1:52 PM, Tom White
> <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> > I'm trying to filter on the number of columns in a relation as
> > suggested in the FAQ, but I get the following error. This is in the
> > types branch. Has the syntax changed or does this look like a bug?
> >
> > A = LOAD 'foo' USING PigStorage('\t'); B = FILTER A BY
> ARITY(*) < 5;
> > DUMP B;
> > 2009-01-05 21:46:56,355 [main] ERROR
> >
> >
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expression
> > Operators.POUserFunc
> > - Caught error from UDF
> >
> org.apache.pig.builtin.ARITY[org.apache.pig.data.DataByteArray cannot
> > be cast to org.apache.pig.data.Tuple
> > [org.apache.pig.data.DataByteArray cannot be cast to
> > org.apache.pig.data.Tuple]]
> > 2009-01-05 21:46:56,356 [main] INFO
> >
> org.apache.pig.backend.local.executionengine.LocalPigLauncher
> - Failed
> > jobs!!
> > 2009-01-05 21:46:56,356 [main] INFO
> >
> org.apache.pig.backend.local.executionengine.LocalPigLauncher - 1 out
> > of 1 failed!
> > 2009-01-05 21:46:56,357 [main] ERROR
> > org.apache.pig.tools.grunt.GruntParser -
> java.io.IOException: Unable
> > to open iterator for alias: B [Job terminated with anomalous status
> > FAILED]
> >        at org.apache.pig.PigServer.openIterator(PigServer.java:389)
> >        at
> >
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser
> .java:269)
> >        at
> >
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(Pi
> gScriptParser.java:178)
> >        at
> >
> org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntP
> arser.java:94)
> >        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
> >        at org.apache.pig.Main.main(Main.java:282)
> > Caused by: java.io.IOException: Job terminated with
> anomalous status FAILED
> >        ... 6 more
> >
> > 2009-01-05 21:46:56,357 [main] ERROR
> > org.apache.pig.tools.grunt.GruntParser - Unable to open iterator for
> > alias: B [Job terminated with anomalous status FAILED]
> > 2009-01-05 21:46:56,357 [main] ERROR
> > org.apache.pig.tools.grunt.GruntParser -
> java.io.IOException: Unable
> > to open iterator for alias: B [Job terminated with anomalous status
> > FAILED]
> >
> > Thanks,
> > Tom
> >
>

https://issues.apache.org/jira/browse/PIG-597

> -----Original Message-----
> From: Tom White [mailto:[EMAIL PROTECTED]]
> Sent: Monday, January 05, 2009 1:53 PM
> To: [EMAIL PROTECTED]
> Subject: Number of columns in a relation
>
> Hi,
>
> I'm trying to filter on the number of columns in a relation
> as suggested in the FAQ, but I get the following error. This
> is in the types branch. Has the syntax changed or does this
> look like a bug?
>
> A = LOAD 'foo' USING PigStorage('\t');
> B = FILTER A BY ARITY(*) < 5;
> DUMP B;
> 2009-01-05 21:46:56,355 [main] ERROR
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.ex
> pressionOperators.POUserFunc
> - Caught error from UDF
> org.apache.pig.builtin.ARITY[org.apache.pig.data.DataByteArray
>  cannot be cast to org.apache.pig.data.Tuple
> [org.apache.pig.data.DataByteArray cannot be cast to
> org.apache.pig.data.Tuple]]
> 2009-01-05 21:46:56,356 [main] INFO
> org.apache.pig.backend.local.executionengine.LocalPigLauncher
> - Failed jobs!!
> 2009-01-05 21:46:56,356 [main] INFO
> org.apache.pig.backend.local.executionengine.LocalPigLauncher
> - 1 out of 1 failed!
> 2009-01-05 21:46:56,357 [main] ERROR
> org.apache.pig.tools.grunt.GruntParser - java.io.IOException:
> Unable to open iterator for alias: B [Job terminated with
> anomalous status FAILED]
> at org.apache.pig.PigServer.openIterator(PigServer.java:389)
> at
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser
> .java:269)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(Pi
> gScriptParser.java:178)
> at
> org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntP
> arser.java:94)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
> at org.apache.pig.Main.main(Main.java:282)
> Caused by: java.io.IOException: Job terminated with anomalous
> status FAILED
> ... 6 more
>
> 2009-01-05 21:46:56,357 [main] ERROR
> org.apache.pig.tools.grunt.GruntParser - Unable to open iterator for
> alias: B [Job terminated with anomalous status FAILED]
> 2009-01-05 21:46:56,357 [main] ERROR
> org.apache.pig.tools.grunt.GruntParser - java.io.IOException:
> Unable to open iterator for alias: B [Job terminated with
> anomalous status FAILED]
>
> Thanks,
> Tom
>