Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> SUM


You're getting a 'ClassCastException' because the contents of the bags
are DataByteArray and not long (or cannot be cast to long). I suspect
that you're generating the contents of the bag in some way from a UDF,
no?

You need to either declare the output schema explicitly in the UDF or
just use the 'AS' statement. For example, say you have a UDF that sums
two numbers:

data   = LOAD 'foobar' AS (int:a, int:b);
summed = FOREACH data GENERATE MyFancySummingUDF(a,b) AS (sum:int);
DUMP summed;

--jacob
@thedatachef

On Sun, 2011-04-24 at 18:02 +0200, pob wrote:
> x = foreach g2 generate group, data.(size);
> dump x;
>
> ((drm,0),{(464868)})
> ((drm,1),{(464868)})
> ((snezz,0),{(8073),(8073)})
>
> but:
> x = foreach g2 generate group, SUM(data.size);
>
>
>
>
> 2011-04-24 18:02:18,910 [Thread-793] WARN
>  org.apache.hadoop.mapred.LocalJobRunner - job_local_0038
> org.apache.pig.backend.executionengine.ExecException: ERROR 2106: Error
> while computing sum in Initial
> at org.apache.pig.builtin.LongSum$Initial.exec(LongSum.java:87)
> at org.apache.pig.builtin.LongSum$Initial.exec(LongSum.java:65)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:229)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:273)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:343)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:291)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:256)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> Caused by: java.lang.ClassCastException: org.apache.pig.data.DataByteArray
> cannot be cast to java.lang.Long
> at org.apache.pig.builtin.LongSum$Initial.exec(LongSum.java:79)
> ... 14 more
> 2011-04-24 18:02:19,213 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - HadoopJobId: job_local_0038
> 2011-04-24 18:02:19,213 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 0% complete
> 2011-04-24 18:02:24,215 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - job job_local_0038 has failed! Stop running all dependent jobs
> 2011-04-24 18:02:24,216 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 100% complete
> 2011-04-24 18:02:24,216 [main] ERROR
> org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
> 2011-04-24 18:02:24,216 [main] INFO  org.apache.pig.tools.pigstats.PigStats
> - Detected Local mode. Stats reported below may be incomplete
> 2011-04-24 18:02:24,216 [main] INFO  org.apache.pig.tools.pigstats.PigStats
> - Script Statistics:
>
>
>
>
> Pig Stack Trace
> ---------------
> ERROR 1066: Unable to open iterator for alias x
>
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
> open iterator for alias x
>         at org.apache.pig.PigServer.openIterator(PigServer.java:754)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB