Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> ClassCastException


Copy link to this message
-
RE: ClassCastException
If you want to count the number of records per group then replace
COUNT(*) with COUNT(B).

Santhosh

-----Original Message-----
From: Shirley Cohen [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, February 03, 2009 12:06 PM
To: [EMAIL PROTECTED]
Subject: ClassCastException

Hi,

I'm trying to run the following query in mapreduce mode:

A = LOAD 'item.tbl' USING PigStorage('|') AS (item: int, quantity:
double, price: double, discount: double, tax: double, returnflag:
chararray, status: chararray);
B = FOREACH A GENERATE *, price * (1 - discount) AS disc_price, price *
(1 - discount) * (1 + tax) AS charge;
C = GROUP B BY (returnflag, status);
D = FOREACH C GENERATE group.returnflag AS returnflag, group.status AS
status,      
SUM($1.quantity) as sum_qty,
AVG($1.discount) as avg_disc,
COUNT(*) AS count_order;
E = ORDER D BY returnflag, status;
STORE E INTO 'q1.out';

And I'm getting the following error message:

2009-02-03 12:41:09,118 [main] INFO  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLa
uncher
- 0% complete
2009-02-03 12:41:14,121 [main] INFO  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLa
uncher
- 16% complete
2009-02-03 12:41:44,139 [main] INFO  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLa
uncher
- 33% complete
2009-02-03 12:41:49,140 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLa
uncher
- Map reduce job failed
2009-02-03 12:41:49,140 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLa
uncher
- depending job 0 with jobID execute0 failed. Job failed!
2009-02-03 12:41:49,144 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLa
uncher
- Job failed!
2009-02-03 12:41:49,157 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher -
Error message from task (reduce)
task_200902031240_0001_r_000000java.lang.ClassCastException:
org.apache.pig.data.DefaultTuple cannot be cast to
org.apache.pig.data.DataBag
        at org.apache.pig.builtin.COUNT.exec(COUNT.java:44)
        at org.apache.pig.builtin.COUNT.exec(COUNT.java:38)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOp
erators.POUserFunc.getNext(POUserFunc.java:186)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOp
erators.POUserFunc.getNext(POUserFunc.java:248)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOp
erators.POForEach.processPlan(POForEach.java:265)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOp
erators.POForEach.getNext(POForEach.java:197)
        at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduc
e$Reduce.runPipeline(PigMapReduce.java:283)
        at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduc
e$Reduce.processOnePackageOutput(PigMapReduce.java:250)
        at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduc
e$Reduce.reduce(PigMapReduce.java:230)
        at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduc
e$Reduce.reduce(PigMapReduce.java:136)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
        at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)

Is there anything wrong with the query? Does anyone know what could be
causing this error?

Thanks,

Shirley
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB