Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - ClassCastException


Copy link to this message
-
RE: ClassCastException
Santhosh Srinivasan 2009-02-03, 23:36
If you want to count the number of records per group then replace
COUNT(*) with COUNT(B).

Santhosh

-----Original Message-----
From: Shirley Cohen [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, February 03, 2009 12:06 PM
To: [EMAIL PROTECTED]
Subject: ClassCastException

Hi,

I'm trying to run the following query in mapreduce mode:

A = LOAD 'item.tbl' USING PigStorage('|') AS (item: int, quantity:
double, price: double, discount: double, tax: double, returnflag:
chararray, status: chararray);
B = FOREACH A GENERATE *, price * (1 - discount) AS disc_price, price *
(1 - discount) * (1 + tax) AS charge;
C = GROUP B BY (returnflag, status);
D = FOREACH C GENERATE group.returnflag AS returnflag, group.status AS
status,      
SUM($1.quantity) as sum_qty,
AVG($1.discount) as avg_disc,
COUNT(*) AS count_order;
E = ORDER D BY returnflag, status;
STORE E INTO 'q1.out';

And I'm getting the following error message:

2009-02-03 12:41:09,118 [main] INFO  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLa
uncher
- 0% complete
2009-02-03 12:41:14,121 [main] INFO  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLa
uncher
- 16% complete
2009-02-03 12:41:44,139 [main] INFO  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLa
uncher
- 33% complete
2009-02-03 12:41:49,140 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLa
uncher
- Map reduce job failed
2009-02-03 12:41:49,140 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLa
uncher
- depending job 0 with jobID execute0 failed. Job failed!
2009-02-03 12:41:49,144 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLa
uncher
- Job failed!
2009-02-03 12:41:49,157 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher -
Error message from task (reduce)
task_200902031240_0001_r_000000java.lang.ClassCastException:
org.apache.pig.data.DefaultTuple cannot be cast to
org.apache.pig.data.DataBag
        at org.apache.pig.builtin.COUNT.exec(COUNT.java:44)
        at org.apache.pig.builtin.COUNT.exec(COUNT.java:38)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOp
erators.POUserFunc.getNext(POUserFunc.java:186)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOp
erators.POUserFunc.getNext(POUserFunc.java:248)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOp
erators.POForEach.processPlan(POForEach.java:265)
        at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOp
erators.POForEach.getNext(POForEach.java:197)
        at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduc
e$Reduce.runPipeline(PigMapReduce.java:283)
        at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduc
e$Reduce.processOnePackageOutput(PigMapReduce.java:250)
        at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduc
e$Reduce.reduce(PigMapReduce.java:230)
        at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduc
e$Reduce.reduce(PigMapReduce.java:136)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
        at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)

Is there anything wrong with the query? Does anyone know what could be
causing this error?

Thanks,

Shirley