Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> ClassCastException with bincond operator


Copy link to this message
-
ClassCastException with bincond operator
I'm new to pig and I'm getting a ClassCastException when I try to run the following script in pig 0.11.1:
A = LOAD 'test.log' AS (timestamp:long, pk_id:int, array_field:chararray, fk_id:int);

B = FOREACH A GENERATE timestamp, pk_id, FLATTEN(STRSPLIT(array_field,',')) AS (field1:chararray, field2:chararray, age:long), fk_id;

C = FOREACH B GENERATE timestamp-(age IS NULL ? (long) 0 : age) AS adjusted_timestamp:long, pk_id, fk_id;

DUMP C;
test.log contains only one row (I replaced tabs with \t):
1362178873552\t15404\ta,b,5\t6

I'm able to do a dump on B and get the results I expect, but when I do a dump/store on C I get this error:
java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Number
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Subtract.genericGetNext(Subtract.java:96)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Subtract.getNext(Subtract.java:119)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:348)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:372)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:297)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:283)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:278)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)

I've tried casting every timestamp and age to a long in the definition of C and it still doesn't work. 

Any ideas on how to fix the script?

Thanks,
Peter
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB