Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Best practices for LOAD statement


Copy link to this message
-
Best practices for LOAD statement
praveenesh kumar 2012-02-07, 06:06
Is it good to keep schema as bytearray in Load statements ?
I was trying to put datatypes to my column headers, I getting casting
errors like below  in group by and join statements ?

Can anyone tell me what are the best practices to write Pig scripts ?

java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be
cast to java.lang.Integer

     at
org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:81)

     at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:117)

     at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:273)

     at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:266)

     at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)

     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)

     at
org.apache.pig.pen.LocalMapReduceSimulator.launchPig(LocalMapReduceSimulator.java:205)

     at
org.apache.pig.pen.ExampleGenerator.getData(ExampleGenerator.java:257)

     at
org.apache.pig.pen.ExampleGenerator.getData(ExampleGenerator.java:238)

     at
org.apache.pig.pen.LineageTrimmingVisitor.init(LineageTrimmingVisitor.java:103)

     at
org.apache.pig.pen.LineageTrimmingVisitor.<init>(LineageTrimmingVisitor.java:98)

     at
org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:166)

     at org.apache.pig.PigServer.getExamples(PigServer.java:1202)

     at
org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:700)

     at
org.apache.pig.tools.pigscript.parser.PigScriptParser.Illustrate(PigScriptParser.java:597)

     at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:308)

     at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:190)

     at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166)

     at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)

     at org.apache.pig.Main.run(Main.java:523)

     at org.apache.pig.Main.main(Main.java:148)

     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

     at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

     at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

     at java.lang.reflect.Method.invoke(Method.java:601)

     at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

2012-02-06 22:47:31,936 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 2997: Encountered IOException. Exception :
org.apache.pig.data.DataByteArray cannot be cast to java.lang.Integer

Details at logfile:
/usr/local/hadoop/pig/trunk/learning/insurance/pig_1328548575504.log
Thanks,
Praveenesh