Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Best practices for LOAD statement


Copy link to this message
-
Best practices for LOAD statement
Is it good to keep schema as bytearray in Load statements ?
I was trying to put datatypes to my column headers, I getting casting
errors like below  in group by and join statements ?

Can anyone tell me what are the best practices to write Pig scripts ?

java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be
cast to java.lang.Integer

     at
org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:81)

     at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:117)

     at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:273)

     at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:266)

     at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)

     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)

     at
org.apache.pig.pen.LocalMapReduceSimulator.launchPig(LocalMapReduceSimulator.java:205)

     at
org.apache.pig.pen.ExampleGenerator.getData(ExampleGenerator.java:257)

     at
org.apache.pig.pen.ExampleGenerator.getData(ExampleGenerator.java:238)

     at
org.apache.pig.pen.LineageTrimmingVisitor.init(LineageTrimmingVisitor.java:103)

     at
org.apache.pig.pen.LineageTrimmingVisitor.<init>(LineageTrimmingVisitor.java:98)

     at
org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:166)

     at org.apache.pig.PigServer.getExamples(PigServer.java:1202)

     at
org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:700)

     at
org.apache.pig.tools.pigscript.parser.PigScriptParser.Illustrate(PigScriptParser.java:597)

     at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:308)

     at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:190)

     at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166)

     at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)

     at org.apache.pig.Main.run(Main.java:523)

     at org.apache.pig.Main.main(Main.java:148)

     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

     at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

     at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

     at java.lang.reflect.Method.invoke(Method.java:601)

     at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

2012-02-06 22:47:31,936 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 2997: Encountered IOException. Exception :
org.apache.pig.data.DataByteArray cannot be cast to java.lang.Integer

Details at logfile:
/usr/local/hadoop/pig/trunk/learning/insurance/pig_1328548575504.log
Thanks,
Praveenesh
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB