Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Best practices for LOAD statement


Copy link to this message
-
Re: Best practices for LOAD statement
Defer type cast should be good. What is your script?

Daniel

On Mon, Feb 6, 2012 at 10:06 PM, praveenesh kumar <[EMAIL PROTECTED]> wrote:
> Is it good to keep schema as bytearray in Load statements ?
> I was trying to put datatypes to my column headers, I getting casting
> errors like below  in group by and join statements ?
>
> Can anyone tell me what are the best practices to write Pig scripts ?
>
> java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be
> cast to java.lang.Integer
>
>     at
> org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:81)
>
>     at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:117)
>
>     at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:273)
>
>     at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:266)
>
>     at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
>
>     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>
>     at
> org.apache.pig.pen.LocalMapReduceSimulator.launchPig(LocalMapReduceSimulator.java:205)
>
>     at
> org.apache.pig.pen.ExampleGenerator.getData(ExampleGenerator.java:257)
>
>     at
> org.apache.pig.pen.ExampleGenerator.getData(ExampleGenerator.java:238)
>
>     at
> org.apache.pig.pen.LineageTrimmingVisitor.init(LineageTrimmingVisitor.java:103)
>
>     at
> org.apache.pig.pen.LineageTrimmingVisitor.<init>(LineageTrimmingVisitor.java:98)
>
>     at
> org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:166)
>
>     at org.apache.pig.PigServer.getExamples(PigServer.java:1202)
>
>     at
> org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:700)
>
>     at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.Illustrate(PigScriptParser.java:597)
>
>     at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:308)
>
>     at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:190)
>
>     at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166)
>
>     at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>
>     at org.apache.pig.Main.run(Main.java:523)
>
>     at org.apache.pig.Main.main(Main.java:148)
>
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
>     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
>     at java.lang.reflect.Method.invoke(Method.java:601)
>
>     at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
> 2012-02-06 22:47:31,936 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 2997: Encountered IOException. Exception :
> org.apache.pig.data.DataByteArray cannot be cast to java.lang.Integer
>
> Details at logfile:
> /usr/local/hadoop/pig/trunk/learning/insurance/pig_1328548575504.log
>
>
> Thanks,
> Praveenesh
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB