|
|
-
ColumnMapKeyPrune bug?Haitao Yao 2012-08-10, 03:20
hey, all
I got this error on my pig script: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2000: Error processing rule ColumnMapKeyPrune. Try -t ColumnMapKeyPrune at org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:122) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:287) at org.apache.pig.PigServer.compilePp(PigServer.java:1360) at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1297) at org.apache.pig.PigServer.execute(PigServer.java:1289) at org.apache.pig.PigServer.executeBatch(PigServer.java:360) at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:131) at org.apache.pig.tools.grunt.GruntParser.processFsCommand(GruntParser.java:972) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:117) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164) at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84) at he.pig.PigWorker.call(BatchPig.java:426) at he.pig.PigWorker.call(BatchPig.java:400) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) Caused by: java.lang.IndexOutOfBoundsException: Index: 6, Size: 6 at java.util.ArrayList.rangeCheck(ArrayList.java:604) at java.util.ArrayList.get(ArrayList.java:382) at org.apache.pig.newplan.logical.relational.LogicalSchema.getField(LogicalSchema.java:651) at org.apache.pig.newplan.logical.expression.DereferenceExpression.getFieldSchema(DereferenceExpression.java:153) at org.apache.pig.newplan.logical.optimizer.FieldSchemaResetter.execute(SchemaResetter.java:192) at org.apache.pig.newplan.logical.expression.AllSameExpressionVisitor.visit(AllSameExpressionVisitor.java:148) at org.apache.pig.newplan.logical.expression.DereferenceExpression.accept(DereferenceExpression.java:84) at org.apache.pig.newplan.ReverseDependencyOrderWalker.walk(ReverseDependencyOrderWalker.java:70) at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50) at org.apache.pig.newplan.logical.optimizer.SchemaResetter.visit(SchemaResetter.java:104) at org.apache.pig.newplan.logical.relational.LOGenerate.accept(LOGenerate.java:240) at org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75) at org.apache.pig.newplan.logical.optimizer.SchemaResetter.visit(SchemaResetter.java:93) at org.apache.pig.newplan.logical.relational.LOForEach.accept(LOForEach.java:74) at org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75) at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50) at org.apache.pig.newplan.logical.optimizer.SchemaPatcher.transformed(SchemaPatcher.java:43) at org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:113) ... 18 more In order by continue running the script, I have to use -t ColumnMapKeyPrune to turn off the optimization. It seems like there's a simple bug java.lang.IndexOutOfBoundsException here? Thanks Haitao Yao [EMAIL PROTECTED] weibo: @haitao_yao Skype: haitao.yao.final |