+
Julien Le Dem 2012-11-15, 17:36
-
Re: [tw-hadoop-users] ERROR 2017: Internal error creating job configuration.Gabor Szabo 2012-11-15, 17:53
Kurt, there's a lot of dependencies that this job expects at different
places, and if you run it as a normal user, it will get confused. Most
likely some input files are not found.
On 11/15/2012 09:36 AM, Julien Le Dem wrote:
> Hi Kurt,
> From the stack trace, it looks like it runs into an error while
> estimating the size of the input.
> Are all of the paths it's looking for there in hdfs:///user/kurt ?
> does it work with pig_11 ? add --pig_version pig_11 to the oink command
> also send out the command line you are using.
> Thanks,
> Julien
>
> On Thu, Nov 15, 2012 at 9:15 AM, Kurt Smith <[EMAIL PROTECTED]> wrote:
>> I'm getting this error when doing a manual run of the search_simplified
>> twadoop query. This query has run fine before. Any idea what the issue is?
>>
>> Pig Stack Trace
>> ---------------
>> ERROR 2017: Internal error creating job configuration.
>>
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException:
>> ERROR 2017: Internal error creating job configuration.
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:738)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:264)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:150)
>> at org.apache.pig.PigServer.launchPlan(PigServer.java:1267)
>> at
>> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1252)
>> at org.apache.pig.PigServer.execute(PigServer.java:1242)
>> at org.apache.pig.PigServer.executeBatch(PigServer.java:356)
>> at
>> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:132)
>> at
>> org.apache.pig.tools.grunt.GruntParser.processScript(GruntParser.java:452)
>> at
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.Script(PigScriptParser.java:752)
>> at
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:423)
>> at
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
>> at
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
>> at org.apache.pig.Main.run(Main.java:561)
>> at org.apache.pig.Main.main(Main.java:111)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
>> Caused by: java.lang.NullPointerException
>> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:971)
>> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:944)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getInputSize(JobControlCompiler.java:840)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.estimateNumberOfReducers(JobControlCompiler.java:810)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.adjustNumReducers(JobControlCompiler.java:750)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:396)
>> ... 20 more
>> ===============================================================================>>
>>
>> from the output:
>> ---
>>
>> 2012-11-15 07:54:02,554 [main] INFO
>> org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to
>> the job
>> 2012-11-15 07:54:02,555 [main] INFO
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
places, and if you run it as a normal user, it will get confused. Most
likely some input files are not found.
On 11/15/2012 09:36 AM, Julien Le Dem wrote:
> Hi Kurt,
> From the stack trace, it looks like it runs into an error while
> estimating the size of the input.
> Are all of the paths it's looking for there in hdfs:///user/kurt ?
> does it work with pig_11 ? add --pig_version pig_11 to the oink command
> also send out the command line you are using.
> Thanks,
> Julien
>
> On Thu, Nov 15, 2012 at 9:15 AM, Kurt Smith <[EMAIL PROTECTED]> wrote:
>> I'm getting this error when doing a manual run of the search_simplified
>> twadoop query. This query has run fine before. Any idea what the issue is?
>>
>> Pig Stack Trace
>> ---------------
>> ERROR 2017: Internal error creating job configuration.
>>
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException:
>> ERROR 2017: Internal error creating job configuration.
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:738)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:264)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:150)
>> at org.apache.pig.PigServer.launchPlan(PigServer.java:1267)
>> at
>> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1252)
>> at org.apache.pig.PigServer.execute(PigServer.java:1242)
>> at org.apache.pig.PigServer.executeBatch(PigServer.java:356)
>> at
>> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:132)
>> at
>> org.apache.pig.tools.grunt.GruntParser.processScript(GruntParser.java:452)
>> at
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.Script(PigScriptParser.java:752)
>> at
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:423)
>> at
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
>> at
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
>> at org.apache.pig.Main.run(Main.java:561)
>> at org.apache.pig.Main.main(Main.java:111)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
>> Caused by: java.lang.NullPointerException
>> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:971)
>> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:944)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getInputSize(JobControlCompiler.java:840)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.estimateNumberOfReducers(JobControlCompiler.java:810)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.adjustNumReducers(JobControlCompiler.java:750)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:396)
>> ... 20 more
>> ===============================================================================>>
>>
>> from the output:
>> ---
>>
>> 2012-11-15 07:54:02,554 [main] INFO
>> org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to
>> the job
>> 2012-11-15 07:54:02,555 [main] INFO
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
+
Kurt Smith 2012-11-15, 17:57
