Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> Re: [tw-hadoop-users] ERROR 2017: Internal error creating job configuration.


Copy link to this message
-
Re: [tw-hadoop-users] ERROR 2017: Internal error creating job configuration.
Hi Kurt,
>From the stack trace, it looks like it runs into an error while
estimating the size of the input.
Are all of the paths it's looking for there in hdfs:///user/kurt ?
does it work with pig_11 ? add --pig_version pig_11 to the oink command
also send out the command line you are using.
Thanks,
Julien

On Thu, Nov 15, 2012 at 9:15 AM, Kurt Smith <[EMAIL PROTECTED]> wrote:
> I'm getting this error when doing a manual run of the search_simplified
> twadoop query. This query has run fine before. Any idea what the issue is?
>
> Pig Stack Trace
> ---------------
> ERROR 2017: Internal error creating job configuration.
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException:
> ERROR 2017: Internal error creating job configuration.
>         at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:738)
>         at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:264)
>         at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:150)
>         at org.apache.pig.PigServer.launchPlan(PigServer.java:1267)
>         at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1252)
>         at org.apache.pig.PigServer.execute(PigServer.java:1242)
>         at org.apache.pig.PigServer.executeBatch(PigServer.java:356)
>         at
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:132)
>         at
> org.apache.pig.tools.grunt.GruntParser.processScript(GruntParser.java:452)
>         at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.Script(PigScriptParser.java:752)
>         at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:423)
>         at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
>         at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>         at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
>         at org.apache.pig.Main.run(Main.java:561)
>         at org.apache.pig.Main.main(Main.java:111)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> Caused by: java.lang.NullPointerException
>         at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:971)
>         at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:944)
>         at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getInputSize(JobControlCompiler.java:840)
>         at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.estimateNumberOfReducers(JobControlCompiler.java:810)
>         at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.adjustNumReducers(JobControlCompiler.java:750)
>         at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:396)
>         ... 20 more
> ===============================================================================>
>
> from the output:
> ---
>
> 2012-11-15 07:54:02,554 [main] INFO
> org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to
> the job
> 2012-11-15 07:54:02,555 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
> 2012-11-15 07:54:03,005 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - BytesPerReducer=1610612736 maxReducers=999
> totalInputFileSize=1456250098769
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB