Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> No matter what I do, pig is trying to run locally


Copy link to this message
-
Re: No matter what I do, pig is trying to run locally
Yes. I believe those conf files are in pig's path. This is the error
message that leads me to believe so:

2013-08-23 11:47:42,757 [main] INFO  org.apache.hadoop.mapreduce.Cluster -
Failed to use org.apache.hadoop.mapred.LocalClientProtocolProvider due to
error: Invalid "mapreduce.jobtracker.address" configuration value for
LocalJobRunner : "dl1rhd401.internal.edmunds.com:8021"

My confusion is why it is trying to use LocalJobRunner.

When I remove the mapred-site.xml, it will read the files from HDFS, but
then run the job locally, even after I use the -x mapreduce option for the
pig command.
On Thu, Aug 22, 2013 at 11:35 PM, Serega Sheypak
<[EMAIL PROTECTED]>wrote:

> Are you sure that your core-site, hdfs-site, maped-site are in pig's
> classpath?
>
>
> 2013/8/23 Tim Chan <[EMAIL PROTECTED]>
>
> > Apache Pig version 0.11.0-cdh4.3.0
> > Hadoop 2.0.0-cdh4.3.0
> >
> > Here is the error I'm getting:
> >
> > 2013-08-22 19:27:50,304 [main] INFO  org.apache.hadoop.mapreduce.Cluster
> -
> > Failed to use org.apache.hadoop.mapred.LocalClientProtocolProvider due to
> > error: Invalid "mapreduce.jobtracker.address" configuration value for
> > LocalJobRunner : "dl1rhd401.internal.edmunds.com:8021"
> > 2013-08-22 19:27:50,320 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> > ERROR 2043: Unexpected error during execution.
> > 2013-08-22 19:27:50,320 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
> > open iterator for alias mercedes_glk
> > at org.apache.pig.PigServer.openIterator(PigServer.java:838)
> > at
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:696)
> > at
> >
> >
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:320)
> > at
> >
> >
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> > at
> >
> >
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
> > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
> > at org.apache.pig.Main.run(Main.java:604)
> > at org.apache.pig.Main.main(Main.java:157)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> > Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias
> > mercedes_glk
> > at org.apache.pig.PigServer.storeEx(PigServer.java:937)
> > at org.apache.pig.PigServer.store(PigServer.java:900)
> > at org.apache.pig.PigServer.openIterator(PigServer.java:813)
> > ... 12 more
> > Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR
> > 2043: Unexpected error during execution.
> > at org.apache.pig.PigServer.launchPlan(PigServer.java:1277)
> > at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1251)
> > at org.apache.pig.PigServer.storeEx(PigServer.java:933)
> > ... 14 more
> > Caused by: java.io.IOException: Cannot initialize Cluster. Please check
> > your configuration for mapreduce.framework.name and the correspond
> server
> > addresses.
> > at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:121)
> > at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:83)
> > at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:76)
> > at org.apache.hadoop.mapred.JobClient.init(JobClient.java:478)
> > at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:457)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:152)
> > at org.apache.pig.PigServer.launchPlan(PigServer.java:1266)
> > ... 16 more
> >
> >
> >
> > I have mapred-site.xml configured to point to our job tracker.
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB