Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Help on running pig in local mode


+
lei tang 2012-10-15, 22:43
+
Prashant Kommireddi 2012-10-15, 22:45
Copy link to this message
-
Re: Help on running pig in local mode
Yes. it is there.  Tried the full path as well, still not working.

On Mon, Oct 15, 2012 at 3:45 PM, Prashant Kommireddi <[EMAIL PROTECTED]>wrote:

> Is this file present on your local FS?
> file:///data1/home/ltang01/mf/prep/in/toy.txt
>
> On Mon, Oct 15, 2012 at 3:43 PM, lei tang <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> > I'm using Pig for my daily job. Pig dose a good job in mapreduce mode in
> > our internal hadoop cluster.  But it signals an error whenever I want to
> > run it in local mode.  I believe this is due to some permission or config
> > issue. Does anybody know how I can address this problem?  Any idea is
> > appreciated.
> >
> > [ltang01@stg-trgt00 in]$ pig -x local
> > 2012-10-11 17:29:25,707 [main] INFO  org.apache.pig.Main - Apache Pig
> > version 0.10.0 (r1328203) compiled Apr 19 2012, 22:54:12
> > 2012-10-11 17:29:25,708 [main] INFO  org.apache.pig.Main - Logging error
> > messages to: /somewhere/in/pig_1350001765703.log
> > 2012-10-11 17:29:26,068 [main] INFO
> > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> Connecting
> > to hadoop file system at: file:///
> > grunt> A = load 'toy.txt';
> > grunt> dump A;
> > 2012-10-11 17:29:36,056 [main] INFO
> > org.apache.pig.tools.pigstats.ScriptState - Pig features used in the
> > script: UNKNOWN
> > 2012-10-11 17:29:36,218 [main] INFO
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler -
> > File concatenation threshold: 100 optimistic? false
> > 2012-10-11 17:29:36,249 [main] INFO
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> > - MR plan size before optimization: 1
> > 2012-10-11 17:29:36,250 [main] INFO
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> > - MR plan size after optimization: 1
> > 2012-10-11 17:29:36,382 [main] INFO
> > org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added
> > to the job
> > 2012-10-11 17:29:36,403 [main] INFO
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> > - mapred.job.reduce.markreset.buffer.percent is not set, set to default
> 0.3
> > 2012-10-11 17:29:36,445 [main] INFO
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> > - Setting up single store job
> > 2012-10-11 17:29:36,486 [main] INFO
> > org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with
> > processName=JobTracker, sessionId> > 2012-10-11 17:29:36,493 [main] INFO
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 1 map-reduce job(s) waiting for submission.
> > 2012-10-11 17:29:36,995 [main] INFO
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 0% complete
> > 2012-10-11 17:29:37,002 [main] INFO
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - job null has failed! Stop running all dependent jobs
> > 2012-10-11 17:29:37,002 [main] INFO
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 100% complete
> > 2012-10-11 17:29:37,007 [main] ERROR
> > org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
> > recreate exception from backend error: ENOENT: No such file or directory
> >         at org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method)
> >         at
> >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:513)
> >         at
> >
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:325)
> >         at
> > org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
> >         at
> >
> >
> org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:126)
> >         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:873)
> >         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:867)
> >         at java.security.AccessController.doPrivileged(Native Method)
+
Dmitriy Ryaboy 2012-10-18, 04:05
+
lei tang 2012-10-19, 18:11
+
Dmitriy Ryaboy 2012-10-23, 00:39