Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Help on running pig in local mode


Copy link to this message
-
Re: Help on running pig in local mode
I think it's trying to find the staging directory set in your
configuration, not finding it, and isn't able to create it.
depending on your configs, that could be in different places, but
usually it's looking under /tmp/mapred . Check permissions there.

D

On Mon, Oct 15, 2012 at 4:35 PM, lei tang <[EMAIL PROTECTED]> wrote:
> Yes. it is there.  Tried the full path as well, still not working.
>
> On Mon, Oct 15, 2012 at 3:45 PM, Prashant Kommireddi <[EMAIL PROTECTED]>wrote:
>
>> Is this file present on your local FS?
>> file:///data1/home/ltang01/mf/prep/in/toy.txt
>>
>> On Mon, Oct 15, 2012 at 3:43 PM, lei tang <[EMAIL PROTECTED]> wrote:
>>
>> > Hi,
>> >
>> > I'm using Pig for my daily job. Pig dose a good job in mapreduce mode in
>> > our internal hadoop cluster.  But it signals an error whenever I want to
>> > run it in local mode.  I believe this is due to some permission or config
>> > issue. Does anybody know how I can address this problem?  Any idea is
>> > appreciated.
>> >
>> > [ltang01@stg-trgt00 in]$ pig -x local
>> > 2012-10-11 17:29:25,707 [main] INFO  org.apache.pig.Main - Apache Pig
>> > version 0.10.0 (r1328203) compiled Apr 19 2012, 22:54:12
>> > 2012-10-11 17:29:25,708 [main] INFO  org.apache.pig.Main - Logging error
>> > messages to: /somewhere/in/pig_1350001765703.log
>> > 2012-10-11 17:29:26,068 [main] INFO
>> > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
>> Connecting
>> > to hadoop file system at: file:///
>> > grunt> A = load 'toy.txt';
>> > grunt> dump A;
>> > 2012-10-11 17:29:36,056 [main] INFO
>> > org.apache.pig.tools.pigstats.ScriptState - Pig features used in the
>> > script: UNKNOWN
>> > 2012-10-11 17:29:36,218 [main] INFO
>> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler -
>> > File concatenation threshold: 100 optimistic? false
>> > 2012-10-11 17:29:36,249 [main] INFO
>> >
>> >
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
>> > - MR plan size before optimization: 1
>> > 2012-10-11 17:29:36,250 [main] INFO
>> >
>> >
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
>> > - MR plan size after optimization: 1
>> > 2012-10-11 17:29:36,382 [main] INFO
>> > org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added
>> > to the job
>> > 2012-10-11 17:29:36,403 [main] INFO
>> >
>> >
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
>> > - mapred.job.reduce.markreset.buffer.percent is not set, set to default
>> 0.3
>> > 2012-10-11 17:29:36,445 [main] INFO
>> >
>> >
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
>> > - Setting up single store job
>> > 2012-10-11 17:29:36,486 [main] INFO
>> > org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with
>> > processName=JobTracker, sessionId>> > 2012-10-11 17:29:36,493 [main] INFO
>> >
>> >
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
>> > - 1 map-reduce job(s) waiting for submission.
>> > 2012-10-11 17:29:36,995 [main] INFO
>> >
>> >
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
>> > - 0% complete
>> > 2012-10-11 17:29:37,002 [main] INFO
>> >
>> >
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
>> > - job null has failed! Stop running all dependent jobs
>> > 2012-10-11 17:29:37,002 [main] INFO
>> >
>> >
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
>> > - 100% complete
>> > 2012-10-11 17:29:37,007 [main] ERROR
>> > org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
>> > recreate exception from backend error: ENOENT: No such file or directory
>> >         at org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method)
>> >         at
>> >
>> >
>> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:513)
>> >         at
>> >
>> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:325)