Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Unable to run PIG local as regular user (non-root)


Copy link to this message
-
Re: Unable to run PIG local as regular user (non-root)
Hi Daniel,

>From the console output, I can see Pig ran against Hadoop 1.x as root while
it ran against Hadoop 2.x as watrous. I don't know what your setup is like,
but please double check what version of Hadoop jars are present in
classpath. Running "pig -secretDebugCmd" should show HADOOP_CLASSPATH.

Apparently, your Pig jar is compiled against Hadoop 1.x, so it only works
against Hadoop 1.x (as root).

Thanks,
Cheolsoo

On Wed, Nov 27, 2013 at 11:01 AM, Watrous, Daniel
<[EMAIL PROTECTED]>wrote:

> Hi,
>
> I've been following the book Hadoop Definitive Guide and a few other
> resources while exploring PIG. I am able to run as root, but not as a
> regular user. I am running in local mode. I've found a few links, but they
> haven't helped me solve my issue. The most relevant seems to be:
> http://answers.mapr.com/questions/5135/cannot-run-pig-in-local-mode
>
> My first experiment is just to load in a text file and dump it to the
> console. The files exist and are owned by the user running PIG. I show the
> failed attempt as a regular user followed by a successful run as root on
> the same system. Any help is appreciated.
>
> Here's what I'm trying:
> [watrous@c0003913 ~]$ pig -x local
> 2013-11-27 18:53:18,691 [main] INFO  org.apache.pig.Main - Apache Pig
> version 0.12.0 (r1529718) compiled Oct 07 2013, 12:20:14
> 2013-11-27 18:53:18,692 [main] INFO  org.apache.pig.Main - Logging error
> messages to: /home/watrous/pig_1385578398689.log
> 2013-11-27 18:53:18,747 [main] INFO  org.apache.pig.impl.util.Utils -
> Default bootup file /home/watrous/.pigbootup not found
> 2013-11-27 18:53:19,025 [main] INFO
>  org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 2013-11-27 18:53:19,025 [main] INFO
>  org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is
> deprecated. Instead, use mapreduce.jobtracker.address
> 2013-11-27 18:53:19,027 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> Connecting to hadoop file system at: file:///
> 2013-11-27 18:53:19,031 [main] INFO
>  org.apache.hadoop.conf.Configuration.deprecation -
> mapred.used.genericoptionsparser is deprecated. Instead, use
> mapreduce.client.genericoptionsparser.used
> 2013-11-27 18:53:19,451 [main] WARN
>  org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 2013-11-27 18:53:19,517 [main] INFO
>  org.apache.hadoop.conf.Configuration.deprecation - io.bytes.per.checksum
> is deprecated. Instead, use dfs.bytes-per-checksum
> 2013-11-27 18:53:19,519 [main] INFO
>  org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> deprecated. Instead, use fs.defaultFS
> grunt> ls /home/watrous/input/
> file:/home/watrous/input/sample.txt<r 1>        530
> grunt> cat /home/watrous/input/sample.txt
>
> 0067011990999991950051507004+68750+023550FM-12+038299999V0203301N00671220001CN9999999N9+00001+99999999999
>
> 0043011990999991950051512004+68750+023550FM-12+038299999V0203201N00671220001CN9999999N9+00221+99999999999
>
> 0043011990999991950051518004+68750+023550FM-12+038299999V0203201N00261220001CN9999999N9-00111+99999999999
>
> 0043012650999991949032412004+62300+010750FM-12+048599999V0202701N00461220001CN0500001N9+01111+99999999999
>
> 0043012650999991949032418004+62300+010750FM-12+048599999V0202701N00461220001CN0500001N9+00781+99999999999
> grunt> records = load '/home/watrous/input/sample.txt';
> 2013-11-27 18:53:25,086 [main] INFO
>  org.apache.hadoop.conf.Configuration.deprecation - job.end.retry.interval
> is deprecated. Instead, use mapreduce.job.end-notification.retry.interval
> 2013-11-27 18:53:25,086 [main] INFO
>  org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.tracker.retiredjobs.cache.size is deprecated. Instead, use
> mapreduce.jobtracker.retiredjobs.cache.size
> 2013-11-27 18:53:25,086 [main] INFO
>  org.apache.hadoop.conf.Configuration.deprecation -