Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Unable to run PIG local as regular user (non-root)


Copy link to this message
-
RE: Unable to run PIG local as regular user (non-root)
Cheolsoo,

I hadn't noticed that. I do have hadoop 2.2.0 installed. Does the PIG distribution include hadoop v1? I suppose that if my environment variables aren't set correctly for hadoop 2 as root, then it may default to version 1 of hadoop. The reason for this is that the path to hadoop/bin is part of PATH as my user, but not as root.

I have confirmed this by removing the path to hadoop binaries from my PATH environment variable and running it locally. Everything worked identically. To correct this permanently I should either move backward to version 1.x of hadoop or compile PIG with hadoop 2.x support. Has PIG been tested against hadoop 2.x?

Daniel

-----Original Message-----
From: Cheolsoo Park [mailto:[EMAIL PROTECTED]]
Sent: Sunday, December 01, 2013 9:55 PM
To: [EMAIL PROTECTED]
Subject: Re: Unable to run PIG local as regular user (non-root)

Hi Daniel,

>From the console output, I can see Pig ran against Hadoop 1.x as root while it ran against Hadoop 2.x as watrous. I don't know what your setup is like, but please double check what version of Hadoop jars are present in classpath. Running "pig -secretDebugCmd" should show HADOOP_CLASSPATH.

Apparently, your Pig jar is compiled against Hadoop 1.x, so it only works against Hadoop 1.x (as root).

Thanks,
Cheolsoo

On Wed, Nov 27, 2013 at 11:01 AM, Watrous, Daniel
<[EMAIL PROTECTED]>wrote:

> Hi,
>
> I've been following the book Hadoop Definitive Guide and a few other
> resources while exploring PIG. I am able to run as root, but not as a
> regular user. I am running in local mode. I've found a few links, but
> they haven't helped me solve my issue. The most relevant seems to be:
> http://answers.mapr.com/questions/5135/cannot-run-pig-in-local-mode
>
> My first experiment is just to load in a text file and dump it to the
> console. The files exist and are owned by the user running PIG. I show
> the failed attempt as a regular user followed by a successful run as
> root on the same system. Any help is appreciated.
>
> Here's what I'm trying:
> [watrous@c0003913 ~]$ pig -x local
> 2013-11-27 18:53:18,691 [main] INFO  org.apache.pig.Main - Apache Pig
> version 0.12.0 (r1529718) compiled Oct 07 2013, 12:20:14
> 2013-11-27 18:53:18,692 [main] INFO  org.apache.pig.Main - Logging
> error messages to: /home/watrous/pig_1385578398689.log
> 2013-11-27 18:53:18,747 [main] INFO  org.apache.pig.impl.util.Utils -
> Default bootup file /home/watrous/.pigbootup not found
> 2013-11-27 18:53:19,025 [main] INFO
>  org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 2013-11-27 18:53:19,025 [main] INFO
>  org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker
> is deprecated. Instead, use mapreduce.jobtracker.address
> 2013-11-27 18:53:19,027 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> Connecting to hadoop file system at: file:///
> 2013-11-27 18:53:19,031 [main] INFO
>  org.apache.hadoop.conf.Configuration.deprecation -
> mapred.used.genericoptionsparser is deprecated. Instead, use
> mapreduce.client.genericoptionsparser.used
> 2013-11-27 18:53:19,451 [main] WARN
>  org.apache.hadoop.util.NativeCodeLoader - Unable to load
> native-hadoop library for your platform... using builtin-java classes
> where applicable
> 2013-11-27 18:53:19,517 [main] INFO
>  org.apache.hadoop.conf.Configuration.deprecation -
> io.bytes.per.checksum is deprecated. Instead, use
> dfs.bytes-per-checksum
> 2013-11-27 18:53:19,519 [main] INFO
>  org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> deprecated. Instead, use fs.defaultFS
> grunt> ls /home/watrous/input/
> file:/home/watrous/input/sample.txt<r 1>        530
> grunt> cat /home/watrous/input/sample.txt
>
> 0067011990999991950051507004+68750+023550FM-12+038299999V0203301N00671220001CN9999999N9+00001+99999999999
>
> 0043011990999991950051512004+68750+023550FM-12+038299999V0203201N00671220001CN9999999N9+00221+99999999999
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB