Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> "Failed to create DataStorage" ?


Copy link to this message
-
Re: "Failed to create DataStorage" ?
Apologies, just read your email again to realize your Java program is
what's not working. Your command-line setup seems fine.

Can you try running your program with
/home/rfcompton/Downloads/pig-0.11.0-withouthadoop.jar instead of
/home/rfcompton/Downloads/pig-0.11.0-src/pig.jar and make sure 0.20.2
hadoop is on the classpath.
On Thu, Mar 21, 2013 at 1:36 PM, Ryan Compton <[EMAIL PROTECTED]>wrote:

> -bash-3.2$ pig -secretDebugCmd
> Find hadoop at /usr/bin/hadoop
> dry run:
> HADOOP_CLASSPATH:
>
> /data/osi/cm-gui/hbase-global-clientconfig/:/home/rfcompton/Downloads/pig-0.11.0-src/bin/../conf:/usr/java/jdk1.6.0_26/lib/tools.jar:/data/osi/cm-gui/global-clientconfig/hadoop-conf/:/home/rfcompton/Downloads/pig-0.11.0-src/bin/../build/ivy/lib/Pig/jython-standalone-2.5.2.jar:/home/isl/rfcompton/Downloads/pig-0.11.0-src/bin/../build/ivy/lib/Pig/jruby-complete-1.6.7.jar:/home/rfcompton/Downloads/pig-0.11.0-src/bin/../pig-withouthadoop.jar:
> HADOOP_OPTS: -Xmx1000m
> -Dpig.log.dir=/home/rfcompton/Downloads/pig-0.11.0-src/bin/../logs
> -Dpig.log.file=pig.log
> -Dpig.home.dir=/home/rfcompton/Downloads/pig-0.11.0-src/bin/..
> /usr/bin/hadoop jar
> /home/rfcompton/Downloads/pig-0.11.0-src/bin/../pig-withouthadoop.jar
>
> On Thu, Mar 21, 2013 at 1:34 PM, Prashant Kommireddi
> <[EMAIL PROTECTED]> wrote:
> > What is the output of "pig -secretDebugCmd"
> >
> > On Thu, Mar 21, 2013 at 1:29 PM, Ryan Compton <[EMAIL PROTECTED]
> >wrote:
> >
> >> Hi,
> >>
> >> Hmm, I've got that much:
> >>
> >> -bash-3.2$ ls $HADOOP_HOME | grep cdh3u3
> >> hadoop-0.20.2-cdh3u3-ant.jar
> >> hadoop-0.20.2-cdh3u3-core.jar
> >> hadoop-0.20.2-cdh3u3-examples.jar
> >> hadoop-0.20.2-cdh3u3-test.jar
> >> hadoop-0.20.2-cdh3u3-tools.jar
> >> hadoop-ant-0.20.2-cdh3u3.jar
> >> hadoop-core-0.20.2-cdh3u3.jar
> >> hadoop-examples-0.20.2-cdh3u3.jar
> >> hadoop-test-0.20.2-cdh3u3.jar
> >> hadoop-tools-0.20.2-cdh3u3.jar
> >>
> >> but I still have the same problem. More info:
> http://pastebin.com/MfUHwu0X
> >>
> >> On Thu, Mar 21, 2013 at 1:16 PM, Prashant Kommireddi
> >> <[EMAIL PROTECTED]> wrote:
> >> > Hi Ryan,
> >> >
> >> > Seems like you are trying to connect to a hadoop cluster running
> 0.20.2.
> >> > Pig by default uses hadoop 1.0 unless you specify otherwise.
> >> >
> >> > You should point HADOOP_HOME to your hadoop installation dir
> >> > (0.20.2-cdh3u3) from where you are launching pig
> >> >
> >> > export HADOOP_HOME=<path_to_0.20.2-cdh3u3>
> >> >
> >> >
> >> > On Thu, Mar 21, 2013 at 12:59 PM, Ryan Compton <
> [EMAIL PROTECTED]
> >> >wrote:
> >> >
> >> >> I can start a grunt shell just fine:
> >> >>
> >> >> -bash-3.2$ pwd
> >> >> /home/rfcompton/Downloads/pig-0.11.0-src
> >> >> -bash-3.2$ ./bin/pig
> >> >> 2013-03-21 12:55:00,048 [main] INFO  org.apache.pig.Main - Apache Pig
> >> >> version 0.11.1-SNAPSHOT (rexported) compiled Mar 21 2013, 12:49:21
> >> >> 2013-03-21 12:55:00,049 [main] INFO  org.apache.pig.Main - Logging
> >> >> error messages to:
> >> >> /home/rfcompton/Downloads/pig-0.11.0-src/pig_1363895700046.log
> >> >> 2013-03-21 12:55:00,076 [main] INFO  org.apache.pig.impl.util.Utils -
> >> >> Default bootup file /home/rfcompton/.pigbootup not found
> >> >> 2013-03-21 12:55:00,330 [main] INFO
> >> >> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> >> >> Connecting to hadoop file system at: hdfs://master:8020/
> >> >> 2013-03-21 12:55:00,581 [main] INFO
> >> >> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> >> >> Connecting to map-reduce job tracker at: node4:8021
> >> >> grunt>
> >> >>
> >> >> But I can't run a java program:
> >> >>
> >> >> -bash-3.2$ java -cp
> >> >>
> >> >>
> >>
> target/mybigjar-1.1-SNAPSHOT-jar-with-dependencies.jar:$HADOOP_CONF_DIR:/home/rfcompton/Downloads/pig-0.11.0-src/pig.jar
> >> >> org.apache.pig.Main
> >> >> 2013-03-21 12:55:58,806 [main] INFO  org.apache.pig.Main - Apache Pig
> >> >> version 0.11.1-SNAPSHOT (rexported) compiled Mar 21 2013, 12:49:21
> >> >> 2013-03-21 12:55:58,806 [main] INFO  org.apache.pig.Main - Logging
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB