Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - "Failed to create DataStorage" ?


Copy link to this message
-
Re: "Failed to create DataStorage" ?
Ryan Compton 2013-03-21, 21:15
java -cp target/mybigjar-1.1-SNAPSHOT-jar-with-dependencies.jar:$HADOOP_CONF_DIR:/home/rfcompton/Downloads/pig-0.11.0-src/pig-withouthadoop.jar
org.apache.pig.Main

Still can't create DataStorage. How far will I be able to get with pig
if I restrict myself to the command line? UDFs will still work, right?

On Thu, Mar 21, 2013 at 1:54 PM, Prashant Kommireddi
<[EMAIL PROTECTED]> wrote:
> Apologies, just read your email again to realize your Java program is
> what's not working. Your command-line setup seems fine.
>
> Can you try running your program with
> /home/rfcompton/Downloads/pig-0.11.0-withouthadoop.jar instead of
> /home/rfcompton/Downloads/pig-0.11.0-src/pig.jar and make sure 0.20.2
> hadoop is on the classpath.
>
>
> On Thu, Mar 21, 2013 at 1:36 PM, Ryan Compton <[EMAIL PROTECTED]>wrote:
>
>> -bash-3.2$ pig -secretDebugCmd
>> Find hadoop at /usr/bin/hadoop
>> dry run:
>> HADOOP_CLASSPATH:
>>
>> /data/osi/cm-gui/hbase-global-clientconfig/:/home/rfcompton/Downloads/pig-0.11.0-src/bin/../conf:/usr/java/jdk1.6.0_26/lib/tools.jar:/data/osi/cm-gui/global-clientconfig/hadoop-conf/:/home/rfcompton/Downloads/pig-0.11.0-src/bin/../build/ivy/lib/Pig/jython-standalone-2.5.2.jar:/home/isl/rfcompton/Downloads/pig-0.11.0-src/bin/../build/ivy/lib/Pig/jruby-complete-1.6.7.jar:/home/rfcompton/Downloads/pig-0.11.0-src/bin/../pig-withouthadoop.jar:
>> HADOOP_OPTS: -Xmx1000m
>> -Dpig.log.dir=/home/rfcompton/Downloads/pig-0.11.0-src/bin/../logs
>> -Dpig.log.file=pig.log
>> -Dpig.home.dir=/home/rfcompton/Downloads/pig-0.11.0-src/bin/..
>> /usr/bin/hadoop jar
>> /home/rfcompton/Downloads/pig-0.11.0-src/bin/../pig-withouthadoop.jar
>>
>> On Thu, Mar 21, 2013 at 1:34 PM, Prashant Kommireddi
>> <[EMAIL PROTECTED]> wrote:
>> > What is the output of "pig -secretDebugCmd"
>> >
>> > On Thu, Mar 21, 2013 at 1:29 PM, Ryan Compton <[EMAIL PROTECTED]
>> >wrote:
>> >
>> >> Hi,
>> >>
>> >> Hmm, I've got that much:
>> >>
>> >> -bash-3.2$ ls $HADOOP_HOME | grep cdh3u3
>> >> hadoop-0.20.2-cdh3u3-ant.jar
>> >> hadoop-0.20.2-cdh3u3-core.jar
>> >> hadoop-0.20.2-cdh3u3-examples.jar
>> >> hadoop-0.20.2-cdh3u3-test.jar
>> >> hadoop-0.20.2-cdh3u3-tools.jar
>> >> hadoop-ant-0.20.2-cdh3u3.jar
>> >> hadoop-core-0.20.2-cdh3u3.jar
>> >> hadoop-examples-0.20.2-cdh3u3.jar
>> >> hadoop-test-0.20.2-cdh3u3.jar
>> >> hadoop-tools-0.20.2-cdh3u3.jar
>> >>
>> >> but I still have the same problem. More info:
>> http://pastebin.com/MfUHwu0X
>> >>
>> >> On Thu, Mar 21, 2013 at 1:16 PM, Prashant Kommireddi
>> >> <[EMAIL PROTECTED]> wrote:
>> >> > Hi Ryan,
>> >> >
>> >> > Seems like you are trying to connect to a hadoop cluster running
>> 0.20.2.
>> >> > Pig by default uses hadoop 1.0 unless you specify otherwise.
>> >> >
>> >> > You should point HADOOP_HOME to your hadoop installation dir
>> >> > (0.20.2-cdh3u3) from where you are launching pig
>> >> >
>> >> > export HADOOP_HOME=<path_to_0.20.2-cdh3u3>
>> >> >
>> >> >
>> >> > On Thu, Mar 21, 2013 at 12:59 PM, Ryan Compton <
>> [EMAIL PROTECTED]
>> >> >wrote:
>> >> >
>> >> >> I can start a grunt shell just fine:
>> >> >>
>> >> >> -bash-3.2$ pwd
>> >> >> /home/rfcompton/Downloads/pig-0.11.0-src
>> >> >> -bash-3.2$ ./bin/pig
>> >> >> 2013-03-21 12:55:00,048 [main] INFO  org.apache.pig.Main - Apache Pig
>> >> >> version 0.11.1-SNAPSHOT (rexported) compiled Mar 21 2013, 12:49:21
>> >> >> 2013-03-21 12:55:00,049 [main] INFO  org.apache.pig.Main - Logging
>> >> >> error messages to:
>> >> >> /home/rfcompton/Downloads/pig-0.11.0-src/pig_1363895700046.log
>> >> >> 2013-03-21 12:55:00,076 [main] INFO  org.apache.pig.impl.util.Utils -
>> >> >> Default bootup file /home/rfcompton/.pigbootup not found
>> >> >> 2013-03-21 12:55:00,330 [main] INFO
>> >> >> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
>> >> >> Connecting to hadoop file system at: hdfs://master:8020/
>> >> >> 2013-03-21 12:55:00,581 [main] INFO
>> >> >> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
>> >> >> Connecting to map-reduce job tracker at: node4:8021