Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> "Failed to create DataStorage" ?


Copy link to this message
-
Re: "Failed to create DataStorage" ?
java -cp target/mybigjar-1.1-SNAPSHOT-jar-with-dependencies.jar:$HADOOP_CONF_DIR:/home/rfcompton/Downloads/pig-0.11.0-src/pig-withouthadoop.jar
org.apache.pig.Main

Still can't create DataStorage. How far will I be able to get with pig
if I restrict myself to the command line? UDFs will still work, right?

On Thu, Mar 21, 2013 at 1:54 PM, Prashant Kommireddi
<[EMAIL PROTECTED]> wrote:
> Apologies, just read your email again to realize your Java program is
> what's not working. Your command-line setup seems fine.
>
> Can you try running your program with
> /home/rfcompton/Downloads/pig-0.11.0-withouthadoop.jar instead of
> /home/rfcompton/Downloads/pig-0.11.0-src/pig.jar and make sure 0.20.2
> hadoop is on the classpath.
>
>
> On Thu, Mar 21, 2013 at 1:36 PM, Ryan Compton <[EMAIL PROTECTED]>wrote:
>
>> -bash-3.2$ pig -secretDebugCmd
>> Find hadoop at /usr/bin/hadoop
>> dry run:
>> HADOOP_CLASSPATH:
>>
>> /data/osi/cm-gui/hbase-global-clientconfig/:/home/rfcompton/Downloads/pig-0.11.0-src/bin/../conf:/usr/java/jdk1.6.0_26/lib/tools.jar:/data/osi/cm-gui/global-clientconfig/hadoop-conf/:/home/rfcompton/Downloads/pig-0.11.0-src/bin/../build/ivy/lib/Pig/jython-standalone-2.5.2.jar:/home/isl/rfcompton/Downloads/pig-0.11.0-src/bin/../build/ivy/lib/Pig/jruby-complete-1.6.7.jar:/home/rfcompton/Downloads/pig-0.11.0-src/bin/../pig-withouthadoop.jar:
>> HADOOP_OPTS: -Xmx1000m
>> -Dpig.log.dir=/home/rfcompton/Downloads/pig-0.11.0-src/bin/../logs
>> -Dpig.log.file=pig.log
>> -Dpig.home.dir=/home/rfcompton/Downloads/pig-0.11.0-src/bin/..
>> /usr/bin/hadoop jar
>> /home/rfcompton/Downloads/pig-0.11.0-src/bin/../pig-withouthadoop.jar
>>
>> On Thu, Mar 21, 2013 at 1:34 PM, Prashant Kommireddi
>> <[EMAIL PROTECTED]> wrote:
>> > What is the output of "pig -secretDebugCmd"
>> >
>> > On Thu, Mar 21, 2013 at 1:29 PM, Ryan Compton <[EMAIL PROTECTED]
>> >wrote:
>> >
>> >> Hi,
>> >>
>> >> Hmm, I've got that much:
>> >>
>> >> -bash-3.2$ ls $HADOOP_HOME | grep cdh3u3
>> >> hadoop-0.20.2-cdh3u3-ant.jar
>> >> hadoop-0.20.2-cdh3u3-core.jar
>> >> hadoop-0.20.2-cdh3u3-examples.jar
>> >> hadoop-0.20.2-cdh3u3-test.jar
>> >> hadoop-0.20.2-cdh3u3-tools.jar
>> >> hadoop-ant-0.20.2-cdh3u3.jar
>> >> hadoop-core-0.20.2-cdh3u3.jar
>> >> hadoop-examples-0.20.2-cdh3u3.jar
>> >> hadoop-test-0.20.2-cdh3u3.jar
>> >> hadoop-tools-0.20.2-cdh3u3.jar
>> >>
>> >> but I still have the same problem. More info:
>> http://pastebin.com/MfUHwu0X
>> >>
>> >> On Thu, Mar 21, 2013 at 1:16 PM, Prashant Kommireddi
>> >> <[EMAIL PROTECTED]> wrote:
>> >> > Hi Ryan,
>> >> >
>> >> > Seems like you are trying to connect to a hadoop cluster running
>> 0.20.2.
>> >> > Pig by default uses hadoop 1.0 unless you specify otherwise.
>> >> >
>> >> > You should point HADOOP_HOME to your hadoop installation dir
>> >> > (0.20.2-cdh3u3) from where you are launching pig
>> >> >
>> >> > export HADOOP_HOME=<path_to_0.20.2-cdh3u3>
>> >> >
>> >> >
>> >> > On Thu, Mar 21, 2013 at 12:59 PM, Ryan Compton <
>> [EMAIL PROTECTED]
>> >> >wrote:
>> >> >
>> >> >> I can start a grunt shell just fine:
>> >> >>
>> >> >> -bash-3.2$ pwd
>> >> >> /home/rfcompton/Downloads/pig-0.11.0-src
>> >> >> -bash-3.2$ ./bin/pig
>> >> >> 2013-03-21 12:55:00,048 [main] INFO  org.apache.pig.Main - Apache Pig
>> >> >> version 0.11.1-SNAPSHOT (rexported) compiled Mar 21 2013, 12:49:21
>> >> >> 2013-03-21 12:55:00,049 [main] INFO  org.apache.pig.Main - Logging
>> >> >> error messages to:
>> >> >> /home/rfcompton/Downloads/pig-0.11.0-src/pig_1363895700046.log
>> >> >> 2013-03-21 12:55:00,076 [main] INFO  org.apache.pig.impl.util.Utils -
>> >> >> Default bootup file /home/rfcompton/.pigbootup not found
>> >> >> 2013-03-21 12:55:00,330 [main] INFO
>> >> >> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
>> >> >> Connecting to hadoop file system at: hdfs://master:8020/
>> >> >> 2013-03-21 12:55:00,581 [main] INFO
>> >> >> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
>> >> >> Connecting to map-reduce job tracker at: node4:8021
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB