Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> "Failed to create DataStorage" ?


Copy link to this message
-
Re: "Failed to create DataStorage" ?
It has to be a classpath issue with a different hadoop version being picked
up. I can very easily reproduce the same issue on my end (copied at the end
of this email)

Regarding your other question, running pig from command-line will let you
run UDFs and provide most functionality you would have otherwise. Note that
complex looping sometimes requires java/python or other pig supported
languages. If you don't have such a requirement command-line will work just
fine (it is also what's generally used more than Pig's Java APIs).

I would recommend trying to check on the classpath for your issue though in
any case.

Error before Pig is launched
----------------------------
ERROR 2999: Unexpected internal error. Failed to create DataStorage

java.lang.RuntimeException: Failed to create DataStorage
    at
org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75)
    at
org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:58)
    at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:205)
    at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:118)
    at org.apache.pig.impl.PigContext.connect(PigContext.java:208)
    at org.apache.pig.PigServer.<init>(PigServer.java:246)
    at org.apache.pig.PigServer.<init>(PigServer.java:231)
    at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:47)
    at org.apache.pig.Main.run(Main.java:487)
    at org.apache.pig.Main.main(Main.java:111)
Caused by: java.io.IOException: Call to x.y.z.net/XX.XX.XX.XX:54310 failed
on local exception: java.io.EOFException
    at org.apache.hadoop.ipc.Client.wrapException(Client.java:1103)
    at org.apache.hadoop.ipc.Client.call(Client.java:1071)
    at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
    at $Proxy1.getProtocolVersion(Unknown Source)
    at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
    at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)
    at
org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:119)
    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:238)
    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:203)
    at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89)
    at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1404)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:123)
    at
org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72)
    ... 9 more
Caused by: java.io.EOFException
    at java.io.DataInputStream.readInt(DataInputStream.java:375)
    at
org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:800)
    at org.apache.hadoop.ipc.Client$Connection.run(Client.java:745)
===============================================================================

On Thu, Mar 21, 2013 at 2:15 PM, Ryan Compton <[EMAIL PROTECTED]>wrote:

> java -cp
> target/mybigjar-1.1-SNAPSHOT-jar-with-dependencies.jar:$HADOOP_CONF_DIR:/home/rfcompton/Downloads/pig-0.11.0-src/pig-withouthadoop.jar
> org.apache.pig.Main
>
> Still can't create DataStorage. How far will I be able to get with pig
> if I restrict myself to the command line? UDFs will still work, right?
>
> On Thu, Mar 21, 2013 at 1:54 PM, Prashant Kommireddi
> <[EMAIL PROTECTED]> wrote:
> > Apologies, just read your email again to realize your Java program is
> > what's not working. Your command-line setup seems fine.
> >
> > Can you try running your program with
> > /home/rfcompton/Downloads/pig-0.11.0-withouthadoop.jar instead of
> > /home/rfcompton/Downloads/pig-0.11.0-src/pig.jar and make sure 0.20.2
> > hadoop is on the classpath.
> >
> >
> > On Thu, Mar 21, 2013 at 1:36 PM, Ryan Compton <[EMAIL PROTECTED]
> >wrote:
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB