Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> HBaseStorage in pig 0.8


Copy link to this message
-
Re: HBaseStorage in pig 0.8
Hey Jeff,

It wasn't starting a job but I got a bit further by registering the pig8 jar in my pig script.  It seemed to have a bunch of dependencies on google common collections; zookeeper etc... built into that jar.

Now I am seeing this in the web ui logs:
2010-11-19 23:19:44,200 INFO org.apache.zookeeper.ClientCnxn: Attempting connection to server localhost/127.0.0.1:2181
2010-11-19 23:19:44,201 WARN org.apache.zookeeper.ClientCnxn: Exception closing session 0x0 to sun.nio.ch.SelectionKeyImpl@65efb4be
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:885)
2010-11-19 23:19:44,201 WARN org.apache.zookeeper.ClientCnxn: Ignoring exception during shutdown input
java.nio.channels.ClosedChannelException
at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:638)
at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:951)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:922)
2010-11-19 23:19:44,201 WARN org.apache.zookeeper.ClientCnxn: Ignoring exception during shutdown output
java.nio.channels.ClosedChannelException
at sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:649)
at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368)
at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:956)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:922)
2010-11-19 23:19:44,303 WARN org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Failed to create /hbase -- check quorum servers, currently=localhost:2181
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase
Looks like it doesn't know where my hbase/conf/hbase-site.xml file is?  Not sure how would this get passed to the HBaseStorage class?

On Nov 19, 2010, at 5:09 PM, Jeff Zhang wrote:

> Does the mapreduce job start ? Could you check the logs on hadoop side ?
>
>
> On Sat, Nov 20, 2010 at 7:56 AM, Corbin Hoenes <[EMAIL PROTECTED]> wrote:
>> We are trying to use the HBaseStorage LoadFunc in pig 0.8 and getting an exception.
>>
>> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias raw
>> at org.apache.pig.PigServer.openIterator(PigServer.java:754)
>> at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
>> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
>> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
>> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:76)
>> at org.apache.pig.Main.run(Main.java:465)
>> at org.apache.pig.Main.main(Main.java:107)
>> Caused by: java.io.IOException: Couldn't retrieve job.
>> at org.apache.pig.PigServer.store(PigServer.java:818)
>> at org.apache.pig.PigServer.openIterator(PigServer.java:728)
>> ... 7 more
>>
>>
>> Other jobs seem to work.
>>
>> What are the requirements for getting hbase storage to work?
>>
>> This is what I am doing:
>> 1 - added hbase config and hadoop config to my PIG_CLASSPATH
>> 2 - pig this script:
>>
>> REGISTER ../lib/hbase-0.20.6.jar
>>
>> raw = LOAD 'hbase://piggytest' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('content:field1 anchor:field1a anchor:field2a') as (content_field1, anchor_field1a, anchor_field2a);
>>
>> dump raw;
>>
>> ---
>> what else am I missing?
>
>
>
> --
> Best Regards
>
> Jeff Zhang
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB