Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> How pig get hadoop and hbase configuration?


+
lulynn_2008 2012-06-13, 07:04
+
Mohammad Tariq 2012-06-13, 07:35
Copy link to this message
-
Re:Re: How pig get hadoop and hbase configuration?
Hello,
hadoop-core-*.jar and commons-configuration-1.6.jar have been in hbase lib directory. jobtracker node can get correct hbase configuration, but tasktracker node can not.
At 2012-06-13 15:35:21,"Mohammad Tariq" <[EMAIL PROTECTED]> wrote:
>Hello,
>
>   Copy the hadoop-core-*.jar from your hadoop folder to the hbase/lib
>folder.Also copy commons-configuration-1.6.jar from hadoop/lib folder
>to hbase/lib folder...Some times due to incompatible jars this may
>happen..do it and see if it works for you.
>
>Regards,
>    Mohammad Tariq
>
>
>On Wed, Jun 13, 2012 at 12:34 PM, lulynn_2008 <[EMAIL PROTECTED]> wrote:
>>  Hi everyone,
>> Following is mine test environment:
>> node 1:namenode, secondarynamenode, jobtracker, hbase master
>> node 2:datanode, tasktracker
>> In node 1, I run following COMMANDS in pig shell, but I found map task failed in tasktracker node with error "HBase is able to connect to ZooKeeper but the connection closes immediately.". This mean tasktracker did not get current hbase configuration. But I can find the correct hbase configuration in jobtracker node. Seems tasktracker node did not get configuration from jobtracker node, but get configuration from hadoop classpath in tasktracker node.
>> I think tasktracker node should get hbase configuration from jobtracker node, but not from local hadoop classpath. Am I correct?
>>
>> In tasktracker side, after I add hbase-site.xml to hadoop classpath, the test case passed.
>> My question is: how tasktracker node get hbase configuration from tasktracker side? From jobtracker side(included in *.jar file transferred by jobtracker node) or local hadoop classpath?
>>
>> COMMANDS:
>> REGISTER /home/pig/Rules.jar;
>> REGISTER '/home/pig/zookeeper.jar';
>> REGISTER '/home/pig/guava-r06.jar';
>> REGISTER '/home/pig/hbase-0.90.5.jar';
>>
>> test = LOAD 'hbase://table' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage( 'd:sWords','-loadKey true') AS (ID: bytearray  , Words:chararray );
>> result = FOREACH test GENERATE ID, com.nice.rules.RunRules(Words);
>> --result = FOREACH AA GENERATE com.nice.rules.RunRules(Words), ID;
>> --dump result;
>>
>> store result into 'table' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('d:drools_cat');
>> --store result into 'AA_10_categs' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('d:cat');
>>
+
Mohammad Tariq 2012-06-13, 07:48
+
Mohammad Tariq 2012-06-13, 07:50
+
lulynn_2008 2012-06-13, 09:15
+
shashwat shriparv 2012-06-13, 09:34
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB