Hi everyone,
Following is mine test environment:
node 1:namenode, secondarynamenode, jobtracker, hbase master
node 2:datanode, tasktracker
In node 1, I run following COMMANDS in pig shell, but I found map task failed in tasktracker node with error "HBase is able to connect to ZooKeeper but the connection closes immediately.". This mean tasktracker did not get current hbase configuration. But I can find the correct hbase configuration in jobtracker node. Seems tasktracker node did not get configuration from jobtracker node, but get configuration from hadoop classpath in tasktracker node.
I think tasktracker node should get hbase configuration from jobtracker node, but not from local hadoop classpath. Am I correct?

In tasktracker side, after I add hbase-site.xml to hadoop classpath, the test case passed.
My question is: how tasktracker node get hbase configuration from tasktracker side? From jobtracker side(included in *.jar file transferred by jobtracker node) or local hadoop classpath?

COMMANDS:
REGISTER /home/pig/Rules.jar;
REGISTER '/home/pig/zookeeper.jar';
REGISTER '/home/pig/guava-r06.jar';
REGISTER '/home/pig/hbase-0.90.5.jar';

test = LOAD 'hbase://table' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage( 'd:sWords','-loadKey true') AS (ID: bytearray  , Words:chararray );
result = FOREACH test GENERATE ID, com.nice.rules.RunRules(Words);
--result = FOREACH AA GENERATE com.nice.rules.RunRules(Words), ID;
--dump result;

store result into 'table' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('d:drools_cat');
--store result into 'AA_10_categs' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('d:cat');

Hello,

   Copy the hadoop-core-*.jar from your hadoop folder to the hbase/lib
folder.Also copy commons-configuration-1.6.jar from hadoop/lib folder
to hbase/lib folder...Some times due to incompatible jars this may
happen..do it and see if it works for you.

Regards,
    Mohammad Tariq
On Wed, Jun 13, 2012 at 12:34 PM, lulynn_2008 <[EMAIL PROTECTED]> wrote:
>  Hi everyone,
> Following is mine test environment:
> node 1:namenode, secondarynamenode, jobtracker, hbase master
> node 2:datanode, tasktracker
> In node 1, I run following COMMANDS in pig shell, but I found map task failed in tasktracker node with error "HBase is able to connect to ZooKeeper but the connection closes immediately.". This mean tasktracker did not get current hbase configuration. But I can find the correct hbase configuration in jobtracker node. Seems tasktracker node did not get configuration from jobtracker node, but get configuration from hadoop classpath in tasktracker node.
> I think tasktracker node should get hbase configuration from jobtracker node, but not from local hadoop classpath. Am I correct?
>
> In tasktracker side, after I add hbase-site.xml to hadoop classpath, the test case passed.
> My question is: how tasktracker node get hbase configuration from tasktracker side? From jobtracker side(included in *.jar file transferred by jobtracker node) or local hadoop classpath?
>
> COMMANDS:
> REGISTER /home/pig/Rules.jar;
> REGISTER '/home/pig/zookeeper.jar';
> REGISTER '/home/pig/guava-r06.jar';
> REGISTER '/home/pig/hbase-0.90.5.jar';
>
> test = LOAD 'hbase://table' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage( 'd:sWords','-loadKey true') AS (ID: bytearray  , Words:chararray );
> result = FOREACH test GENERATE ID, com.nice.rules.RunRules(Words);
> --result = FOREACH AA GENERATE com.nice.rules.RunRules(Words), ID;
> --dump result;
>
> store result into 'table' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('d:drools_cat');
> --store result into 'AA_10_categs' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('d:cat');
>

Hello,
hadoop-core-*.jar and commons-configuration-1.6.jar have been in hbase lib directory. jobtracker node can get correct hbase configuration, but tasktracker node can not.
At 2012-06-13 15:35:21,"Mohammad Tariq" <[EMAIL PROTECTED]> wrote:
>Hello,
>
>   Copy the hadoop-core-*.jar from your hadoop folder to the hbase/lib
>folder.Also copy commons-configuration-1.6.jar from hadoop/lib folder
>to hbase/lib folder...Some times due to incompatible jars this may
>happen..do it and see if it works for you.
>
>Regards,
>    Mohammad Tariq
>
>
>On Wed, Jun 13, 2012 at 12:34 PM, lulynn_2008 <[EMAIL PROTECTED]> wrote:
>>  Hi everyone,
>> Following is mine test environment:
>> node 1:namenode, secondarynamenode, jobtracker, hbase master
>> node 2:datanode, tasktracker
>> In node 1, I run following COMMANDS in pig shell, but I found map task failed in tasktracker node with error "HBase is able to connect to ZooKeeper but the connection closes immediately.". This mean tasktracker did not get current hbase configuration. But I can find the correct hbase configuration in jobtracker node. Seems tasktracker node did not get configuration from jobtracker node, but get configuration from hadoop classpath in tasktracker node.
>> I think tasktracker node should get hbase configuration from jobtracker node, but not from local hadoop classpath. Am I correct?
>>
>> In tasktracker side, after I add hbase-site.xml to hadoop classpath, the test case passed.
>> My question is: how tasktracker node get hbase configuration from tasktracker side? From jobtracker side(included in *.jar file transferred by jobtracker node) or local hadoop classpath?
>>
>> COMMANDS:
>> REGISTER /home/pig/Rules.jar;
>> REGISTER '/home/pig/zookeeper.jar';
>> REGISTER '/home/pig/guava-r06.jar';
>> REGISTER '/home/pig/hbase-0.90.5.jar';
>>
>> test = LOAD 'hbase://table' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage( 'd:sWords','-loadKey true') AS (ID: bytearray  , Words:chararray );
>> result = FOREACH test GENERATE ID, com.nice.rules.RunRules(Words);
>> --result = FOREACH AA GENERATE com.nice.rules.RunRules(Words), ID;
>> --dump result;
>>
>> store result into 'table' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('d:drools_cat');
>> --store result into 'AA_10_categs' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('d:cat');
>>

"HBase is able to connect to ZooKeeper but the connection closes
immediately." - This error means that your HMaster is not able to talk
to your Namenode.

Regards,
    Mohammad Tariq
On Wed, Jun 13, 2012 at 1:12 PM, lulynn_2008 <[EMAIL PROTECTED]> wrote:
> Hello,
> hadoop-core-*.jar and commons-configuration-1.6.jar have been in hbase lib directory. jobtracker node can get correct hbase configuration, but tasktracker node can not.
>
>
>
>
> At 2012-06-13 15:35:21,"Mohammad Tariq" <[EMAIL PROTECTED]> wrote:
>>Hello,
>>
>>   Copy the hadoop-core-*.jar from your hadoop folder to the hbase/lib
>>folder.Also copy commons-configuration-1.6.jar from hadoop/lib folder
>>to hbase/lib folder...Some times due to incompatible jars this may
>>happen..do it and see if it works for you.
>>
>>Regards,
>>    Mohammad Tariq
>>
>>
>>On Wed, Jun 13, 2012 at 12:34 PM, lulynn_2008 <[EMAIL PROTECTED]> wrote:
>>>  Hi everyone,
>>> Following is mine test environment:
>>> node 1:namenode, secondarynamenode, jobtracker, hbase master
>>> node 2:datanode, tasktracker
>>> In node 1, I run following COMMANDS in pig shell, but I found map task failed in tasktracker node with error "HBase is able to connect to ZooKeeper but the connection closes immediately.". This mean tasktracker did not get current hbase configuration. But I can find the correct hbase configuration in jobtracker node. Seems tasktracker node did not get configuration from jobtracker node, but get configuration from hadoop classpath in tasktracker node.
>>> I think tasktracker node should get hbase configuration from jobtracker node, but not from local hadoop classpath. Am I correct?
>>>
>>> In tasktracker side, after I add hbase-site.xml to hadoop classpath, the test case passed.
>>> My question is: how tasktracker node get hbase configuration from tasktracker side? From jobtracker side(included in *.jar file transferred by jobtracker node) or local hadoop classpath?
>>>
>>> COMMANDS:
>>> REGISTER /home/pig/Rules.jar;
>>> REGISTER '/home/pig/zookeeper.jar';
>>> REGISTER '/home/pig/guava-r06.jar';
>>> REGISTER '/home/pig/hbase-0.90.5.jar';
>>>
>>> test = LOAD 'hbase://table' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage( 'd:sWords','-loadKey true') AS (ID: bytearray  , Words:chararray );
>>> result = FOREACH test GENERATE ID, com.nice.rules.RunRules(Words);
>>> --result = FOREACH AA GENERATE com.nice.rules.RunRules(Words), ID;
>>> --dump result;
>>>
>>> store result into 'table' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('d:drools_cat');
>>> --store result into 'AA_10_categs' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('d:cat');
>>>

Could you send me your hadoop and hbase config files???

Regards,
    Mohammad Tariq
On Wed, Jun 13, 2012 at 1:18 PM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:
> "HBase is able to connect to ZooKeeper but the connection closes
> immediately." - This error means that your HMaster is not able to talk
> to your Namenode.
>
> Regards,
>     Mohammad Tariq
>
>
> On Wed, Jun 13, 2012 at 1:12 PM, lulynn_2008 <[EMAIL PROTECTED]> wrote:
>> Hello,
>> hadoop-core-*.jar and commons-configuration-1.6.jar have been in hbase lib directory. jobtracker node can get correct hbase configuration, but tasktracker node can not.
>>
>>
>>
>>
>> At 2012-06-13 15:35:21,"Mohammad Tariq" <[EMAIL PROTECTED]> wrote:
>>>Hello,
>>>
>>>   Copy the hadoop-core-*.jar from your hadoop folder to the hbase/lib
>>>folder.Also copy commons-configuration-1.6.jar from hadoop/lib folder
>>>to hbase/lib folder...Some times due to incompatible jars this may
>>>happen..do it and see if it works for you.
>>>
>>>Regards,
>>>    Mohammad Tariq
>>>
>>>
>>>On Wed, Jun 13, 2012 at 12:34 PM, lulynn_2008 <[EMAIL PROTECTED]> wrote:
>>>>  Hi everyone,
>>>> Following is mine test environment:
>>>> node 1:namenode, secondarynamenode, jobtracker, hbase master
>>>> node 2:datanode, tasktracker
>>>> In node 1, I run following COMMANDS in pig shell, but I found map task failed in tasktracker node with error "HBase is able to connect to ZooKeeper but the connection closes immediately.". This mean tasktracker did not get current hbase configuration. But I can find the correct hbase configuration in jobtracker node. Seems tasktracker node did not get configuration from jobtracker node, but get configuration from hadoop classpath in tasktracker node.
>>>> I think tasktracker node should get hbase configuration from jobtracker node, but not from local hadoop classpath. Am I correct?
>>>>
>>>> In tasktracker side, after I add hbase-site.xml to hadoop classpath, the test case passed.
>>>> My question is: how tasktracker node get hbase configuration from tasktracker side? From jobtracker side(included in *.jar file transferred by jobtracker node) or local hadoop classpath?
>>>>
>>>> COMMANDS:
>>>> REGISTER /home/pig/Rules.jar;
>>>> REGISTER '/home/pig/zookeeper.jar';
>>>> REGISTER '/home/pig/guava-r06.jar';
>>>> REGISTER '/home/pig/hbase-0.90.5.jar';
>>>>
>>>> test = LOAD 'hbase://table' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage( 'd:sWords','-loadKey true') AS (ID: bytearray  , Words:chararray );
>>>> result = FOREACH test GENERATE ID, com.nice.rules.RunRules(Words);
>>>> --result = FOREACH AA GENERATE com.nice.rules.RunRules(Words), ID;
>>>> --dump result;
>>>>
>>>> store result into 'table' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('d:drools_cat');
>>>> --store result into 'AA_10_categs' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('d:cat');
>>>>

I found that there is no hbase configuration in tasktracker classpath.  After we add hbase conf directory into tasktracker hadoop classpath, the test case passed. But I think the hbase configuration should be passed by jobtracker node, and I can find correct hbase configuration in jobtracker node: job*.xml.
Besides I can find the right hbase configuration in tasktrack:${mapred.local.dir}/taskTracker/jobcache/$jobid/job.xml, but this job.xml file is not in task java.class.path. Maybe that is why task can not read the correct hbase configuration.

node 1:namenode, secondarynamenode, jobtracker, hbase master
node 2:datanode, tasktracker

Hbase configurations in node1:
<configuration>
   <property>
      <name>hbase.cluster.distributed</name>
      <value>true</value>
   </property>
   <property>
      <name>hbase.hstore.compaction.min.size</name>
      <value>4194304</value>
   </property>
   <property>
      <name>dfs.support.append</name>
      <value>true</value>
   </property>
   <property>
      <name>hbase.rootdir</name>
      <value>hdfs://node1:9000/hbase</value>
   </property>
   <property>
      <name>hbase.zookeeper.quorum</name>
      <value>node1</value>
   </property>
   <property>
      <name>hbase.zookeeper.property.clientPort</name>
      <value>2181</value>
   </property>
   <property>
      <name>hbase.master.port</name>
      <value>60000</value>
   </property>
   <property>
      <name>hbase.master.info.port</name>
      <value>60010</value>
   </property>
   <property>
      <name>hbase.regionserver.port</name>
      <value>60020</value>
   </property>
   <property>
      <name>hbase.regionserver.info.port</name>
      <value>60030</value>
   </property>
</configuration>

At 2012-06-13 15:50:51,"Mohammad Tariq" <[EMAIL PROTECTED]> wrote:
>Could you send me your hadoop and hbase config files???
>
>Regards,
>    Mohammad Tariq
>
>
>On Wed, Jun 13, 2012 at 1:18 PM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:
>> "HBase is able to connect to ZooKeeper but the connection closes
>> immediately." - This error means that your HMaster is not able to talk
>> to your Namenode.
>>
>> Regards,
>>     Mohammad Tariq
>>
>>
>> On Wed, Jun 13, 2012 at 1:12 PM, lulynn_2008 <[EMAIL PROTECTED]> wrote:
>>> Hello,
>>> hadoop-core-*.jar and commons-configuration-1.6.jar have been in hbase lib directory. jobtracker node can get correct hbase configuration, but tasktracker node can not.
>>>
>>>
>>>
>>>
>>> At 2012-06-13 15:35:21,"Mohammad Tariq" <[EMAIL PROTECTED]> wrote:
>>>>Hello,
>>>>
>>>>   Copy the hadoop-core-*.jar from your hadoop folder to the hbase/lib
>>>>folder.Also copy commons-configuration-1.6.jar from hadoop/lib folder
>>>>to hbase/lib folder...Some times due to incompatible jars this may
>>>>happen..do it and see if it works for you.
>>>>
>>>>Regards,
>>>>    Mohammad Tariq
>>>>
>>>>
>>>>On Wed, Jun 13, 2012 at 12:34 PM, lulynn_2008 <[EMAIL PROTECTED]> wrote:
>>>>>  Hi everyone,
>>>>> Following is mine test environment:
>>>>> node 1:namenode, secondarynamenode, jobtracker, hbase master
>>>>> node 2:datanode, tasktracker
>>>>> In node 1, I run following COMMANDS in pig shell, but I found map task failed in tasktracker node with error "HBase is able to connect to ZooKeeper but the connection closes immediately.". This mean tasktracker did not get current hbase configuration. But I can find the correct hbase configuration in jobtracker node. Seems tasktracker node did not get configuration from jobtracker node, but get configuration from hadoop classpath in tasktracker node.
>>>>> I think tasktracker node should get hbase configuration from jobtracker node, but not from local hadoop classpath. Am I correct?
>>>>>
>>>>> In tasktracker side, after I add hbase-site.xml to hadoop classpath, the test case passed.
>>>>> My question is: how tasktracker node get hbase configuration from tasktracker side? From jobtracker side(included in *.jar file transferred by jobtracker node) or local hadoop classpath?
>>>>>
>>>>> COMMANDS:

You need to connect hbase to hadoop not hadoop to hbase....

On Wed, Jun 13, 2012 at 2:45 PM, lulynn_2008 <[EMAIL PROTECTED]> wrote:

> I found that there is no hbase configuration in tasktracker classpath.
>  After we add hbase conf directory into tasktracker hadoop classpath, the
> test case passed. But I think the hbase configuration should be passed by
> jobtracker node, and I can find correct hbase configuration in jobtracker
> node: job*.xml.
> Besides I can find the right hbase configuration in
> tasktrack:${mapred.local.dir}/taskTracker/jobcache/$jobid/job.xml, but this
> job.xml file is not in task java.class.path. Maybe that is why task can not
> read the correct hbase configuration.
>
> node 1:namenode, secondarynamenode, jobtracker, hbase master
> node 2:datanode, tasktracker
>
> Hbase configurations in node1:
> <configuration>
>   <property>
>      <name>hbase.cluster.distributed</name>
>      <value>true</value>
>   </property>
>   <property>
>      <name>hbase.hstore.compaction.min.size</name>
>      <value>4194304</value>
>   </property>
>   <property>
>      <name>dfs.support.append</name>
>      <value>true</value>
>   </property>
>   <property>
>      <name>hbase.rootdir</name>
>      <value>hdfs://node1:9000/hbase</value>
>   </property>
>   <property>
>      <name>hbase.zookeeper.quorum</name>
>      <value>node1</value>
>   </property>
>   <property>
>      <name>hbase.zookeeper.property.clientPort</name>
>      <value>2181</value>
>   </property>
>   <property>
>      <name>hbase.master.port</name>
>      <value>60000</value>
>   </property>
>   <property>
>      <name>hbase.master.info.port</name>
>      <value>60010</value>
>   </property>
>   <property>
>      <name>hbase.regionserver.port</name>
>      <value>60020</value>
>   </property>
>   <property>
>      <name>hbase.regionserver.info.port</name>
>      <value>60030</value>
>   </property>
> </configuration>
>
>
>
>
>
> At 2012-06-13 15:50:51,"Mohammad Tariq" <[EMAIL PROTECTED]> wrote:
> >Could you send me your hadoop and hbase config files???
> >
> >Regards,
> >    Mohammad Tariq
> >
> >
> >On Wed, Jun 13, 2012 at 1:18 PM, Mohammad Tariq <[EMAIL PROTECTED]>
> wrote:
> >> "HBase is able to connect to ZooKeeper but the connection closes
> >> immediately." - This error means that your HMaster is not able to talk
> >> to your Namenode.
> >>
> >> Regards,
> >>     Mohammad Tariq
> >>
> >>
> >> On Wed, Jun 13, 2012 at 1:12 PM, lulynn_2008 <[EMAIL PROTECTED]>
> wrote:
> >>> Hello,
> >>> hadoop-core-*.jar and commons-configuration-1.6.jar have been in hbase
> lib directory. jobtracker node can get correct hbase configuration, but
> tasktracker node can not.
> >>>
> >>>
> >>>
> >>>
> >>> At 2012-06-13 15:35:21,"Mohammad Tariq" <[EMAIL PROTECTED]> wrote:
> >>>>Hello,
> >>>>
> >>>>   Copy the hadoop-core-*.jar from your hadoop folder to the hbase/lib
> >>>>folder.Also copy commons-configuration-1.6.jar from hadoop/lib folder
> >>>>to hbase/lib folder...Some times due to incompatible jars this may
> >>>>happen..do it and see if it works for you.
> >>>>
> >>>>Regards,
> >>>>    Mohammad Tariq
> >>>>
> >>>>
> >>>>On Wed, Jun 13, 2012 at 12:34 PM, lulynn_2008 <[EMAIL PROTECTED]>
> wrote:
> >>>>>  Hi everyone,
> >>>>> Following is mine test environment:
> >>>>> node 1:namenode, secondarynamenode, jobtracker, hbase master
> >>>>> node 2:datanode, tasktracker
> >>>>> In node 1, I run following COMMANDS in pig shell, but I found map
> task failed in tasktracker node with error "HBase is able to connect to
> ZooKeeper but the connection closes immediately.". This mean tasktracker
> did not get current hbase configuration. But I can find the correct hbase
> configuration in jobtracker node. Seems tasktracker node did not get
> configuration from jobtracker node, but get configuration from hadoop
> classpath in tasktracker node.
> >>>>> I think tasktracker node should get hbase configuration from
> jobtracker node, but not from local hadoop classpath. Am I correct?

∞
Shashwat Shriparv