Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - [pig-0.10.0 failed to load hbase table]Re:Re: Re: Re: pig-0.10.0: TestJobSubmission failed with: Internal error creating job configuration.


+
lulynn_2008 2012-09-12, 07:39
Copy link to this message
-
Re: [pig-0.10.0 failed to load hbase table]Re:Re: Re: Re: pig-0.10.0: TestJobSubmission failed with: Internal error creating job configuration.
Cheolsoo Park 2012-09-12, 18:24
Hi,

2012-09-12 00:30:07,198 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server /127.0.0.1:2181
This message seems wrong. I assume that you're running the ZK quorum on
remote machines, but it is trying to connect to localhost. Can you try to
set "hbase.zookeeper.quorum" in "pig.properties" as follows:
"hbase.zookeeper.quorum=<your
ZK quorum host:port>" ?

Thanks,
Cheolsoo

On Wed, Sep 12, 2012 at 12:39 AM, lulynn_2008 <[EMAIL PROTECTED]> wrote:

> Hi Cheolsoo,
> Current TestJobSubmission and TestHBaseStorage passed. But if I run
> following scripts with hbase-0.94/zookeeper-3.4.3 cluster:
>
> the same issue happened.
>
> 2012-09-12 00:30:07,198 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server /127.0.0.1:2181
> 2012-09-12 00:30:07,199 INFO
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier of
> this process is [EMAIL PROTECTED]
> 2012-09-12 00:30:07,212 WARN
> org.apache.zookeeper.client.ZooKeeperSaslClient: SecurityException:
> java.lang.SecurityException: Unable to locate a login configuration
> occurred when trying to find JAAS configuration.
> 2012-09-12 00:30:07,213 INFO
> org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 2012-09-12 00:30:07,221 WARN org.apache.zookeeper.ClientCnxn: Session 0x0
> for server null, unexpected error, closing socket connection and attempting
> reconnect
> java.net.ConnectException: Connection refused
>         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>         at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:610)
>         at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
>         at
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
>
> Reproduce steps:
>
> Create hbase table:
> ./hbase shell:
> create 'employees', 'SN', 'department', 'address'
> put 'employees', 'Hong', 'address:country', 'China'
>
> Run following pig commands:
> ./pig
> A = load 'hbase://employees' using
> org.apache.pig.backend.hadoop.hbase.HBaseStorage( 'address:country',
> '-loadKey true') as (SN:bytearray,country:bytearray);
> B = filter A by SN == 'Hong';
> dump B;
>
>
>
>
>
> At 2012-08-22 05:51:38,"Cheolsoo Park" <[EMAIL PROTECTED]> wrote:
> >OK, I got TestJobSubmission passing. Please apply the following diff to
> >your Pig:
> >
> >
> >diff --git test/org/apache/pig/test/TestJobSubmission.java
> >test/org/apache/pig/test/TestJobSubmission.java
> >index 89ac132..211c4ae 100644
> >--- test/org/apache/pig/test/TestJobSubmission.java
> >+++ test/org/apache/pig/test/TestJobSubmission.java
> >@@ -26,8 +26,10 @@ import java.util.Random;
> >
> > import org.apache.hadoop.conf.Configuration;
> > import org.apache.hadoop.hbase.HBaseTestingUtility;
> >*+import org.apache.hadoop.hbase.HConstants;*
> > import org.apache.hadoop.hbase.MiniHBaseCluster;
> > import org.apache.hadoop.hbase.util.Bytes;
> >*+import org.apache.hadoop.hbase.zookeeper.MiniZooKeeperCluster;*
> > import org.apache.hadoop.mapred.jobcontrol.Job;
> > import org.apache.hadoop.mapred.jobcontrol.JobControl;
> > import org.apache.pig.ExecType;
> >@@ -541,7 +543,7 @@ public class TestJobSubmission {
> >         // use the estimation
> >         Configuration conf = cluster.getConfiguration();
> >         HBaseTestingUtility util = new HBaseTestingUtility(conf);
> >*-        util.startMiniZKCluster();*
> >*+        int clientPort = util.startMiniZKCluster().getClientPort();*
> >         util.startMiniHBaseCluster(1, 1);
> >
> >         String query = "a = load '/passwd';" +
> >@@ -553,6 +555,7 @@ public class TestJobSubmission {
> >
> >         pc.getConf().setProperty("pig.exec.reducers.bytes.per.reducer",
> >"100");
> >         pc.getConf().setProperty("pig.exec.reducers.max", "10");
+
lulynn_2008 2012-09-13, 08:10
+
Cheolsoo Park 2012-09-13, 09:24