|
|
-
pig-0.10.0: TestJobSubmission failed with: Internal error creating job configuration.
lulynn_2008 2012-08-15, 07:56
Hi All,
In TestJobSubmission, testReducerNumEstimation failed with following error information. Please give a glance to check what the problem is. Thanks.
Error information:
Testcase: testReducerNumEstimation took 1,809.419 sec
Caused an ERROR
Internal error creating job configuration.
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException: ERROR 2017: Internal error creating job configuration.
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:731)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:259)
at org.apache.pig.test.TestJobSubmission.testReducerNumEstimation(TestJobSubmission.java:491)
Caused by: org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find region for passwd,,99999999999999 after 10 tries.
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:926)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:832)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:933)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:836)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:234)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:174)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:133)
at org.apache.pig.backend.hadoop.hbase.HBaseStorage.setLocation(HBaseStorage.java:545)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:384)
+
lulynn_2008 2012-08-15, 07:56
-
Re: pig-0.10.0: TestJobSubmission failed with: Internal error creating job configuration.
Cheolsoo Park 2012-08-15, 17:30
Hi,
Caused by: org.apache.hadoop.hbase.client.NoServerForRegionException:
> Unable to find region for passwd,,99999999999999 after 10 tries.
It looks like the region server is not available.
1) Does TestHBaseStorage pass? I guess that it fails with the same error.
2) In your log, do you see any errors related to the HBase region server?
For example, java.lang.IllegalStateException: Could not instantiate a
region instance.
If the region server failed to start, you will see this failure.
Thanks,
Cheolsoo
On Wed, Aug 15, 2012 at 12:56 AM, lulynn_2008 <[EMAIL PROTECTED]> wrote:
> Hi All,
>
> In TestJobSubmission, testReducerNumEstimation failed with following error
> information. Please give a glance to check what the problem is. Thanks.
>
> Error information:
> Testcase: testReducerNumEstimation took 1,809.419 sec
> Caused an ERROR
> Internal error creating job configuration.
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException:
> ERROR 2017: Internal error creating job configuration.
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:731)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:259)
> at
> org.apache.pig.test.TestJobSubmission.testReducerNumEstimation(TestJobSubmission.java:491)
> Caused by: org.apache.hadoop.hbase.client.NoServerForRegionException:
> Unable to find region for passwd,,99999999999999 after 10 tries.
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:926)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:832)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:933)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:836)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
> at
> org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:234)
> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:174)
> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:133)
> at
> org.apache.pig.backend.hadoop.hbase.HBaseStorage.setLocation(HBaseStorage.java:545)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:384)
>
>
+
Cheolsoo Park 2012-08-15, 17:30
-
Re:Re: pig-0.10.0: TestJobSubmission failed with: Internal error creating job configuration.
lulynn_2008 2012-08-17, 06:02
BTW, I am using hadoop-1.0.3, hbase-0.94.0 and zookeeper-3.4.3.
Hi Cheolsoo,
You are right. TestHBaseStorage also failed with the same error.
I got following region server information, and seems hbase region has been started successfully. But hbase can not connect to zookeeper
12/08/13 06:34:12 INFO server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x1391cf90aed0000 type:delete cxid:0x24 zxid:0x10 txntype:-1 reqpath:n/a Error Path:/hbase/root-region-server Error:KeeperErrorCode = NoNode for /hbase/root-region-server
12/08/13 06:34:16 INFO regionserver.HRegion: creating HRegion pigtable_1 HTD == {NAME => 'pigtable_1', FAMILIES => [{NAME => 'pig', REPLICATION_SCOPE => '0', KEEP_DELETED_CELLS => 'false', COMPRESSION => 'NONE', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true', MIN_VERSIONS => '0', DATA_BLOCK_ENCODING => 'NONE', IN_MEMORY => 'false', BLOOMFILTER => 'NONE', TTL => '2147483647', VERSIONS => '3', BLOCKSIZE => '65536'}]} RootDir = hdfs://shihc015:54670/user/root/hbase Table name == pigtable_1
12/08/13 06:34:43 INFO zookeeper.ClientCnxn: Opening socket connection to server shihc015/127.0.0.1:21818
12/08/13 06:34:43 WARN client.ZooKeeperSaslClient: SecurityException: java.lang.SecurityException: Unable to locate a login configuration occurred when trying to find JAAS configuration.
12/08/13 06:34:43 INFO client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
12/08/13 06:34:43 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:610)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
12/08/13 06:34:43 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/root-region-server
12/08/16 22:33:05 ERROR zookeeper.ZooKeeperWatcher: hconnection Received unexpected KeeperException, re-throwing exception
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/root-region-server
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1131)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(RecoverableZooKeeper.java:264)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataInternal(ZKUtil.java:522)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:498)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.getData(ZooKeeperNodeTracker.java:156)
at org.apache.hadoop.hbase.zookeeper.RootRegionTracker.getRootRegionLocation(RootRegionTracker.java:62)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:821)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:933)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:832)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:933)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:836)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:234)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:174)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:133)
at org.apache.pig.backend.hadoop.hbase.HBaseStorage.setLocation(HBaseStorage.java:551)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:385)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:259)
at org.apache.pig.test.TestJobSubmission.testReducerNumEstimation(TestJobSubmission.java:492)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at org.junit.runners.BlockJUnit4ClassRunner.runNotIg
+
lulynn_2008 2012-08-17, 06:02
-
Re: Re: pig-0.10.0: TestJobSubmission failed with: Internal error creating job configuration.
Cheolsoo Park 2012-08-18, 06:26
Hi,
I tried the same version of libraries as you're using and was able to get
TestHBaseStorage passing in trunk. Here is what I did:
1) Updated the version of hadoop, hbase, and zookeeper in
ivy/libraries.properties:
hadoop-core.version=1.0.3
> hadoop-test.version=1.0.3
> hbase.version=0.94.0
> zookeeper.version=3.4.3
2) Updated TestHBaseStorage.java since it doesn't compile with hbase-0.94.0:
HTable table = new HTable(tableName); => HTable table = new HTable(conf,
> tableName);
There are 6 places to fix.
3) Added the following lines to ivy.xml:
<dependency org="com.github.stephenc.high-scale-lib" name="high-scale-lib"
> rev="${high-scale-lib.version}" conf="test->default"/>
> <dependency org="com.google.protobuf" name="protobuf-java"
> rev="${protobuf-java.version}" conf="test->default"/>
I also added the following line to ivy/libraries.properties:
high-scale-lib.version=1.1.1
In particular, I was able to reproduce your error without this line in
ivy.xml:
<dependency org="com.google.protobuf" name="protobuf-java"
> rev="${protobuf-java.version}" conf="test->default"/>
With all these changes, TestHBaseStorage passes for me. Please let me know
if this works for you.
Thanks,
Cheolsoo
On Thu, Aug 16, 2012 at 11:02 PM, lulynn_2008 <[EMAIL PROTECTED]> wrote:
> BTW, I am using hadoop-1.0.3, hbase-0.94.0 and zookeeper-3.4.3.
>
>
> Hi Cheolsoo,
> You are right. TestHBaseStorage also failed with the same error.
>
> I got following region server information, and seems hbase region has been
> started successfully. But hbase can not connect to zookeeper
>
> 12/08/13 06:34:12 INFO server.PrepRequestProcessor: Got user-level
> KeeperException when processing sessionid:0x1391cf90aed0000 type:delete
> cxid:0x24 zxid:0x10 txntype:-1 reqpath:n/a Error
> Path:/hbase/root-region-server Error:KeeperErrorCode = NoNode for
> /hbase/root-region-server
>
> 12/08/13 06:34:16 INFO regionserver.HRegion: creating HRegion pigtable_1
> HTD == {NAME => 'pigtable_1', FAMILIES => [{NAME => 'pig',
> REPLICATION_SCOPE => '0', KEEP_DELETED_CELLS => 'false', COMPRESSION =>
> 'NONE', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true', MIN_VERSIONS =>
> '0', DATA_BLOCK_ENCODING => 'NONE', IN_MEMORY => 'false', BLOOMFILTER =>
> 'NONE', TTL => '2147483647', VERSIONS => '3', BLOCKSIZE => '65536'}]}
> RootDir = hdfs://shihc015:54670/user/root/hbase Table name == pigtable_1
>
> 12/08/13 06:34:43 INFO zookeeper.ClientCnxn: Opening socket connection to
> server shihc015/127.0.0.1:21818
> 12/08/13 06:34:43 WARN client.ZooKeeperSaslClient: SecurityException:
> java.lang.SecurityException: Unable to locate a login configuration
> occurred when trying to find JAAS configuration.
> 12/08/13 06:34:43 INFO client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section 'Client'
> could not be found. If you are not using SASL, you may ignore this. On the
> other hand, if you expected SASL to work, please fix your JAAS
> configuration.
> 12/08/13 06:34:43 WARN zookeeper.ClientCnxn: Session 0x0 for server null,
> unexpected error, closing socket connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:610)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
> 12/08/13 06:34:43 WARN zookeeper.RecoverableZooKeeper: Possibly transient
> ZooKeeper exception:
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/root-region-server
>
>
> 12/08/16 22:33:05 ERROR zookeeper.ZooKeeperWatcher: hconnection Received
> unexpected KeeperException, re-throwing exception
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/root-region-server
+
Cheolsoo Park 2012-08-18, 06:26
-
Re:Re: Re: pig-0.10.0: TestJobSubmission failed with: Internal error creating job configuration.
lulynn_2008 2012-08-20, 03:01
Hi Cheolsoo,
TestHBaseStorage passed now with your fix. Thanks.
But TestJobSubmission still failed with the same previous error with TestHBaseStorage. Please help to check. Thanks
At 2012-08-18 14:26:32,"Cheolsoo Park" <[EMAIL PROTECTED]> wrote:
>Hi,
>
>I tried the same version of libraries as you're using and was able to get
>TestHBaseStorage passing in trunk. Here is what I did:
>
>1) Updated the version of hadoop, hbase, and zookeeper in
>ivy/libraries.properties:
>
>hadoop-core.version=1.0.3
>> hadoop-test.version=1.0.3
>> hbase.version=0.94.0
>> zookeeper.version=3.4.3
>
>
>2) Updated TestHBaseStorage.java since it doesn't compile with hbase-0.94.0:
>
>HTable table = new HTable(tableName); => HTable table = new HTable(conf,
>> tableName);
>
>
>There are 6 places to fix.
>
>3) Added the following lines to ivy.xml:
>
><dependency org="com.github.stephenc.high-scale-lib" name="high-scale-lib"
>> rev="${high-scale-lib.version}" conf="test->default"/>
>> <dependency org="com.google.protobuf" name="protobuf-java"
>> rev="${protobuf-java.version}" conf="test->default"/>
>
>
>I also added the following line to ivy/libraries.properties:
>
>high-scale-lib.version=1.1.1
>
>
>In particular, I was able to reproduce your error without this line in
>ivy.xml:
>
><dependency org="com.google.protobuf" name="protobuf-java"
>> rev="${protobuf-java.version}" conf="test->default"/>
>
>
>With all these changes, TestHBaseStorage passes for me. Please let me know
>if this works for you.
>
>Thanks,
>Cheolsoo
>
>On Thu, Aug 16, 2012 at 11:02 PM, lulynn_2008 <[EMAIL PROTECTED]> wrote:
>
>> BTW, I am using hadoop-1.0.3, hbase-0.94.0 and zookeeper-3.4.3.
>>
>>
>> Hi Cheolsoo,
>> You are right. TestHBaseStorage also failed with the same error.
>>
>> I got following region server information, and seems hbase region has been
>> started successfully. But hbase can not connect to zookeeper
>>
>> 12/08/13 06:34:12 INFO server.PrepRequestProcessor: Got user-level
>> KeeperException when processing sessionid:0x1391cf90aed0000 type:delete
>> cxid:0x24 zxid:0x10 txntype:-1 reqpath:n/a Error
>> Path:/hbase/root-region-server Error:KeeperErrorCode = NoNode for
>> /hbase/root-region-server
>>
>> 12/08/13 06:34:16 INFO regionserver.HRegion: creating HRegion pigtable_1
>> HTD == {NAME => 'pigtable_1', FAMILIES => [{NAME => 'pig',
>> REPLICATION_SCOPE => '0', KEEP_DELETED_CELLS => 'false', COMPRESSION =>
>> 'NONE', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true', MIN_VERSIONS =>
>> '0', DATA_BLOCK_ENCODING => 'NONE', IN_MEMORY => 'false', BLOOMFILTER =>
>> 'NONE', TTL => '2147483647', VERSIONS => '3', BLOCKSIZE => '65536'}]}
>> RootDir = hdfs://shihc015:54670/user/root/hbase Table name == pigtable_1
>>
>> 12/08/13 06:34:43 INFO zookeeper.ClientCnxn: Opening socket connection to
>> server shihc015/127.0.0.1:21818
>> 12/08/13 06:34:43 WARN client.ZooKeeperSaslClient: SecurityException:
>> java.lang.SecurityException: Unable to locate a login configuration
>> occurred when trying to find JAAS configuration.
>> 12/08/13 06:34:43 INFO client.ZooKeeperSaslClient: Client will not
>> SASL-authenticate because the default JAAS configuration section 'Client'
>> could not be found. If you are not using SASL, you may ignore this. On the
>> other hand, if you expected SASL to work, please fix your JAAS
>> configuration.
>> 12/08/13 06:34:43 WARN zookeeper.ClientCnxn: Session 0x0 for server null,
>> unexpected error, closing socket connection and attempting reconnect
>> java.net.ConnectException: Connection refused
>> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>> at
>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:610)
>> at
>> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
>> 12/08/13 06:34:43 WARN zookeeper.RecoverableZooKeeper: Possibly transient
>> ZooKeeper exception:
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
+
lulynn_2008 2012-08-20, 03:01
-
Re:Re: Re: pig-0.10.0: TestJobSubmission failed with: Internal error creating job configuration.
lulynn_2008 2012-08-21, 07:24
Hi Cheolsoo,
Please help to check whether TestJobSunbmission passed in your environment. Thanks
Hi Cheolsoo,
TestHBaseStorage passed now with your fix. Thanks.
But TestJobSubmission still failed with the same previous error with TestHBaseStorage. Please help to check. Thanks
At 2012-08-18 14:26:32,"Cheolsoo Park" <[EMAIL PROTECTED]> wrote:
>Hi,
>
>I tried the same version of libraries as you're using and was able to get
>TestHBaseStorage passing in trunk. Here is what I did:
>
>1) Updated the version of hadoop, hbase, and zookeeper in
>ivy/libraries.properties:
>
>hadoop-core.version=1.0.3
>> hadoop-test.version=1.0.3
>> hbase.version=0.94.0
>> zookeeper.version=3.4.3
>
>
>2) Updated TestHBaseStorage.java since it doesn't compile with hbase-0.94.0:
>
>HTable table = new HTable(tableName); => HTable table = new HTable(conf,
>> tableName);
>
>
>There are 6 places to fix.
>
>3) Added the following lines to ivy.xml:
>
><dependency org="com.github.stephenc.high-scale-lib" name="high-scale-lib"
>> rev="${high-scale-lib.version}" conf="test->default"/>
>> <dependency org="com.google.protobuf" name="protobuf-java"
>> rev="${protobuf-java.version}" conf="test->default"/>
>
>
>I also added the following line to ivy/libraries.properties:
>
>high-scale-lib.version=1.1.1
>
>
>In particular, I was able to reproduce your error without this line in
>ivy.xml:
>
><dependency org="com.google.protobuf" name="protobuf-java"
>> rev="${protobuf-java.version}" conf="test->default"/>
>
>
>With all these changes, TestHBaseStorage passes for me. Please let me know
>if this works for you.
>
>Thanks,
>Cheolsoo
>
>On Thu, Aug 16, 2012 at 11:02 PM, lulynn_2008 <[EMAIL PROTECTED]> wrote:
>
>> BTW, I am using hadoop-1.0.3, hbase-0.94.0 and zookeeper-3.4.3.
>>
>>
>> Hi Cheolsoo,
>> You are right. TestHBaseStorage also failed with the same error.
>>
>> I got following region server information, and seems hbase region has been
>> started successfully. But hbase can not connect to zookeeper
>>
>> 12/08/13 06:34:12 INFO server.PrepRequestProcessor: Got user-level
>> KeeperException when processing sessionid:0x1391cf90aed0000 type:delete
>> cxid:0x24 zxid:0x10 txntype:-1 reqpath:n/a Error
>> Path:/hbase/root-region-server Error:KeeperErrorCode = NoNode for
>> /hbase/root-region-server
>>
>> 12/08/13 06:34:16 INFO regionserver.HRegion: creating HRegion pigtable_1
>> HTD == {NAME => 'pigtable_1', FAMILIES => [{NAME => 'pig',
>> REPLICATION_SCOPE => '0', KEEP_DELETED_CELLS => 'false', COMPRESSION =>
>> 'NONE', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true', MIN_VERSIONS =>
>> '0', DATA_BLOCK_ENCODING => 'NONE', IN_MEMORY => 'false', BLOOMFILTER =>
>> 'NONE', TTL => '2147483647', VERSIONS => '3', BLOCKSIZE => '65536'}]}
>> RootDir = hdfs://shihc015:54670/user/root/hbase Table name == pigtable_1
>>
>> 12/08/13 06:34:43 INFO zookeeper.ClientCnxn: Opening socket connection to
>> server shihc015/127.0.0.1:21818
>> 12/08/13 06:34:43 WARN client.ZooKeeperSaslClient: SecurityException:
>> java.lang.SecurityException: Unable to locate a login configuration
>> occurred when trying to find JAAS configuration.
>> 12/08/13 06:34:43 INFO client.ZooKeeperSaslClient: Client will not
>> SASL-authenticate because the default JAAS configuration section 'Client'
>> could not be found. If you are not using SASL, you may ignore this. On the
>> other hand, if you expected SASL to work, please fix your JAAS
>> configuration.
>> 12/08/13 06:34:43 WARN zookeeper.ClientCnxn: Session 0x0 for server null,
>> unexpected error, closing socket connection and attempting reconnect
>> java.net.ConnectException: Connection refused
>> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>> at
>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:610)
>> at
>> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
>> 12/08/13 06:34:43 WARN zookeeper.RecoverableZooKeeper: Possibly transient
+
lulynn_2008 2012-08-21, 07:24
-
Re: Re: Re: pig-0.10.0: TestJobSubmission failed with: Internal error creating job configuration.
Cheolsoo Park 2012-08-21, 08:25
Hi,
TestJobSubmission doesn't pass for me either. But it doesn't throw the
error that you're seeing. Instead, it hangs at line 583
in TestJobSubmission.java. I am not sure yet what's happening though.
Thanks,
Cheolsoo
On Tue, Aug 21, 2012 at 12:24 AM, lulynn_2008 <[EMAIL PROTECTED]> wrote:
> Hi Cheolsoo,
> Please help to check whether TestJobSunbmission passed in your
> environment. Thanks
>
>
> Hi Cheolsoo,
> TestHBaseStorage passed now with your fix. Thanks.
> But TestJobSubmission still failed with the same previous error with
> TestHBaseStorage. Please help to check. Thanks
>
>
>
>
>
> At 2012-08-18 14:26:32,"Cheolsoo Park" <[EMAIL PROTECTED]> wrote:
> >Hi,
> >
> >I tried the same version of libraries as you're using and was able to get
> >TestHBaseStorage passing in trunk. Here is what I did:
> >
> >1) Updated the version of hadoop, hbase, and zookeeper in
> >ivy/libraries.properties:
> >
> >hadoop-core.version=1.0.3
> >> hadoop-test.version=1.0.3
> >> hbase.version=0.94.0
> >> zookeeper.version=3.4.3
> >
> >
> >2) Updated TestHBaseStorage.java since it doesn't compile with
> hbase-0.94.0:
> >
> >HTable table = new HTable(tableName); => HTable table = new HTable(conf,
> >> tableName);
> >
> >
> >There are 6 places to fix.
> >
> >3) Added the following lines to ivy.xml:
> >
> ><dependency org="com.github.stephenc.high-scale-lib" name="high-scale-lib"
> >> rev="${high-scale-lib.version}" conf="test->default"/>
> >> <dependency org="com.google.protobuf" name="protobuf-java"
> >> rev="${protobuf-java.version}" conf="test->default"/>
> >
> >
> >I also added the following line to ivy/libraries.properties:
> >
> >high-scale-lib.version=1.1.1
> >
> >
> >In particular, I was able to reproduce your error without this line in
> >ivy.xml:
> >
> ><dependency org="com.google.protobuf" name="protobuf-java"
> >> rev="${protobuf-java.version}" conf="test->default"/>
> >
> >
> >With all these changes, TestHBaseStorage passes for me. Please let me know
> >if this works for you.
> >
> >Thanks,
> >Cheolsoo
> >
> >On Thu, Aug 16, 2012 at 11:02 PM, lulynn_2008 <[EMAIL PROTECTED]>
> wrote:
> >
> >> BTW, I am using hadoop-1.0.3, hbase-0.94.0 and zookeeper-3.4.3.
> >>
> >>
> >> Hi Cheolsoo,
> >> You are right. TestHBaseStorage also failed with the same error.
> >>
> >> I got following region server information, and seems hbase region has
> been
> >> started successfully. But hbase can not connect to zookeeper
> >>
> >> 12/08/13 06:34:12 INFO server.PrepRequestProcessor: Got user-level
> >> KeeperException when processing sessionid:0x1391cf90aed0000 type:delete
> >> cxid:0x24 zxid:0x10 txntype:-1 reqpath:n/a Error
> >> Path:/hbase/root-region-server Error:KeeperErrorCode = NoNode for
> >> /hbase/root-region-server
> >>
> >> 12/08/13 06:34:16 INFO regionserver.HRegion: creating HRegion pigtable_1
> >> HTD == {NAME => 'pigtable_1', FAMILIES => [{NAME => 'pig',
> >> REPLICATION_SCOPE => '0', KEEP_DELETED_CELLS => 'false', COMPRESSION =>
> >> 'NONE', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true', MIN_VERSIONS =>
> >> '0', DATA_BLOCK_ENCODING => 'NONE', IN_MEMORY => 'false', BLOOMFILTER =>
> >> 'NONE', TTL => '2147483647', VERSIONS => '3', BLOCKSIZE => '65536'}]}
> >> RootDir = hdfs://shihc015:54670/user/root/hbase Table name == pigtable_1
> >>
> >> 12/08/13 06:34:43 INFO zookeeper.ClientCnxn: Opening socket connection
> to
> >> server shihc015/127.0.0.1:21818
> >> 12/08/13 06:34:43 WARN client.ZooKeeperSaslClient: SecurityException:
> >> java.lang.SecurityException: Unable to locate a login configuration
> >> occurred when trying to find JAAS configuration.
> >> 12/08/13 06:34:43 INFO client.ZooKeeperSaslClient: Client will not
> >> SASL-authenticate because the default JAAS configuration section
> 'Client'
> >> could not be found. If you are not using SASL, you may ignore this. On
> the
> >> other hand, if you expected SASL to work, please fix your JAAS
> >> configuration.
> >> 12/08/13 06:34:43 WARN zookeeper.ClientCnxn: Session 0x0 for server
+
Cheolsoo Park 2012-08-21, 08:25
-
Re: Re: Re: pig-0.10.0: TestJobSubmission failed with: Internal error creating job configuration.
Cheolsoo Park 2012-08-21, 21:51
OK, I got TestJobSubmission passing. Please apply the following diff to
your Pig:
diff --git test/org/apache/pig/test/TestJobSubmission.java
test/org/apache/pig/test/TestJobSubmission.java
index 89ac132..211c4ae 100644
--- test/org/apache/pig/test/TestJobSubmission.java
+++ test/org/apache/pig/test/TestJobSubmission.java
@@ -26,8 +26,10 @@ import java.util.Random;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseTestingUtility;
*+import org.apache.hadoop.hbase.HConstants;*
import org.apache.hadoop.hbase.MiniHBaseCluster;
import org.apache.hadoop.hbase.util.Bytes;
*+import org.apache.hadoop.hbase.zookeeper.MiniZooKeeperCluster;*
import org.apache.hadoop.mapred.jobcontrol.Job;
import org.apache.hadoop.mapred.jobcontrol.JobControl;
import org.apache.pig.ExecType;
@@ -541,7 +543,7 @@ public class TestJobSubmission {
// use the estimation
Configuration conf = cluster.getConfiguration();
HBaseTestingUtility util = new HBaseTestingUtility(conf);
*- util.startMiniZKCluster();*
*+ int clientPort = util.startMiniZKCluster().getClientPort();*
util.startMiniHBaseCluster(1, 1);
String query = "a = load '/passwd';" +
@@ -553,6 +555,7 @@ public class TestJobSubmission {
pc.getConf().setProperty("pig.exec.reducers.bytes.per.reducer",
"100");
pc.getConf().setProperty("pig.exec.reducers.max", "10");
*+ pc.getConf().setProperty(HConstants.ZOOKEEPER_CLIENT_PORT,
Integer.toString(clientPort));*
ConfigurationValidator.validatePigProperties(pc.getProperties());
conf = ConfigurationUtil.toConfiguration(pc.getProperties());
JobControlCompiler jcc = new JobControlCompiler(pc, conf);
There are several changes to MiniZooKeeperCluster in HBase 0.94 compared to
0.90. In particular, the ZK client port is no longer 21818 by default. In
fact, it is randomized in 0.94. So the client port has to be explicitly set
in PigContext at runtime.
Now I am able to run both TestJobSubmission and TestHBaseStorage. Please
let me know if this works for you. I am going to file a jira to capture
what I found, so we can make use of it when bumping the Hbase version in
the future.
Thanks,
Cheolsoo
On Tue, Aug 21, 2012 at 1:25 AM, Cheolsoo Park <[EMAIL PROTECTED]>wrote:
> Hi,
>
> TestJobSubmission doesn't pass for me either. But it doesn't throw the
> error that you're seeing. Instead, it hangs at line 583
> in TestJobSubmission.java. I am not sure yet what's happening though.
>
> Thanks,
> Cheolsoo
>
> On Tue, Aug 21, 2012 at 12:24 AM, lulynn_2008 <[EMAIL PROTECTED]> wrote:
>
>> Hi Cheolsoo,
>> Please help to check whether TestJobSunbmission passed in your
>> environment. Thanks
>>
>>
>> Hi Cheolsoo,
>> TestHBaseStorage passed now with your fix. Thanks.
>> But TestJobSubmission still failed with the same previous error with
>> TestHBaseStorage. Please help to check. Thanks
>>
>>
>>
>>
>>
>> At 2012-08-18 14:26:32,"Cheolsoo Park" <[EMAIL PROTECTED]> wrote:
>> >Hi,
>> >
>> >I tried the same version of libraries as you're using and was able to get
>> >TestHBaseStorage passing in trunk. Here is what I did:
>> >
>> >1) Updated the version of hadoop, hbase, and zookeeper in
>> >ivy/libraries.properties:
>> >
>> >hadoop-core.version=1.0.3
>> >> hadoop-test.version=1.0.3
>> >> hbase.version=0.94.0
>> >> zookeeper.version=3.4.3
>> >
>> >
>> >2) Updated TestHBaseStorage.java since it doesn't compile with
>> hbase-0.94.0:
>> >
>> >HTable table = new HTable(tableName); => HTable table = new HTable(conf,
>> >> tableName);
>> >
>> >
>> >There are 6 places to fix.
>> >
>> >3) Added the following lines to ivy.xml:
>> >
>> ><dependency org="com.github.stephenc.high-scale-lib"
>> name="high-scale-lib"
>> >> rev="${high-scale-lib.version}" conf="test->default"/>
>> >> <dependency org="com.google.protobuf" name="protobuf-java"
>> >> rev="${protobuf-java.version}" conf="test->default"/>
>> >
>> >
>> >I also added the following line to ivy/libraries.properties:
+
Cheolsoo Park 2012-08-21, 21:51
-
Re:Re: Re: Re: pig-0.10.0: TestJobSubmission failed with: Internal error creating job configuration.
lulynn_2008 2012-08-22, 02:55
Both UT passed now:-)
Thanks Cheolsoo.
At 2012-08-22 05:51:38,"Cheolsoo Park" <[EMAIL PROTECTED]> wrote:
>OK, I got TestJobSubmission passing. Please apply the following diff to
>your Pig:
>
>
>diff --git test/org/apache/pig/test/TestJobSubmission.java
>test/org/apache/pig/test/TestJobSubmission.java
>index 89ac132..211c4ae 100644
>--- test/org/apache/pig/test/TestJobSubmission.java
>+++ test/org/apache/pig/test/TestJobSubmission.java
>@@ -26,8 +26,10 @@ import java.util.Random;
>
> import org.apache.hadoop.conf.Configuration;
> import org.apache.hadoop.hbase.HBaseTestingUtility;
>*+import org.apache.hadoop.hbase.HConstants;*
> import org.apache.hadoop.hbase.MiniHBaseCluster;
> import org.apache.hadoop.hbase.util.Bytes;
>*+import org.apache.hadoop.hbase.zookeeper.MiniZooKeeperCluster;*
> import org.apache.hadoop.mapred.jobcontrol.Job;
> import org.apache.hadoop.mapred.jobcontrol.JobControl;
> import org.apache.pig.ExecType;
>@@ -541,7 +543,7 @@ public class TestJobSubmission {
> // use the estimation
> Configuration conf = cluster.getConfiguration();
> HBaseTestingUtility util = new HBaseTestingUtility(conf);
>*- util.startMiniZKCluster();*
>*+ int clientPort = util.startMiniZKCluster().getClientPort();*
> util.startMiniHBaseCluster(1, 1);
>
> String query = "a = load '/passwd';" +
>@@ -553,6 +555,7 @@ public class TestJobSubmission {
>
> pc.getConf().setProperty("pig.exec.reducers.bytes.per.reducer",
>"100");
> pc.getConf().setProperty("pig.exec.reducers.max", "10");
>*+ pc.getConf().setProperty(HConstants.ZOOKEEPER_CLIENT_PORT,
>Integer.toString(clientPort));*
> ConfigurationValidator.validatePigProperties(pc.getProperties());
> conf = ConfigurationUtil.toConfiguration(pc.getProperties());
> JobControlCompiler jcc = new JobControlCompiler(pc, conf);
>
>
>There are several changes to MiniZooKeeperCluster in HBase 0.94 compared to
>0.90. In particular, the ZK client port is no longer 21818 by default. In
>fact, it is randomized in 0.94. So the client port has to be explicitly set
>in PigContext at runtime.
>
>Now I am able to run both TestJobSubmission and TestHBaseStorage. Please
>let me know if this works for you. I am going to file a jira to capture
>what I found, so we can make use of it when bumping the Hbase version in
>the future.
>
>Thanks,
>Cheolsoo
>
>
>On Tue, Aug 21, 2012 at 1:25 AM, Cheolsoo Park <[EMAIL PROTECTED]>wrote:
>
>> Hi,
>>
>> TestJobSubmission doesn't pass for me either. But it doesn't throw the
>> error that you're seeing. Instead, it hangs at line 583
>> in TestJobSubmission.java. I am not sure yet what's happening though.
>>
>> Thanks,
>> Cheolsoo
>>
>> On Tue, Aug 21, 2012 at 12:24 AM, lulynn_2008 <[EMAIL PROTECTED]> wrote:
>>
>>> Hi Cheolsoo,
>>> Please help to check whether TestJobSunbmission passed in your
>>> environment. Thanks
>>>
>>>
>>> Hi Cheolsoo,
>>> TestHBaseStorage passed now with your fix. Thanks.
>>> But TestJobSubmission still failed with the same previous error with
>>> TestHBaseStorage. Please help to check. Thanks
>>>
>>>
>>>
>>>
>>>
>>> At 2012-08-18 14:26:32,"Cheolsoo Park" <[EMAIL PROTECTED]> wrote:
>>> >Hi,
>>> >
>>> >I tried the same version of libraries as you're using and was able to get
>>> >TestHBaseStorage passing in trunk. Here is what I did:
>>> >
>>> >1) Updated the version of hadoop, hbase, and zookeeper in
>>> >ivy/libraries.properties:
>>> >
>>> >hadoop-core.version=1.0.3
>>> >> hadoop-test.version=1.0.3
>>> >> hbase.version=0.94.0
>>> >> zookeeper.version=3.4.3
>>> >
>>> >
>>> >2) Updated TestHBaseStorage.java since it doesn't compile with
>>> hbase-0.94.0:
>>> >
>>> >HTable table = new HTable(tableName); => HTable table = new HTable(conf,
>>> >> tableName);
>>> >
>>> >
>>> >There are 6 places to fix.
>>> >
>>> >3) Added the following lines to ivy.xml:
>>> >
>>> ><dependency org="com.github.stephenc.high-scale-lib"
>>> name="high-scale-lib"
>>> >> rev="${high-scale-lib.version}" conf="test->default"/>
+
lulynn_2008 2012-08-22, 02:55
-
Re:Re: pig-0.10.0: TestJobSubmission failed with: Internal error creating job configuration.
lulynn_2008 2012-08-17, 05:59
Hi Cheolsoo,
You are right. TestHBaseStorage also failed with the same error.
I got following region server information, and seems hbase region has been started successfully. But hbase can not connect to zookeeper
12/08/13 06:34:12 INFO server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x1391cf90aed0000 type:delete cxid:0x24 zxid:0x10 txntype:-1 reqpath:n/a Error Path:/hbase/root-region-server Error:KeeperErrorCode = NoNode for /hbase/root-region-server
12/08/13 06:34:16 INFO regionserver.HRegion: creating HRegion pigtable_1 HTD == {NAME => 'pigtable_1', FAMILIES => [{NAME => 'pig', REPLICATION_SCOPE => '0', KEEP_DELETED_CELLS => 'false', COMPRESSION => 'NONE', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true', MIN_VERSIONS => '0', DATA_BLOCK_ENCODING => 'NONE', IN_MEMORY => 'false', BLOOMFILTER => 'NONE', TTL => '2147483647', VERSIONS => '3', BLOCKSIZE => '65536'}]} RootDir = hdfs://shihc015:54670/user/root/hbase Table name == pigtable_1
12/08/13 06:34:43 INFO zookeeper.ClientCnxn: Opening socket connection to server shihc015/127.0.0.1:21818
12/08/13 06:34:43 WARN client.ZooKeeperSaslClient: SecurityException: java.lang.SecurityException: Unable to locate a login configuration occurred when trying to find JAAS configuration.
12/08/13 06:34:43 INFO client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
12/08/13 06:34:43 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:610)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
12/08/13 06:34:43 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/root-region-server
12/08/16 22:33:05 ERROR zookeeper.ZooKeeperWatcher: hconnection Received unexpected KeeperException, re-throwing exception
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/root-region-server
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1131)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(RecoverableZooKeeper.java:264)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataInternal(ZKUtil.java:522)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:498)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.getData(ZooKeeperNodeTracker.java:156)
at org.apache.hadoop.hbase.zookeeper.RootRegionTracker.getRootRegionLocation(RootRegionTracker.java:62)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:821)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:933)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:832)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:933)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:836)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:234)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:174)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:133)
at org.apache.pig.backend.hadoop.hbase.HBaseStorage.setLocation(HBaseStorage.java:551)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:385)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:259)
at org.apache.pig.test.TestJobSubmission.testReducerNumEstimation(TestJobSubmission.java:492)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
at org.junit.runners
+
lulynn_2008 2012-08-17, 05:59
|
|
