Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - RE: Hadoop/HBase Upgrade from 0.20.3 to 0.90.2


+
Zhong, Andy 2011-07-15, 04:43
Copy link to this message
-
Re: Hadoop/HBase Upgrade from 0.20.3 to 0.90.2
Joey Echeverria 2011-07-19, 16:16
Hey Andy,

This looks like the log from the regions server. Any chance you can post the
log for the HMaster?

-Joey

On Fri, Jul 15, 2011 at 12:43 AM, Zhong, Andy <[EMAIL PROTECTED]>wrote:

> St.Ack,
>
> It's weird. I did Hbase upgrade from Hbase 0.20.3 to Hbase 0.90.2 by
> following "replace the Hadoop JAR file from Hadoop 0.20.2 release with the
> sync-supporting Hadoop JAR file found in HBase 0.90.2". It was suceesfully
> before, but now we are facing error from Hbase shell: NativeException:
> org.apache.hadoop.hbase.MasterNotRunningException:
> hofvmecln15.ecom.sears.com:60000, even though the processes of Hmaster and
> Region are running, and 60010 can be accessible via 60010.
>
> Could you please help to find out the possible root cause? Below is the
> logging messages from the hbase.log from Master Node:
>
> 2011-07-14 22:54:55,903 INFO  zookeeper.ZooKeeper - Client
> environment:user.dir=/appl/hbase/share/hbase-0.90.2
> 2011-07-14 22:54:55,904 INFO  zookeeper.ZooKeeper - Initiating client
> connection, connectString=hofvmecln19:2181,hofvmecln18:2181,hofvmecln17:2181
> sessionTimeout=60000
> watcher=org.apache.hadoop.hbase.regionserver.HRegionServer@51b4a74b
> 2011-07-14 22:54:55,905 INFO  zookeeper.ClientCnxn -
> zookeeper.disableAutoWatchReset is false
> 2011-07-14 22:54:55,932 INFO  zookeeper.ClientCnxn - Attempting connection
> to server hofvmecln18/151.149.116.46:2181
> 2011-07-14 22:54:55,938 INFO  zookeeper.ClientCnxn - Priming connection to
> java.nio.channels.SocketChannel[connected local=/151.149.119.93:52156remote=hofvmecln18/
> 151.149.116.46:2181]
> 2011-07-14 22:54:55,939 INFO  zookeeper.ClientCnxn - Server connection
> successful
> 2011-07-14 22:54:56,030 INFO  regionserver.HRegionServer - Got ZooKeeper
> event, state: SyncConnected, type: None, path: null
> 2011-07-14 22:54:56,202 INFO  regionserver.HRegionServer - Telling master
> at hofvmecln15.ecom.sears.com:60000 that we are up
> 2011-07-14 22:54:56,407 WARN  ipc.HBaseServer - IPC Server listener on
> 60000: readAndProcess threw exception java.io.EOFException. Count of bytes
> read: 0
> java.io.EOFException
>        at java.io.DataInputStream.readFully(DataInputStream.java:180)
>        at java.io.DataInputStream.readUTF(DataInputStream.java:592)
>        at java.io.DataInputStream.readUTF(DataInputStream.java:547)
>        at
> org.apache.hadoop.hbase.ipc.HBaseRPC$Invocation.readFields(HBaseRPC.java:122)
>        at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:978)
>        at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:946)
>        at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:522)
>        at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:316)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>        at java.lang.Thread.run(Thread.java:619)
> 2011-07-14 22:54:56,412 WARN  regionserver.HRegionServer - Unable to
> connect to master. Retrying. Error was:
> java.io.IOException: Call to
> hofvmecln15.ecom.sears.com/151.149.119.93:60000 failed on local exception:
> java.io.EOFException
>        at
> org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:757)
>        at
> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:727)
>        at
> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:333)
>        at $Proxy0.getProtocolVersion(Unknown Source)
>        at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:489)
>        at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:465)
>        at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:512)
>        at
> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:423)
>        at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getMaster(HRegionServer.java:1296)

Joseph Echeverria
Cloudera, Inc.
443.305.9434