Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> RE: Hadoop/HBase Upgrade from 0.20.3 to 0.90.2


+
Zhong, Andy 2011-07-15, 04:43
Copy link to this message
-
Re: Hadoop/HBase Upgrade from 0.20.3 to 0.90.2
Hey Andy,

This looks like the log from the regions server. Any chance you can post the
log for the HMaster?

-Joey

On Fri, Jul 15, 2011 at 12:43 AM, Zhong, Andy <[EMAIL PROTECTED]>wrote:

> St.Ack,
>
> It's weird. I did Hbase upgrade from Hbase 0.20.3 to Hbase 0.90.2 by
> following "replace the Hadoop JAR file from Hadoop 0.20.2 release with the
> sync-supporting Hadoop JAR file found in HBase 0.90.2". It was suceesfully
> before, but now we are facing error from Hbase shell: NativeException:
> org.apache.hadoop.hbase.MasterNotRunningException:
> hofvmecln15.ecom.sears.com:60000, even though the processes of Hmaster and
> Region are running, and 60010 can be accessible via 60010.
>
> Could you please help to find out the possible root cause? Below is the
> logging messages from the hbase.log from Master Node:
>
> 2011-07-14 22:54:55,903 INFO  zookeeper.ZooKeeper - Client
> environment:user.dir=/appl/hbase/share/hbase-0.90.2
> 2011-07-14 22:54:55,904 INFO  zookeeper.ZooKeeper - Initiating client
> connection, connectString=hofvmecln19:2181,hofvmecln18:2181,hofvmecln17:2181
> sessionTimeout=60000
> watcher=org.apache.hadoop.hbase.regionserver.HRegionServer@51b4a74b
> 2011-07-14 22:54:55,905 INFO  zookeeper.ClientCnxn -
> zookeeper.disableAutoWatchReset is false
> 2011-07-14 22:54:55,932 INFO  zookeeper.ClientCnxn - Attempting connection
> to server hofvmecln18/151.149.116.46:2181
> 2011-07-14 22:54:55,938 INFO  zookeeper.ClientCnxn - Priming connection to
> java.nio.channels.SocketChannel[connected local=/151.149.119.93:52156remote=hofvmecln18/
> 151.149.116.46:2181]
> 2011-07-14 22:54:55,939 INFO  zookeeper.ClientCnxn - Server connection
> successful
> 2011-07-14 22:54:56,030 INFO  regionserver.HRegionServer - Got ZooKeeper
> event, state: SyncConnected, type: None, path: null
> 2011-07-14 22:54:56,202 INFO  regionserver.HRegionServer - Telling master
> at hofvmecln15.ecom.sears.com:60000 that we are up
> 2011-07-14 22:54:56,407 WARN  ipc.HBaseServer - IPC Server listener on
> 60000: readAndProcess threw exception java.io.EOFException. Count of bytes
> read: 0
> java.io.EOFException
>        at java.io.DataInputStream.readFully(DataInputStream.java:180)
>        at java.io.DataInputStream.readUTF(DataInputStream.java:592)
>        at java.io.DataInputStream.readUTF(DataInputStream.java:547)
>        at
> org.apache.hadoop.hbase.ipc.HBaseRPC$Invocation.readFields(HBaseRPC.java:122)
>        at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:978)
>        at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:946)
>        at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:522)
>        at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:316)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>        at java.lang.Thread.run(Thread.java:619)
> 2011-07-14 22:54:56,412 WARN  regionserver.HRegionServer - Unable to
> connect to master. Retrying. Error was:
> java.io.IOException: Call to
> hofvmecln15.ecom.sears.com/151.149.119.93:60000 failed on local exception:
> java.io.EOFException
>        at
> org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:757)
>        at
> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:727)
>        at
> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:333)
>        at $Proxy0.getProtocolVersion(Unknown Source)
>        at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:489)
>        at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:465)
>        at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:512)
>        at
> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:423)
>        at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getMaster(HRegionServer.java:1296)

Joseph Echeverria
Cloudera, Inc.
443.305.9434
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB