Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - 0.95.2 upgrade errors


Copy link to this message
-
Re: 0.95.2 upgrade errors
Kiru Pakkirisamy 2013-09-16, 06:47
Himanshu et al,
I see this error in one region. the .meta file is having a problem  I guess. Thanks for all the help.

13/09/16 02:13:39 WARN wal.HLogSplitter: Could not open hdfs://ssp-master:8020/apps/hbase/data/WALs/ssp-region3,60020,1379105062356-splitting/ssp-region3%2C60020%2C1379105062356.1379105067148.meta for reading. File is empty
java.io.EOFException
        at java.io.DataInputStream.readFully(DataInputStream.java:180)
        at java.io.DataInputStream.readFully(DataInputStream.java:152)
        at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1512)
        at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1490)
        at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1479)
        at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1474)
        at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.<init>(SequenceFileLogReader.java:69)
        at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.reset(SequenceFileLogReader.java:174)
        at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.initReader(SequenceFileLogReader.java:183)
        at org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:64)
        at org.apache.hadoop.hbase.regionserver.wal.HLogFactory.createReader(HLogFactory.java:126)
        at org.apache.hadoop.hbase.regionserver.wal.HLogFactory.createReader(HLogFactory.java:89)

and in the master I see 

 2013-09-16 02:33:28,755 [RpcServer.handler=27,port=60000] ERROR org.apache.hadoop.ipc.RpcServer.call - Unexpected throwable object
java.lang.NullPointerException
        at org.apache.hadoop.hbase.master.TableNamespaceManager.get(TableNamespaceManager.java:111)
        at org.apache.hadoop.hbase.master.HMaster.getNamespaceDescriptor(HMaster.java:3116)
        at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1779)
        at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1820)
        at org.apache.hadoop.hbase.protobuf.generated.MasterAdminProtos$MasterAdminService$2.callBlockingMethod(MasterAdminProtos.java:27720)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2156)
        at org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1861)
Regards,
- kiru
________________________________
 From: Himanshu Vashishtha <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]; Kiru Pakkirisamy <[EMAIL PROTECTED]>
Cc: Ted Yu <[EMAIL PROTECTED]>
Sent: Friday, September 13, 2013 4:18 PM
Subject: Re: 0.95.2 upgrade errors
 
Is there something odd (exceptions on starting) in Master logs? Does it start normally? You removed the above jar from HMaster node too? Could you restart master and see if that changes anything?

On Fri, Sep 13, 2013 at 1:47 PM, Kiru Pakkirisamy <[EMAIL PROTECTED]> wrote:
>
>I see a hadoop-core-1.1.2 jar in the hbase/lib dir. Is this the same as the hadoop 'core'  jar in the hadoop directory ?
>If I remove this, the regionservers seem to stay up. But create table fails. 
>
>ERROR: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=7, exceptions:
>Fri Sep 13 16:44:48 EDT 2013, org.apache.hadoop.hbase.client.RpcRetryingCaller@734893da, org.apache.hadoop.hbase.ipc.RemoteWithExtrasException: java.io.IOException
>at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2194)
>at org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1861)
>Caused by: java.lang.NullPointerException
>at org.apache.hadoop.hbase.master.TableNamespaceManager.get(TableNamespaceManager.java:111)
>

>Regards,
>- kiru
>
>
>
>
>________________________________
> From: Himanshu Vashishtha <[EMAIL PROTECTED]>
>To: [EMAIL PROTECTED]; Kiru Pakkirisamy <[EMAIL PROTECTED]>
>Cc: Ted Yu <[EMAIL PROTECTED]>
>Sent: Friday, September 13, 2013 10:22 AM
>
>Subject: Re: 0.95.2 upgrade errors
>
>
>Sounds odd, but just to be sure you are picking right hadoop jars, you