Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - NullPointerException while loading large amount of new rows into HBase, exception is thrown when trying to obtain lock for RowKey


+
Tianying Chang 2013-05-16, 17:12
Copy link to this message
-
Re: NullPointerException while loading large amount of new rows into HBase, exception is thrown when trying to obtain lock for RowKey
ramkrishna vasudevan 2013-05-16, 17:16
Which version of HBase?

Regards
Ram
On Thu, May 16, 2013 at 10:42 PM, Tianying Chang <[EMAIL PROTECTED]> wrote:

> Hi,
>
> When our customers(using TSDB) loads large amount of data into HBase, we
> saw many NullPointerException in the RS logs as below. I checked the source
> code, it seems when trying to obtain the lock for a rowKey, if the entry
> for that row already exists and the "waitfoBlock" is false (therefore it
> won't retry, but just return a NULL value). I can see in
> doMiniBatchMutation(), the waitForBlock is set to be false. (most other
> places "waitForBlock" is always set to true).
>
> This exception is thrown from function lockRow(), which has been
> deprecated. I am not sure why it is deprecated, and what is used to replace
> it. Is this normal? which implies the HBase should not throw this
> misleading error message to log.  Or should the client call some other API?
>
> Thanks
> Tian-Ying
>
> 2013-05-14 12:45:30,911 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer: Row lock
> -3430274391270203797 explicitly acquired by client
> 2013-05-14 12:45:30,911 WARN org.apache.hadoop.ipc.HBaseServer:
> (responseTooSlow): {"processingtimems":29783,"call":"lockRow([B@339a6a5c,
> [B@5ebcd87b), rpc version=1, client version=29,
> methodsFingerPrint=0","client":"10.53.106.37:58892
> ","starttimems":1368560701128,"queuetimems":847,"class":"HRegionServer","responsesize":0,"method":"lockRow"}
> 2013-05-14 12:46:00,911 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row
> lock (fsOk: true)
> java.lang.NullPointerException
>                 at
> java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:881)
>                 at
> org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:2346)
>                 at
> org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:2332)
>                 at sun.reflect.GeneratedMethodAccessor156.invoke(Unknown
> Source)
>                 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>                 at java.lang.reflect.Method.invoke(Method.java:597)
>                 at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:384)
>                 at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1336)
> 2013-05-14 12:46:02,514 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server
> Responder, call get([B@2166c821,
> {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"id":["tagv"]},"maxVersions":1,"row":"
> slcsn-s00314.slc.ebay.com"}), rpc version=1, client version=29,
> methodsFingerPrint=0 from 10.53.106.37:58892: output error
> 2013-05-14 12:46:02,514 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 3 on 60020 caught: java.io.IOException: Connection reset by peer
>                 at sun.nio.ch.FileDispatcher.write0(Native Method)
>                 at
> sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
>
+
ramkrishna vasudevan 2013-05-16, 17:41
+
Tianying Chang 2013-05-16, 18:35
+
Tianying Chang 2013-05-16, 18:41
+
ramkrishna vasudevan 2013-05-17, 03:49
+
Tianying Chang 2013-05-20, 17:32