Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - How to adjust hbase settings when too many store files?


Copy link to this message
-
Re: 答复: How to adjust hbase settings when too many store files?
ramkrishna vasudevan 2012-10-30, 06:46
Hi

Can you see if your hTable instances are shared across different threads.
 that could be the reason for you null pointer excepiton.

Regards
Ram

On Tue, Oct 30, 2012 at 7:22 AM, xkwang bruce <[EMAIL PROTECTED]>wrote:

> Hi,苏铖.
>
> U may need presplit you htable when the load is heavy or there should be
> some problem in your client code.
> Just a suggestion.
>
> bruce
>
>
> 2012/10/29 苏铖 <[EMAIL PROTECTED]>
>
> > Hi, everyone.
> >
> > I changed the max size of hbase store file and increased region servers.
> > The
> > former exception doesn't happen again.
> > But there is another exception at the client side.
> >
> > 2012-10-29 19:06:27:758 WARN [pool-2-thread-2]
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation
> > | Failed all from
> >
> >
> region=statistic_visit_detail1,,1351508069797.3272dd30817191d9d393d1d6e1b99d
> > 1b., hostname=hadoop02, port=60020
> > java.util.concurrent.ExecutionException: java.lang.RuntimeException:
> > java.lang.NullPointerException
> >         at
> > java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> >         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> >         at
> >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.
> > processBatchCallback(HConnectionManager.java:1557)
> >         at
> >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.
> > processBatch(HConnectionManager.java:1409)
> >         at
> > org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:900)
> >         at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:773)
> >         at org.apache.hadoop.hbase.client.HTable.put(HTable.java:760)
> >         at
> > com.lietou.datawarehouse.imp.HBaseImporter$ActualHBaseImporter$1.
> > process(HBaseImporter.java:150)
> >         at
> > com.lietou.datawarehouse.imp.HBaseImporter$ActualHBaseImporter$1.
> > process(HBaseImporter.java:133)
> >         at
> >
> >
> com.lietou.datawarehouse.common.range.Repeater.rangeRepeat(Repeater.java:48)
> >         at
> >
> >
> com.lietou.datawarehouse.common.range.Repeater.rangeRepeat(Repeater.java:30)
> >         at
> >
> >
> com.lietou.datawarehouse.imp.HBaseImporter$ActualHBaseImporter.run(HBaseImpo
> > rter.java:162)
> >         at
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.ja
> > va:886)
> >         at
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9
> > 08)
> >         at java.lang.Thread.run(Thread.java:662)
> > Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> >         at
> >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.
> > getRegionServerWithoutRetries(HConnectionManager.java:1371)
> >         at
> >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$
> > 3.call(HConnectionManager.java:1383)
> >         at
> >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$
> > 3.call(HConnectionManager.java:1381)
> >         at
> > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> >         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> >         ... 3 more
> > Caused by: java.lang.NullPointerException
> >         at
> >
> >
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngi
> > ne.java:158)
> >         at $Proxy10.multi(Unknown Source)
> >         at
> >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$
> > 3$1.call(HConnectionManager.java:1386)
> >         at
> >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$
> > 3$1.call(HConnectionManager.java:1384)
> >         at
> >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.
> > getRegionServerWithoutRetries(HConnectionManager.java:1365)
> >         ... 7 more
> >
> >
> > This error happens quite offen. And on the server side, here are some