Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> How to adjust hbase settings when too many store files?


Copy link to this message
-
How to adjust hbase settings when too many store files?
Hello. I encounter a region server error when I try to put bulk data from a
java client.

The java client extracts data from a relational database and puts those data
into hbase.

When I try to extract data from a large table(say, 1 billion records), the
error happens.

 

The region server's log says:

 

>2012-10-28 00:00:02,169 WARN
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Region
statistic_visit_detail,20120804|72495|8549

56,1351353594195.ad2592ee7a3610c60c47cf8be77496c8. has too many store files;
delaying flush up to 90000ms

>2012-10-28 00:00:02,791 DEBUG
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
because memory above low wa

ter=347.1m

>2012-10-28 00:00:02,791 DEBUG
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Under global heap
pressure: Region statistic_vis

it_detail,20120804|72495|854956,1351353594195.ad2592ee7a3610c60c47cf8be77496
c8. has too many store files, but is 141.5m vs best flus

hable region's 46.8m. Choosing the bigger.

>2012-10-28 00:00:02,791 INFO
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region
statistic_visit_detail,20120804|7

2495|854956,1351353594195.ad2592ee7a3610c60c47cf8be77496c8. due to global
heap pressure

...

 

And finally,

 

>2012-10-28 00:00:43,511 INFO org.apache.hadoop.hbase.regionserver.HRegion:
compaction interrupted by user

>java.io.InterruptedIOException: Aborting compaction of store cf1 in region
statistic_visit_detail,20120804|72495|854956,135135359419

5.ad2592ee7a3610c60c47cf8be77496c8. because user requested stop.

        at
org.apache.hadoop.hbase.regionserver.Store.compactStore(Store.java:1275)

        at
org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:765)

        at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1023)

        at
org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest.run(Compa
ctionRequest.java:177)

        at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.ja
va:886)

        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9
08)

        at java.lang.Thread.run(Thread.java:662)

 

Then the region server shuts down.

 

It seems that too many store files(due to too many records from
relational-db) consumed too many memories, if I'm right.

I'm new to hbase, what settings should I adjust? Or even increase region
servers?

I'm going to do some research by myself, and any advise will be appreciated.

Best regards,

 

Su

 

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB