Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> WHY REGIONS STOP SPLITTING


Copy link to this message
-
Re: WHY REGIONS STOP SPLITTING
below is the region server's log:
   too many flush and compaction in these regionservers, below is excerpt
from one of the regionserver's log:

2013-06-15 23:17:30,719 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
No existing file in archive
for:hdfs://mycluster/hbase/.archive/receipt/1cb5ac9af3886f629ec6602d357158f1/cf/73d0c818a5ff43c8bfe0c66b33afb374,
free to archive original file.
2013-06-15 23:17:30,729 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Finished archiving file from: class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/1cb5ac9af3886f629ec6602d357158f1/cf/73d0c818a5ff43c8bfe0c66b33afb374,
to:
hdfs://mycluster/hbase/.archive/receipt/1cb5ac9af3886f629ec6602d357158f1/cf/73d0c818a5ff43c8bfe0c66b33afb374
2013-06-15 23:17:30,729 INFO org.apache.hadoop.hbase.regionserver.Store:
Completed compaction of 3 file(s) in cf of
receipt,0000000000000000000021010503901002,1371297880334.1cb5ac9af3886f629ec6602d357158f1.
into 6389524ff5784370a7d5e3c5f4cacfec, size=541.1m; total size for store is
2.3g
2013-06-15 23:17:30,729 INFO
org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
completed compaction:
regionName=receipt,0000000000000000000021010503901002,1371297880334.1cb5ac9af3886f629ec6602d357158f1.,
storeName=cf, fileCount=3, fileSize=541.1m, priority=8,
time=1726303500804089; duration=20sec
2013-06-15 23:17:30,729 DEBUG
org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
CompactSplitThread status: compaction_queue=(39:1), split_queue=0
2013-06-15 23:17:30,730 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Starting compaction on cf in region
receipt,0000000000000000000021250036011002,1371307135776.66fd719d546ad77358f0f8aa42dbca40.
2013-06-15 23:17:30,730 INFO org.apache.hadoop.hbase.regionserver.Store:
Starting compaction of 3 file(s) in cf of
receipt,0000000000000000000021250036011002,1371307135776.66fd719d546ad77358f0f8aa42dbca40.
into
tmpdir=hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp,
seqid=332549270, totalSize=300.9m
2013-06-15 23:17:30,730 DEBUG
org.apache.hadoop.hbase.regionserver.Compactor: Compacting
hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
keycount=3411, bloomtype=NONE, size=127.7m, encoding=NONE
2013-06-15 23:17:30,730 DEBUG
org.apache.hadoop.hbase.regionserver.Compactor: Compacting
hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50,
keycount=3411, bloomtype=NONE, size=127.7m, encoding=NONE
2013-06-15 23:17:30,730 DEBUG
org.apache.hadoop.hbase.regionserver.Compactor: Compacting
hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40,
keycount=1217, bloomtype=NONE, size=45.5m, encoding=NONE
2013-06-15 23:17:30,977 DEBUG org.apache.hadoop.hbase.util.FSUtils:
Creating
file=hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1
with permission=rwxrwxrwx
2013-06-15 23:17:30,988 DEBUG
org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
[cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
[cacheEvictOnClose=false] [cacheCompressed=false]
2013-06-15 23:17:30,988 INFO
org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
type for
hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1:
CompoundBloomFilterWriter
2013-06-15 23:17:40,944 INFO
org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
DeleteFamily was added to HFile
(hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1)

2013-06-15 23:17:40,955 INFO org.apache.hadoop.hbase.regionserver.Store:
Renaming compacted file at
hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1
to
hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/a0739ed1108a4a0dbf5055169b9c2ce1
2013-06-15 23:17:40,977 DEBUG org.apache.hadoop.hbase.regionserver.Store:
Removing store files after compaction...
2013-06-15 23:17:40,983 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Archiving compacted store files.
2013-06-15 23:17:40,983 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Starting to archive files:[class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50,
class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40]
2013-06-15 23:17:40,983 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
moving files to the archive directory:
hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf
2013-06-15 23:17:40,985 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Archiving:class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff
2013-06-15 23:17:40,986 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
No existing file in archive
for:hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
free to archive original file.
2013-06-15 23:17:40,994 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Finished archiving file from: class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
to:
hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca4
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB