Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> WHY REGIONS STOP SPLITTING


Copy link to this message
-
Re: WHY REGIONS STOP SPLITTING
below is the region server's log:
   too many flush and compaction in these regionservers, below is excerpt
from one of the regionserver's log:

2013-06-15 23:17:30,719 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
No existing file in archive
for:hdfs://mycluster/hbase/.archive/receipt/1cb5ac9af3886f629ec6602d357158f1/cf/73d0c818a5ff43c8bfe0c66b33afb374,
free to archive original file.
2013-06-15 23:17:30,729 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Finished archiving file from: class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/1cb5ac9af3886f629ec6602d357158f1/cf/73d0c818a5ff43c8bfe0c66b33afb374,
to:
hdfs://mycluster/hbase/.archive/receipt/1cb5ac9af3886f629ec6602d357158f1/cf/73d0c818a5ff43c8bfe0c66b33afb374
2013-06-15 23:17:30,729 INFO org.apache.hadoop.hbase.regionserver.Store:
Completed compaction of 3 file(s) in cf of
receipt,0000000000000000000021010503901002,1371297880334.1cb5ac9af3886f629ec6602d357158f1.
into 6389524ff5784370a7d5e3c5f4cacfec, size=541.1m; total size for store is
2.3g
2013-06-15 23:17:30,729 INFO
org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
completed compaction:
regionName=receipt,0000000000000000000021010503901002,1371297880334.1cb5ac9af3886f629ec6602d357158f1.,
storeName=cf, fileCount=3, fileSize=541.1m, priority=8,
time=1726303500804089; duration=20sec
2013-06-15 23:17:30,729 DEBUG
org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
CompactSplitThread status: compaction_queue=(39:1), split_queue=0
2013-06-15 23:17:30,730 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Starting compaction on cf in region
receipt,0000000000000000000021250036011002,1371307135776.66fd719d546ad77358f0f8aa42dbca40.
2013-06-15 23:17:30,730 INFO org.apache.hadoop.hbase.regionserver.Store:
Starting compaction of 3 file(s) in cf of
receipt,0000000000000000000021250036011002,1371307135776.66fd719d546ad77358f0f8aa42dbca40.
into
tmpdir=hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp,
seqid=332549270, totalSize=300.9m
2013-06-15 23:17:30,730 DEBUG
org.apache.hadoop.hbase.regionserver.Compactor: Compacting
hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
keycount=3411, bloomtype=NONE, size=127.7m, encoding=NONE
2013-06-15 23:17:30,730 DEBUG
org.apache.hadoop.hbase.regionserver.Compactor: Compacting
hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50,
keycount=3411, bloomtype=NONE, size=127.7m, encoding=NONE
2013-06-15 23:17:30,730 DEBUG
org.apache.hadoop.hbase.regionserver.Compactor: Compacting
hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40,
keycount=1217, bloomtype=NONE, size=45.5m, encoding=NONE
2013-06-15 23:17:30,977 DEBUG org.apache.hadoop.hbase.util.FSUtils:
Creating
file=hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1
with permission=rwxrwxrwx
2013-06-15 23:17:30,988 DEBUG
org.apache.hadoop.hbase.io.hfile.HFileWriterV2: Initialized with
CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false]
[cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false]
[cacheEvictOnClose=false] [cacheCompressed=false]
2013-06-15 23:17:30,988 INFO
org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family Bloom filter
type for
hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1:
CompoundBloomFilterWriter
2013-06-15 23:17:40,944 INFO
org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom and NO
DeleteFamily was added to HFile
(hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1)

2013-06-15 23:17:40,955 INFO org.apache.hadoop.hbase.regionserver.Store:
Renaming compacted file at
hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/.tmp/a0739ed1108a4a0dbf5055169b9c2ce1
to
hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/a0739ed1108a4a0dbf5055169b9c2ce1
2013-06-15 23:17:40,977 DEBUG org.apache.hadoop.hbase.regionserver.Store:
Removing store files after compaction...
2013-06-15 23:17:40,983 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Archiving compacted store files.
2013-06-15 23:17:40,983 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Starting to archive files:[class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/412fb567bc6a459dafdda5531e42ff50,
class org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/defbd01d065044f1bdcf1ae6161b3a40]
2013-06-15 23:17:40,983 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
moving files to the archive directory:
hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf
2013-06-15 23:17:40,985 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Archiving:class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff
2013-06-15 23:17:40,986 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
No existing file in archive
for:hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
free to archive original file.
2013-06-15 23:17:40,994 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Finished archiving file from: class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://mycluster/hbase/receipt/66fd719d546ad77358f0f8aa42dbca40/cf/39d6834a9189425ebc6f0b1855b4bbff,
to:
hdfs://mycluster/hbase/.archive/receipt/66fd719d546ad77358f0f8aa42dbca4