Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Errors after major compaction


Copy link to this message
-
Re: Errors after major compaction
It does seem that both servers opened the same region around the same time.
The region was offline because I disabled the table so I can change its TTL.

Here is the log from haddop1-s05 :
2011-06-29 16:37:12,576 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Received request to open
region:
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
2011-06-29 16:37:12,680 DEBUG
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Processing
open of
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
2011-06-29 16:37:12,680 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x33004a38816050b Attempting to transition node
584dac5cc70d8682f71c4675a843c309 from M_ZK_REGION_OFFLINE to
RS_ZK_REGION_OPENING
2011-06-29 16:37:12,711 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x33004a38816050b Successfully transitioned node
584dac5cc70d8682f71c4675a843c309 from M_ZK_REGION_OFFLINE to
RS_ZK_REGION_OPENING
2011-06-29 16:37:12,711 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Opening region: REGION => {NAME =>
'gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.',
STARTKEY => 'GSLoad_1308518553_168_WEB204', ENDKEY =>
'GSLoad_1308518810_1249_WEB204', ENCODED =>
584dac5cc70d8682f71c4675a843c309, TABLE => {{NAME => 'gs_raw_events',
FAMILIES => [{NAME => 'events', BLOOMFILTER => 'NONE', REPLICATION_SCOPE =>
'1', VERSIONS => '3', COMPRESSION => 'LZO', TTL => '604800', BLOCKSIZE =>
'65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}}
2011-06-29 16:37:12,711 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Instantiated
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
2011-06-29 16:37:12,847 DEBUG org.apache.hadoop.hbase.regionserver.Store:
loaded
hdfs://hadoop1-m1:8020/hbase/gs_raw_events/584dac5cc70d8682f71c4675a843c309/events/1971818821800304360,
isReference=false, isBulkLoadResult=false, seqid=1162228062,
majorCompaction=false
2011-06-29 16:37:12,848 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Onlined
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.;
next sequenceid=1162228063
2011-06-29 16:37:12,849 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x33004a38816050b Attempting to transition node
584dac5cc70d8682f71c4675a843c309 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENING
2011-06-29 16:37:12,875 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x33004a38816050b Successfully transitioned node
584dac5cc70d8682f71c4675a843c309 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENING
2011-06-29 16:37:12,951 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
Updated row
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
in region .META.,,1 with server=hadoop1-s05.farm-ny.gigya.com:60020,
startcode=1307349217076
2011-06-29 16:37:12,951 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x33004a38816050b Attempting to transition node
584dac5cc70d8682f71c4675a843c309 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENED
2011-06-29 16:37:12,964 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x33004a38816050b Successfully transitioned node
584dac5cc70d8682f71c4675a843c309 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENED
2011-06-29 16:37:12,964 DEBUG
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Opened
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
2011-06-29 16:40:00,878 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Received request to open
region:
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
2011-06-29 16:40:00,878 DEBUG
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Processing
open of
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
2011-06-29 16:40:01,079 WARN
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Attempted
open of
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
but already online on this server
2011-06-29 16:43:50,395 DEBUG
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction (major)
requested for
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
because User-triggered major compaction; priority=1, compaction queue
size=1248
2011-06-29 20:19:49,906 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Starting major compaction on region
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
2011-06-29 20:19:49,906 INFO org.apache.hadoop.hbase.regionserver.Store:
Started compaction of 1 file(s) in cf=events  into
hdfs://hadoop1-m1:8020/hbase/gs_raw_events/584dac5cc70d8682f71c4675a843c309/.tmp,
seqid=1162228062, totalSize=98.3m
2011-06-29 20:19:49,906 DEBUG org.apache.hadoop.hbase.regionserver.Store:
Compacting
hdfs://hadoop1-m1:8020/hbase/gs_raw_events/584dac5cc70d8682f71c4675a843c309/events/1971818821800304360,
keycount=6882816, bloomtype=NONE, size=98.3m
2011-06-29 20:19:59,920 INFO org.apache.hadoop.hbase.regionserver.HRegion:
completed compaction on region
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
after 10sec

And here is the one from hadoop1-s02:
2011-06-29 16:43:57,935 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Received request to open
region:
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
2011-06-29 16:43:58,990 DEBUG
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Processing
open of
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
2011-06-29 16:43:58,990 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x23004
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB