Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Errors after major compaction


Copy link to this message
-
Re: Errors after major compaction
It does seem that both servers opened the same region around the same time.
The region was offline because I disabled the table so I can change its TTL.

Here is the log from haddop1-s05 :
2011-06-29 16:37:12,576 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Received request to open
region:
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
2011-06-29 16:37:12,680 DEBUG
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Processing
open of
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
2011-06-29 16:37:12,680 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x33004a38816050b Attempting to transition node
584dac5cc70d8682f71c4675a843c309 from M_ZK_REGION_OFFLINE to
RS_ZK_REGION_OPENING
2011-06-29 16:37:12,711 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x33004a38816050b Successfully transitioned node
584dac5cc70d8682f71c4675a843c309 from M_ZK_REGION_OFFLINE to
RS_ZK_REGION_OPENING
2011-06-29 16:37:12,711 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Opening region: REGION => {NAME =>
'gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.',
STARTKEY => 'GSLoad_1308518553_168_WEB204', ENDKEY =>
'GSLoad_1308518810_1249_WEB204', ENCODED =>
584dac5cc70d8682f71c4675a843c309, TABLE => {{NAME => 'gs_raw_events',
FAMILIES => [{NAME => 'events', BLOOMFILTER => 'NONE', REPLICATION_SCOPE =>
'1', VERSIONS => '3', COMPRESSION => 'LZO', TTL => '604800', BLOCKSIZE =>
'65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}}
2011-06-29 16:37:12,711 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Instantiated
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
2011-06-29 16:37:12,847 DEBUG org.apache.hadoop.hbase.regionserver.Store:
loaded
hdfs://hadoop1-m1:8020/hbase/gs_raw_events/584dac5cc70d8682f71c4675a843c309/events/1971818821800304360,
isReference=false, isBulkLoadResult=false, seqid=1162228062,
majorCompaction=false
2011-06-29 16:37:12,848 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Onlined
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.;
next sequenceid=1162228063
2011-06-29 16:37:12,849 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x33004a38816050b Attempting to transition node
584dac5cc70d8682f71c4675a843c309 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENING
2011-06-29 16:37:12,875 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x33004a38816050b Successfully transitioned node
584dac5cc70d8682f71c4675a843c309 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENING
2011-06-29 16:37:12,951 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
Updated row
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
in region .META.,,1 with server=hadoop1-s05.farm-ny.gigya.com:60020,
startcode=1307349217076
2011-06-29 16:37:12,951 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x33004a38816050b Attempting to transition node
584dac5cc70d8682f71c4675a843c309 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENED
2011-06-29 16:37:12,964 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x33004a38816050b Successfully transitioned node
584dac5cc70d8682f71c4675a843c309 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENED
2011-06-29 16:37:12,964 DEBUG
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Opened
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
2011-06-29 16:40:00,878 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Received request to open
region:
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
2011-06-29 16:40:00,878 DEBUG
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Processing
open of
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
2011-06-29 16:40:01,079 WARN
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Attempted
open of
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
but already online on this server
2011-06-29 16:43:50,395 DEBUG
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction (major)
requested for
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
because User-triggered major compaction; priority=1, compaction queue
size=1248
2011-06-29 20:19:49,906 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Starting major compaction on region
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
2011-06-29 20:19:49,906 INFO org.apache.hadoop.hbase.regionserver.Store:
Started compaction of 1 file(s) in cf=events  into
hdfs://hadoop1-m1:8020/hbase/gs_raw_events/584dac5cc70d8682f71c4675a843c309/.tmp,
seqid=1162228062, totalSize=98.3m
2011-06-29 20:19:49,906 DEBUG org.apache.hadoop.hbase.regionserver.Store:
Compacting
hdfs://hadoop1-m1:8020/hbase/gs_raw_events/584dac5cc70d8682f71c4675a843c309/events/1971818821800304360,
keycount=6882816, bloomtype=NONE, size=98.3m
2011-06-29 20:19:59,920 INFO org.apache.hadoop.hbase.regionserver.HRegion:
completed compaction on region
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
after 10sec

And here is the one from hadoop1-s02:
2011-06-29 16:43:57,935 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Received request to open
region:
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
2011-06-29 16:43:58,990 DEBUG
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Processing
open of
gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309.
2011-06-29 16:43:58,990 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:60020-0x23004