Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> A dropped table has region still deployed on region server


+
Shrijeet Paliwal 2012-04-20, 17:26
+
Shrijeet Paliwal 2012-04-20, 17:33
+
Ted Yu 2012-04-20, 18:23
Copy link to this message
-
Re: A dropped table has region still deployed on region server
What happened in region server :

2012-04-19 19:32:31,752 INFO
org.apache.hadoop.hbase.regionserver.SplitRequest: Successful rollback of
failed split of
TestUserBTProfile,141018963342131767,1334870127822.5fe2189f32a52f3f2bce55a92ae5df9d.
2012-04-19 19:50:10,392 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Flushing
.META.,,1.1028785192
2012-04-19 19:50:10,529 INFO org.apache.hadoop.hbase.regionserver.Store:
Added hdfs://
master-hadoop.example.com:8020/hbase/.META./1028785192/info/270b515bf7d34ab88552f0e244419a0d,
entries=823, sequenceid=1825092, memsize=243.3k, filesize=102.2k
2012-04-19 19:50:10,535 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Finished memstore flush of ~483.5k/495080, currentsize=0.0/0 for region
.META.,,1.1028785192 in 88ms, sequenceid=1825092, compaction requested=false
2012-04-19 19:50:10,614 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Starting compaction on info in region .META.,,1.1028785192
2012-04-19 19:50:10,615 INFO org.apache.hadoop.hbase.regionserver.Store:
Starting compaction of 2 file(s) in info of .META.,,1.1028785192 into
tmpdir=hdfs://master-hadoop.example.com:8020/hbase/.META./1028785192/.tmp,
seqid=1825092, totalSize=541.0k
2012-04-19 19:50:10,813 INFO org.apache.hadoop.hbase.regionserver.Store:
Renaming compacted file at hdfs://
master-hadoop.example.com:8020/hbase/.META./1028785192/.tmp/5f7bea0f8f8142109c0ef41ada5a509eto
hdfs://
master-hadoop.example.com:8020/hbase/.META./1028785192/info/5f7bea0f8f8142109c0ef41ada5a509e2012-04-1919:50:10,927
INFO org.apache.hadoop.hbase.regionserver.Store: Completed
major compaction of 2 file(s) in info of .META.,,1.1028785192 into
5f7bea0f8f8142109c0ef41ada5a509e, size=442.0k; total size for store is
442.0k2012-04-19 19:50:10,927 INFO
org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
completed compaction: regionName=.META.,,1.1028785192, storeName=info,
fileCount=2, fileSize=541.0k, priority=1, time=6327679323413783;
duration=0sec2012-04-19 20:04:14,620 INFO
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush of region
userprofile,671599294440030433,1334171984301.075898f82c75fa049e7bfb88b7ca567d.
due to global heap pressure

On Fri, Apr 20, 2012 at 11:23 AM, Ted Yu <[EMAIL PROTECTED]> wrote:

> bq. Successful rollback of failed split of
> Can you look for 'TestUserBTProfile,141018963342131767' in region / master
> logs to see what happened next ?
>
> Thanks
>
> On Fri, Apr 20, 2012 at 10:33 AM, Shrijeet Paliwal
> <[EMAIL PROTECTED]>wrote:
>
> > Here are the version details:
> >
> > Hadoop : CDH3u0
> > HBase: 0.92.1 (last commit sha :
> 355e14491c78b56d9f2e270e9995d666d4ce6767)
> >
> > On Fri, Apr 20, 2012 at 10:26 AM, Shrijeet Paliwal
> > <[EMAIL PROTECTED]>wrote:
> >
> > > Hi All,
> > >
> > > hbck is reporting this error:
> > >
> > > ERROR: Region UNKNOWN_REGION on ip-4.example.com,60020,1334705737039,
> > > key=5fe2189f32a52f3f2bce55a92ae5df9d, not on HDFS or in META but
> deployed
> > > on ip-4.example.com,60020,1334705737039
> > >
> > > Region server log :
> > > http://pastie.org/3823850
> > >
> > > Namenode audit log:
> > > http://pastie.org/3823867
> > >
> > > Master log:
> > > http://pastie.org/3823871
> > >
> > > From the region server log it seems there was a hiccup during a split
> > > event as seen here:
> > >
> > > 2189f32a52f3f2bce55a92ae5df9d.; Took too long to split the files and
> > > create the references, aborting split
> > > java.io.IOException: Took too long to split the files and create the
> > > references, aborting split
> > > >-at
> > >
> >
> org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:613)
> > > >-at
> > >
> >
> org.apache.hadoop.hbase.regionserver.SplitTransaction.createDaughters(SplitTransaction.java:285)
> > > >-at
> > >
> >
> org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:450)
> > > >-at
> > >
> >
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:67)
> > > >-at
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB