Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Never ending transtionning regions.


Copy link to this message
-
Re: Never ending transtionning regions.
JM,

  How are you doing today?  Right before the file does not exist should be
another path.  Can you let me know if in that path there are a pointers
from a split to 2ebfef593a3d715b59b85670909182c9?  The directory may
already exist.  I have seen this a couple times now and am trying to ferret
out a root cause to open a JIRA with.  I suspect we have a split code bug
in .92+

On Sat, Feb 23, 2013 at 4:10 PM, Jean-Marc Spaggiari <
[EMAIL PROTECTED]> wrote:

> Hi,
>
> I have 2 regions transitionning from servers to servers for 15 minutes now.
>
> I have nothing in the master logs about those 2 regions but on the region
> server logs I have some files notfound2013-02-23 16:02:07,347 ERROR
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed open
> of region=entry,theykey,1361651769136.6dd77bc9ff91e0e6d413f74e670ab435.,
> starting to roll back the global memstore size.
> java.io.IOException: java.io.IOException: java.io.FileNotFoundException:
> File does not exist:
>
> /hbase/entry/2ebfef593a3d715b59b85670909182c9/a/62b0aae45d59408dbcfc513954efabc7
>     at
>
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:597)
>     at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:510)
>     at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4177)
>     at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4125)
>     at
>
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:328)
>     at
>
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:100)
>     at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
>     at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>     at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>     at java.lang.Thread.run(Thread.java:722)
> Caused by: java.io.IOException: java.io.FileNotFoundException: File does
> not exist:
>
> /hbase/entry/2ebfef593a3d715b59b85670909182c9/a/62b0aae45d59408dbcfc513954efabc7
>     at
> org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:433)
>     at org.apache.hadoop.hbase.regionserver.Store.<init>(Store.java:240)
>     at
>
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:3141)
>     at
> org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:572)
>     at
> org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:570)
>     at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>     at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>     at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>     at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>     at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>     ... 3 more
> Caused by: java.io.FileNotFoundException: File does not exist:
>
> /hbase/entry/2ebfef593a3d715b59b85670909182c9/a/62b0aae45d59408dbcfc513954efabc7
>     at
>
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1843)
>     at
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.<init>(DFSClient.java:1834)
>     at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:578)
>     at
>
> org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:154)
>     at
> org.apache.hadoop.fs.FilterFileSystem.open(FilterFileSystem.java:108)
>     at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)
>     at
>
> org.apache.hadoop.hbase.io.hfile.HFile.createReaderWithEncoding(HFile.java:573)
>     at
>
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.<init>(StoreFile.java:1261)
>     at
>
> org.apache.hadoop.hbase.io.HalfStoreFileReader.<init>(HalfStoreFileReader.java:70)
>     at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:508)
>     at
>
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:603)

Kevin O'Dell
Customer Operations Engineer, Cloudera