Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - getting splitmanager debug logs continuously


Copy link to this message
-
Re: getting splitmanager debug logs continuously
Jimmy Xiang 2013-08-08, 21:48
Can you start the master as well (besides region servers)?
On Thu, Aug 8, 2013 at 2:41 PM, oc tsdb <[EMAIL PROTECTED]> wrote:

> I am using hbase-0.92
>
> Region server was not running on any of the nodes.
>
> Restarted the cluster. It started region server on all nodes except
> HMaster but still unresponsive.
>
> processes running on master are
> TSDMain
> HMaster
> SecondaryNameNode
> NameNode
> JobTracker
> HQuorumPeer
>
> processes running on all other nodes are
> DataNode
> TaskTracker
> RegionServer
> TSDMain
>
> This time, I see the error messages in the attached log.
>
> Could you please suggest if I can recover/restore the data and get the
> cluster up.
>
> Thanks & Regards,
> VSR
>
>
>
> On Thu, Aug 8, 2013 at 1:40 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> Can you tell us the version of HBase you're using ?
>>
>> Do you find something in region server logs on the 4 remaining nodes ?
>>
>> Cheers
>>
>> On Thu, Aug 8, 2013 at 1:36 PM, oc tsdb <[EMAIL PROTECTED]> wrote:
>>
>> > Hi,
>> >
>> > I am running a cluster with 6 nodes;
>> > Two of 6 nodes in my cluster went down (due to other application
>> failure)
>> > and came back after some time (had to do a power reboot).
>> > When these nodes are back I use to get "WARN
>> org.apache.hadoop.DFSClient:
>> > Failed to connect to , add to deadnodes and continue".
>> > Now these messages are stopped and getting continuous debug message as
>> > follows.
>> >
>> > 2013-08-08 12:57:36,628 DEBUG org.apache.hadoop.hbase.
>> > master.SplitLogManager: total tasks = 14 unassigned = 14
>> > 2013-08-08 12:57:37,628 DEBUG
>> > org.apache.hadoop.hbase.master.SplitLogManager: total tasks = 14
>> unassigned
>> > = 14
>> > 2013-08-08 12:57:37,629 DEBUG
>> > org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
>> > /hbase/splitlog/hdfs%3A%2F%2Fmb-1.corp.oc.com%3A54310%2Fhbase%2F.logs%
>> > 2Fmb-3.corp.oc.com%2C60020%2C1375466447768-splitting%2Fmb-3.corp.oc.com
>> > %252C60020%252C1375466447768.1375631802971
>> > ver = 0
>> > 2013-08-08 12:57:37,629 DEBUG
>> > org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
>> > /hbase/splitlog/hdfs%3A%2F%2Fmb-1.corp.oc.com%3A54310%2Fhbase%2F.logs%
>> > 2Fmb-6.corp.oc.com%2C60020%2C1375466460755-splitting%2Fmb-6.corp.oc.com
>> > %252C60020%252C1375466460755.1375623787557
>> > ver = 0
>> > 2013-08-08 12:57:37,629 DEBUG
>> > org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
>> > /hbase/splitlog/hdfs%3A%2F%2Fmb-1.corp.oc.com%3A54310%2Fhbase%2F.logs%
>> > 2Fmb-6.corp.oc.com%2C60020%2C1375466460755-splitting%2Fmb-6.corp.oc.com
>> > %252C60020%252C1375466460755.1375619231059
>> > ver = 3
>> > 2013-08-08 12:57:37,629 DEBUG
>> > org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
>> > /hbase/splitlog/hdfs%3A%2F%2Fmb-1.corp.oc.com%3A54310%2Fhbase%2F.logs%
>> > 2Fmb-2.corp.oc.com%2C60020%2C1375466479427-splitting%2Fmb-2.corp.oc.com
>> > %252C60020%252C1375466479427.1375639017535
>> > ver = 0
>> > 2013-08-08 12:57:37,629 DEBUG
>> > org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
>> > /hbase/splitlog/hdfs%3A%2F%2Fmb-1.corp.oc.com%3A54310%2Fhbase%2F.logs%
>> > 2Fmb-6.corp.oc.com%2C60020%2C1375466460755-splitting%2Fmb-6.corp.oc.com
>> > %252C60020%252C1375466460755.1375623021175
>> > ver = 0
>> > 2013-08-08 12:57:37,629 DEBUG
>> > org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
>> > /hbase/splitlog/hdfs%3A%2F%2Fmb-1.corp.oc.com%3A54310%2Fhbase%2F.logs%
>> > 2Fmb-3.corp.oc.com%2C60020%2C1375466447768-splitting%2Fmb-3.corp.oc.com
>> > %252C60020%252C1375466447768.1375630425141
>> > ver = 0
>> > 2013-08-08 12:57:37,629 DEBUG
>> > org.apache.hadoop.hbase.master.SplitLogManager: resubmitting unassigned
>> > task(s) after timeout
>> > 2013-08-08 12:57:37,629 DEBUG
>> > org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
>> > /hbase/splitlog/hdfs%3A%2F%2Fmb-1.corp.oc.com%3A54310%2Fhbase%2F.logs%
>> > 2Fmb-6.corp.oc.com%2C60020%2C1375466460755-splitting%2Fmb-6.corp.oc.com