Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> getting splitmanager debug logs continuously


+
oc tsdb 2013-08-08, 20:36
+
Ted Yu 2013-08-08, 20:40
+
oc tsdb 2013-08-08, 21:41
Copy link to this message
-
Re: getting splitmanager debug logs continuously
Can you start the master as well (besides region servers)?
On Thu, Aug 8, 2013 at 2:41 PM, oc tsdb <[EMAIL PROTECTED]> wrote:

> I am using hbase-0.92
>
> Region server was not running on any of the nodes.
>
> Restarted the cluster. It started region server on all nodes except
> HMaster but still unresponsive.
>
> processes running on master are
> TSDMain
> HMaster
> SecondaryNameNode
> NameNode
> JobTracker
> HQuorumPeer
>
> processes running on all other nodes are
> DataNode
> TaskTracker
> RegionServer
> TSDMain
>
> This time, I see the error messages in the attached log.
>
> Could you please suggest if I can recover/restore the data and get the
> cluster up.
>
> Thanks & Regards,
> VSR
>
>
>
> On Thu, Aug 8, 2013 at 1:40 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> Can you tell us the version of HBase you're using ?
>>
>> Do you find something in region server logs on the 4 remaining nodes ?
>>
>> Cheers
>>
>> On Thu, Aug 8, 2013 at 1:36 PM, oc tsdb <[EMAIL PROTECTED]> wrote:
>>
>> > Hi,
>> >
>> > I am running a cluster with 6 nodes;
>> > Two of 6 nodes in my cluster went down (due to other application
>> failure)
>> > and came back after some time (had to do a power reboot).
>> > When these nodes are back I use to get "WARN
>> org.apache.hadoop.DFSClient:
>> > Failed to connect to , add to deadnodes and continue".
>> > Now these messages are stopped and getting continuous debug message as
>> > follows.
>> >
>> > 2013-08-08 12:57:36,628 DEBUG org.apache.hadoop.hbase.
>> > master.SplitLogManager: total tasks = 14 unassigned = 14
>> > 2013-08-08 12:57:37,628 DEBUG
>> > org.apache.hadoop.hbase.master.SplitLogManager: total tasks = 14
>> unassigned
>> > = 14
>> > 2013-08-08 12:57:37,629 DEBUG
>> > org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
>> > /hbase/splitlog/hdfs%3A%2F%2Fmb-1.corp.oc.com%3A54310%2Fhbase%2F.logs%
>> > 2Fmb-3.corp.oc.com%2C60020%2C1375466447768-splitting%2Fmb-3.corp.oc.com
>> > %252C60020%252C1375466447768.1375631802971
>> > ver = 0
>> > 2013-08-08 12:57:37,629 DEBUG
>> > org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
>> > /hbase/splitlog/hdfs%3A%2F%2Fmb-1.corp.oc.com%3A54310%2Fhbase%2F.logs%
>> > 2Fmb-6.corp.oc.com%2C60020%2C1375466460755-splitting%2Fmb-6.corp.oc.com
>> > %252C60020%252C1375466460755.1375623787557
>> > ver = 0
>> > 2013-08-08 12:57:37,629 DEBUG
>> > org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
>> > /hbase/splitlog/hdfs%3A%2F%2Fmb-1.corp.oc.com%3A54310%2Fhbase%2F.logs%
>> > 2Fmb-6.corp.oc.com%2C60020%2C1375466460755-splitting%2Fmb-6.corp.oc.com
>> > %252C60020%252C1375466460755.1375619231059
>> > ver = 3
>> > 2013-08-08 12:57:37,629 DEBUG
>> > org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
>> > /hbase/splitlog/hdfs%3A%2F%2Fmb-1.corp.oc.com%3A54310%2Fhbase%2F.logs%
>> > 2Fmb-2.corp.oc.com%2C60020%2C1375466479427-splitting%2Fmb-2.corp.oc.com
>> > %252C60020%252C1375466479427.1375639017535
>> > ver = 0
>> > 2013-08-08 12:57:37,629 DEBUG
>> > org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
>> > /hbase/splitlog/hdfs%3A%2F%2Fmb-1.corp.oc.com%3A54310%2Fhbase%2F.logs%
>> > 2Fmb-6.corp.oc.com%2C60020%2C1375466460755-splitting%2Fmb-6.corp.oc.com
>> > %252C60020%252C1375466460755.1375623021175
>> > ver = 0
>> > 2013-08-08 12:57:37,629 DEBUG
>> > org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
>> > /hbase/splitlog/hdfs%3A%2F%2Fmb-1.corp.oc.com%3A54310%2Fhbase%2F.logs%
>> > 2Fmb-3.corp.oc.com%2C60020%2C1375466447768-splitting%2Fmb-3.corp.oc.com
>> > %252C60020%252C1375466447768.1375630425141
>> > ver = 0
>> > 2013-08-08 12:57:37,629 DEBUG
>> > org.apache.hadoop.hbase.master.SplitLogManager: resubmitting unassigned
>> > task(s) after timeout
>> > 2013-08-08 12:57:37,629 DEBUG
>> > org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
>> > /hbase/splitlog/hdfs%3A%2F%2Fmb-1.corp.oc.com%3A54310%2Fhbase%2F.logs%
>> > 2Fmb-6.corp.oc.com%2C60020%2C1375466460755-splitting%2Fmb-6.corp.oc.com
+
oc tsdb 2013-08-08, 22:50
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB