Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Hbase Takes Long time to Restart - Whats the correct way to restart Hbase cluster?


Copy link to this message
-
RE: Hbase Takes Long time to Restart - Whats the correct way to
Viral Bajaria 2013-01-15, 00:56
restart Hbase cluster?
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary=0015175cb4cee635fe04d348eb19

--0015175cb4cee635fe04d348eb19
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit

Is that one unassigned task even getting assigned or it errored out and
won't even run on retries ? I have seen this happen before when our
region server crashed. On restart, there are a few splitlog tasks that
never finish and error out. If you check the regionserver where the
task should run you will see that it is having trouble acquiring lease
on that file. Can you verify that ?

Viral
From: Ameya Kantikar
Sent: 1/14/2013 4:34 PM
To: [EMAIL PROTECTED]
Subject: Hbase Takes Long time to Restart - Whats the correct way to
restart Hbase cluster?
I restarted Hbase master, and its taking long time (at least 30 minutes) to
come back.

In master-status page I am seeing over 400 regions in transition.

In the hbase master log I am seeing following:
DEBUG org.apache.hadoop.hbase.master.SplitLogManager: total tasks = 1
unassigned = 1

(This is down from over 50 to 1 as in the log).

My setup is as follows:

I am running cdh4, hbase 0.92.1.

My question is, what is the right way to restart hbase-master machine? All
this time master is restarting, my cluster is unavailable.
>From hbase shell if I hit any table its giving error such as:

INFO ipc.HBaseRPC: Server at hbase1/XX.XX.XX.XX:60020 could not be reached
after 1 tries, giving up

Any ideas?

Ameya