HBase, mail # user - Re: always assinging root

always assinging root

jing wang 2012-12-07, 03:29

Hi there,

Something is wrong with our hbase cluster. In mater:60010 web ui, it
shows bellow after we restart the mater, using bin/hbase-daemon.sh start
master.
What's wrong ?
Recent tasksStart TimeDescriptionStateStatusFri Dec 07 10:06:57 CST 2012Master
startupRUNNING (since 1hrs, 15mins, 29sec ago)Assigning ROOT region (since
1hrs, 3mins, 41sec ago)
HBase Version0.90.6-cdh3u5, r
Hadoop Version0.20.2-cdh3u5, rde14a95e895a72e7b2501bbe628c1e23578aae29

Thanks,
Jing Wang

Re: always assinging root

ramkrishna vasudevan 2012-12-07, 03:33

Check out your master logs.. It may give us more clue.

Regards
Ram

On Fri, Dec 7, 2012 at 8:59 AM, jing wang <[EMAIL PROTECTED]>wrote:

> Hi there,
>
> Something is wrong with our hbase cluster. In mater:60010 web ui, it
> shows bellow after we restart the mater, using bin/hbase-daemon.sh start
> master.
> What's wrong ?
> Recent tasksStart TimeDescriptionStateStatusFri Dec 07 10:06:57 CST
> 2012Master
> startupRUNNING (since 1hrs, 15mins, 29sec ago)Assigning ROOT region (since
> 1hrs, 3mins, 41sec ago)
> HBase Version0.90.6-cdh3u5, r
> Hadoop Version0.20.2-cdh3u5, rde14a95e895a72e7b2501bbe628c1e23578aae29
>
> Thanks,
> Jing Wang
>

Re: always assinging root

jing wang 2012-12-07, 03:49

thanks for your reply.
here is the link for the full log since starting master:
http://pastebin.com/EjRvbk0g
ps:svr36 is master, others are regionserver

Thanks
Jing Wang
2012/12/7 ramkrishna vasudevan <[EMAIL PROTECTED]>

> Check out your master logs.. It may give us more clue.
>
> Regards
> Ram
>
> On Fri, Dec 7, 2012 at 8:59 AM, jing wang <[EMAIL PROTECTED]
> >wrote:
>
> > Hi there,
> >
> > Something is wrong with our hbase cluster. In mater:60010 web ui, it
> > shows bellow after we restart the mater, using bin/hbase-daemon.sh start
> > master.
> > What's wrong ?
> > Recent tasksStart TimeDescriptionStateStatusFri Dec 07 10:06:57 CST
> > 2012Master
> > startupRUNNING (since 1hrs, 15mins, 29sec ago)Assigning ROOT region
> (since
> > 1hrs, 3mins, 41sec ago)
> > HBase Version0.90.6-cdh3u5, r
> > Hadoop Version0.20.2-cdh3u5, rde14a95e895a72e7b2501bbe628c1e23578aae29
> >
> > Thanks,
> > Jing Wang
> >
>

Re: always assinging root

Yi Liang 2012-12-07, 05:16

Jing,

I think you could follow this kind of log to check rs log to find why it
could not open root region.

2012-12-07 10:48:48,708 INFO
org.apache.hadoop.hbase.master.AssignmentManager: Assigning region
-ROOT-,,0.70236052 to dn004,60020,1353922884530

Regards,
Yi
On Fri, Dec 7, 2012 at 11:49 AM, jing wang <[EMAIL PROTECTED]>wrote:

> thanks for your reply.
> here is the link for the full log since starting master:
> http://pastebin.com/EjRvbk0g
> ps:svr36 is master, others are regionserver
>
> Thanks
> Jing Wang
>
>
> 2012/12/7 ramkrishna vasudevan <[EMAIL PROTECTED]>
>
> > Check out your master logs.. It may give us more clue.
> >
> > Regards
> > Ram
> >
> > On Fri, Dec 7, 2012 at 8:59 AM, jing wang <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Hi there,
> > >
> > > Something is wrong with our hbase cluster. In mater:60010 web ui, it
> > > shows bellow after we restart the mater, using bin/hbase-daemon.sh
> start
> > > master.
> > > What's wrong ?
> > > Recent tasksStart TimeDescriptionStateStatusFri Dec 07 10:06:57 CST
> > > 2012Master
> > > startupRUNNING (since 1hrs, 15mins, 29sec ago)Assigning ROOT region
> > (since
> > > 1hrs, 3mins, 41sec ago)
> > > HBase Version0.90.6-cdh3u5, r
> > > Hadoop Version0.20.2-cdh3u5, rde14a95e895a72e7b2501bbe628c1e23578aae29
> > >
> > > Thanks,
> > > Jing Wang
> > >
> >
>

Re:Re: always assinging root

happygodwithwang 2012-12-07, 05:53

Yi,

Thanks for your reply.
following is the link for regionserver 'dn004' log :http://pastebin.com/224CKwsu
I can't find anything useful.

Thanks,
Jing Wang

At 2012-12-07 13:16:14,"Yi Liang" <[EMAIL PROTECTED]> wrote:
Jing,
I think you could follow this kind of log to check rs log to find why it could not open root region.
2012-12-07 10:48:48,708 INFO org.apache.hadoop.hbase.master.AssignmentManager: Assigning region -ROOT-,,0.70236052 to dn004,60020,1353922884530

Regards,
Yi

On Fri, Dec 7, 2012 at 11:49 AM, jing wang <[EMAIL PROTECTED]> wrote:
thanks for your reply.
here is the link for the full log since starting master:
http://pastebin.com/EjRvbk0g
ps:svr36 is master, others are regionserver

Thanks
Jing Wang

2012/12/7 ramkrishna vasudevan <[EMAIL PROTECTED]>

> Check out your master logs.. It may give us more clue.
>
> Regards
> Ram
>
> On Fri, Dec 7, 2012 at 8:59 AM, jing wang <[EMAIL PROTECTED]
> >wrote:
>
> > Hi there,
> >
> > Something is wrong with our hbase cluster. In mater:60010 web ui, it
> > shows bellow after we restart the mater, using bin/hbase-daemon.sh start
> > master.
> > What's wrong ?
> > Recent tasksStart TimeDescriptionStateStatusFri Dec 07 10:06:57 CST
> > 2012Master
> > startupRUNNING (since 1hrs, 15mins, 29sec ago)Assigning ROOT region
> (since
> > 1hrs, 3mins, 41sec ago)
> > HBase Version0.90.6-cdh3u5, r
> > Hadoop Version0.20.2-cdh3u5, rde14a95e895a72e7b2501bbe628c1e23578aae29
> >
> > Thanks,
> > Jing Wang
> >
>

Re: Re: always assinging root

Jimmy Xiang 2012-12-07, 18:24

That regionserver is stopping? If not, you may need to restart it.

2012-12-07 13:18:53,947 DEBUG
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler:
Processing open of -ROOT-,,0.70236052
2012-12-07 13:18:53,947 INFO
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Server
stopping or stopped, skipping open of -ROOT-,,0.70236052

Jimmy

On Thu, Dec 6, 2012 at 9:53 PM, happygodwithwang
<[EMAIL PROTECTED]> wrote:
> Yi,
>
> Thanks for your reply.
> following is the link for regionserver 'dn004' log :http://pastebin.com/224CKwsu
> I can't find anything useful.
>
>
>
> Thanks,
> Jing Wang
>
> At 2012-12-07 13:16:14,"Yi Liang" <[EMAIL PROTECTED]> wrote:
> Jing,
>
>
> I think you could follow this kind of log to check rs log to find why it could not open root region.
>
>
> 2012-12-07 10:48:48,708 INFO org.apache.hadoop.hbase.master.AssignmentManager: Assigning region -ROOT-,,0.70236052 to dn004,60020,1353922884530
>
> Regards,
> Yi
>
> On Fri, Dec 7, 2012 at 11:49 AM, jing wang <[EMAIL PROTECTED]> wrote:
> thanks for your reply.
> here is the link for the full log since starting master:
> http://pastebin.com/EjRvbk0g
> ps:svr36 is master, others are regionserver
>
> Thanks
> Jing Wang
>
>
>
> 2012/12/7 ramkrishna vasudevan <[EMAIL PROTECTED]>
>
>> Check out your master logs.. It may give us more clue.
>>
>> Regards
>> Ram
>>
>> On Fri, Dec 7, 2012 at 8:59 AM, jing wang <[EMAIL PROTECTED]
>> >wrote:
>>
>> > Hi there,
>> >
>> > Something is wrong with our hbase cluster. In mater:60010 web ui, it
>> > shows bellow after we restart the mater, using bin/hbase-daemon.sh start
>> > master.
>> > What's wrong ?
>> > Recent tasksStart TimeDescriptionStateStatusFri Dec 07 10:06:57 CST
>> > 2012Master
>> > startupRUNNING (since 1hrs, 15mins, 29sec ago)Assigning ROOT region
>> (since
>> > 1hrs, 3mins, 41sec ago)
>> > HBase Version0.90.6-cdh3u5, r
>> > Hadoop Version0.20.2-cdh3u5, rde14a95e895a72e7b2501bbe628c1e23578aae29
>> >
>> > Thanks,
>> > Jing Wang
>> >
>>
>
>

Re: Re: always assinging root

jing wang 2012-12-10, 09:29

Hi Jimmy,

Thanks for you reply. We solved the problem by bin/stop-hbase.sh and
bin/start-hbase.sh. It was a long time.
But we still didn't get the root cause.

2012/12/8 Jimmy Xiang <[EMAIL PROTECTED]>
>
> That regionserver is stopping? If not, you may need to restart it.
>
> 2012-12-07 13:18:53,947 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler:
> Processing open of -ROOT-,,0.70236052
> 2012-12-07 13:18:53,947 INFO
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Server
> stopping or stopped, skipping open of -ROOT-,,0.70236052
>
> Jimmy
>
> On Thu, Dec 6, 2012 at 9:53 PM, happygodwithwang
> <[EMAIL PROTECTED]> wrote:
> > Yi,
> >
> > Thanks for your reply.
> > following is the link for regionserver 'dn004' log :
http://pastebin.com/224CKwsu
> > I can't find anything useful.
> >
> >
> >
> > Thanks,
> > Jing Wang
> >
> > At 2012-12-07 13:16:14,"Yi Liang" <[EMAIL PROTECTED]> wrote:
> > Jing,
> >
> >
> > I think you could follow this kind of log to check rs log to find why
it could not open root region.
> >
> >
> > 2012-12-07 10:48:48,708 INFO
org.apache.hadoop.hbase.master.AssignmentManager: Assigning region
-ROOT-,,0.70236052 to dn004,60020,1353922884530
> >
> > Regards,
> > Yi
> >
> > On Fri, Dec 7, 2012 at 11:49 AM, jing wang <[EMAIL PROTECTED]>
wrote:
> > thanks for your reply.
> > here is the link for the full log since starting master:
> > http://pastebin.com/EjRvbk0g
> > ps:svr36 is master, others are regionserver
> >
> > Thanks
> > Jing Wang
> >
> >
> >
> > 2012/12/7 ramkrishna vasudevan <[EMAIL PROTECTED]>
> >
> >> Check out your master logs.. It may give us more clue.
> >>
> >> Regards
> >> Ram
> >>
> >> On Fri, Dec 7, 2012 at 8:59 AM, jing wang <[EMAIL PROTECTED]
> >> >wrote:
> >>
> >> > Hi there,
> >> >
> >> > Something is wrong with our hbase cluster. In mater:60010 web ui,
it
> >> > shows bellow after we restart the mater, using bin/hbase-daemon.sh
start
> >> > master.
> >> > What's wrong ?
> >> > Recent tasksStart TimeDescriptionStateStatusFri Dec 07 10:06:57 CST
> >> > 2012Master
> >> > startupRUNNING (since 1hrs, 15mins, 29sec ago)Assigning ROOT region
> >> (since
> >> > 1hrs, 3mins, 41sec ago)
> >> > HBase Version0.90.6-cdh3u5, r
> >> > Hadoop Version0.20.2-cdh3u5,
rde14a95e895a72e7b2501bbe628c1e23578aae29
> >> >
> >> > Thanks,
> >> > Jing Wang
> >> >
> >>
> >
> >