Hello list,

       I have a pseudo distributed setup on my laptop. Everything was
working fine untill now. But lately HDFS has started taking a lot of time
to leave the safemode. Infact, I have to it manuaaly most of the times as
TT and Hbase daemons get disturbed because of this.

I am using hadoop-1.0.4. Is it a problem with this version? I have never
faced any such issue with older versions. Or, is something going wrong on
my side??

Thank you so much for your precious time.

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com

Hi Tariq,

I often have to force HDFS to go out of safe mode manually when I
restart my cluster (or after power outage).... I never tought about
reporting that ;)

I'm using hadoop-1.0.3. I think it was because of the MR files still
not replicated on enought nodes. But not 100% sure.

JM

2013/1/19, Mohammad Tariq <[EMAIL PROTECTED]>:
> Hello list,
>
>        I have a pseudo distributed setup on my laptop. Everything was
> working fine untill now. But lately HDFS has started taking a lot of time
> to leave the safemode. Infact, I have to it manuaaly most of the times as
> TT and Hbase daemons get disturbed because of this.
>
> I am using hadoop-1.0.4. Is it a problem with this version? I have never
> faced any such issue with older versions. Or, is something going wrong on
> my side??
>
> Thank you so much for your precious time.
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>

Hey Jean,

        Feels good to hear that ;) I don't have to feel
myself a solitary yonker anymore.

Since I am working on a single node, the problem
becomes more sever. I don't have any other node
where MR files could get replicated.

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Sun, Jan 20, 2013 at 5:08 AM, Jean-Marc Spaggiari <
[EMAIL PROTECTED]> wrote:

> Hi Tariq,
>
> I often have to force HDFS to go out of safe mode manually when I
> restart my cluster (or after power outage).... I never tought about
> reporting that ;)
>
> I'm using hadoop-1.0.3. I think it was because of the MR files still
> not replicated on enought nodes. But not 100% sure.
>
> JM
>
> 2013/1/19, Mohammad Tariq <[EMAIL PROTECTED]>:
> > Hello list,
> >
> >        I have a pseudo distributed setup on my laptop. Everything was
> > working fine untill now. But lately HDFS has started taking a lot of time
> > to leave the safemode. Infact, I have to it manuaaly most of the times as
> > TT and Hbase daemons get disturbed because of this.
> >
> > I am using hadoop-1.0.4. Is it a problem with this version? I have never
> > faced any such issue with older versions. Or, is something going wrong on
> > my side??
> >
> > Thank you so much for your precious time.
> >
> > Warm Regards,
> > Tariq
> > https://mtariq.jux.com/
> > cloudfront.blogspot.com
> >
>

On Sun, Jan 20, 2013 at 7:50 AM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:

> Hey Jean,
>
>         Feels good to hear that ;) I don't have to feel
> myself a solitary yonker anymore.
>
> Since I am working on a single node, the problem
> becomes more sever. I don't have any other node
> where MR files could get replicated.
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Sun, Jan 20, 2013 at 5:08 AM, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]> wrote:
>
>> Hi Tariq,
>>
>> I often have to force HDFS to go out of safe mode manually when I
>> restart my cluster (or after power outage).... I never tought about
>> reporting that ;)
>>
>> I'm using hadoop-1.0.3. I think it was because of the MR files still
>> not replicated on enought nodes. But not 100% sure.
>>
>> JM
>>
>> 2013/1/19, Mohammad Tariq <[EMAIL PROTECTED]>:
>> > Hello list,
>> >
>> >        I have a pseudo distributed setup on my laptop. Everything was
>> > working fine untill now. But lately HDFS has started taking a lot of
>> time
>> > to leave the safemode. Infact, I have to it manuaaly most of the times
>> as
>> > TT and Hbase daemons get disturbed because of this.
>> >
>> > I am using hadoop-1.0.4. Is it a problem with this version? I have never
>> > faced any such issue with older versions. Or, is something going wrong
>> on
>> > my side??
>> >
>> > Thank you so much for your precious time.
>> >
>> > Warm Regards,
>> > Tariq
>> > https://mtariq.jux.com/
>> > cloudfront.blogspot.com
>> >
>>
>
>

If your DN is starting too slow, then you should investigate why.

In any case, Apache Bigtop's (http://bigtop.apache.org) pseudo-distributed
configs provide good values for 1-node setups. In your case, you seem to be
missing dfs.safemode.min.datanodes set to 1, and dfs.safemode.extension set
to 0.
On Sun, Jan 20, 2013 at 4:05 AM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:

> Hello list,
>
>        I have a pseudo distributed setup on my laptop. Everything was
> working fine untill now. But lately HDFS has started taking a lot of time
> to leave the safemode. Infact, I have to it manuaaly most of the times as
> TT and Hbase daemons get disturbed because of this.
>
> I am using hadoop-1.0.4. Is it a problem with this version? I have never
> faced any such issue with older versions. Or, is something going wrong on
> my side??
>
> Thank you so much for your precious time.
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>

--
Harsh J

Hi Tariq,

When you start your namenode,Is it able to come out of Safemode
Automatically.

If no then there are under replicated blocks or corrupted blocks where
namenode is trying to fetch it.

Try to remove corrupted blocks.

Regards,
Varun Kumar.P

On Sun, Jan 20, 2013 at 4:05 AM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:

> Hello list,
>
>        I have a pseudo distributed setup on my laptop. Everything was
> working fine untill now. But lately HDFS has started taking a lot of time
> to leave the safemode. Infact, I have to it manuaaly most of the times as
> TT and Hbase daemons get disturbed because of this.
>
> I am using hadoop-1.0.4. Is it a problem with this version? I have never
> faced any such issue with older versions. Or, is something going wrong on
> my side??
>
> Thank you so much for your precious time.
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>

--
Regards,
Varun Kumar.P

Thank you so much for the valuable reply Harsh. I'll
look into it. One quick question, why it it happening
with 1.0.4? Is there any compulsion to set these two
props, you have specified above. Earlier version were
doing absolutely fine without these props?

I am Sorry to be a pest of questions. But, I am kinda
curious about this. Thank you so much.

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Sun, Jan 20, 2013 at 4:29 PM, Harsh J <[EMAIL PROTECTED]> wrote:

> If your DN is starting too slow, then you should investigate why.
>
> In any case, Apache Bigtop's (http://bigtop.apache.org)
> pseudo-distributed configs provide good values for 1-node setups. In your
> case, you seem to be missing dfs.safemode.min.datanodes set to 1,
> and dfs.safemode.extension set to 0.
>
>
> On Sun, Jan 20, 2013 at 4:05 AM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>
>> Hello list,
>>
>>        I have a pseudo distributed setup on my laptop. Everything was
>> working fine untill now. But lately HDFS has started taking a lot of time
>> to leave the safemode. Infact, I have to it manuaaly most of the times as
>> TT and Hbase daemons get disturbed because of this.
>>
>> I am using hadoop-1.0.4. Is it a problem with this version? I have never
>> faced any such issue with older versions. Or, is something going wrong on
>> my side??
>>
>> Thank you so much for your precious time.
>>
>> Warm Regards,
>> Tariq
>> https://mtariq.jux.com/
>> cloudfront.blogspot.com
>>
>
>
>
> --
> Harsh J
>

Hello Varun,

   Thank you so much for your reply. In most of the
cases, it is not. But apart from that everything seems
to be fine. I am not getting any notification about
under replicated blocks or corrupted blocks. I will do
a recheck though.

Thank you.

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Sun, Jan 20, 2013 at 5:43 PM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:

> Thank you so much for the valuable reply Harsh. I'll
> look into it. One quick question, why it it happening
> with 1.0.4? Is there any compulsion to set these two
> props, you have specified above. Earlier version were
> doing absolutely fine without these props?
>
> I am Sorry to be a pest of questions. But, I am kinda
> curious about this. Thank you so much.
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Sun, Jan 20, 2013 at 4:29 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> If your DN is starting too slow, then you should investigate why.
>>
>> In any case, Apache Bigtop's (http://bigtop.apache.org)
>> pseudo-distributed configs provide good values for 1-node setups. In your
>> case, you seem to be missing dfs.safemode.min.datanodes set to 1,
>> and dfs.safemode.extension set to 0.
>>
>>
>> On Sun, Jan 20, 2013 at 4:05 AM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>>
>>> Hello list,
>>>
>>>        I have a pseudo distributed setup on my laptop. Everything was
>>> working fine untill now. But lately HDFS has started taking a lot of time
>>> to leave the safemode. Infact, I have to it manuaaly most of the times as
>>> TT and Hbase daemons get disturbed because of this.
>>>
>>> I am using hadoop-1.0.4. Is it a problem with this version? I have never
>>> faced any such issue with older versions. Or, is something going wrong on
>>> my side??
>>>
>>> Thank you so much for your precious time.
>>>
>>> Warm Regards,
>>> Tariq
>>> https://mtariq.jux.com/
>>> cloudfront.blogspot.com
>>>
>>
>>
>>
>> --
>> Harsh J
>>
>
>

I am not aware of a direct regression in DN startup slowdown or block
report slowdown; its hard to tell what exactly the regression is without
more notes or logs on behavior.
On Sun, Jan 20, 2013 at 5:43 PM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:

> Thank you so much for the valuable reply Harsh. I'll
> look into it. One quick question, why it it happening
> with 1.0.4? Is there any compulsion to set these two
> props, you have specified above. Earlier version were
> doing absolutely fine without these props?
>
> I am Sorry to be a pest of questions. But, I am kinda
> curious about this. Thank you so much.
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Sun, Jan 20, 2013 at 4:29 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> If your DN is starting too slow, then you should investigate why.
>>
>> In any case, Apache Bigtop's (http://bigtop.apache.org)
>> pseudo-distributed configs provide good values for 1-node setups. In your
>> case, you seem to be missing dfs.safemode.min.datanodes set to 1,
>> and dfs.safemode.extension set to 0.
>>
>>
>> On Sun, Jan 20, 2013 at 4:05 AM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>>
>>> Hello list,
>>>
>>>        I have a pseudo distributed setup on my laptop. Everything was
>>> working fine untill now. But lately HDFS has started taking a lot of time
>>> to leave the safemode. Infact, I have to it manuaaly most of the times as
>>> TT and Hbase daemons get disturbed because of this.
>>>
>>> I am using hadoop-1.0.4. Is it a problem with this version? I have never
>>> faced any such issue with older versions. Or, is something going wrong on
>>> my side??
>>>
>>> Thank you so much for your precious time.
>>>
>>> Warm Regards,
>>> Tariq
>>> https://mtariq.jux.com/
>>> cloudfront.blogspot.com
>>>
>>
>>
>>
>> --
>> Harsh J
>>
>
>
--
Harsh J