Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - Managing space in Master Node


+
Arko Provo Mukherjee 2013-02-13, 19:32
+
Mohammad Tariq 2013-02-13, 19:37
+
Charles Baker 2013-02-13, 19:38
+
Arko Provo Mukherjee 2013-02-13, 19:52
+
Mohammad Tariq 2013-02-13, 19:55
+
Arko Provo Mukherjee 2013-02-13, 20:01
+
Mohammad Tariq 2013-02-13, 20:06
+
Arko Provo Mukherjee 2013-02-13, 20:12
+
Charles Baker 2013-02-13, 20:21
+
Mohammad Tariq 2013-02-13, 20:14
+
Arko Provo Mukherjee 2013-02-13, 20:28
+
Arko Provo Mukherjee 2013-02-13, 20:31
+
Arko Provo Mukherjee 2013-02-13, 23:06
+
Mohammad Tariq 2013-02-13, 23:22
+
Arko Provo Mukherjee 2013-02-13, 23:32
+
Arko Provo Mukherjee 2013-02-13, 23:35
+
Mohammad Tariq 2013-02-13, 23:43
+
Arko Provo Mukherjee 2013-02-13, 23:48
+
Mohammad Tariq 2013-02-13, 23:51
Copy link to this message
-
Re: Managing space in Master Node
Arko Provo Mukherjee 2013-02-13, 23:54
Sure, thanks!
One last question before I go ahead and make the changes.
My hdfs-site.xml doesn't have an entry for dfs.replication.
So can I just assume that the system is using the default of 3?
I ask this because if it is 1, then I will first have to increase it
so as to not loose data when I stop the NN being a DN.
Thanks & regards
Arko

On Wed, Feb 13, 2013 at 5:51 PM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:
> Decommissioning is for removing DNs and not the NN.
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Thu, Feb 14, 2013 at 5:18 AM, Arko Provo Mukherjee
> <[EMAIL PROTECTED]> wrote:
>>
>> Hi,
>>
>> I was just wondering if it is okay to exclude the NN IP.
>>
>> I hope it won't create any "side effects".
>>
>> This is because I don't want to actually decommission the NN node, but
>> just make it stop working as a DataNode.
>>
>> However, if you are sure there are no problems, I will go ahead and
>> make the changes.
>>
>> Thanks & regards
>> Arko
>>
>> On Wed, Feb 13, 2013 at 5:43 PM, Mohammad Tariq <[EMAIL PROTECTED]>
>> wrote:
>> > It should.
>> >
>> > BTW what's the problem with decommissioning?It's the correct way of
>> > removing
>> > the DN, as said by Charles earlier. Create the exclude file and add the
>> > property in the hdfs-site.xml file.
>> >
>> > Warm Regards,
>> > Tariq
>> > https://mtariq.jux.com/
>> > cloudfront.blogspot.com
>> >
>> >
>> > On Thu, Feb 14, 2013 at 5:05 AM, Arko Provo Mukherjee
>> > <[EMAIL PROTECTED]> wrote:
>> >>
>> >> Hi,
>> >>
>> >> I see that the NameNode IP is present in conf/slaves file.
>> >>
>> >> Will removing the IP of the NN from the conf/slaves file and
>> >> restarting hadoop help?
>> >>
>> >> Thanks & regards
>> >> Arko
>> >>
>> >> On Wed, Feb 13, 2013 at 5:32 PM, Arko Provo Mukherjee
>> >> <[EMAIL PROTECTED]> wrote:
>> >> > Hi,
>> >> >
>> >> > I have 10 nodes. 1 Master Node (running all deamons) and 9 Slave
>> >> > Nodes
>> >> > (running DN and Task Tracker).
>> >> >
>> >> > My issue is that the my NN is getting filled up pretty fast as the NN
>> >> > acts as a DN.
>> >> >
>> >> > I want my NN not to function as the DN.
>> >> >
>> >> > Does including the IP for the NN in the "dfs.hosts.exclude" and
>> >> > running "$ hadoop dfsadmin -refreshNodes" solve my problem?
>> >> >
>> >> > Thanks & regards
>> >> > Arko
>> >> >
>> >> > On Wed, Feb 13, 2013 at 5:22 PM, Mohammad Tariq <[EMAIL PROTECTED]>
>> >> > wrote:
>> >> >> Stopping DN on one machine means excluding it. What exactly is the
>> >> >> requirement? How many nodes do you have right now?
>> >> >>
>> >> >> Warm Regards,
>> >> >> Tariq
>> >> >> https://mtariq.jux.com/
>> >> >> cloudfront.blogspot.com
>> >> >>
>> >> >>
>> >> >> On Thu, Feb 14, 2013 at 4:36 AM, Arko Provo Mukherjee
>> >> >> <[EMAIL PROTECTED]> wrote:
>> >> >>>
>> >> >>> Hi,
>> >> >>>
>> >> >>> I am not sure the provided link matches my situation.
>> >> >>>
>> >> >>> That is because, I am not trying to exclude any node but just
>> >> >>> trying
>> >> >>> to stop the NN act as a DataNode.
>> >> >>>
>> >> >>> Thanks & regards
>> >> >>> Arko
>> >> >>>
>> >> >>> On Wed, Feb 13, 2013 at 2:31 PM, Arko Provo Mukherjee
>> >> >>> <[EMAIL PROTECTED]> wrote:
>> >> >>> > Thanks! Let me study and find out if it works for me.
>> >> >>> > Thanks & regards
>> >> >>> > Arko
>> >> >>> >
>> >> >>> > On Wed, Feb 13, 2013 at 2:28 PM, Arko Provo Mukherjee
>> >> >>> > <[EMAIL PROTECTED]> wrote:
>> >> >>> >> Hi,
>> >> >>> >>
>> >> >>> >> But the new location will also be the same NN physical server.
>> >> >>> >> So how does that help?
>> >> >>> >>
>> >> >>> >> I would like to clear up the space in the master by removing all
>> >> >>> >> the
>> >> >>> >> HDFS data to the slave nodes.
>> >> >>> >>
>> >> >>> >> Thanks & regards
>> >> >>> >> Arko
>> >> >>> >>
>> >> >>> >> On Wed, Feb 13, 2013 at 2:14 PM, Mohammad Tariq
>> >> >>> >> <[EMAIL PROTECTED]>
>> >> >>> >> wrote:
>> >>
+
Mohammad Tariq 2013-02-14, 00:02
+
ramon.pin@... 2013-02-14, 09:31
+
Arko Provo Mukherjee 2013-02-14, 17:48
+
Arko Provo Mukherjee 2013-02-15, 00:21
+
Charles Baker 2013-02-15, 02:16