Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - Managing space in Master Node


+
Arko Provo Mukherjee 2013-02-13, 19:32
+
Mohammad Tariq 2013-02-13, 19:37
+
Charles Baker 2013-02-13, 19:38
+
Arko Provo Mukherjee 2013-02-13, 19:52
+
Mohammad Tariq 2013-02-13, 19:55
+
Arko Provo Mukherjee 2013-02-13, 20:01
+
Mohammad Tariq 2013-02-13, 20:06
+
Arko Provo Mukherjee 2013-02-13, 20:12
+
Charles Baker 2013-02-13, 20:21
+
Mohammad Tariq 2013-02-13, 20:14
+
Arko Provo Mukherjee 2013-02-13, 20:28
+
Arko Provo Mukherjee 2013-02-13, 20:31
+
Arko Provo Mukherjee 2013-02-13, 23:06
+
Mohammad Tariq 2013-02-13, 23:22
+
Arko Provo Mukherjee 2013-02-13, 23:32
+
Arko Provo Mukherjee 2013-02-13, 23:35
+
Mohammad Tariq 2013-02-13, 23:43
Copy link to this message
-
Re: Managing space in Master Node
Arko Provo Mukherjee 2013-02-13, 23:48
Hi,

I was just wondering if it is okay to exclude the NN IP.

I hope it won't create any "side effects".

This is because I don't want to actually decommission the NN node, but
just make it stop working as a DataNode.

However, if you are sure there are no problems, I will go ahead and
make the changes.

Thanks & regards
Arko

On Wed, Feb 13, 2013 at 5:43 PM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:
> It should.
>
> BTW what's the problem with decommissioning?It's the correct way of removing
> the DN, as said by Charles earlier. Create the exclude file and add the
> property in the hdfs-site.xml file.
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Thu, Feb 14, 2013 at 5:05 AM, Arko Provo Mukherjee
> <[EMAIL PROTECTED]> wrote:
>>
>> Hi,
>>
>> I see that the NameNode IP is present in conf/slaves file.
>>
>> Will removing the IP of the NN from the conf/slaves file and
>> restarting hadoop help?
>>
>> Thanks & regards
>> Arko
>>
>> On Wed, Feb 13, 2013 at 5:32 PM, Arko Provo Mukherjee
>> <[EMAIL PROTECTED]> wrote:
>> > Hi,
>> >
>> > I have 10 nodes. 1 Master Node (running all deamons) and 9 Slave Nodes
>> > (running DN and Task Tracker).
>> >
>> > My issue is that the my NN is getting filled up pretty fast as the NN
>> > acts as a DN.
>> >
>> > I want my NN not to function as the DN.
>> >
>> > Does including the IP for the NN in the "dfs.hosts.exclude" and
>> > running "$ hadoop dfsadmin -refreshNodes" solve my problem?
>> >
>> > Thanks & regards
>> > Arko
>> >
>> > On Wed, Feb 13, 2013 at 5:22 PM, Mohammad Tariq <[EMAIL PROTECTED]>
>> > wrote:
>> >> Stopping DN on one machine means excluding it. What exactly is the
>> >> requirement? How many nodes do you have right now?
>> >>
>> >> Warm Regards,
>> >> Tariq
>> >> https://mtariq.jux.com/
>> >> cloudfront.blogspot.com
>> >>
>> >>
>> >> On Thu, Feb 14, 2013 at 4:36 AM, Arko Provo Mukherjee
>> >> <[EMAIL PROTECTED]> wrote:
>> >>>
>> >>> Hi,
>> >>>
>> >>> I am not sure the provided link matches my situation.
>> >>>
>> >>> That is because, I am not trying to exclude any node but just trying
>> >>> to stop the NN act as a DataNode.
>> >>>
>> >>> Thanks & regards
>> >>> Arko
>> >>>
>> >>> On Wed, Feb 13, 2013 at 2:31 PM, Arko Provo Mukherjee
>> >>> <[EMAIL PROTECTED]> wrote:
>> >>> > Thanks! Let me study and find out if it works for me.
>> >>> > Thanks & regards
>> >>> > Arko
>> >>> >
>> >>> > On Wed, Feb 13, 2013 at 2:28 PM, Arko Provo Mukherjee
>> >>> > <[EMAIL PROTECTED]> wrote:
>> >>> >> Hi,
>> >>> >>
>> >>> >> But the new location will also be the same NN physical server.
>> >>> >> So how does that help?
>> >>> >>
>> >>> >> I would like to clear up the space in the master by removing all
>> >>> >> the
>> >>> >> HDFS data to the slave nodes.
>> >>> >>
>> >>> >> Thanks & regards
>> >>> >> Arko
>> >>> >>
>> >>> >> On Wed, Feb 13, 2013 at 2:14 PM, Mohammad Tariq
>> >>> >> <[EMAIL PROTECTED]>
>> >>> >> wrote:
>> >>> >>> No, copy this directory to the new location of your choice.
>> >>> >>>
>> >>> >>> Warm Regards,
>> >>> >>> Tariq
>> >>> >>> https://mtariq.jux.com/
>> >>> >>> cloudfront.blogspot.com
>> >>> >>>
>> >>> >>>
>> >>> >>> On Thu, Feb 14, 2013 at 1:42 AM, Arko Provo Mukherjee
>> >>> >>> <[EMAIL PROTECTED]> wrote:
>> >>> >>>>
>> >>> >>>> Thanks!
>> >>> >>>> However, in that case, how do I handle the current data that is
>> >>> >>>> already in
>> >>> >>>> NN?
>> >>> >>>> Do I have to reformat HDFS?
>> >>> >>>> Thanks & regards
>> >>> >>>> Arko
>> >>> >>>>
>> >>> >>>> On Wed, Feb 13, 2013 at 2:06 PM, Mohammad Tariq
>> >>> >>>> <[EMAIL PROTECTED]>
>> >>> >>>> wrote:
>> >>> >>>> > Just don't run the DN daemon on that machine. Dedicate that
>> >>> >>>> > machine
>> >>> >>>> > to
>> >>> >>>> > NN.
>> >>> >>>> > Remove the hostname of the NN machine from the 'slaves' file.
>> >>> >>>> >
>> >>> >>>> > Warm Regards,
>> >>> >>>> > Tariq
>> >>> >>>> > https://mtariq.jux.com/
>> >
+
Mohammad Tariq 2013-02-13, 23:51
+
Arko Provo Mukherjee 2013-02-13, 23:54
+
Mohammad Tariq 2013-02-14, 00:02
+
ramon.pin@... 2013-02-14, 09:31
+
Arko Provo Mukherjee 2013-02-14, 17:48
+
Arko Provo Mukherjee 2013-02-15, 00:21
+
Charles Baker 2013-02-15, 02:16