Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - Managing space in Master Node


+
Arko Provo Mukherjee 2013-02-13, 19:32
+
Mohammad Tariq 2013-02-13, 19:37
+
Charles Baker 2013-02-13, 19:38
+
Arko Provo Mukherjee 2013-02-13, 19:52
+
Mohammad Tariq 2013-02-13, 19:55
+
Arko Provo Mukherjee 2013-02-13, 20:01
+
Mohammad Tariq 2013-02-13, 20:06
+
Arko Provo Mukherjee 2013-02-13, 20:12
+
Charles Baker 2013-02-13, 20:21
+
Mohammad Tariq 2013-02-13, 20:14
+
Arko Provo Mukherjee 2013-02-13, 20:28
+
Arko Provo Mukherjee 2013-02-13, 20:31
+
Arko Provo Mukherjee 2013-02-13, 23:06
+
Mohammad Tariq 2013-02-13, 23:22
+
Arko Provo Mukherjee 2013-02-13, 23:32
+
Arko Provo Mukherjee 2013-02-13, 23:35
Copy link to this message
-
Re: Managing space in Master Node
Mohammad Tariq 2013-02-13, 23:43
It should.

BTW what's the problem with decommissioning?It's the correct way of
removing the DN, as said by Charles earlier. Create the exclude file and
add the property in the hdfs-site.xml file.

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Thu, Feb 14, 2013 at 5:05 AM, Arko Provo Mukherjee <
[EMAIL PROTECTED]> wrote:

> Hi,
>
> I see that the NameNode IP is present in conf/slaves file.
>
> Will removing the IP of the NN from the conf/slaves file and
> restarting hadoop help?
>
> Thanks & regards
> Arko
>
> On Wed, Feb 13, 2013 at 5:32 PM, Arko Provo Mukherjee
> <[EMAIL PROTECTED]> wrote:
> > Hi,
> >
> > I have 10 nodes. 1 Master Node (running all deamons) and 9 Slave Nodes
> > (running DN and Task Tracker).
> >
> > My issue is that the my NN is getting filled up pretty fast as the NN
> > acts as a DN.
> >
> > I want my NN not to function as the DN.
> >
> > Does including the IP for the NN in the "dfs.hosts.exclude" and
> > running "$ hadoop dfsadmin -refreshNodes" solve my problem?
> >
> > Thanks & regards
> > Arko
> >
> > On Wed, Feb 13, 2013 at 5:22 PM, Mohammad Tariq <[EMAIL PROTECTED]>
> wrote:
> >> Stopping DN on one machine means excluding it. What exactly is the
> >> requirement? How many nodes do you have right now?
> >>
> >> Warm Regards,
> >> Tariq
> >> https://mtariq.jux.com/
> >> cloudfront.blogspot.com
> >>
> >>
> >> On Thu, Feb 14, 2013 at 4:36 AM, Arko Provo Mukherjee
> >> <[EMAIL PROTECTED]> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I am not sure the provided link matches my situation.
> >>>
> >>> That is because, I am not trying to exclude any node but just trying
> >>> to stop the NN act as a DataNode.
> >>>
> >>> Thanks & regards
> >>> Arko
> >>>
> >>> On Wed, Feb 13, 2013 at 2:31 PM, Arko Provo Mukherjee
> >>> <[EMAIL PROTECTED]> wrote:
> >>> > Thanks! Let me study and find out if it works for me.
> >>> > Thanks & regards
> >>> > Arko
> >>> >
> >>> > On Wed, Feb 13, 2013 at 2:28 PM, Arko Provo Mukherjee
> >>> > <[EMAIL PROTECTED]> wrote:
> >>> >> Hi,
> >>> >>
> >>> >> But the new location will also be the same NN physical server.
> >>> >> So how does that help?
> >>> >>
> >>> >> I would like to clear up the space in the master by removing all the
> >>> >> HDFS data to the slave nodes.
> >>> >>
> >>> >> Thanks & regards
> >>> >> Arko
> >>> >>
> >>> >> On Wed, Feb 13, 2013 at 2:14 PM, Mohammad Tariq <[EMAIL PROTECTED]
> >
> >>> >> wrote:
> >>> >>> No, copy this directory to the new location of your choice.
> >>> >>>
> >>> >>> Warm Regards,
> >>> >>> Tariq
> >>> >>> https://mtariq.jux.com/
> >>> >>> cloudfront.blogspot.com
> >>> >>>
> >>> >>>
> >>> >>> On Thu, Feb 14, 2013 at 1:42 AM, Arko Provo Mukherjee
> >>> >>> <[EMAIL PROTECTED]> wrote:
> >>> >>>>
> >>> >>>> Thanks!
> >>> >>>> However, in that case, how do I handle the current data that is
> >>> >>>> already in
> >>> >>>> NN?
> >>> >>>> Do I have to reformat HDFS?
> >>> >>>> Thanks & regards
> >>> >>>> Arko
> >>> >>>>
> >>> >>>> On Wed, Feb 13, 2013 at 2:06 PM, Mohammad Tariq <
> [EMAIL PROTECTED]>
> >>> >>>> wrote:
> >>> >>>> > Just don't run the DN daemon on that machine. Dedicate that
> machine
> >>> >>>> > to
> >>> >>>> > NN.
> >>> >>>> > Remove the hostname of the NN machine from the 'slaves' file.
> >>> >>>> >
> >>> >>>> > Warm Regards,
> >>> >>>> > Tariq
> >>> >>>> > https://mtariq.jux.com/
> >>> >>>> > cloudfront.blogspot.com
> >>> >>>> >
> >>> >>>> >
> >>> >>>> > On Thu, Feb 14, 2013 at 1:31 AM, Arko Provo Mukherjee
> >>> >>>> > <[EMAIL PROTECTED]> wrote:
> >>> >>>> >>
> >>> >>>> >> Hi,
> >>> >>>> >>
> >>> >>>> >> Thanks for the help!
> >>> >>>> >>
> >>> >>>> >> However, I am still unsure about how to "turn off" the datanode
> >>> >>>> >> feature on the NN.
> >>> >>>> >>
> >>> >>>> >> I checked the "hdfs-site.xml" file and the dfs.data.dir is
> pointed
> >>> >>>> >> to
> >>> >>>> >> a directory.
> >>> >>>> >>
> >>> >>>> >> Should I just comment out that property? What would happen to
+
Arko Provo Mukherjee 2013-02-13, 23:48
+
Mohammad Tariq 2013-02-13, 23:51
+
Arko Provo Mukherjee 2013-02-13, 23:54
+
Mohammad Tariq 2013-02-14, 00:02
+
ramon.pin@... 2013-02-14, 09:31
+
Arko Provo Mukherjee 2013-02-14, 17:48
+
Arko Provo Mukherjee 2013-02-15, 00:21
+
Charles Baker 2013-02-15, 02:16