Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Managing space in Master Node


+
Arko Provo Mukherjee 2013-02-13, 19:32
+
Mohammad Tariq 2013-02-13, 19:37
+
Charles Baker 2013-02-13, 19:38
+
Arko Provo Mukherjee 2013-02-13, 19:52
+
Mohammad Tariq 2013-02-13, 19:55
+
Arko Provo Mukherjee 2013-02-13, 20:01
+
Mohammad Tariq 2013-02-13, 20:06
+
Arko Provo Mukherjee 2013-02-13, 20:12
+
Charles Baker 2013-02-13, 20:21
+
Mohammad Tariq 2013-02-13, 20:14
+
Arko Provo Mukherjee 2013-02-13, 20:28
+
Arko Provo Mukherjee 2013-02-13, 20:31
+
Arko Provo Mukherjee 2013-02-13, 23:06
+
Mohammad Tariq 2013-02-13, 23:22
+
Arko Provo Mukherjee 2013-02-13, 23:32
+
Arko Provo Mukherjee 2013-02-13, 23:35
+
Mohammad Tariq 2013-02-13, 23:43
+
Arko Provo Mukherjee 2013-02-13, 23:48
Copy link to this message
-
Re: Managing space in Master Node
Decommissioning is for removing DNs and not the NN.

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Thu, Feb 14, 2013 at 5:18 AM, Arko Provo Mukherjee <
[EMAIL PROTECTED]> wrote:

> Hi,
>
> I was just wondering if it is okay to exclude the NN IP.
>
> I hope it won't create any "side effects".
>
> This is because I don't want to actually decommission the NN node, but
> just make it stop working as a DataNode.
>
> However, if you are sure there are no problems, I will go ahead and
> make the changes.
>
> Thanks & regards
> Arko
>
> On Wed, Feb 13, 2013 at 5:43 PM, Mohammad Tariq <[EMAIL PROTECTED]>
> wrote:
> > It should.
> >
> > BTW what's the problem with decommissioning?It's the correct way of
> removing
> > the DN, as said by Charles earlier. Create the exclude file and add the
> > property in the hdfs-site.xml file.
> >
> > Warm Regards,
> > Tariq
> > https://mtariq.jux.com/
> > cloudfront.blogspot.com
> >
> >
> > On Thu, Feb 14, 2013 at 5:05 AM, Arko Provo Mukherjee
> > <[EMAIL PROTECTED]> wrote:
> >>
> >> Hi,
> >>
> >> I see that the NameNode IP is present in conf/slaves file.
> >>
> >> Will removing the IP of the NN from the conf/slaves file and
> >> restarting hadoop help?
> >>
> >> Thanks & regards
> >> Arko
> >>
> >> On Wed, Feb 13, 2013 at 5:32 PM, Arko Provo Mukherjee
> >> <[EMAIL PROTECTED]> wrote:
> >> > Hi,
> >> >
> >> > I have 10 nodes. 1 Master Node (running all deamons) and 9 Slave Nodes
> >> > (running DN and Task Tracker).
> >> >
> >> > My issue is that the my NN is getting filled up pretty fast as the NN
> >> > acts as a DN.
> >> >
> >> > I want my NN not to function as the DN.
> >> >
> >> > Does including the IP for the NN in the "dfs.hosts.exclude" and
> >> > running "$ hadoop dfsadmin -refreshNodes" solve my problem?
> >> >
> >> > Thanks & regards
> >> > Arko
> >> >
> >> > On Wed, Feb 13, 2013 at 5:22 PM, Mohammad Tariq <[EMAIL PROTECTED]>
> >> > wrote:
> >> >> Stopping DN on one machine means excluding it. What exactly is the
> >> >> requirement? How many nodes do you have right now?
> >> >>
> >> >> Warm Regards,
> >> >> Tariq
> >> >> https://mtariq.jux.com/
> >> >> cloudfront.blogspot.com
> >> >>
> >> >>
> >> >> On Thu, Feb 14, 2013 at 4:36 AM, Arko Provo Mukherjee
> >> >> <[EMAIL PROTECTED]> wrote:
> >> >>>
> >> >>> Hi,
> >> >>>
> >> >>> I am not sure the provided link matches my situation.
> >> >>>
> >> >>> That is because, I am not trying to exclude any node but just trying
> >> >>> to stop the NN act as a DataNode.
> >> >>>
> >> >>> Thanks & regards
> >> >>> Arko
> >> >>>
> >> >>> On Wed, Feb 13, 2013 at 2:31 PM, Arko Provo Mukherjee
> >> >>> <[EMAIL PROTECTED]> wrote:
> >> >>> > Thanks! Let me study and find out if it works for me.
> >> >>> > Thanks & regards
> >> >>> > Arko
> >> >>> >
> >> >>> > On Wed, Feb 13, 2013 at 2:28 PM, Arko Provo Mukherjee
> >> >>> > <[EMAIL PROTECTED]> wrote:
> >> >>> >> Hi,
> >> >>> >>
> >> >>> >> But the new location will also be the same NN physical server.
> >> >>> >> So how does that help?
> >> >>> >>
> >> >>> >> I would like to clear up the space in the master by removing all
> >> >>> >> the
> >> >>> >> HDFS data to the slave nodes.
> >> >>> >>
> >> >>> >> Thanks & regards
> >> >>> >> Arko
> >> >>> >>
> >> >>> >> On Wed, Feb 13, 2013 at 2:14 PM, Mohammad Tariq
> >> >>> >> <[EMAIL PROTECTED]>
> >> >>> >> wrote:
> >> >>> >>> No, copy this directory to the new location of your choice.
> >> >>> >>>
> >> >>> >>> Warm Regards,
> >> >>> >>> Tariq
> >> >>> >>> https://mtariq.jux.com/
> >> >>> >>> cloudfront.blogspot.com
> >> >>> >>>
> >> >>> >>>
> >> >>> >>> On Thu, Feb 14, 2013 at 1:42 AM, Arko Provo Mukherjee
> >> >>> >>> <[EMAIL PROTECTED]> wrote:
> >> >>> >>>>
> >> >>> >>>> Thanks!
> >> >>> >>>> However, in that case, how do I handle the current data that is
> >> >>> >>>> already in
> >> >>> >>>> NN?
> >> >>> >>>> Do I have to reformat HDFS?
> >> >>> >>>> Thanks & regards
+
Arko Provo Mukherjee 2013-02-13, 23:54
+
Mohammad Tariq 2013-02-14, 00:02
+
ramon.pin@... 2013-02-14, 09:31
+
Arko Provo Mukherjee 2013-02-14, 17:48
+
Arko Provo Mukherjee 2013-02-15, 00:21
+
Charles Baker 2013-02-15, 02:16
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB