Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Managing space in Master Node


Copy link to this message
-
Re: Managing space in Master Node
Stopping DN on one machine means excluding it. What exactly is the
requirement? How many nodes do you have right now?

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Thu, Feb 14, 2013 at 4:36 AM, Arko Provo Mukherjee <
[EMAIL PROTECTED]> wrote:

> Hi,
>
> I am not sure the provided link matches my situation.
>
> That is because, I am not trying to exclude any node but just trying
> to stop the NN act as a DataNode.
>
> Thanks & regards
> Arko
>
> On Wed, Feb 13, 2013 at 2:31 PM, Arko Provo Mukherjee
> <[EMAIL PROTECTED]> wrote:
> > Thanks! Let me study and find out if it works for me.
> > Thanks & regards
> > Arko
> >
> > On Wed, Feb 13, 2013 at 2:28 PM, Arko Provo Mukherjee
> > <[EMAIL PROTECTED]> wrote:
> >> Hi,
> >>
> >> But the new location will also be the same NN physical server.
> >> So how does that help?
> >>
> >> I would like to clear up the space in the master by removing all the
> >> HDFS data to the slave nodes.
> >>
> >> Thanks & regards
> >> Arko
> >>
> >> On Wed, Feb 13, 2013 at 2:14 PM, Mohammad Tariq <[EMAIL PROTECTED]>
> wrote:
> >>> No, copy this directory to the new location of your choice.
> >>>
> >>> Warm Regards,
> >>> Tariq
> >>> https://mtariq.jux.com/
> >>> cloudfront.blogspot.com
> >>>
> >>>
> >>> On Thu, Feb 14, 2013 at 1:42 AM, Arko Provo Mukherjee
> >>> <[EMAIL PROTECTED]> wrote:
> >>>>
> >>>> Thanks!
> >>>> However, in that case, how do I handle the current data that is
> already in
> >>>> NN?
> >>>> Do I have to reformat HDFS?
> >>>> Thanks & regards
> >>>> Arko
> >>>>
> >>>> On Wed, Feb 13, 2013 at 2:06 PM, Mohammad Tariq <[EMAIL PROTECTED]>
> >>>> wrote:
> >>>> > Just don't run the DN daemon on that machine. Dedicate that machine
> to
> >>>> > NN.
> >>>> > Remove the hostname of the NN machine from the 'slaves' file.
> >>>> >
> >>>> > Warm Regards,
> >>>> > Tariq
> >>>> > https://mtariq.jux.com/
> >>>> > cloudfront.blogspot.com
> >>>> >
> >>>> >
> >>>> > On Thu, Feb 14, 2013 at 1:31 AM, Arko Provo Mukherjee
> >>>> > <[EMAIL PROTECTED]> wrote:
> >>>> >>
> >>>> >> Hi,
> >>>> >>
> >>>> >> Thanks for the help!
> >>>> >>
> >>>> >> However, I am still unsure about how to "turn off" the datanode
> >>>> >> feature on the NN.
> >>>> >>
> >>>> >> I checked the "hdfs-site.xml" file and the dfs.data.dir is pointed
> to
> >>>> >> a directory.
> >>>> >>
> >>>> >> Should I just comment out that property? What would happen to the
> >>>> >> current data that there is in the Master? Will it get removed
> >>>> >> automatically?
> >>>> >>
> >>>> >> Thanks & regards
> >>>> >> Arko
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >> On Wed, Feb 13, 2013 at 1:55 PM, Mohammad Tariq <
> [EMAIL PROTECTED]>
> >>>> >> wrote:
> >>>> >> > You can specify the logging level as specified by Charles. But
> >>>> >> > turning
> >>>> >> > logs
> >>>> >> > off is never a good idea. Logs are really helpful in problem
> >>>> >> > diagnosis,
> >>>> >> > which are eventual.
> >>>> >> >
> >>>> >> > Warm Regards,
> >>>> >> > Tariq
> >>>> >> > https://mtariq.jux.com/
> >>>> >> > cloudfront.blogspot.com
> >>>> >> >
> >>>> >> >
> >>>> >> > On Thu, Feb 14, 2013 at 1:22 AM, Arko Provo Mukherjee
> >>>> >> > <[EMAIL PROTECTED]> wrote:
> >>>> >> >>
> >>>> >> >> Hi,
> >>>> >> >>
> >>>> >> >> Yeah, my NameNode is also seconding as a DataNode.
> >>>> >> >>
> >>>> >> >> I would like to "turn off" this feature.
> >>>> >> >>
> >>>> >> >> Request help regarding the same.
> >>>> >> >>
> >>>> >> >> Thanks & regards
> >>>> >> >> Arko
> >>>> >> >>
> >>>> >> >> On Wed, Feb 13, 2013 at 1:38 PM, Charles Baker <[EMAIL PROTECTED]>
> >>>> >> >> wrote:
> >>>> >> >> > Hi Arko. Sounds like you may be running a DataNode on the
> NameNode
> >>>> >> >> > which
> >>>> >> >> > is
> >>>> >> >> > not recommended practice. Normally, the only files the NN
> stores
> >>>> >> >> > are
> >>>> >> >> > the
> >>>> >> >> > image and edits files. It does not store any actual HDFS
> data. If
> >>>> >>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB