Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Managing space in Master Node


Copy link to this message
-
Re: Managing space in Master Node
Mohammad Tariq 2013-02-13, 19:37
Hello Arko,

          Add the dfs.data.dir property in your hdfs-site.xml file and
point it to
some other location.

For logs, do the same thing by modifying the following line in
hadoop-env.sh file :

# Where log files are stored.  $HADOOP_HOME/logs by default.
export HADOOP_LOG_DIR=/hadoop/hdfs/logs

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Thu, Feb 14, 2013 at 1:02 AM, Arko Provo Mukherjee <
[EMAIL PROTECTED]> wrote:

> Hello Gurus,
>
> I am managing a Hadoop Cluster to run some experiments.
>
> The issue I am continuously facing is that the Master Node runs out of
> disk space due to logs and data files.
>
> I can monitor and delete log files. However, I cannot delete the HDFS data.
>
> Thus, is there a way to force Hadoop not to save any HDFS data in the
> Master Node?
>
> Then I can use my master to handle the metadata only and store the logs.
>
> Thanks & regards
> Arko
>