Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - HDFS DataNode daily log growing really high and fast


+
Ronen Itkin 2011-10-30, 09:02
+
Ronen Itkin 2011-10-31, 05:59
Copy link to this message
-
Re: HDFS DataNode daily log growing really high and fast
Brock Noland 2011-10-31, 12:27
Hi,

On Mon, Oct 31, 2011 at 12:59 AM, Ronen Itkin <[EMAIL PROTECTED]> wrote:
> For instance, yesterday's daily log:
> "/var/log/hadoop/hadoop-hadoop-datanode-ip-10-10-10-4.log"
> on the problematic Node03 was in the size of 1.1 GB while on other Nodes
> the same log was in the size of 87 MB.
>
> Again, nothing is being run specifically on Node03, I have 3 nodes, with
> replication of 3 - means that all the data is being saved on every node,
> All nodes are connected to the same switch (and on the same subnet) - so no
> advantages to Node03 in any Job.
>
> I am being suspicious regarding HBase...
Does that servers regionserver have more regions assigned to it?
Check the HBase GUI.

Also, you can turn that message off with:

log4j.logger.org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace=WARN
Brock
+
Ronen Itkin 2011-11-01, 12:50