Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> DFS respond very slow


+
Alexey 2012-10-09, 10:12
Copy link to this message
-
Re: DFS respond very slow
Additional info: I also tried to use openjdk instead of sun's - issue
still persists

On 10/09/12 03:12 AM, Alexey wrote:
> Hi,
>
> I have an issues with hadoop dfs, I have 3 servers (24Gb RAM on each).
> The servers are not overloaded, they just have hadoop installed. One
> have datanode and namenode, second - datanode only, third - datanode and
> secondarynamenode.
>
> Hadoop datanodes have a max memory limit 8Gb. Default replication factor
> - 2. Limited replication bandwidth between datanodes - 5Mb.
>
> I've setupped hadoop to communicate between nodes by IP address.
> Everything is works - I can read/write files on each datanode, etc. But
> the issue is that hadoop dfs commands are executing very slow, even
> "hadoop dfs -ls /" takes about 3 seconds to execute, but it have only
> one folder /user in it.
> Files are also uploading to the hdfs very slow - hundreds kilobytes/second.
>
> I'm using Debian stable x86-64 distribution and hadoop running through
> sun-java6-jdk 6.26-0squeeze1
>
> Please give me any suggestions what I need to adjust/check to arrange
> this issue.
>
> As I said before - overall hdfs configuration is correct, because
> everything works except performance.
>
> --
> Best regards
> Alexey
>

--
Best regards
Alexey
+
Harsh J 2012-10-10, 06:50
+
Alexey 2012-10-10, 06:54
+
Harsh J 2012-10-10, 06:56
+
Alexey 2012-10-10, 07:20
+
Vinod Kumar Vavilapalli 2012-10-16, 00:22
+
Vinod Kumar Vavilapalli 2012-10-16, 02:36
+
Andy Isaacson 2012-10-16, 01:56
+
Ted Dunning 2012-10-16, 02:23
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB