Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Distributing the code to multiple nodes


Copy link to this message
-
Distributing the code to multiple nodes
Hello All,

I have a 2 node hadoop cluster running with a replication factor of 2. I
have a file of size around 1 GB which when copied to HDFS is replicated to
both the nodes. Seeing the block info I can see the file has been
subdivided into 8 parts which means it has been subdivided into 8 blocks
each of size 128 MB.  I use this file as input to run the word count
program. Some how I feel only one node is doing all the work and the code
is not distributed to other node. How can I make sure code is distributed
to both the nodes? Also is there a log or GUI which can be used for this?
Please note I am using the latest stable release that is 2.2.0.

++Ashish
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB