Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - how do i view the local file system output of a mapper on cygwin + windows?


Copy link to this message
-
how do i view the local file system output of a mapper on cygwin + windows?
Jane Wayne 2012-04-05, 20:55
i am currently testing my map reduce job on Windows + Cygwin + Hadoop
v0.20.205. for some strange reason, the list of values (i.e.
Iterable<T> values) going into the reducer looks all wrong. i have
tracked the map reduce process with logging statements (i.e. logged
the input to the map, logged the output from the map, logged the
partitioner, logged the input to the reducer). at all stages,
everything looks correct except at the reducer.

is there anyway (using Windows  + Cygwin) to view the local map
outputs before they are shuffled/sorted to the reducer? i need to know
why the values are incorrect.
+
Jane Wayne 2012-04-06, 00:56
+
Harsh J 2012-04-06, 02:50