Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> Shuffling over the network for local map data.


Copy link to this message
-
Re: Shuffling over the network for local map data.
I have a patch that tries to use file links instead of making a copy of the
data that is already available locally. I tested it on the a single machine
cluster configuration running 48 mappers and reducers. I unfortunately do
not have access to a cluster even a small one. Can some on review and test
run my patch ?

I created the patch using Eclipse against 1.0.3. My knowledge in Java in
limited and the code is not well written in some classes. So please let me
know if I need to make changes to the code along with a short explanation
of the change.  I will happily do so.

Thanks,
Suresh.