Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> number of reducers

Copy link to this message
number of reducers

  I wrote a simple map reduce job in hadoop streaming.

I am wondering if I am doing something wrong ..

While number of mappers are projected to be around 1700.. reducers.. just 1?

It’s couple of TB’s worth of data.

What can I do to address this.

Basically mapper looks like this

For line in sys.stdin:

    Print line


For line in sys.stdin:

    New_line = process_line(line)

    Print new_line

Bejoy KS 2012-11-20, 20:09
Kartashov, Andy 2012-11-20, 21:50
alxsss@... 2012-11-20, 22:00
jamal sasha 2012-11-20, 20:24
Harsh J 2012-11-21, 04:08