Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - number of reducers


+
jamal sasha 2012-11-20, 19:38
+
Bejoy KS 2012-11-20, 20:09
+
Kartashov, Andy 2012-11-20, 21:50
+
alxsss@... 2012-11-20, 22:00
+
jamal sasha 2012-11-20, 20:24
Copy link to this message
-
Re: number of reducers
Harsh J 2012-11-21, 04:08
Hey Jamal,

I'd recommend first going over the whole tutorial to get a good grip
on how Hadoop MR is designed to work:
http://hadoop.apache.org/docs/stable/mapred_tutorial.html

On Wed, Nov 21, 2012 at 1:08 AM, jamal sasha <[EMAIL PROTECTED]> wrote:
>
>
> Hi,
>
>   I wrote a simple map reduce job in hadoop streaming.
>
>
>
> I am wondering if I am doing something wrong ..
>
> While number of mappers are projected to be around 1700.. reducers.. just 1?
>
> It’s couple of TB’s worth of data.
>
> What can I do to address this.
>
> Basically mapper looks like this
>
>
>
> For line in sys.stdin:
>
>     Print line
>
>
>
> Reducer
>
> For line in sys.stdin:
>
>     New_line = process_line(line)
>
>     Print new_line
>
>
>
>
>
> Thanks
>
>

--
Harsh J