Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - number of map and reduce task does not change in M/R program


Copy link to this message
-
Re: number of map and reduce task does not change in M/R program
Dieter De Witte 2013-10-19, 10:00
Hi anesh,

It doesn't depend on the number of map tasks and since your reducer doesn't
start yet it doesn't depend on that as well. Maybe check the counters of
your jobs, is the number of map input records going up, if not then you're
stuck somewhere otherwise you might have a really big dataset :).. If you
want to provide more details, you can also put it on stackoverflow, it is
easier to look at your code then.

Regards, irW
2013/10/19 Anseh Danesh <[EMAIL PROTECTED]>

> Hi all.. I have a question.. I have a mapreduce program that get input
> from cassandra. my input is a little big, about 100000000 data. my problem
> is that my program takes too long to process, but I think mapreduce is good
> and fast for large volume of data. so I think maybe I have problems in
> number of map and reduce tasks.. I set the number of map and reduce asks
> with JobConf, with Job, and also in conf/mapred-site.xml, but I don't see
> any changes.. in my logs at first there is map 0% reduce 0% and after about
> 2 hours working it shows map 1% reduce 0%..!! what should I do? please Help
> me I really get confused...
>