Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Terasort problem


Copy link to this message
-
Re: Terasort problem
Ted Yu 2010-07-11, 15:55
mapred.tasktracker.reduce.tasks.maximum and
mapred.tasktracker.map.tasks.maximum are configured in mapred-site.xml
They're cluster-wide.

Hadoop would sync configuation from name node to data nodes upon startup,
you don't need to configure for individual datanode.

"Too many fetch-failures..." error appeared in previous discussions and I
don't see definitive cause from them.

On Sat, Jul 10, 2010 at 4:29 AM, Tonci Buljan <[EMAIL PROTECTED]>wrote:

> Thank you for your response Owen. It is true, I haven't done that, figured
> that few hours after posting here.
>
> I'm having problems with understanding these variables:
>
> mapred.tasktracker.reduce.tasks.maximum <- Is this configured on every
> datanode separately? What number shall I put here?
>
> mapred.tasktracker.map.tasks.maximum <- same question  as
> mapred.tasktracker.reduce.tasks.maximum
>
> mapred.reduce.tasks <- Is this configured ONLY on Namenode and what value
> should it have for my 8 node cluster?
>
> mapred.map.tasks <- same question as mapred.reduce.tasks
>
>
> I've tried playing with these variables but getting error:"Too many
> fetch-failures..."
>
> Please, if anyone have any idea how to setup this the right way.
>
> Thank you.
>
> On 9 July 2010 15:33, Owen O'Malley <[EMAIL PROTECTED]> wrote:
>
> > I would guess that you didn't set the number of reducers for the job,
> > and it defaulted to 2.
> >
> > -- Owen
> >
>