Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Terasort problem


Copy link to this message
-
Re: Terasort problem
mapred.tasktracker.reduce.tasks.maximum and
mapred.tasktracker.map.tasks.maximum are configured in mapred-site.xml
They're cluster-wide.

Hadoop would sync configuation from name node to data nodes upon startup,
you don't need to configure for individual datanode.

"Too many fetch-failures..." error appeared in previous discussions and I
don't see definitive cause from them.

On Sat, Jul 10, 2010 at 4:29 AM, Tonci Buljan <[EMAIL PROTECTED]>wrote:

> Thank you for your response Owen. It is true, I haven't done that, figured
> that few hours after posting here.
>
> I'm having problems with understanding these variables:
>
> mapred.tasktracker.reduce.tasks.maximum <- Is this configured on every
> datanode separately? What number shall I put here?
>
> mapred.tasktracker.map.tasks.maximum <- same question  as
> mapred.tasktracker.reduce.tasks.maximum
>
> mapred.reduce.tasks <- Is this configured ONLY on Namenode and what value
> should it have for my 8 node cluster?
>
> mapred.map.tasks <- same question as mapred.reduce.tasks
>
>
> I've tried playing with these variables but getting error:"Too many
> fetch-failures..."
>
> Please, if anyone have any idea how to setup this the right way.
>
> Thank you.
>
> On 9 July 2010 15:33, Owen O'Malley <[EMAIL PROTECTED]> wrote:
>
> > I would guess that you didn't set the number of reducers for the job,
> > and it defaulted to 2.
> >
> > -- Owen
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB