Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> One mapper/reducer runs on a single JVM


Copy link to this message
-
Re: One mapper/reducer runs on a single JVM
Thanks Michael,

"If you are running just Hadoop, you could have a little swap. Running
HBase, fuggit about it." -- could you give a bit more information about
what do you mean swap and why forget for HBase?

regards,
Lin
On Tue, Nov 6, 2012 at 12:46 PM, Michael Segel <[EMAIL PROTECTED]>wrote:

> Mappers and Reducers are separate JVM processes.
> And yes you need to take in to account the amount of memory the machine(s)
> when you configure the number of slots.
>
> If you are running just Hadoop, you could have a little swap. Running
> HBase, fuggit about it.
>
>
> On Nov 5, 2012, at 7:12 PM, Lin Ma <[EMAIL PROTECTED]> wrote:
>
> > Hello Hadoop experts,
> >
> > I have a question in my mind for a long time. Supposing I am developing
> M-R program, and it is Java based (Java UDF, implements mapper or reducer
> interface). My question is, in this scenario, whether a mapper or a reducer
> is a separate JVM process? E.g. supposing on a machine, there are 4
> mappers, they are 4 individual processes? I am also wondering whether the
> processes on a single machine will impact each other when each JVM wants to
> get more memory to run faster?
> >
> > thanks in advance,
> > Lin
> >
> >
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB