Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Hadoop overhead


Copy link to this message
-
Re: Hadoop overhead
If its too many short duration jobs, you might want to keep an eye on
jobtracker and tweak number of heartbeats processed per second &
outofbandheartbeat option. JobTracker might be bombarded with events
otherwise.

On Thu, Apr 8, 2010 at 8:07 PM, Jeff Zhang <[EMAIL PROTECTED]> wrote:

> By default, for each task hadoop will create a new jvm process which will
> be
> the major cost in my opinion. You can customize configuration to let
> tasktracker reuse the jvm to eliminate the overhead to some extend.
>
> On Thu, Apr 8, 2010 at 8:55 PM, Aleksandar Stupar <
> [EMAIL PROTECTED]> wrote:
>
> > Hi all,
> >
> > As I realize hadoop is mainly used for tasks that take long
> > time to execute. I'm considering to use hadoop for task
> > whose lower bound in distributed execution is like 5 to 10
> > seconds. Am wondering what would the overhead be with
> > using hadoop.
> >
> > Does anyone have an idea? Any link where I can find this out?
> >
> > Thanks,
> > Aleksandar.
> >
> >
> >
>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

--
~Rajesh.B
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB