Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # dev >> Adding Elasticity to Hadoop MapReduce


+
Bharath Ravi 2011-09-14, 20:27
+
Amandeep Khurana 2011-09-14, 21:18
Copy link to this message
-
Re: Adding Elasticity to Hadoop MapReduce
This makes a bit of sense, but you have to worry about the inertia of the
data.  Adding compute resources is easy.  Adding data resources, not so
much.  And if the computation is not near the data, then it is likely to be
much less effective.

On Wed, Sep 14, 2011 at 4:27 PM, Bharath Ravi <[EMAIL PROTECTED]>wrote:

> Hi all,
>
> I'm a newcomer to Hadoop development, and I'm planning to work on an idea
> that I wanted to run by the dev community.
>
> My apologies if this is not the right place to post this.
>
> Amazon has an "Elastic MapReduce" Service (
> http://aws.amazon.com/elasticmapreduce/) that runs on Hadoop.
> The service allows dynamic/runtime changes in resource allocation: more
> specifically, varying the number of
> compute nodes that a job is running on.
>
> I was wondering if such a facility could be added to the publicly available
> Hadoop MapReduce.
>
> Does this idea make sense, has any previous work been done on this?
> I'd appreciate it if someone could point me the right way to find out more!
>
> Thanks a lot in advance!
> --
> Bharath Ravi
>
+
Steve Loughran 2011-09-15, 09:23
+
Arun C Murthy 2011-09-14, 21:24
+
Bharath Ravi 2011-09-15, 01:01
+
Steve Loughran 2011-09-15, 09:15
+
Junping Du 2011-09-15, 09:14
+
Steve Loughran 2011-09-15, 09:26
+
Milind.Bhandarkar@... 2011-09-19, 19:37
+
Arun C Murthy 2011-09-19, 19:46