Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Why Hadoop is slow in Cloud


Copy link to this message
-
Re: Why Hadoop is slow in Cloud
On Wed, Jan 19, 2011 at 1:32 PM, Marc Farnum Rendino <[EMAIL PROTECTED]> wrote:
> On Tue, Jan 18, 2011 at 8:59 AM, Adarsh Sharma <[EMAIL PROTECTED]> wrote:
>> I want to know *AT WHAT COSTS  *it comes.
>> 10-15% is tolerable but at this rate, it needs some work.
>>
>> As Steve rightly suggest , I am in some CPU bound testing work to  know the
>>  exact stats.
>
> Yep; you've got to test your own workflow to see how it's affected by
> your conditions - lots of variables.
>
> BTW: For AWS (Amazon) there are significant differences in I/O, for
> different instance types; if I recall correctly, for best I/O, start
> no lower than m1.large. And the three storage types (instance, EBS,
> and S3) have different characteristics as well; I'd start with EBS,
> though I haven't worked much with S3 yet, and that does offer some
> benefits.
>
As for virtualization,paravirtualization,emulation.....(whatever ulization)
There are always a lot of variables, but the net result is always
less. It may be 2% 10% or 15%, but it is always less. A $50,000 server
and such a solution takes 10% performance right off the top. There
goes $5,000.00 performance right out the window. I never think
throwing away performance was acceptable ( I was born without a silver
SSD in my crib).  Plus some people even pay for virtualization
software (vendors will remain nameless) Truly paying for less.