Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Hosting Hadoop


Copy link to this message
-
Re: Hosting Hadoop
Alex, did you run into funky issues with EC2/EMR? The kind of issues that would come up because its a virtualized environment? We currently own our hardware and are just trying to do an ROI analysis on whether moving to Amazon can reduce our admin costs. Currently administering a Hadoop cluster is a bit expensive (in terms of man hours spent trying to replace disks and so on) and we are exploring whether its possible to avoid some of those costs
 
Regards,
Dhaval
________________________________
 From: alex bohr <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Cc: Dhaval Shah <[EMAIL PROTECTED]>
Sent: Monday, 12 August 2013 1:41 PM
Subject: Re: Hosting Hadoop
 
I've had good experience running a large hadoop cluster on EC2 instances.  After almost 1 year we haven't had any significant down time, just lost a small # of data nodes.  
I don't think EMR is an ideal solution if your cluster will be running 24/7.

But for running a large cluster, I don't see how you it's more cost efficient to run in the cloud than to own the hardware and we're trying to move off the cloud onto our own hardware.  Can I ask why you're looking to move to the cloud?

On Fri, Aug 9, 2013 at 10:42 AM, Nitin Pawar <[EMAIL PROTECTED]> wrote:

check altiscale as well
>
>
>
>On Fri, Aug 9, 2013 at 3:05 AM, Dhaval Shah <[EMAIL PROTECTED]> wrote:
>
>Thanks for the list Marcos. I will go through the slides/links. I think that's helpful
>> 
>>Regards,
>>Dhaval
>>
>>
>>
>>________________________________
>> From: Marcos Luis Ortiz Valmaseda <[EMAIL PROTECTED]>
>>To: Dhaval Shah <[EMAIL PROTECTED]>
>>Cc: [EMAIL PROTECTED]
>>Sent: Thursday, 8 August 2013 4:50 PM
>>Subject: Re: Hosting Hadoop
>>
>>
>>Well, all depends, because many companies use Cloud Computing
>>platforms like Amazon EMR. Vmware, Rackscpace Cloud for Hadoop
>>hosting:
>>http://aws.amazon.com/elasticmapreduce
>>http://www.vmware.com/company/news/releases/vmw-mapr-hadoop-062013.html
>>http://bitrefinery.com/services/hadoop-hosting
>>http://www.joyent.com/products/compute-service/features/hadoop
>>
>>There a lot of companies using HBase hosted in Cloud. The last
>>HBaseCon was full of great use-cases:
>>HBase at
 Pinterest:
>>http://www.hbasecon.com/sessions/apache-hbase-operations-at-pinterest/
>>
>>HBase at Groupon
>>http://www.hbasecon.com/sessions/apache-hbase-at-groupon/
>>
>>A great talk by Benoit for Networking design for HBase:
>>http://www.hbasecon.com/sessions/scalable-network-designs-for-apache-hbase/
>>
>>Using Coprocessors to Index Columns in an Elasticsearch Cluster
>>http://www.hbasecon.com/sessions/using-coprocessors-to-index-columns/
>>
>>2013/8/8, Dhaval Shah <[EMAIL PROTECTED]>:
>>> We are exploring the possibility of hosting Hadoop outside of our data
>>> centers. I am aware that Hadoop in general isn't exactly designed to run on
>>> virtual hardware. So a few questions:
>>> 1. Are there any providers out there who would host Hadoop on dedicated
>>> physical hardware?
>>> 2. Has anyone had success hosting Hadoop on virtualized hardware where 100%
>>> uptime and performance/stability are very important (we use HBase as a real
>>> time database and it needs to be up all the time)?
>>>
>>> Thanks,
>>> Dhaval
>>
>>
>>--
>>Marcos Ortiz Valmaseda
>>Product Manager at PDVSA
>>http://about.me/marcosortiz
>>
>>
>>
>
>
>
>--
>Nitin Pawar
>