Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> Any related paper on how to resolve hadoop SPOF issue?


Copy link to this message
-
Re: Any related paper on how to resolve hadoop SPOF issue?
Hi Alex,

> UC Santa Cruz contributed a ;login: article describing replacing HDFS with
> Ceph.  (I was one of the authors.)  One of the key architectural advantages of
> Ceph over HDFS is that Ceph distributes its metadata service over multiple
> metadata servers.

Were you able to maintain stable operation with Ceph at a high load? Did you try running HBase?

Best regards,
       - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
----- Original Message -----
> From: Alex Nelson <[EMAIL PROTECTED]>
> To: George Kousiouris <[EMAIL PROTECTED]>
> Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; Carlos Maltzahn <[EMAIL PROTECTED]>
> Sent: Friday, August 26, 2011 2:44 AM
> Subject: Re: Any related paper on how to resolve hadoop SPOF issue?
>
> Hi George,
>
> UC Santa Cruz contributed a ;login: article describing replacing HDFS with
> Ceph.  (I was one of the authors.)  One of the key architectural advantages of
> Ceph over HDFS is that Ceph distributes its metadata service over multiple
> metadata servers.
>
> I hope that helps.
>
> --Alex
>
>
> On Aug 25, 2011, at 02:54 , George Kousiouris wrote:
>
>>
>>  Hi,
>>
>>  many thanks!!
>>
>>  I also found this one, which has some related work also with other efforts,
> in case it is helpful for someone else:
>>
> https://ritdml.rit.edu/bitstream/handle/1850/13321/ATalwalkarThesis1-2011.pdf?sequence=1
>>
>>
>>  BR,
>>  George
>>
>>  On 8/25/2011 12:46 PM, Nan Zhu wrote:
>>>  Hope it helps
>>>
>>>  http://www.springerlink.com/content/h17r882710314147/
>>>
>>>  Best,
>>>
>>>  Nan
>>>
>>>  On Thu, Aug 25, 2011 at 5:43 PM, George
> Kousiouris<[EMAIL PROTECTED]>wrote:
>>>
>>>>  Hi guys,
>>>>
>>>>  We are currently in the process of writing a paper regarding hadoop
> and we
>>>>  would like to reference any attempt to remove the single point of
> failure of
>>>>  the Namenode. We have found in various presentations some efforts
> (like
>>>>  dividing the namespace between more than one namenodes if i
> remember
>>>>  correctly) but the search for a concrete paper on the issue came to
> nothing.
>>>>
>>>>  Is anyone aware or has participated in such an effort?
>>>>
>>>>  Thanks,
>>>>  George
>>>>
>>>>  --
>>>>
>>>>  ---------------------------
>>>>
>>>>  George Kousiouris
>>>>  Electrical and Computer Engineer
>>>>  Division of Communications,
>>>>  Electronics and Information Engineering
>>>>  School of Electrical and Computer Engineering
>>>>  Tel: +30 210 772 2546
>>>>  Mobile: +30 6939354121
>>>>  Fax: +30 210 772 2569
>>>>  Email: [EMAIL PROTECTED]
>>>>  Site: http://users.ntua.gr/gkousiou/
>>>>
>>>>  National Technical University of Athens
>>>>  9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>>>>
>>>>
>>>
>>
>>
>>  --
>>
>>  ---------------------------
>>
>>  George Kousiouris
>>  Electrical and Computer Engineer
>>  Division of Communications,
>>  Electronics and Information Engineering
>>  School of Electrical and Computer Engineering
>>  Tel: +30 210 772 2546
>>  Mobile: +30 6939354121
>>  Fax: +30 210 772 2569
>>  Email: [EMAIL PROTECTED]
>>  Site: http://users.ntua.gr/gkousiou/
>>
>>  National Technical University of Athens
>>  9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB