Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # dev - Any related paper on how to resolve hadoop SPOF issue?


Copy link to this message
-
Re: Any related paper on how to resolve hadoop SPOF issue?
Andrew Purtell 2011-08-27, 09:07
Hi Alex,

> UC Santa Cruz contributed a ;login: article describing replacing HDFS with
> Ceph.  (I was one of the authors.)  One of the key architectural advantages of
> Ceph over HDFS is that Ceph distributes its metadata service over multiple
> metadata servers.

Were you able to maintain stable operation with Ceph at a high load? Did you try running HBase?

Best regards,
       - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
----- Original Message -----
> From: Alex Nelson <[EMAIL PROTECTED]>
> To: George Kousiouris <[EMAIL PROTECTED]>
> Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; Carlos Maltzahn <[EMAIL PROTECTED]>
> Sent: Friday, August 26, 2011 2:44 AM
> Subject: Re: Any related paper on how to resolve hadoop SPOF issue?
>
> Hi George,
>
> UC Santa Cruz contributed a ;login: article describing replacing HDFS with
> Ceph.  (I was one of the authors.)  One of the key architectural advantages of
> Ceph over HDFS is that Ceph distributes its metadata service over multiple
> metadata servers.
>
> I hope that helps.
>
> --Alex
>
>
> On Aug 25, 2011, at 02:54 , George Kousiouris wrote:
>
>>
>>  Hi,
>>
>>  many thanks!!
>>
>>  I also found this one, which has some related work also with other efforts,
> in case it is helpful for someone else:
>>
> https://ritdml.rit.edu/bitstream/handle/1850/13321/ATalwalkarThesis1-2011.pdf?sequence=1
>>
>>
>>  BR,
>>  George
>>
>>  On 8/25/2011 12:46 PM, Nan Zhu wrote:
>>>  Hope it helps
>>>
>>>  http://www.springerlink.com/content/h17r882710314147/
>>>
>>>  Best,
>>>
>>>  Nan
>>>
>>>  On Thu, Aug 25, 2011 at 5:43 PM, George
> Kousiouris<[EMAIL PROTECTED]>wrote:
>>>
>>>>  Hi guys,
>>>>
>>>>  We are currently in the process of writing a paper regarding hadoop
> and we
>>>>  would like to reference any attempt to remove the single point of
> failure of
>>>>  the Namenode. We have found in various presentations some efforts
> (like
>>>>  dividing the namespace between more than one namenodes if i
> remember
>>>>  correctly) but the search for a concrete paper on the issue came to
> nothing.
>>>>
>>>>  Is anyone aware or has participated in such an effort?
>>>>
>>>>  Thanks,
>>>>  George
>>>>
>>>>  --
>>>>
>>>>  ---------------------------
>>>>
>>>>  George Kousiouris
>>>>  Electrical and Computer Engineer
>>>>  Division of Communications,
>>>>  Electronics and Information Engineering
>>>>  School of Electrical and Computer Engineering
>>>>  Tel: +30 210 772 2546
>>>>  Mobile: +30 6939354121
>>>>  Fax: +30 210 772 2569
>>>>  Email: [EMAIL PROTECTED]
>>>>  Site: http://users.ntua.gr/gkousiou/
>>>>
>>>>  National Technical University of Athens
>>>>  9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>>>>
>>>>
>>>
>>
>>
>>  --
>>
>>  ---------------------------
>>
>>  George Kousiouris
>>  Electrical and Computer Engineer
>>  Division of Communications,
>>  Electronics and Information Engineering
>>  School of Electrical and Computer Engineering
>>  Tel: +30 210 772 2546
>>  Mobile: +30 6939354121
>>  Fax: +30 210 772 2569
>>  Email: [EMAIL PROTECTED]
>>  Site: http://users.ntua.gr/gkousiou/
>>
>>  National Technical University of Athens
>>  9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
>>
>