Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Using a hard drive instead of


Copy link to this message
-
Re: Using a hard drive instead of
Hi Mark,

HDFS contains a write-ahead log which will protect you from power
failure.  It's called the edit log.  If you want warm failover, you
can use HDFS HA, which is available in recent versions of HDFS.

Hope this helps.
Colin
On Wed, Oct 17, 2012 at 3:44 PM, Mark Kerzner <[EMAIL PROTECTED]> wrote:
> Colin,
>
> swap space would give me very high memory, up to 1 TB, say, but it won't
> protect me from power failure. Very large clusters is only one application
> of this idea, warm failover is another. The drive is not theoretical, just
> look at the price tag :)
>
> Mark
>
>
> On Wed, Oct 17, 2012 at 5:37 PM, Colin Patrick McCabe <[EMAIL PROTECTED]>
> wrote:
>>
>> The direct answer to your question us to use this theoretical super-fast
>> hard drive as Linux swap space.
>>
>> The better answer is to use federation or another solution if your needs
>> exceed those servable by a single NameNode.
>>
>> Cheers.
>> Colin
>>
>> On Oct 11, 2012 9:00 PM, "Mark Kerzner" <[EMAIL PROTECTED]> wrote:
>>>
>>> Hi,
>>>
>>> Imagine I have a very fast hard drive that I want to use for the
>>> NameNode. That is, I want the NameNode to store its blocks information on
>>> this hard drive instead of in memory.
>>>
>>> Why would I do it? Scalability (no federation needed), many files are not
>>> a problem, and warm fail-over is automatic. What would I need to change in
>>> the NameNode to tell it to use the hard drive?
>>>
>>> Thank you,
>>> Mark
>
>