Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: Using a hard drive instead of


+
Ravi Prakash 2012-10-13, 03:46
+
Mark Kerzner 2012-10-14, 04:07
+
Mark Kerzner 2012-10-12, 03:59
+
Harsh J 2012-10-12, 04:16
+
Mark Kerzner 2012-10-12, 04:27
+
Gaurav Sharma 2012-10-12, 04:34
+
Lance Norskog 2012-10-12, 05:01
+
Colin Patrick McCabe 2012-10-17, 22:37
+
Michael Segel 2012-10-17, 23:27
+
Mark Kerzner 2012-10-17, 22:44
Copy link to this message
-
Re: Using a hard drive instead of
Hi Mark,

HDFS contains a write-ahead log which will protect you from power
failure.  It's called the edit log.  If you want warm failover, you
can use HDFS HA, which is available in recent versions of HDFS.

Hope this helps.
Colin
On Wed, Oct 17, 2012 at 3:44 PM, Mark Kerzner <[EMAIL PROTECTED]> wrote:
> Colin,
>
> swap space would give me very high memory, up to 1 TB, say, but it won't
> protect me from power failure. Very large clusters is only one application
> of this idea, warm failover is another. The drive is not theoretical, just
> look at the price tag :)
>
> Mark
>
>
> On Wed, Oct 17, 2012 at 5:37 PM, Colin Patrick McCabe <[EMAIL PROTECTED]>
> wrote:
>>
>> The direct answer to your question us to use this theoretical super-fast
>> hard drive as Linux swap space.
>>
>> The better answer is to use federation or another solution if your needs
>> exceed those servable by a single NameNode.
>>
>> Cheers.
>> Colin
>>
>> On Oct 11, 2012 9:00 PM, "Mark Kerzner" <[EMAIL PROTECTED]> wrote:
>>>
>>> Hi,
>>>
>>> Imagine I have a very fast hard drive that I want to use for the
>>> NameNode. That is, I want the NameNode to store its blocks information on
>>> this hard drive instead of in memory.
>>>
>>> Why would I do it? Scalability (no federation needed), many files are not
>>> a problem, and warm fail-over is automatic. What would I need to change in
>>> the NameNode to tell it to use the hard drive?
>>>
>>> Thank you,
>>> Mark
>
>