Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - HA for hadoop-0.20.2


+
lei liu 2012-11-13, 08:48
+
Harsh J 2012-11-13, 09:19
Copy link to this message
-
Re: HA for hadoop-0.20.2
ac@...) 2012-11-13, 11:25
Hi, A question, is 2.x ready for production deployment?  Thanks

On 13 Nov 2012, at 5:19 PM, Harsh J wrote:

> Hi,
>
> Why not just use the 2.x releases for HA-NNs? There is quite a wide
> delta between 0.20.x and 2.x, especially around the edit log areas
> after HDFS-1073.
>
> In any case, I think your question suits [EMAIL PROTECTED]
> more than the user lists, although I don't quite understand what
> you're attempting to do (or point).
>
> On Tue, Nov 13, 2012 at 2:18 PM, lei liu <[EMAIL PROTECTED]> wrote:
>> I want to implement HA function for hadoop-0.20.2.
>>
>> When I learn the hadoop-2.0 code, I meet some question like this:
>>
>> Thera are below code in FSEditLogLoader.loadEditRecords method.
>>
>>   if (op.hasTransactionId()) {
>>            if (op.getTransactionId() > expectedTxId) {
>>              MetaRecoveryContext.editLogLoaderPrompt("There appears " +
>>                  "to be a gap in the edit log.  We expected txid " +
>>                  expectedTxId + ", but got txid " +
>>                  op.getTransactionId() + ".", recovery, "ignoring missing "
>> +
>>                  " transaction IDs");
>>            } else if (op.getTransactionId() < expectedTxId) {
>>              MetaRecoveryContext.editLogLoaderPrompt("There appears " +
>>                  "to be an out-of-order edit in the edit log.  We " +
>>                  "expected txid " + expectedTxId + ", but got txid " +
>>                  op.getTransactionId() + ".", recovery,
>>                  "skipping the out-of-order edit");
>>              continue;
>>            }
>>          }
>>
>> The method use transaction id to  guarantee same transaction log is not
>> applied to namespace more than once.
>>
>> But in hadoop-0.20.2, FSEditLog don't store the transaction id into edits
>> log file. So I want to know if  StandbyNN apply same transaction log  to
>> namespace more than once, that will lead to the namespace of StandbyNN is
>> corrupt?
>>
>> Please give me some advice,Thanks.
>>
>>
>>
>> Best Regards
>>
>> LiuLei
>
>
>
> --
> Harsh J