I want to implement HA function for hadoop-0.20.2.

When I learn the hadoop-2.0 code, I meet some question like this:

Thera are below code in FSEditLogLoader.loadEditRecords method.
   if (op.hasTransactionId()) {
            if (op.getTransactionId() > expectedTxId) {
              MetaRecoveryContext.editLogLoaderPrompt("There appears " +
                  "to be a gap in the edit log.  We expected txid " +
                  expectedTxId + ", but got txid " +
                  op.getTransactionId() + ".", recovery, "ignoring missing
" +
                  " transaction IDs");
            } *else if (op.getTransactionId() < expectedTxId) {
              MetaRecoveryContext.editLogLoaderPrompt("There appears " +
                  "to be an out-of-order edit in the edit log.  We " +
                  "expected txid " + expectedTxId + ", but got txid " +
                  op.getTransactionId() + ".", recovery,
                  "skipping the out-of-order edit");
              continue;*
            }
          }

The method use transaction id to  guarantee same transaction log is not
applied to namespace more than once.

But in hadoop-0.20.2, FSEditLog don't store the transaction id into edits
log file. So I want to know if  StandbyNN apply same transaction log  to
namespace more than once, that will lead to the namespace of StandbyNN is
corrupt?

Please give me some advice，Thanks.

Best Regards
LiuLei

Hi,

Why not just use the 2.x releases for HA-NNs? There is quite a wide
delta between 0.20.x and 2.x, especially around the edit log areas
after HDFS-1073.

In any case, I think your question suits [EMAIL PROTECTED]
more than the user lists, although I don't quite understand what
you're attempting to do (or point).

On Tue, Nov 13, 2012 at 2:18 PM, lei liu <[EMAIL PROTECTED]> wrote:
> I want to implement HA function for hadoop-0.20.2.
>
> When I learn the hadoop-2.0 code, I meet some question like this:
>
> Thera are below code in FSEditLogLoader.loadEditRecords method.
>
>    if (op.hasTransactionId()) {
>             if (op.getTransactionId() > expectedTxId) {
>               MetaRecoveryContext.editLogLoaderPrompt("There appears " +
>                   "to be a gap in the edit log.  We expected txid " +
>                   expectedTxId + ", but got txid " +
>                   op.getTransactionId() + ".", recovery, "ignoring missing "
> +
>                   " transaction IDs");
>             } else if (op.getTransactionId() < expectedTxId) {
>               MetaRecoveryContext.editLogLoaderPrompt("There appears " +
>                   "to be an out-of-order edit in the edit log.  We " +
>                   "expected txid " + expectedTxId + ", but got txid " +
>                   op.getTransactionId() + ".", recovery,
>                   "skipping the out-of-order edit");
>               continue;
>             }
>           }
>
> The method use transaction id to  guarantee same transaction log is not
> applied to namespace more than once.
>
> But in hadoop-0.20.2, FSEditLog don't store the transaction id into edits
> log file. So I want to know if  StandbyNN apply same transaction log  to
> namespace more than once, that will lead to the namespace of StandbyNN is
> corrupt?
>
> Please give me some advice，Thanks.
>
>
>
> Best Regards
>
> LiuLei

--
Harsh J

Hi, A question, is 2.x ready for production deployment?  Thanks

On 13 Nov 2012, at 5:19 PM, Harsh J wrote:

> Hi,
>
> Why not just use the 2.x releases for HA-NNs? There is quite a wide
> delta between 0.20.x and 2.x, especially around the edit log areas
> after HDFS-1073.
>
> In any case, I think your question suits [EMAIL PROTECTED]
> more than the user lists, although I don't quite understand what
> you're attempting to do (or point).
>
> On Tue, Nov 13, 2012 at 2:18 PM, lei liu <[EMAIL PROTECTED]> wrote:
>> I want to implement HA function for hadoop-0.20.2.
>>
>> When I learn the hadoop-2.0 code, I meet some question like this:
>>
>> Thera are below code in FSEditLogLoader.loadEditRecords method.
>>
>>   if (op.hasTransactionId()) {
>>            if (op.getTransactionId() > expectedTxId) {
>>              MetaRecoveryContext.editLogLoaderPrompt("There appears " +
>>                  "to be a gap in the edit log.  We expected txid " +
>>                  expectedTxId + ", but got txid " +
>>                  op.getTransactionId() + ".", recovery, "ignoring missing "
>> +
>>                  " transaction IDs");
>>            } else if (op.getTransactionId() < expectedTxId) {
>>              MetaRecoveryContext.editLogLoaderPrompt("There appears " +
>>                  "to be an out-of-order edit in the edit log.  We " +
>>                  "expected txid " + expectedTxId + ", but got txid " +
>>                  op.getTransactionId() + ".", recovery,
>>                  "skipping the out-of-order edit");
>>              continue;
>>            }
>>          }
>>
>> The method use transaction id to  guarantee same transaction log is not
>> applied to namespace more than once.
>>
>> But in hadoop-0.20.2, FSEditLog don't store the transaction id into edits
>> log file. So I want to know if  StandbyNN apply same transaction log  to
>> namespace more than once, that will lead to the namespace of StandbyNN is
>> corrupt?
>>
>> Please give me some advice，Thanks.
>>
>>
>>
>> Best Regards
>>
>> LiuLei
>
>
>
> --
> Harsh J