Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> HA for hadoop-0.20.2


+
lei liu 2012-11-13, 08:48
+
Harsh J 2012-11-13, 09:19
Copy link to this message
-
Re: HA for hadoop-0.20.2
Hi, A question, is 2.x ready for production deployment?  Thanks

On 13 Nov 2012, at 5:19 PM, Harsh J wrote:

> Hi,
>
> Why not just use the 2.x releases for HA-NNs? There is quite a wide
> delta between 0.20.x and 2.x, especially around the edit log areas
> after HDFS-1073.
>
> In any case, I think your question suits [EMAIL PROTECTED]
> more than the user lists, although I don't quite understand what
> you're attempting to do (or point).
>
> On Tue, Nov 13, 2012 at 2:18 PM, lei liu <[EMAIL PROTECTED]> wrote:
>> I want to implement HA function for hadoop-0.20.2.
>>
>> When I learn the hadoop-2.0 code, I meet some question like this:
>>
>> Thera are below code in FSEditLogLoader.loadEditRecords method.
>>
>>   if (op.hasTransactionId()) {
>>            if (op.getTransactionId() > expectedTxId) {
>>              MetaRecoveryContext.editLogLoaderPrompt("There appears " +
>>                  "to be a gap in the edit log.  We expected txid " +
>>                  expectedTxId + ", but got txid " +
>>                  op.getTransactionId() + ".", recovery, "ignoring missing "
>> +
>>                  " transaction IDs");
>>            } else if (op.getTransactionId() < expectedTxId) {
>>              MetaRecoveryContext.editLogLoaderPrompt("There appears " +
>>                  "to be an out-of-order edit in the edit log.  We " +
>>                  "expected txid " + expectedTxId + ", but got txid " +
>>                  op.getTransactionId() + ".", recovery,
>>                  "skipping the out-of-order edit");
>>              continue;
>>            }
>>          }
>>
>> The method use transaction id to  guarantee same transaction log is not
>> applied to namespace more than once.
>>
>> But in hadoop-0.20.2, FSEditLog don't store the transaction id into edits
>> log file. So I want to know if  StandbyNN apply same transaction log  to
>> namespace more than once, that will lead to the namespace of StandbyNN is
>> corrupt?
>>
>> Please give me some advice,Thanks.
>>
>>
>>
>> Best Regards
>>
>> LiuLei
>
>
>
> --
> Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB