Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> Question for edit log retain


Copy link to this message
-
Re: Question for edit log retain
That's detailed. Thanks ATM.
On Tue, Jul 2, 2013 at 7:02 AM, Aaron T. Myers <[EMAIL PROTECTED]> wrote:

> Hi Azuryy,
>
> On Wed, Jun 26, 2013 at 6:12 PM, Azuryy Yu <[EMAIL PROTECTED]> wrote:
>
> > Hi Dear all,
> >
> > I have some confusion for edit log retain,
> >
> > NNStorageRetentionManager.java:
> >
> > 1)
> > purgeCheckpointsOlderThan()
> >   What's mean of check point here?
> >
>
> Here, "checkpoint" is referring to the fsimage files on the NN.
>
>
> >
> > 2)purgeOldStorage()
> >   I cannot understand the calculation of minimum txid, I think I can
> > understand it if I know these keys indicates.
> > DFS_NAMENODE_NUM_CHECKPOINTS_RETAINED_KEY
> >
>
> This is the number of old fsimage files to retain on the NN during purging.
>
>
> > DFS_NAMENODE_NUM_EXTRA_EDITS_RETAINED_KEY
> >
>
> This is the number of extra transactions to retain during purging. Here,
> "extra" is referring to transactions beyond what is strictly required to
> play back all FS changes since the last checkpoint (fsimage.)
>
>
> > DFS_NAMENODE_MAX_EXTRA_EDITS_SEGMENTS_RETAINED_KEY
> >
>
> This is the maximum number of extra edit _segments_ to retain during
> purging. Segments refer to finalized edit log files, with a start and end
> transaction ID. A single edit file is a "segment."
>
>
> >
> > Can any developer give me a short detail explanation? Thanks much.
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB