Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # dev - Question for edit log retain


Copy link to this message
-
Re: Question for edit log retain
Azuryy Yu 2013-07-02, 02:05
That's detailed. Thanks ATM.
On Tue, Jul 2, 2013 at 7:02 AM, Aaron T. Myers <[EMAIL PROTECTED]> wrote:

> Hi Azuryy,
>
> On Wed, Jun 26, 2013 at 6:12 PM, Azuryy Yu <[EMAIL PROTECTED]> wrote:
>
> > Hi Dear all,
> >
> > I have some confusion for edit log retain,
> >
> > NNStorageRetentionManager.java:
> >
> > 1)
> > purgeCheckpointsOlderThan()
> >   What's mean of check point here?
> >
>
> Here, "checkpoint" is referring to the fsimage files on the NN.
>
>
> >
> > 2)purgeOldStorage()
> >   I cannot understand the calculation of minimum txid, I think I can
> > understand it if I know these keys indicates.
> > DFS_NAMENODE_NUM_CHECKPOINTS_RETAINED_KEY
> >
>
> This is the number of old fsimage files to retain on the NN during purging.
>
>
> > DFS_NAMENODE_NUM_EXTRA_EDITS_RETAINED_KEY
> >
>
> This is the number of extra transactions to retain during purging. Here,
> "extra" is referring to transactions beyond what is strictly required to
> play back all FS changes since the last checkpoint (fsimage.)
>
>
> > DFS_NAMENODE_MAX_EXTRA_EDITS_SEGMENTS_RETAINED_KEY
> >
>
> This is the maximum number of extra edit _segments_ to retain during
> purging. Segments refer to finalized edit log files, with a start and end
> transaction ID. A single edit file is a "segment."
>
>
> >
> > Can any developer give me a short detail explanation? Thanks much.
> >
>