hi guys,
    Currently, NameNode uses read-write lock at top root folder. In my
opinion, it's too huge for whole namespace with million of file/folder.
Meanwhile, a few level folder on top of the namespace directory has been
set with different application. Does we need lesser lock level for such
application sub-directory to reduce impact each other? It can be used with
five(or less) top level to HDFS path with individual lock tree-like
structure.
     Can anybody provide your feedback or advice? Thanks

-Regards
Denny Ye

Hi Denny,

Do you see an issue in practice?

Fine grained locking makes sense if you hold the lock while doing disk
IO. But since the structure is all in-memory, it's far less important,
and in fact could reduce performance in some cases.

Until you hit a few thousand nodes in your cluster, lock contention in
the NN is not a bottleneck. And the read-write lock introduced in
later versions gets past this issue even for ~4k node clusters for the
most part.

Not to say it won't ever make sense, but seems like a low priority change.

-Todd

On Tue, May 8, 2012 at 1:15 AM, Denny Ye <[EMAIL PROTECTED]> wrote:
> hi guys,
>     Currently, NameNode uses read-write lock at top root folder. In my
> opinion, it's too huge for whole namespace with million of file/folder.
> Meanwhile, a few level folder on top of the namespace directory has been set
> with different application. Does we need lesser lock level for such
> application sub-directory to reduce impact each other? It can be used with
> five(or less) top level to HDFS path with individual lock tree-like
> structure.
>      Can anybody provide your feedback or advice? Thanks
>
> -Regards
> Denny Ye

--
Todd Lipcon
Software Engineer, Cloudera