-Re: Is FileSystem thread-safe?
Matthew Farrellee 2013-04-02, 11:41
If you're interested in the semantics of FileSystem operations, have a
look a HADOOP-9371
Depending on what you're trying to do, the thread-safety of a particular
FS implementation in a single JVM instance may not be as important as
the semantics you get across JVM instances.
On 04/01/2013 06:17 PM, Arpit Agarwal wrote:
> Hi John,
> DistributedFileSystem is intended to be thread-safe, true to its name.
> Metadata operations are handled by the NameNode server which
> synchronizes concurrent client requests via locks (you can look at the
> FSNameSystem class).
> Some discussion on the thread-safety aspects of HDFS:
> On Sun, Mar 31, 2013 at 11:52 AM, Ted Yu <[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>> wrote:
> If you look at DistributedFileSystem source code, you would see that
> it calls the DFSClient field member for most of the actions.
> Requests to Namenode are then made through ClientProtocol.
> An hdfs committer would be able to give you affirmative answer.
> On Sun, Mar 31, 2013 at 11:27 AM, John Lilley
> <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> wrote:
> *From:*Ted Yu [mailto:[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>]
> *Subject:* Re: Is FileSystem thread-safe?____
> >>FileSystem is an abstract class, what concrete class are you
> using (DistributedFileSystem, etc) ? ____
> Good point. I am calling FileSystem.get(URI uri, Configuration
> conf) with an URI like ï¿½hdfs://server:port/ï¿½ï¿½ on a remote
> server, so I assume it is creating a DistributedFileSystem.
> However I am not finding any documentation discussing its
> thread-safety (or lack thereof), perhaps you can point me to it?____
> Thanks, john____