Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # dev - symlink support in Hadoop 2 GA


+
Andrew Wang 2013-09-17, 01:49
+
Suresh Srinivas 2013-09-17, 15:23
+
Colin McCabe 2013-09-17, 16:10
+
Andrew Wang 2013-09-17, 20:27
Copy link to this message
-
Re: symlink support in Hadoop 2 GA
Colin McCabe 2013-09-17, 15:14
I think it makes sense to finish symlinks support in the Hadoop 2 GA release.

Colin

On Mon, Sep 16, 2013 at 6:49 PM, Andrew Wang <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> I wanted to broadcast plans for putting the FileSystem symlinks work
> (HADOOP-8040) into branch-2.1 for the pending Hadoop 2 GA release. I think
> it's pretty important we get it in since it's not a compatible change; if
> it misses the GA train, we're not going to have symlinks until the next
> major release.
>
> However, we're still dealing with ongoing issues revealed via testing.
> There's user-code out there that only handles files and directories and
> will barf when given a symlink (perhaps a dangling one!). See HADOOP-9912
> for a nice example where globStatus returning symlinks broke Pig; some of
> us had a conference call to talk it through, and one definite conclusion
> was that this wasn't solvable in a generally compatible manner.
>
> There are also still some gaps in symlink support right now. For example,
> the more esoteric FileSystems like WebHDFS, HttpFS, and HFTP need symlink
> resolution, and tooling like the FsShell and Distcp still need to be
> updated as well.
>
> So, there's definitely work to be done, but there are a lot of users
> interested in the feature, and symlinks really should be in GA. Would
> appreciate any thoughts/input on the matter.
>
> Thanks,
> Andrew
+
Eli Collins 2013-09-17, 22:05
+
Steve Loughran 2013-09-18, 09:29
+
Alejandro Abdelnur 2013-09-18, 11:53
+
Steve Loughran 2013-09-18, 12:45
+
Eli Collins 2013-09-18, 16:24
+
Arun C Murthy 2013-09-18, 16:48
+
J. Rottinghuis 2013-09-19, 00:11
+
Colin McCabe 2013-09-19, 17:40
+
Alejandro Abdelnur 2013-09-19, 11:12
+
sanjay Radia 2013-10-03, 18:39
+
Daryn Sharp 2013-10-03, 21:08
+
Andrew Wang 2013-10-04, 17:58