Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> sym Links in hadoop


Copy link to this message
-
Re: sym Links in hadoop
yes dave u r right man i think symlink support is not available for 1.x.x
rel ... i did searched a lot :)  i cant move to lower versions
act i wanted webhdfs thts why i needed 1.x.x versions its a complete
deadlock for me know     i guess coz i needed webhdfs plus symlink :) ....

On Tue, Oct 9, 2012 at 6:14 PM, Dave Beech <[EMAIL PROTECTED]> wrote:

> Actually, I don't think the FileContext class Colin mentioned is the
> one you are talking about.
>
> Hadoop 1.0.3 doesn't have org.apache.hadoop.fs.FileContext, which is
> the class you would need, but it does have
> org.apache.hadoop.metrics.file.FileContext which is something
> completely different.
>
> Symlink support for HDFS, as far as I'm aware, isn't contained in any
> "stable" release of Hadoop (i.e. 1.x.x)
>
> On 9 October 2012 13:40, Dave Beech <[EMAIL PROTECTED]> wrote:
> > A lot of code in Hadoop is marked "Deprecated". This doesn't mean you
> > shouldn't use it ;)
> >
> > Cheers,
> > Dave
> >
> > On 9 October 2012 09:28, Visioner Sadak <[EMAIL PROTECTED]>
> wrote:
> >> I am using hadoop 1.0.3
> >>
> >> On Tue, Oct 9, 2012 at 12:22 PM, Visioner Sadak <
> [EMAIL PROTECTED]>
> >> wrote:
> >>>
> >>> Thanks Colin I tried using FileContext but the class is showing as
> >>> depricated
> >>>
> >>> On Tue, Oct 9, 2012 at 12:02 AM, Colin McCabe <[EMAIL PROTECTED]>
> >>> wrote:
> >>>>
> >>>> You can create an HDFS symlink by using the FileContext#createSymlink
> >>>> function.  I don't think this can be done through the "hadoop fs"
> >>>> command, so you're going to have to write some Java code to do this.
> >>>>
> >>>> We should consider adding this functionality to the "hadoop fs"
> >>>> command in the future.
> >>>>
> >>>> Colin
> >>>>
> >>>>
> >>>> On Mon, Oct 8, 2012 at 7:20 AM, Visioner Sadak <
> [EMAIL PROTECTED]>
> >>>> wrote:
> >>>> > thanks dave its in hdfs only....any other methods of creating a
> symlink
> >>>> >
> >>>> >
> >>>> > On Mon, Oct 8, 2012 at 7:00 PM, Dave Beech <[EMAIL PROTECTED]>
> wrote:
> >>>> >>
> >>>> >> Hi,
> >>>> >> The FileUtil.symlink command does nothing more than call the unix
> "ln"
> >>>> >> command, so it has no knowledge of how to work with Hadoop archive
> >>>> >> files, only plain files and directories.
> >>>> >> Is your archive on local disk, or in HDFS?
> >>>> >> Cheers,
> >>>> >> Dave
> >>>> >>
> >>>> >> On 8 October 2012 13:43, Visioner Sadak <[EMAIL PROTECTED]>
> >>>> >> wrote:
> >>>> >> > I tried using FileUtil class for creating a symlink within hadoop
> >>>> >> > actually i
> >>>> >> > want to create a symlink for my har directory so my code looks
> like
> >>>> >> >
> >>>> >> > FileUtil.symLink("/user2/","har:///user/5oct2012.har")
> >>>> >> >
> >>>> >> > but getting error like this
> >>>> >> >
> >>>> >> > org.apache.hadoop.fs.FileUtil
> >>>> >> >  - Command 'ln -s /user2/ har://user/5oct2012.har/' failed 1
> with:
> >>>> >> > ln:
> >>>> >> > target `har:///user/5oct2012.har/' is not a directory: No such
> file
> >>>> >> > or
> >>>> >> > directory
> >>>> >> >
> >>>> >> >
> >>>> >> >
> >>>> >> >
> >>>> >> > On Mon, Oct 8, 2012 at 12:13 PM, Visioner Sadak
> >>>> >> > <[EMAIL PROTECTED]>
> >>>> >> > wrote:
> >>>> >> >>
> >>>> >> >> Hello,
> >>>> >> >>
> >>>> >> >>           can we create symlinks within hadoop is ther any shell
> >>>> >> >> commands
> >>>> >> >> or can we do it thru java....
> >>>> >> >
> >>>> >> >
> >>>> >
> >>>> >
> >>>
> >>>
> >>
>