Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - sym Links in hadoop


+
Visioner Sadak 2012-10-08, 06:43
+
Visioner Sadak 2012-10-08, 12:43
+
Dave Beech 2012-10-08, 13:30
+
Visioner Sadak 2012-10-08, 14:20
+
Terry Healy 2012-10-08, 15:33
+
Colin McCabe 2012-10-08, 18:32
+
Visioner Sadak 2012-10-09, 06:52
+
Visioner Sadak 2012-10-09, 08:28
+
Dave Beech 2012-10-09, 12:40
Copy link to this message
-
Re: sym Links in hadoop
Dave Beech 2012-10-09, 12:44
Actually, I don't think the FileContext class Colin mentioned is the
one you are talking about.

Hadoop 1.0.3 doesn't have org.apache.hadoop.fs.FileContext, which is
the class you would need, but it does have
org.apache.hadoop.metrics.file.FileContext which is something
completely different.

Symlink support for HDFS, as far as I'm aware, isn't contained in any
"stable" release of Hadoop (i.e. 1.x.x)

On 9 October 2012 13:40, Dave Beech <[EMAIL PROTECTED]> wrote:
> A lot of code in Hadoop is marked "Deprecated". This doesn't mean you
> shouldn't use it ;)
>
> Cheers,
> Dave
>
> On 9 October 2012 09:28, Visioner Sadak <[EMAIL PROTECTED]> wrote:
>> I am using hadoop 1.0.3
>>
>> On Tue, Oct 9, 2012 at 12:22 PM, Visioner Sadak <[EMAIL PROTECTED]>
>> wrote:
>>>
>>> Thanks Colin I tried using FileContext but the class is showing as
>>> depricated
>>>
>>> On Tue, Oct 9, 2012 at 12:02 AM, Colin McCabe <[EMAIL PROTECTED]>
>>> wrote:
>>>>
>>>> You can create an HDFS symlink by using the FileContext#createSymlink
>>>> function.  I don't think this can be done through the "hadoop fs"
>>>> command, so you're going to have to write some Java code to do this.
>>>>
>>>> We should consider adding this functionality to the "hadoop fs"
>>>> command in the future.
>>>>
>>>> Colin
>>>>
>>>>
>>>> On Mon, Oct 8, 2012 at 7:20 AM, Visioner Sadak <[EMAIL PROTECTED]>
>>>> wrote:
>>>> > thanks dave its in hdfs only....any other methods of creating a symlink
>>>> >
>>>> >
>>>> > On Mon, Oct 8, 2012 at 7:00 PM, Dave Beech <[EMAIL PROTECTED]> wrote:
>>>> >>
>>>> >> Hi,
>>>> >> The FileUtil.symlink command does nothing more than call the unix "ln"
>>>> >> command, so it has no knowledge of how to work with Hadoop archive
>>>> >> files, only plain files and directories.
>>>> >> Is your archive on local disk, or in HDFS?
>>>> >> Cheers,
>>>> >> Dave
>>>> >>
>>>> >> On 8 October 2012 13:43, Visioner Sadak <[EMAIL PROTECTED]>
>>>> >> wrote:
>>>> >> > I tried using FileUtil class for creating a symlink within hadoop
>>>> >> > actually i
>>>> >> > want to create a symlink for my har directory so my code looks like
>>>> >> >
>>>> >> > FileUtil.symLink("/user2/","har:///user/5oct2012.har")
>>>> >> >
>>>> >> > but getting error like this
>>>> >> >
>>>> >> > org.apache.hadoop.fs.FileUtil
>>>> >> >  - Command 'ln -s /user2/ har://user/5oct2012.har/' failed 1 with:
>>>> >> > ln:
>>>> >> > target `har:///user/5oct2012.har/' is not a directory: No such file
>>>> >> > or
>>>> >> > directory
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> > On Mon, Oct 8, 2012 at 12:13 PM, Visioner Sadak
>>>> >> > <[EMAIL PROTECTED]>
>>>> >> > wrote:
>>>> >> >>
>>>> >> >> Hello,
>>>> >> >>
>>>> >> >>           can we create symlinks within hadoop is ther any shell
>>>> >> >> commands
>>>> >> >> or can we do it thru java....
>>>> >> >
>>>> >> >
>>>> >
>>>> >
>>>
>>>
>>
+
Visioner Sadak 2012-10-09, 13:23