Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: recursive list in java without block


Copy link to this message
-
Re: recursive list in java without block
Hi Kun,

So I think I was incorrect as I only checked the 0.20/1.0 APIs.

We do have optimal recursive listing support in 2.0 for both
FileContext and FileSystem APIs:
http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/FileSystem.html#listFiles(org.apache.hadoop.fs.Path,%20boolean).

I'd encourage using this, rather than FsShell which is not a public
API for users and may change its behavior anytime in future (no
guarantee).

On Fri, May 17, 2013 at 2:48 PM, Kun Ling <[EMAIL PROTECTED]> wrote:
> Hi Ankit,
>
>    Following Harsh's advice, I have found out that: although none of the
> FileSystem.java and DistributedFileSystem.java have support recursively
> liststatus().   However, the FsShell.java did have an ls() method which is
> used to support hadoop command like lsr ( that is the ls -R in Linux).
>
>
> yours,
>
> Kun Ling
>
>
> On Fri, May 17, 2013 at 6:59 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>>
>> The FileSystem API doesn't provide a utility to do recursive listing
>> yet, so you'd have to build it on your own.
>>
>> MR and the Fs Shell, both do seem to have inbuilt support for such a
>> utility though.
>>
>> On Fri, May 17, 2013 at 3:25 AM, Ankit Bhatnagar
>> <[EMAIL PROTECTED]> wrote:
>> > Hi folks,
>> >
>> > How can I get a recursive listing of file using java code from HDFS
>> > (hadoop
>> > 0.23.7*)
>> > i mean equivalent to ls  -R?
>> >
>> > Ankit
>>
>>
>>
>> --
>> Harsh J
>
>
>
>
> --
> http://www.lingcc.com

--
Harsh J