Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: recursive list in java without block


Copy link to this message
-
Re: recursive list in java without block
Hi Kun,

So I think I was incorrect as I only checked the 0.20/1.0 APIs.

We do have optimal recursive listing support in 2.0 for both
FileContext and FileSystem APIs:
http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/FileSystem.html#listFiles(org.apache.hadoop.fs.Path,%20boolean).

I'd encourage using this, rather than FsShell which is not a public
API for users and may change its behavior anytime in future (no
guarantee).

On Fri, May 17, 2013 at 2:48 PM, Kun Ling <[EMAIL PROTECTED]> wrote:
> Hi Ankit,
>
>    Following Harsh's advice, I have found out that: although none of the
> FileSystem.java and DistributedFileSystem.java have support recursively
> liststatus().   However, the FsShell.java did have an ls() method which is
> used to support hadoop command like lsr ( that is the ls -R in Linux).
>
>
> yours,
>
> Kun Ling
>
>
> On Fri, May 17, 2013 at 6:59 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>>
>> The FileSystem API doesn't provide a utility to do recursive listing
>> yet, so you'd have to build it on your own.
>>
>> MR and the Fs Shell, both do seem to have inbuilt support for such a
>> utility though.
>>
>> On Fri, May 17, 2013 at 3:25 AM, Ankit Bhatnagar
>> <[EMAIL PROTECTED]> wrote:
>> > Hi folks,
>> >
>> > How can I get a recursive listing of file using java code from HDFS
>> > (hadoop
>> > 0.23.7*)
>> > i mean equivalent to ls  -R?
>> >
>> > Ankit
>>
>>
>>
>> --
>> Harsh J
>
>
>
>
> --
> http://www.lingcc.com

--
Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB