Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> RE: FileSystem iterative or limited alternative to listStatus()


Copy link to this message
-
RE: FileSystem iterative or limited alternative to listStatus()
Hmmm, that's not right either, but this looks like the following will work since I can stop iterating at any time.
RemoteIterator<LocatedFileStatus> listFiles(Path f, boolean recursive);
Thanks for getting me looking the right direction, anyway!
john

From: Ted Yu [mailto:[EMAIL PROTECTED]]
Sent: Saturday, January 11, 2014 4:58 PM
To: [EMAIL PROTECTED]
Subject: Re: FileSystem iterative or limited alternative to listStatus()

Can you utilize the following API ?
  public FileStatus[] listStatus(Path f, PathFilter filter)
Cheers

On Sat, Jan 11, 2014 at 3:52 PM, John Lilley <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Is there an HDFS file system method for listing a directory contents iteratively, or at least stopping at some limit?  We have an application in which the user can type wildcards, which our app expands.  However, during interactive phases we only want the first matching file to check its format.  There may be folders with many 1000s of files...
Thanks
John
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB