Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - Re: Get the tree structure of a HDFS dir, similar to dir/files


+
Harsh J 2013-07-24, 05:24
Copy link to this message
-
Re: Get the tree structure of a HDFS dir, similar to dir/files
Shahab Yunus 2013-07-23, 20:53
See this
https://sites.google.com/site/hadoopandhive/home/how-to-read-all-files-in-a-directory-in-hdfs-using-hadoop-filesystem-api

and

http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/fs/FileSystem.html#isDirectory(org.apache.hadoop.fs.Path)

Basically you can write your own function, possible with recursion, to
iterate over a directory using the idea and combinations from above two
links.

Regards,
Shahab
On Tue, Jul 23, 2013 at 2:05 PM, Huy Pham <[EMAIL PROTECTED]> wrote:

>  Hi All,
>    Do any of you have or can refer me to some sample Java code that get
> the tree structure of a HDFS directory, similar to the file system?
>    For example: I have a HDFS dir, called /data, inside data, there is
> /data/valid and /data/invalid, and so on, so I would need to be able to get
> the whole tree structure of that and know which is is a dir, which one is a
> file. Both program and HDFS are LOCAL.
>    In other words, what I look for is something similar to File class in
> Java, which has isDirectory() and list() to list all the children (files
> and dirs) of a dir. Found something in stackoverflow but it does not work.
> Thanks
> Huy
>
>
>