Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - HDFS block physical location


Copy link to this message
-
Re: Re: HDFS block physical location
Chen He 2012-07-25, 22:20
For block to filename mapping, you can get from my previous answer.

For block to harddisk mapping, you may need to traverse all the directory
that used for HDFS, I am sure your OS has the information about which hard
drive is mounted to which directory.

with these two types of information, you can write a small Perl or Python
script to get what you want.

Or

Take look of the namenode.java and see where and how it saves the table of
block information.

Please correct me if there is any mistake.

Chen
On Wed, Jul 25, 2012 at 6:10 PM, <[EMAIL PROTECTED]> wrote:

>
> Thanks,
> but that just gives me the hostnames or am I overlooking something?
> I actually need the filename/harddisk on the node.
>
> JS
>
> Gesendet: Mittwoch, 25. Juli 2012 um 23:33 Uhr
> Von: "Chen He" <[EMAIL PROTECTED]>
> An: [EMAIL PROTECTED]
> Betreff: Re: HDFS block physical location
> >nohup hadoop fsck / -files -blocks -locations
> >cat nohup.out | grep [your block name]
>
> Hope this helps.
>
> On Wed, Jul 25, 2012 at 5:17 PM, <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> > just a short question. Is there any way to figure out the physical
> storage
> > location of a given block?
> > I don't mean just a list of hostnames (which I know how to obtain), but
> > actually the file where it is being stored in.
> > We use several hard disks for hdfs data on each node, and I would need to
> > know which block ends up on which harddisk.
> >
> > Thanks!
> > JS
> >
> >
>
>
>
>