Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Get Current Block or Split ID, and using it, the Block Path


Copy link to this message
-
Re: Get Current Block or Split ID, and using it, the Block Path
I think if you called getInputFormat on JobConf and then called getSplits
you would atleast get the locations.

http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/InputSplit.html

On Sun, Apr 8, 2012 at 9:16 AM, Deepak Nettem <[EMAIL PROTECTED]>wrote:

> Hi,
>
> Is it possible to get the 'id' of the currently executing split or block
> from within the mapper? Using this block Id / split id, I want to be able
> to query the namenode to get the names of hosts having that block / spllit,
> and the actual path to the data.
>
> I need this for some analytics that I'm doing. Is there a client API that
> allows doing this?  If not, what's the best way to do this?
>
> Best,
> Deepak Nettem
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB