Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Wrong input split locations after enabling reverse DNS


Copy link to this message
-
Wrong input split locations after enabling reverse DNS
I recently enabled reverse DNS on my test cluster.  Now when I run a MR
job, the HBase input split locations are all adding a period to the end.
 For example:

/default-rack/foo-1.
/default-rack/foo-2.

Yet the machine locations are still correct:

/default-rack/foo-1
/default-rack/foo-2

Since those strings don't match, it isn't assigning the tasks locally.  It
actually thinks 100% of the map tasks are rack-local and 0% data-local
(although in reality, some still wind up being data-local due to sheer
luck).

What is the issue here?  Note that I don't have this problem with the MR
tasks using SequenceFile as input, only with HBase's TableMapper.
+
Stack 2012-12-17, 23:11
+
Robert Dyer 2012-12-18, 01:26
+
Jean-Daniel Cryans 2012-12-18, 02:21
+
Robert Dyer 2012-12-18, 02:39
+
Jean-Daniel Cryans 2012-12-18, 02:45
+
Robert Dyer 2013-01-28, 09:37
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB