Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Wrong input split locations after enabling reverse DNS


Copy link to this message
-
Re: Wrong input split locations after enabling reverse DNS
Jean-Daniel Cryans 2012-12-18, 02:21
Maybe TableInputFormatBase.getSplits is missing something similar to HBASE-4109?

J-D

On Mon, Dec 17, 2012 at 5:26 PM, Robert Dyer <[EMAIL PROTECTED]> wrote:
> That's what I thought too.  Except I am running 0.94.2 and this fix was
> released in 0.90.4.
>
>
> On Mon, Dec 17, 2012 at 5:11 PM, Stack <[EMAIL PROTECTED]> wrote:
>
>> On Sun, Dec 16, 2012 at 1:16 AM, Robert Dyer <[EMAIL PROTECTED]> wrote:
>>
>>> I recently enabled reverse DNS on my test cluster.  Now when I run a MR
>>> job, the HBase input split locations are all adding a period to the end.
>>>  For example:
>>>
>>> /default-rack/foo-1.
>>> /default-rack/foo-2.
>>>
>>> Yet the machine locations are still correct:
>>>
>>> /default-rack/foo-1
>>> /default-rack/foo-2
>>>
>>> Since those strings don't match, it isn't assigning the tasks locally.  It
>>> actually thinks 100% of the map tasks are rack-local and 0% data-local
>>> (although in reality, some still wind up being data-local due to sheer
>>> luck).
>>>
>>> What is the issue here?  Note that I don't have this problem with the MR
>>> tasks using SequenceFile as input, only with HBase's TableMapper.
>>>
>>
>>
>> Looks like https://issues.apache.org/jira/browse/HBASE-4109 ?
>> St.Ack
>>
>>