Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: incrCounter doesn't return a value


Copy link to this message
-
Re: incrCounter doesn't return a value
Jens/Ashish,
Thanks for your replies. I missed the point that these counters are
maintained by their respective tasks and periodically sent to the
tasktracker and then to the jobtracker for global aggregation.

Thanks
Manish
On Sun, Aug 4, 2013 at 10:43 AM, Ashish Umrani <[EMAIL PROTECTED]>wrote:

> Manish,
>
> I am not sure if the counter would provide a globally unique id.  To the
> best of my knowledge, Counters are mapper specific.  So even if one could
> see the application working on one mapper, at the end, when deployed on
> production, duplicate ids would cause a problem.  So, unless you are
> looking for uniqueness on a single mapper, I would suggest you to look for
> some other solution, perhaps like the one suggested by Jens.
>
> Regards
> ashish
>
>
> On Sun, Aug 4, 2013 at 10:36 AM, Jens Scheidtmann <
> [EMAIL PROTECTED]> wrote:
>
>> Dear Manish,
>>
>> Use some combination of Job Id and/or IP address and other attributes to
>> make your own unique ids. incrCounter had to synchronize globally across
>> your cluster, if it were to provide incremental IDs.
>>
>> Best regards,
>>
>> Jens
>>
>> Am Donnerstag, 1. August 2013 schrieb Manish Verma :
>>
>>> Hi,
>>>
>>> I wanted to use the incrCounter API to generate auto increment ids but
>>> the problem is that it doesn't return the incremented value. Does anyone
>>> know why this API does not return the incremented value? And whether it
>>> would be possible to change it to return the incremented value?
>>>
>>> Thanks
>>> Manish V
>>>
>>
>