Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Increment Counters in HBase during MapReduce

Copy link to this message
Re: Increment Counters in HBase during MapReduce
Sure, why not?

You can always open a connection to the counter table in your Mapper.setup() method and then increment the counters within the Mapper.map() method.

Your update of the counter is an artifact and not the output of the Mapper.map() method.
On Jun 18, 2012, at 7:49 PM, Sid Kumar wrote:

> Hi everyone,
>    I have a use case in HBase that I was wondering if someone may have
> stumbled upon. I am maintaining an ad impressions table with columns that
> are counters for certain metrics. I started using the incrementColumnValue
> method part of the HTable API to update these metrics and that works great.
>    I was wondering if this function could be used from a MapReduce job.
> The TableOutputFormat supports only Delete and Put operations. Using the
> Incremental counters saves me from doing any aggregations in my Map Reduce
> code. Ideally i would like to just call this function in my mapper and
> wouldn't even need a Reducer.
>    Has anyone run into this use case? I would also love to know if there
> are any better alternatives of solving this too. Any info would be great.
> Thanks
> Sid