Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - increment counters via bulk upload in HBase


Copy link to this message
-
Re: increment counters via bulk upload in HBase
Amit Sela 2013-02-16, 17:09
I don't know if incrementing using MapReduce is the best way to go.... Why
not run a multi threaded java process that reads the sequence file and
executes the increments? The key could be the table name and the value
could be an Increment object. That way you read the table name and execute
the appropriate increment.
You can generate that sequence file with MapReduce and have #reducers
output files and then have a processing thread for each file.

Good luck!
On Feb 16, 2013 6:39 PM, "Ashish Nigam" <[EMAIL PROTECTED]> wrote:

> Hi,
> Is there a way to increment counters in HBase via bulk upload?
> At present, I am storing counters in a sequence file in HDFS. Then I use a
> mapper to read the file and increment counters in HBase via Java API calls.
> I am assuming that if there is a way to increment counters via bulk upload,
> it will be more efficient.
>
> Thanks
> Ashish
>