Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Review Request: FLUME-1649. Improve HBaseSink performance.


Copy link to this message
-
Re: Review Request: FLUME-1649. Improve HBaseSink performance.


> On Dec. 6, 2012, 10:41 p.m., Brock Noland wrote:
> > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 244
> > <https://reviews.apache.org/r/7657/diff/3/?file=177947#file177947line244>
> >
> >     Why do we batch the puts into one runnable but not the increments?

This is because Increments did not implement Row interface in 0.92.1 release (which we use in the hadoop-1 profile), and so they cannot be batched
- Hari
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7657/#review14116
-----------------------------------------------------------
On Oct. 19, 2012, 12:14 a.m., Hari Shreedharan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7657/
> -----------------------------------------------------------
>
> (Updated Oct. 19, 2012, 12:14 a.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> Modify the Hbase sink to use multiple threads to wait on RPC calls. Since the threads are blocked often, this will not cause too many running threads.
>
>
> This addresses bug FLUME-1649.
>     https://issues.apache.org/jira/browse/FLUME-1649
>
>
> Diffs
> -----
>
>   flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java 021ecd0
>
> Diff: https://reviews.apache.org/r/7657/diff/
>
>
> Testing
> -------
>
> All current unit tests pass. Did functional testing.
>
>
> Thanks,
>
> Hari Shreedharan
>
>