Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> coprocessor enabled put very slow, help please~~~


Copy link to this message
-
Re: coprocessor enabled put very slow, help please~~~
Well...

if you look at the OP's code, there's a couple things I think could be causing a bit of the overhead on the writes.

Lets see if he fixes his code and see if there are any changes in performance.

On Feb 18, 2013, at 11:56 AM, Wei Tan <[EMAIL PROTECTED]> wrote:

> Well, my experience shows:
>
> 1. A local read can be >10ms and a remote put can be 1-2ms. Due to the
> nature of LSM a read is always a scan to one or multiple files. A very
> quick experiment can be, if you temporarily disable the check and only do
> the put, will you see performance go up?
>
> 2. In a lot of cases, RPC may NOT be the bottle neck. Remember a "local"
> put also involves RPC -- during WAL to HDFS.
>
>
> Best Regards,
> Wei
>
> Wei Tan
> Research Staff Member
> IBM T. J. Watson Research Center
> Yorktown Heights, NY 10598
> [EMAIL PROTECTED]; 914-945-4386
>
>
>
> From:   Prakash Kadel <[EMAIL PROTECTED]>
> To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
> Date:   02/18/2013 04:04 AM
> Subject:        Re: coprocessor enabled put very slow, help please~~~
>
>
>
> its a local read. i just check the last param of PostCheckAndPut
> indicating if the Put succeeded. Incase if the put success, i insert a row
> in another table
>
> Sincerely,
> Prakash Kadel
>
> On Feb 18, 2013, at 2:52 PM, Wei Tan <[EMAIL PROTECTED]> wrote:
>
>> Is your CheckAndPut involving a local or remote READ? Due to the nature
> of
>> LSM, read is much slower compared to a write...
>>
>>
>> Best Regards,
>> Wei
>>
>>
>>
>>
>> From:   Prakash Kadel <[EMAIL PROTECTED]>
>> To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
>> Date:   02/17/2013 07:49 PM
>> Subject:        coprocessor enabled put very slow, help please~~~
>>
>>
>>
>> hi,
>>  i am trying to insert few million documents to hbase with mapreduce.
> To
>> enable quick search of docs i want to have some indexes, so i tried to
> use
>> the coprocessors, but they are slowing down my inserts. Arent the
>> coprocessors not supposed to increase the latency?
>> my settings:
>>   3 region servers
>>  60 maps
>> each map inserts to doc table.(checkAndPut)
>> regionobserver coprocessor does a postCheckAndPut and inserts some rows
> to
>> a index table.
>>
>>
>> Sincerely,
>> Prakash
>>
>
>