Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> coprocessor enabled put very slow, help please~~~


Copy link to this message
-
Re: coprocessor enabled put very slow, help please~~~
Well...

if you look at the OP's code, there's a couple things I think could be causing a bit of the overhead on the writes.

Lets see if he fixes his code and see if there are any changes in performance.

On Feb 18, 2013, at 11:56 AM, Wei Tan <[EMAIL PROTECTED]> wrote:

> Well, my experience shows:
>
> 1. A local read can be >10ms and a remote put can be 1-2ms. Due to the
> nature of LSM a read is always a scan to one or multiple files. A very
> quick experiment can be, if you temporarily disable the check and only do
> the put, will you see performance go up?
>
> 2. In a lot of cases, RPC may NOT be the bottle neck. Remember a "local"
> put also involves RPC -- during WAL to HDFS.
>
>
> Best Regards,
> Wei
>
> Wei Tan
> Research Staff Member
> IBM T. J. Watson Research Center
> Yorktown Heights, NY 10598
> [EMAIL PROTECTED]; 914-945-4386
>
>
>
> From:   Prakash Kadel <[EMAIL PROTECTED]>
> To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
> Date:   02/18/2013 04:04 AM
> Subject:        Re: coprocessor enabled put very slow, help please~~~
>
>
>
> its a local read. i just check the last param of PostCheckAndPut
> indicating if the Put succeeded. Incase if the put success, i insert a row
> in another table
>
> Sincerely,
> Prakash Kadel
>
> On Feb 18, 2013, at 2:52 PM, Wei Tan <[EMAIL PROTECTED]> wrote:
>
>> Is your CheckAndPut involving a local or remote READ? Due to the nature
> of
>> LSM, read is much slower compared to a write...
>>
>>
>> Best Regards,
>> Wei
>>
>>
>>
>>
>> From:   Prakash Kadel <[EMAIL PROTECTED]>
>> To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
>> Date:   02/17/2013 07:49 PM
>> Subject:        coprocessor enabled put very slow, help please~~~
>>
>>
>>
>> hi,
>>  i am trying to insert few million documents to hbase with mapreduce.
> To
>> enable quick search of docs i want to have some indexes, so i tried to
> use
>> the coprocessors, but they are slowing down my inserts. Arent the
>> coprocessors not supposed to increase the latency?
>> my settings:
>>   3 region servers
>>  60 maps
>> each map inserts to doc table.(checkAndPut)
>> regionobserver coprocessor does a postCheckAndPut and inserts some rows
> to
>> a index table.
>>
>>
>> Sincerely,
>> Prakash
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB