Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - coprocessor enabled put very slow, help please~~~


Copy link to this message
-
Re: coprocessor enabled put very slow, help please~~~
Wei Tan 2013-02-18, 17:56
Well, my experience shows:

1. A local read can be >10ms and a remote put can be 1-2ms. Due to the
nature of LSM a read is always a scan to one or multiple files. A very
quick experiment can be, if you temporarily disable the check and only do
the put, will you see performance go up?
 
2. In a lot of cases, RPC may NOT be the bottle neck. Remember a "local"
put also involves RPC -- during WAL to HDFS.
Best Regards,
Wei

Wei Tan
Research Staff Member
IBM T. J. Watson Research Center
Yorktown Heights, NY 10598
[EMAIL PROTECTED]; 914-945-4386

From:   Prakash Kadel <[EMAIL PROTECTED]>
To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
Date:   02/18/2013 04:04 AM
Subject:        Re: coprocessor enabled put very slow, help please~~~

its a local read. i just check the last param of PostCheckAndPut
indicating if the Put succeeded. Incase if the put success, i insert a row
in another table

Sincerely,
Prakash Kadel

On Feb 18, 2013, at 2:52 PM, Wei Tan <[EMAIL PROTECTED]> wrote:

> Is your CheckAndPut involving a local or remote READ? Due to the nature
of
> LSM, read is much slower compared to a write...
>
>
> Best Regards,
> Wei
>
>
>
>
> From:   Prakash Kadel <[EMAIL PROTECTED]>
> To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
> Date:   02/17/2013 07:49 PM
> Subject:        coprocessor enabled put very slow, help please~~~
>
>
>
> hi,
>   i am trying to insert few million documents to hbase with mapreduce.
To
> enable quick search of docs i want to have some indexes, so i tried to
use
> the coprocessors, but they are slowing down my inserts. Arent the
> coprocessors not supposed to increase the latency?
> my settings:
>    3 region servers
>   60 maps
> each map inserts to doc table.(checkAndPut)
> regionobserver coprocessor does a postCheckAndPut and inserts some rows
to
> a index table.
>
>
> Sincerely,
> Prakash
>