Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> coprocessor enabled put very slow, help please~~~


Copy link to this message
-
Re: coprocessor enabled put very slow, help please~~~
Well, my experience shows:

1. A local read can be >10ms and a remote put can be 1-2ms. Due to the
nature of LSM a read is always a scan to one or multiple files. A very
quick experiment can be, if you temporarily disable the check and only do
the put, will you see performance go up?
 
2. In a lot of cases, RPC may NOT be the bottle neck. Remember a "local"
put also involves RPC -- during WAL to HDFS.
Best Regards,
Wei

Wei Tan
Research Staff Member
IBM T. J. Watson Research Center
Yorktown Heights, NY 10598
[EMAIL PROTECTED]; 914-945-4386

From:   Prakash Kadel <[EMAIL PROTECTED]>
To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
Date:   02/18/2013 04:04 AM
Subject:        Re: coprocessor enabled put very slow, help please~~~

its a local read. i just check the last param of PostCheckAndPut
indicating if the Put succeeded. Incase if the put success, i insert a row
in another table

Sincerely,
Prakash Kadel

On Feb 18, 2013, at 2:52 PM, Wei Tan <[EMAIL PROTECTED]> wrote:

> Is your CheckAndPut involving a local or remote READ? Due to the nature
of
> LSM, read is much slower compared to a write...
>
>
> Best Regards,
> Wei
>
>
>
>
> From:   Prakash Kadel <[EMAIL PROTECTED]>
> To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
> Date:   02/17/2013 07:49 PM
> Subject:        coprocessor enabled put very slow, help please~~~
>
>
>
> hi,
>   i am trying to insert few million documents to hbase with mapreduce.
To
> enable quick search of docs i want to have some indexes, so i tried to
use
> the coprocessors, but they are slowing down my inserts. Arent the
> coprocessors not supposed to increase the latency?
> my settings:
>    3 region servers
>   60 maps
> each map inserts to doc table.(checkAndPut)
> regionobserver coprocessor does a postCheckAndPut and inserts some rows
to
> a index table.
>
>
> Sincerely,
> Prakash
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB