Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - coprocessor enabled put very slow, help please~~~


+
Prakash Kadel 2013-02-18, 00:48
+
Prakash Kadel 2013-02-18, 00:52
+
lars hofhansl 2013-02-18, 01:07
+
Prakash Kadel 2013-02-18, 01:13
+
lars hofhansl 2013-02-18, 01:17
+
Prakash Kadel 2013-02-18, 01:26
+
lars hofhansl 2013-02-18, 02:31
+
Michael Segel 2013-02-18, 09:31
+
Michael Segel 2013-02-18, 01:31
+
Prakash Kadel 2013-02-18, 02:01
+
Prakash Kadel 2013-02-18, 01:32
+
Wei Tan 2013-02-18, 05:52
+
Prakash Kadel 2013-02-18, 09:01
+
Michael Segel 2013-02-18, 09:35
+
yonghu 2013-02-18, 10:57
+
Michael Segel 2013-02-18, 12:11
+
yonghu 2013-02-18, 12:22
+
Michael Segel 2013-02-18, 12:45
+
prakash kadel 2013-02-18, 13:21
+
Michel Segel 2013-02-18, 14:13
+
Wei Tan 2013-02-19, 15:15
+
Michael Segel 2013-02-19, 16:01
+
Michael Segel 2013-02-19, 16:29
+
Andrew Purtell 2013-02-19, 20:05
+
prakash kadel 2013-02-19, 07:41
+
Asaf Mesika 2013-02-19, 21:53
+
Michel Segel 2013-02-20, 13:00
+
Prakash Kadel 2013-02-20, 13:26
+
Michel Segel 2013-02-20, 14:14
+
Prakash Kadel 2013-02-20, 15:10
Copy link to this message
-
Re: coprocessor enabled put very slow, help please~~~
Wei Tan 2013-02-18, 17:56
Well, my experience shows:

1. A local read can be >10ms and a remote put can be 1-2ms. Due to the
nature of LSM a read is always a scan to one or multiple files. A very
quick experiment can be, if you temporarily disable the check and only do
the put, will you see performance go up?
 
2. In a lot of cases, RPC may NOT be the bottle neck. Remember a "local"
put also involves RPC -- during WAL to HDFS.
Best Regards,
Wei

Wei Tan
Research Staff Member
IBM T. J. Watson Research Center
Yorktown Heights, NY 10598
[EMAIL PROTECTED]; 914-945-4386

From:   Prakash Kadel <[EMAIL PROTECTED]>
To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
Date:   02/18/2013 04:04 AM
Subject:        Re: coprocessor enabled put very slow, help please~~~

its a local read. i just check the last param of PostCheckAndPut
indicating if the Put succeeded. Incase if the put success, i insert a row
in another table

Sincerely,
Prakash Kadel

On Feb 18, 2013, at 2:52 PM, Wei Tan <[EMAIL PROTECTED]> wrote:

> Is your CheckAndPut involving a local or remote READ? Due to the nature
of
> LSM, read is much slower compared to a write...
>
>
> Best Regards,
> Wei
>
>
>
>
> From:   Prakash Kadel <[EMAIL PROTECTED]>
> To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
> Date:   02/17/2013 07:49 PM
> Subject:        coprocessor enabled put very slow, help please~~~
>
>
>
> hi,
>   i am trying to insert few million documents to hbase with mapreduce.
To
> enable quick search of docs i want to have some indexes, so i tried to
use
> the coprocessors, but they are slowing down my inserts. Arent the
> coprocessors not supposed to increase the latency?
> my settings:
>    3 region servers
>   60 maps
> each map inserts to doc table.(checkAndPut)
> regionobserver coprocessor does a postCheckAndPut and inserts some rows
to
> a index table.
>
>
> Sincerely,
> Prakash
>
+
Michael Segel 2013-02-18, 18:42
+
yonghu 2013-02-18, 09:01
+
yonghu 2013-02-18, 09:02