Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> HBase insert slow


+
徐歷盛 2013-02-08, 16:23
+
Mesika, Asaf 2013-02-08, 16:38
Copy link to this message
-
Re: HBase insert slow
Could you please show your hbase-site.xml? Also, try the basic tweaks like
usign putlist instead of put, turning wal off(if it is fine with you) etc.

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Fri, Feb 8, 2013 at 10:08 PM, Mesika, Asaf <[EMAIL PROTECTED]> wrote:

> What's the rate you're getting (MB/sec or lines/sec)?
> What's your cluster setup?
>
> On Feb 8, 2013, at 6:23 PM, 徐歷盛 wrote:
>
> > Hi,
> >    I recently suffered HBase insert slow issue (read from mysql, process
> > and then insert into HBase). The partial DEBUG log as below. It looks
> that
> > most of the time were spent near "Cached location" and "Got ping response
> > for sessionid". Is there any way to improve the performance ? Thanks
> >    I tried HTable and HTablePool, both showed the similiar result.
> >
> > [13/02/08 08:59:07:106][][DEBUG]Call: next 2
> > [13/02/08 08:59:11:619][][DEBUG]Cached location for
> > TABLE2_IDX01,,1355799947553.1a08e9f6c702b56d3dc839b83ce86124. is
> > 10.39.128.126:60020
> > ...
> > ...
> > [13/02/08 08:59:16:180][][DEBUG]Call: next 2
> > [13/02/08 08:59:19:298][][DEBUG]Got ping response for sessionid:
> > 0x13c4449defa5228 after 0ms
> >
> >
> > Detail Log as below
> >
> ==============================================================================================================> >
> > [13/02/08 08:58:25:863][][INFO]Initiating client connection,
> > connectString=SERVER1:2181 sessionTimeout=180000 watcher=hconnection
> > [13/02/08 08:58:25:866][][DEBUG]zookeeper.disableAutoWatchReset is false
> > [13/02/08 08:58:25:907][][INFO]Opening socket connection to server
> SERVER1/
> > 10.39.128.115:2181
> > [13/02/08 08:58:25:910][][INFO]Socket connection established to SERVER1/
> > 10.39.128.115:2181, initiating session
> > [13/02/08 08:58:25:912][][DEBUG]Session establishment request sent on
> > SERVER1/10.39.128.115:2181
> > [13/02/08 08:58:25:917][][INFO]Session establishment complete on server
> > SERVER1/10.39.128.115:2181, sessionid = 0x13c4449defa5228, negotiated
> > timeout = 40000
> >
> > [13/02/08 08:58:30:581][][DEBUG]Lookedup root region location,
> >
> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@1df2964
> ;
> > hsa=10.39.128.152:60020
> > [13/02/08 08:58:30:664][][DEBUG]The ping interval is60000ms.
> > [13/02/08 08:58:30:729][][DEBUG]Connecting to
> > 10.39.128.152/10.39.128.152:60020
> > [13/02/08 08:58:30:757][][DEBUG]IPC Client (47) connection to
> > 10.39.128.152/10.39.128.152:60020 from an unknown user sending #0
> > [13/02/08 08:58:30:827][][DEBUG]IPC Client (47) connection to
> > 10.39.128.152/10.39.128.152:60020 from an unknown user: starting, having
> > connections 1
> > [13/02/08 08:58:30:836][][DEBUG]IPC Client (47) connection to
> > 10.39.128.152/10.39.128.152:60020 from an unknown user got value #0
> > [13/02/08 08:58:30:837][][DEBUG]Call: getProtocolVersion 117
> > [13/02/08 08:58:30:838][][DEBUG]IPC Client (47) connection to
> > 10.39.128.152/10.39.128.152:60020 from an unknown user sending #1
> > [13/02/08 08:58:30:846][][DEBUG]IPC Client (47) connection to
> > 10.39.128.152/10.39.128.152:60020 from an unknown user got value #1
> > [13/02/08 08:58:30:848][][DEBUG]Call: getClosestRowBefore 10
> > [13/02/08 08:58:35:396][][DEBUG]Cached location for .META.,,1.1028785192
> is
> > 10.39.128.158:60020
> > [13/02/08 08:58:35:398][][DEBUG]Connecting to
> > 10.39.128.158/10.39.128.158:60020
> > [13/02/08 08:58:35:399][][DEBUG]IPC Client (47) connection to
> > 10.39.128.158/10.39.128.158:60020 from an unknown user sending #2
> > [13/02/08 08:58:35:399][][DEBUG]IPC Client (47) connection to
> > 10.39.128.158/10.39.128.158:60020 from an unknown user: starting, having
> > connections 2
> > [13/02/08 08:58:35:401][][DEBUG]IPC Client (47) connection to
> > 10.39.128.158/10.39.128.158:60020 from an unknown user got value #2
> > [13/02/08 08:58:35:401][][DEBUG]Call: getProtocolVersion 3
> > [13/02/08 08:58:35:417][][DEBUG]IPC Client (47) connection to
+
徐歷盛 2013-02-09, 01:49
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB