Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Many scanner opening


Copy link to this message
-
Re: Many scanner opening
Lars,

Cool stuff! Thanks a lot! I'm not sure I can apply the patch, cause we're
using CDH-4.1.1, but increasing size of internal scanner does the trick -
decreased number of scanners.
At least temporarily it's good enough.

Thanks!

On Wed, Dec 19, 2012 at 6:23 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:

> You might have run into HBASE-7336.
> (Not available in any official release, yet)
>
> If you're using 0.94 (and probably 0.92) you can just apply this patch
> (it's save and simple).
>
>
>
> ________________________________
>  From: Eugeny Morozov <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Tuesday, December 18, 2012 12:01 AM
> Subject: Many scanner opening
>
> Hello!
>
> We faced an issue recently that the more map tasks are completed, the
> longer it takes to complete one more map task.
>
> In our architecture we have two scanners to read the table. The first one,
> which is called 'outer' scanner is reading table and filter some rowkeys.
> These rowkeys are used as a filter for second scanner - 'internal'. Thus we
> constantly open 'internal' scanner with different filters.
>
> As an additional symptoms we see that our cluster practically does nothing
> - there is no CPU loading, no disk loading, no network, etc. Most of the
> time it means we are waiting on some locks, but I'm not sure.
>
> I would appreciate any ideas or suggestions to understand the case.
> Thank you in advance.
> --
> Evgeny Morozov
> Developer Grid Dynamics
> Skype: morozov.evgeny
> www.griddynamics.com
> [EMAIL PROTECTED]
>

--
Evgeny Morozov
Developer Grid Dynamics
Skype: morozov.evgeny
www.griddynamics.com
[EMAIL PROTECTED]
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB