Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Allocating more heap for endpoint coprocessors


Copy link to this message
-
Re: Allocating more heap for endpoint coprocessors
Maybe we should be better with marking improvement as such in jira, and then have a special list in the release announcements highlighting these (otherwise they just be in the noise of all the other bug fixes).

-- Lars

________________________________
 From: Ramkrishna.S.Vasudevan <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]; 'lars hofhansl' <[EMAIL PROTECTED]>
Sent: Thursday, August 30, 2012 9:59 PM
Subject: RE: Allocating more heap for endpoint coprocessors
 
@Lars/@Gary

Do we need to document such things.  Recently someone was asking me a
question like this, if my endpoint impl is so memory intensive it just
affects a running cluster and already the RS
has a huge memory heap associated with it.

So its better we document saying if your endpoint consumes memory and
because it runs along with RS based on your need add that extra amount of
memory that will be taken up by the endpoint impl.

Regards
Ram

> -----Original Message-----
> From: lars hofhansl [mailto:[EMAIL PROTECTED]]
> Sent: Friday, August 31, 2012 3:04 AM
> To: [EMAIL PROTECTED]
> Subject: Re: Allocating more heap for endpoint coprocessors
>
> In the upcoming 0.94.2 release will also have HBASE-6505, which allows
> you to share state between RegionObservers (and Endpoints) within the
> same RegionServer.
>
> -- Lars
>
>
>
> ________________________________
>  From: Gary Helmling <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Thursday, August 30, 2012 1:59 PM
> Subject: Re: Allocating more heap for endpoint coprocessors
>
> Endpoint coprocessors are loaded and run within the HBase RegionServer
> process.  Your endpoint coprocessors will be running on the region
> servers hosting the regions for the table(s) on which the coprocessor
> is configured.
>
> So the way to allocate more memory is by setting either HBASE_HEAPSIZE
> or setting the max heap in HBASE_REGIONSERVER_OPTS in hbase-env.sh on
> the region server.
>
> Note that a separate coprocessor instance is loaded for each table
> region, so, say you want to allocate 10MB for your coprocessor, but
> each region server hosts 20 regions, you would want to increase the
> heap size by 200MB (20x10MB).
>
> --gh
>
> On Thu, Aug 30, 2012 at 1:45 PM, Young Kim <[EMAIL PROTECTED]>
> wrote:
> > Hi,
> >
> > We have some memory intensive endpoint coprocessors running on our
> RegionServers. As a result, we want to allocate more heap for the
> coprocessors, but there doesn't seem to be much documentation on which
> Hbase processes are directly responsible for coprocessors. Does anyone
> happen to know or direct me to some resource that does?
> >
> > Thanks,
> > Young Kim
> >
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB