Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Out Of Memory on localmode.


Copy link to this message
-
Re: Out Of Memory on localmode.
In that situation I might turn the setting off entirely. You would
guess that the common use case is not "really small datasets" better
the query to take longer then to oom a machine.

On Tue, Feb 5, 2013 at 3:12 AM, Bennie Schut <[EMAIL PROTECTED]> wrote:
> Hi,
>
>
>
> Just in case anyone else ever runs into this.
>
> Lately our cluster kept on killing itself with an OOM message in the kernel
> log. It took me a while to realize why this happened since no single process
> was causing it.
>
> I traced it back to a few queries running concurrently on a really small
> datasets. This caused all of these queries to run localmode. Then I realized
> there isn’t a limit to how many queries can run in localmode and since they
> use the normal hadoop memory settings it’s pretty easy to hit OOM on a
> machine this way.
>
> I’m not sure about the long term solution (some kind of limit on the number
> of localmode processes), but for now I’ll probably disable localmode on
> these queries.
>
>
>
> Bennie.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB