Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Issue with Hive and table with lots of column


Copy link to this message
-
Re: Issue with Hive and table with lots of column
thanks.

re #1.  we need to find that Hiveserver2 process. For all i know the one
you reported is hiveserver1 (which works.) chances are they use the same
-Xmx value but we really shouldn't make any assumptions.

try wide format on the ps command (eg. ps -efw | grep -i Hiveserver2)

re.#2.  okay.  so that tells us is not the number of columns blowing the
heap but rather the combination of rows + columns.  There's no way it
stores the full result set on the heap even under normal circumstances so
my guess is there's an internal number of rows it buffers.  sorta like how
unix buffers stdout.  How and where that's set is out of my league.
However, maybe you get around it by upping your heapsize again if you have
the available memory of course.
On Tue, Feb 18, 2014 at 8:39 AM, David Gayou <[EMAIL PROTECTED]> wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB