Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Partition performance


+
Ian 2013-04-04, 23:01
+
Sanjay Subramanian 2013-04-04, 23:06
+
Ramki Palle 2013-04-04, 23:21
+
Owen OMalley 2013-04-04, 23:25
+
Dean Wampler 2013-04-04, 23:28
+
Ian 2013-04-05, 18:36
+
Ramki Palle 2013-04-05, 20:12
+
Ian 2013-04-11, 22:25
+
Peter Marron 2013-07-02, 09:34
+
Owen OMalley 2013-07-02, 14:51
+
David Morel 2013-07-03, 12:19
+
Edward Capriolo 2013-07-03, 14:22
Copy link to this message
-
Re: Partition performance
On Wed, Jul 3, 2013 at 5:19 AM, David Morel <[EMAIL PROTECTED]> wrote:

>
> That is still not really answering the question, which is: why is it slower
> to run a query on a heavily partitioned table than it is on the same number
> of files in a less heavily partitioned table.
>

According to Gopal's investigations in
https://issues.apache.org/jira/browse/HIVE-4051, each time Hive plans a
query, it does a query per a partition to the backing SQL database. That
would explain a lot of the latency for tables with large numbers of
partitions.

-- Owen
+
Peter Marron 2013-07-04, 07:37
+
Peter Marron 2013-07-04, 09:25
+
Dean Wampler 2013-07-03, 13:51
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB