Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> A tool to analyze and tune performance for Hive?


Copy link to this message
-
A tool to analyze and tune performance for Hive?
Hi everyone,

May I know if there is any tool available to analyze and tune the
performance for Hive queries? And what is the state of the art?

I had some experience on tuning Pig, based on manually clicking JT web
pages and collecting pieces of data from here and there, and guessing
what might be wrong. That was a slow and uncomfortable process. So
before I dive into Hive, I'd like to hear any experience from you.

PS: for individual jobs, we built a tool called Starfish:
http://www.cs.duke.edu/starfish/release.html . It can be used to
analyze the job's performance and profile the job for auto-tuning. It
could be used for Hive too, but now doesn't capture the Hive-related
info, as well as the interaction among jobs.

Thanks,
Jie
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB