Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> A tool to analyze and tune performance for Hive?


Copy link to this message
-
RE: A tool to analyze and tune performance for Hive?
You may have a try for HiTune & HiBench. Just google for them.

-----Original Message-----
From: Jie Li [mailto:[EMAIL PROTECTED]]
Sent: Friday, December 14, 2012 10:02 AM
To: [EMAIL PROTECTED]
Subject: A tool to analyze and tune performance for Hive?

Hi everyone,

May I know if there is any tool available to analyze and tune the performance for Hive queries? And what is the state of the art?

I had some experience on tuning Pig, based on manually clicking JT web pages and collecting pieces of data from here and there, and guessing what might be wrong. That was a slow and uncomfortable process. So before I dive into Hive, I'd like to hear any experience from you.

PS: for individual jobs, we built a tool called Starfish:
http://www.cs.duke.edu/starfish/release.html . It can be used to analyze the job's performance and profile the job for auto-tuning. It could be used for Hive too, but now doesn't capture the Hive-related info, as well as the interaction among jobs.

Thanks,
Jie
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB