Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> HADOOP and Query Capabilities


Copy link to this message
-
Re: HADOOP and Query Capabilities
On Mon, May 13, 2013 at 9:34 AM, Nalin Khosla <[EMAIL PROTECTED]>wrote:

> Had a quick question wrt to querying HADOOP data;
>
> 1. What tools are available to Query Hadoop data in real time vs batch?
>

The line between real time and batch isn't that clear. We are working on
substantially speeding up the performance of Hive (
http://www.slideshare.net/Hadoop_Summit/innovations-in-apache-hadoop-mapreduce-pig-hive-for-improving-query-performance).
The better question is whether you have small enough data so that it can
fit in RAM on your cluster. If so, you should look at Shark (
https://amplab.cs.berkeley.edu/2012/11/26/low-latency-sql-queries-at-massive-scale-a-performance-analysis-of-shark/)
or a proprietary MPP database such as Teradata or Impala.
>
> 2. I believe HIVE provides a batch interface, not sure on what tools
> within HIVE support the query capabilities against HADOOP ?
>

Hive currently uses MapReduce to run the queries. We plan on extending to
use Tez, which is a new Apache project that provides a richer framework for
queries.
>
> 3. Besides HIVE, are there any other Query tools to query HADOOP data
> (ad-hoc queries) ?
>

Pig and Cascading are the main open source ones for large data. Shark does
the smaller ad-hoc queries. Drill plans to fit into the ad-hoc space, but
hasn't made a release yet.

4. Finally, what skill set is required to use HIVE or other alternate tools
> ? Can business users uses these tools?
>

 Using Hive requires a learning curve. Business users will be able to run
queries against the data, but it will require someone with more engineering
background to design the table layouts and updating scheme.

-- Owen