Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HBase, Hive, Hive over HBase or Pig over HBase


Copy link to this message
-
Re: HBase, Hive, Hive over HBase or Pig over HBase
Vivek,

Can you elaborate on 4? Storing data in HDFS directly does not give you the
option of updating it. However, that's not a good enough reason to use
HBase. Do you need random reads/writes outside of just the selective
increments? Can you store the increments in a separate file and then do a
resolution in the final results and periodically collapse all the updates
and make a new base table?

Hive over HBase is not yet ready. Pig - HBase integration is relatively more
mature.

Also, like Doug said, 30m records can be handled by an RDBMS. Does that not
solve your purpose? What are the challenges you faced, if any?

-Amandeep
On Wed, Oct 26, 2011 at 12:31 PM, viva v <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I am working on a use case that has the following characteristics.
> 1) Data volume is in the order 30 million records
> 2) Data schema is known & is fixed (for the application we are building)
> 3) Data is NOT multi format. A single key will have integer data for
> different aspects of that key
> 4) Data will be incrementally updated (some column values will be updated
> at
> different points of time)
> 5) There is a need to support adhoc (queries are not known ahead of time)
> querying of data (without writing map reduce jobs)
> 6) Queries are likely to have a lot of joins & aggregations
>
> Could you please help me with suggestions on whether i should use
> 1) Hive
> 2) HBase
> 3) Hive over HBase
> 4) Pig over HBase
>
> Thanks
> Vivek
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB