Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Reflect MySQL updates into Hive


Copy link to this message
-
Reflect MySQL updates into Hive
Hi All,

We are new to hadoop and hive, we are trying to use hive to
run analytical queries and we are using sqoop to import data into hive, in
our RDBMS the data updated very frequently and this needs to be reflected
to hive. Hive does not support update/delete but there are many workarounds
to do this task.

What's in our mind is importing all the tables into hive as is, then we
build the required tables for reporting.

My questions are:

   1. What is the best way to reflect MySQL updates into Hive with minimal
   resources?
   2. Is sqoop the right tool to do the ETL?
   3. Is Hive the right tool to do this kind of queries or we should search
   for alternatives?

Any hint will be useful, thanks in advanced.

--
Ibrahim
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB