Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Sqoop not updating incremental.last.value when used with "--hive-import"


Copy link to this message
-
Sqoop not updating incremental.last.value when used with "--hive-import"
I'm using hive 0.9.0, hadoop 1.1.1, and sqoop 1.4.2.

When I create a saved job to do an incremental import from a MySQL
server and import into Hive using "--hive-import", it appears that the
data is correctly imported, but that the incremental.last.value of the
job is not updated, so re-executing the job just imports the same
data.

If I create a job that's identical in all respects, but leave off
"--hive-import", it finishes with "INFO tool.ImportTool: Saving
incremental import state to the metastore", as I'd expect, and does in
fact update the incremental.last.value.

Is this behavior a bug? Or does Sqoop just not support incremental
imports into Hive?

-Mason
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB