Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Edit/change saved job without losing incremental history?


Copy link to this message
-
Edit/change saved job without losing incremental history?
Hello,

I am exploring Sqoop, and wondering about a hypothetical problem. What
happens if someone needs to change a saved incremental append job in Sqoop
1? Say for example the connection string needed to be changed because of a
network change or something. Shouldn't do that to your cluster, I know,
but, things happen.

From what I have seen, there isn't any way to edit an in-place job...only
to delete it and create a new one. If you have to do that, I'm guessing the
metastore will drop the history of the incremental append last value. If
you build a new incremental append job, that will bring everything over
from the beginning, which may not be good to do. If you run an incremental
import specifying the "from last value" parameter, that won't be a good for
a saved job, will it? Wouldn't it just begin from that value every time?

Is there a way to edit saved job parameters in the metastore directly so
you can update the connection string or inject the "last value" value in
the new job history after you create it but before running it the first
time? Or is something saved by job name?

Just wondering if there is a solution here that isn't really apparent in
the documentation...

Thanks,
*Devin Suiter*
Jr. Data Solutions Software Engineer
100 Sandusky Street | 2nd Floor | Pittsburgh, PA 15212
Google Voice: 412-256-8556 | www.rdx.com

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB