Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Sqoop incremental import ( can any just help me out)


Copy link to this message
-
Re: Sqoop incremental import ( can any just help me out)
I have not tried this but I believe you can change the split by as you
wish.  The split by is used to split the jobs while --check-column and
--last-value are used for incremental import.

I do not know exact scenario but if empno gives a better split, you still
can use that for incremental import instead of changing the split-by field.

I would suggest you do a very simple test to find out.

Hope this helps,
Chalcy
On Mon, Dec 30, 2013 at 1:18 PM, yogesh kumar <[EMAIL PROTECTED]> wrote:

> Hello all,
>
> I have done sqoop import for a particluar table first time say table
> Employee..
>
> sqoop import -libjars .....
> --query "select empno, name, date, loc from table Employee where
> \$CONDITIONS ..  "
> *--split-by empno*
> --fields-terminated-by ','
> .
> .
> .
> .
>
> I have created an external table on hive,
>
> *Now I want to pull data on daily basis by using incremental pull.  can
> I specify the different column for --split-by*
>
> like
>
> sqoop import -libjars .....
> --query "select empno, name, date, loc from table Employee where
> \$CONDITIONS ..  "
> --check-column date
> --incremental append
> --last-value 2013-05-01
> *--split-by date*
> --split-by empno
>
>
> Can I change the column for *split by in incremental sqoop*, if not then
> how to do it.
>
> Pls suggest
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB