Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> Sqoop incremental import ( can any just help me out)


+
yogesh kumar 2013-12-30, 18:18
Copy link to this message
-
Re: Sqoop incremental import ( can any just help me out)
I have not tried this but I believe you can change the split by as you
wish.  The split by is used to split the jobs while --check-column and
--last-value are used for incremental import.

I do not know exact scenario but if empno gives a better split, you still
can use that for incremental import instead of changing the split-by field.

I would suggest you do a very simple test to find out.

Hope this helps,
Chalcy
On Mon, Dec 30, 2013 at 1:18 PM, yogesh kumar <[EMAIL PROTECTED]> wrote:

> Hello all,
>
> I have done sqoop import for a particluar table first time say table
> Employee..
>
> sqoop import -libjars .....
> --query "select empno, name, date, loc from table Employee where
> \$CONDITIONS ..  "
> *--split-by empno*
> --fields-terminated-by ','
> .
> .
> .
> .
>
> I have created an external table on hive,
>
> *Now I want to pull data on daily basis by using incremental pull.  can
> I specify the different column for --split-by*
>
> like
>
> sqoop import -libjars .....
> --query "select empno, name, date, loc from table Employee where
> \$CONDITIONS ..  "
> --check-column date
> --incremental append
> --last-value 2013-05-01
> *--split-by date*
> --split-by empno
>
>
> Can I change the column for *split by in incremental sqoop*, if not then
> how to do it.
>
> Pls suggest
>
+
yogesh kumar 2013-12-30, 19:27
+
Devin Suiter RDX 2013-12-30, 19:57
+
yogesh kumar 2013-12-30, 20:13