-Sqoop import - column removed in the source DB and all the records are off by 1 column.
In my Sqoop import one of the column in the source table got deleted and
that resulting in data issue. This resulting data are off by 1 column.
The removed column was in the middle of the schema. If it were last column
then wouldn't have any worries.
Data is imported from MySql to Hive using Sqoop. I am using sqoop-1.3.0
Here is the syntax.
sqoop import --hive-import
--table 'TABLENAME '
--where 'created between 1353960000000 and 1353963600000'
Now the problem is One of the column in the source DB got removed.
I tried with workaround by including the --columns
1) By hardcoding third column with quotes.
--columns "col1,col2,'col3' as col3,col4"
but this gives error Column name 'col3' not in table
2) Then i tried with (col2 repeated twice)
--columns " col1,col2, col2 , col4"
It threw an error
Imported Failed: Duplicate Column identifier specified:
3) Then i tried with (col2 as col3)
--columns " col1,col2, col2 as col3, col4"
ERROR tool.ImportTool: Imported Failed: Column name 'authid uuid' not in
Could anybody suggest workaround for this.
Jarek Jarcec Cecho 2012-12-01, 18:03
Anandha L Ranganathan 2012-12-02, 04:13
Jarek Jarcec Cecho 2012-12-02, 17:11
Anandha L Ranganathan 2012-12-02, 19:10
Jarek Jarcec Cecho 2012-12-02, 19:35