Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop, mail # user - Sqoop import - column removed in the source DB and all the records are off by 1 column.


Copy link to this message
-
Sqoop import - column removed in the source DB and all the records are off by 1 column.
Anandha L Ranganathan 2012-12-01, 02:15
In my Sqoop import one of the column in the source table got deleted and
that resulting in data issue. This resulting   data are off by 1 column.
The removed column was in the middle of the schema. If it were last column
then wouldn't have any worries.

Data is imported  from MySql to Hive using Sqoop.  I am using  sqoop-1.3.0

Here is the syntax.

sqoop import     --hive-import
                --options-file 'credential.txt'
                --table 'TABLENAME '
                --where 'created between 1353960000000 and 1353963600000'
                --hive-partition-key part
                --hive-partition-value 'PARTITION_VALUE'
                --hive-overwrite
                --hive-delims-replacement

Now the problem is One of the column in the source DB got removed.

I tried with workaround by including  the --columns

 1) By hardcoding third column with quotes.
        --columns "col1,col2,'col3' as col3,col4"

but this gives error Column name 'col3' not in table
2) Then i tried with (col2 repeated twice)
   --columns " col1,col2, col2 , col4"

It threw an error
   Imported Failed: Duplicate Column identifier specified:
3) Then i tried with (col2 as col3)
   --columns " col1,col2, col2 as col3, col4"

ERROR tool.ImportTool: Imported Failed: Column name 'authid uuid' not in
table

Could anybody suggest workaround for this.

Thanks
+
Jarek Jarcec Cecho 2012-12-01, 18:03
+
Anandha L Ranganathan 2012-12-02, 04:13
+
Jarek Jarcec Cecho 2012-12-02, 17:11
+
Anandha L Ranganathan 2012-12-02, 19:10
+
Jarek Jarcec Cecho 2012-12-02, 19:35