Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Re: [sqoop-user] Error while sqoop-import


Copy link to this message
-
Re: [sqoop-user] Error while sqoop-import
This user group for SQOOP is deprecated moving the conversation to Apache
Sqoop user group

Hi Bhavesh
            When you do import all tables, SQOOP does the default import
with the PK column in RDBMS as the --split-by column. In most cases in
RDBMS you have a non numeric PK or a composite Key as your PK, in those
circumstances you have to individually import tables from rdbms using
appropriate --split-by column for *each table*.
In your case I believe AddressUid from one table is a non numeric  field
and hence it is causing issues with the boundary SQL fired from SQOOP. (you
need to specify --split-by) So as the bottom line you need to issue
individual import statements for all tables to be imported. Club those in a
shell script and fire the script if want a single point trigger.

Hope it helps!...

Regards
Bejoy.K.S
On Wed, Feb 1, 2012 at 11:44 AM, Bhavesh Shah <[EMAIL PROTECTED]>wrote:

> Hello Bejoy K S,
> When I am trying for import for single table I executed that one with
> --split-by option  but when I tried for --sqoop-import-all-tables
> I got these errors:
>
>
> hadoop@ubuntu:~/sqoop-1.3.0-cdh3u1/bin$ ./sqoop-import-all-tables
> --connect 'jdbc:sqlserver://192.168.1.1;username=abcd;password=12345;database=HadoopTest'
> --hive-import
> 12/01/31 05:29:49 INFO tool.BaseSqoopTool: Using Hive-specific delimiters
> for output. You can override
> 12/01/31 05:29:49 INFO tool.BaseSqoopTool: delimiters with
> --fields-terminated-by, etc.
> 12/01/31 05:29:49 INFO SqlServer.MSSQLServerManagerFactory: Using
> Microsoft's SQL Server - Hadoop Connector
> 12/01/31 05:29:49 INFO manager.SqlManager: Using default fetchSize of 1000
> 12/01/31 05:29:49 INFO tool.CodeGenTool: Beginning code generation
> 12/01/31 05:29:50 INFO manager.SqlManager: Executing SQL statement: SELECT
> TOP 1 * FROM [Address]
> 12/01/31 05:29:50 INFO manager.SqlManager: Executing SQL statement: SELECT
> TOP 1 * FROM [Address]
> 12/01/31 05:29:50 INFO orm.CompilationManager: HADOOP_HOME is
> /home/hadoop/hadoop-0.20.2-cdh3u2
> 12/01/31 05:29:51 ERROR orm.CompilationManager: Could not rename
> /tmp/sqoop-hadoop/compile/6c3bc3e91230270163a8c4d1c5491071/Address.java to
> /home/hadoop/sqoop-1.3.0-cdh3u1/bin/./Address.java
>
> java.io.IOException: Destination
> '/home/hadoop/sqoop-1.3.0-cdh3u1/bin/./Address.java' already exists
>     at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:1811)
>     at
> com.cloudera.sqoop.orm.CompilationManager.compile(CompilationManager.java:227)
>     at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83)
>     at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:337)
>     at
> com.cloudera.sqoop.tool.ImportAllTablesTool.run(ImportAllTablesTool.java:64)
>
>     at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
>     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>     at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
>     at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
>     at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
>     at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)
> 12/01/31 05:29:51 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-hadoop/compile/6c3bc3e91230270163a8c4d1c5491071/Address.jar
> 12/01/31 05:29:51 INFO mapreduce.ImportJobBase: Beginning import of Address
> 12/01/31 05:29:51 INFO manager.SqlManager: Executing SQL statement: SELECT
> TOP 1 * FROM [Address]
> 12/01/31 05:29:52 INFO db.DataDrivenDBInputFormat: BoundingValsQuery:
> SELECT MIN([AddressUid]), MAX([AddressUid]) FROM [Address]
> 12/01/31 05:29:52 INFO mapred.JobClient: Cleaning up the staging area
> hdfs://localhost:54310/home/hadoop/hadoopTemp/mapred/staging/hadoop/.staging/job_201201301559_0287
> 12/01/31 05:29:52 ERROR tool.ImportAllTablesTool: Encountered IOException
> running import job: java.io.IOException:
> com.microsoft.sqlserver.jdbc.SQLServerException: Operand data type
> uniqueidentifier is invalid for min operator.
>
> -Thanks
> Bhavesh Shah
>
>
>
> On Wed, Feb 1, 2012 at 10:51 AM, Bhavesh Shah <[EMAIL PROTECTED]>wrote:

Regards
       Bejoy
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB