Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Hive uploading


Copy link to this message
-
RE: Hive uploading
yogesh.kumar13@... 2012-07-05, 13:26
Hi Bejoy,

I used it. after creating userdata/....   dir in hdfs

sqoop import --verbose --connect jdbc:mysql://localhost:3306/Demo --username sqoop1 --password SQOOP1 --table yesrdbms1 -num-mappers 1 --warehouse-dir /userdata/yogesh/sqoop/imports --hive-import  --hive-table yeshivee

outcome is

12/07/05 18:48:56 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
12/07/05 18:48:56 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
12/07/05 18:48:56 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
12/07/05 18:48:56 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
12/07/05 18:48:56 INFO tool.CodeGenTool: Beginning code generation
12/07/05 18:48:56 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `yesrdbms1` AS t LIMIT 1
12/07/05 18:48:56 INFO orm.CompilationManager: HADOOP_HOME is /HADOOP/hadoop-0.20.2/bin/..
12/07/05 18:48:56 INFO orm.CompilationManager: Found hadoop core jar at: /HADOOP/hadoop-0.20.2/bin/../hadoop-0.20.2-core.jar
Note: /tmp/sqoop-mediaadmin/compile/603b6e26b2fd1160693ba0a66786d12e/yesrdbms1.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
12/07/05 18:48:57 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-mediaadmin/compile/603b6e26b2fd1160693ba0a66786d12e/yesrdbms1.jar
12/07/05 18:48:57 WARN manager.MySQLManager: It looks like you are importing from mysql.
12/07/05 18:48:57 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
12/07/05 18:48:57 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
12/07/05 18:48:57 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
12/07/05 18:48:57 INFO mapreduce.ImportJobBase: Beginning import of yesrdbms1
12/07/05 18:48:58 INFO mapred.JobClient: Running job: job_201207051104_0011
12/07/05 18:48:59 INFO mapred.JobClient:  map 0% reduce 0%
12/07/05 18:49:09 INFO mapred.JobClient:  map 100% reduce 0%
12/07/05 18:49:11 INFO mapred.JobClient: Job complete: job_201207051104_0011
12/07/05 18:49:11 INFO mapred.JobClient: Counters: 5
12/07/05 18:49:11 INFO mapred.JobClient:   Job Counters
12/07/05 18:49:11 INFO mapred.JobClient:     Launched map tasks=1
12/07/05 18:49:11 INFO mapred.JobClient:   FileSystemCounters
12/07/05 18:49:11 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=9
12/07/05 18:49:11 INFO mapred.JobClient:   Map-Reduce Framework
12/07/05 18:49:11 INFO mapred.JobClient:     Map input records=1
12/07/05 18:49:11 INFO mapred.JobClient:     Spilled Records=0
12/07/05 18:49:11 INFO mapred.JobClient:     Map output records=1
12/07/05 18:49:11 INFO mapreduce.ImportJobBase: Transferred 9 bytes in 13.6291 seconds (0.6604 bytes/sec)
12/07/05 18:49:11 INFO mapreduce.ImportJobBase: Retrieved 1 records.
12/07/05 18:49:11 INFO hive.HiveImport: Removing temporary files from import process: /userdata/yogesh/sqoop/imports/yesrdbms1/_logs
12/07/05 18:49:11 INFO hive.HiveImport: Loading uploaded data into Hive
12/07/05 18:49:11 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `yesrdbms1` AS t LIMIT 1
12/07/05 18:49:13 INFO hive.HiveImport: Logging initialized using configuration in jar:file:/HADOOP/hive/lib/hive-common-0.8.1.jar!/hive-log4j.properties
12/07/05 18:49:13 INFO hive.HiveImport: Hive history file=/tmp/mediaadmin/hive_job_log_mediaadmin_201207051849_2011545354.txt
12/07/05 18:49:17 INFO hive.HiveImport: OK
12/07/05 18:49:17 INFO hive.HiveImport: Time taken: 3.864 seconds
12/07/05 18:49:17 INFO hive.HiveImport: Loading data to table default.yeshivee
12/07/05 18:49:17 INFO hive.HiveImport: OK
12/07/05 18:49:17 INFO hive.HiveImport: Time taken: 0.245 seconds
12/07/05 18:49:17 INFO hive.HiveImport: Hive import complete.

------------------------------------*****************************************------------------------------------
and also this one

sqoop import --connect jdbc:mysql://localhost:3306/Demo --username sqoop1 --password SQOOP1 --table oldtoy --verbose  -num-mappers 1 --warehouse-dir /userdata/yogesh/sqoop/imports --hive-import  --hive-table yeshive2 --create-hive-table

outcome is

Setting your password on the command-line is insecure. Consider using -P instead.
12/07/05 18:53:58 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
12/07/05 18:53:58 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
12/07/05 18:53:59 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
12/07/05 18:53:59 INFO tool.CodeGenTool: Beginning code generation
12/07/05 18:53:59 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `oldtoy` AS t LIMIT 1
12/07/05 18:53:59 INFO orm.CompilationManager: HADOOP_HOME is /HADOOP/hadoop-0.20.2/bin/..
12/07/05 18:53:59 INFO orm.CompilationManager: Found hadoop core jar at: /HADOOP/hadoop-0.20.2/bin/../hadoop-0.20.2-core.jar
Note: /tmp/sqoop-mediaadmin/compile/c5fab2a1adf9725e1c5d556d0cceefd6/oldtoy.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
12/07/05 18:54:00 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-mediaadmin/compile/c5fab2a1adf9725e1c5d556d0cceefd6/oldtoy.jar
12/07/05 18:54:00 WARN manager.MySQLManager: It looks like you are importing from mysql.
12/07/05 18:54:00 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
12/07/05 18:54:00 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
12/07/05 18:54:00 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
12/07/05 18:54:00 INFO mapreduce.ImportJobBase: Beginning import of oldtoy
12/07/05 18:54:01 INFO mapred.JobClient: Running job: job_201207051104_0013
12/07/05 18:54:02 INFO mapred.JobClient:  map 0% reduce 0%
12/07/05 18:54:16 INFO mapred.JobClient:  map 100% reduce 0%
12/07/05 18:54:18 INFO mapred.JobClient: Job complete: job_201207051104_0013
12/07/05