Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Hive uploading


Copy link to this message
-
RE: Hive uploading
Hi Bejoy,

I used it. after creating userdata/....   dir in hdfs

sqoop import --verbose --connect jdbc:mysql://localhost:3306/Demo --username sqoop1 --password SQOOP1 --table yesrdbms1 -num-mappers 1 --warehouse-dir /userdata/yogesh/sqoop/imports --hive-import  --hive-table yeshivee

outcome is

12/07/05 18:48:56 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
12/07/05 18:48:56 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
12/07/05 18:48:56 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
12/07/05 18:48:56 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
12/07/05 18:48:56 INFO tool.CodeGenTool: Beginning code generation
12/07/05 18:48:56 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `yesrdbms1` AS t LIMIT 1
12/07/05 18:48:56 INFO orm.CompilationManager: HADOOP_HOME is /HADOOP/hadoop-0.20.2/bin/..
12/07/05 18:48:56 INFO orm.CompilationManager: Found hadoop core jar at: /HADOOP/hadoop-0.20.2/bin/../hadoop-0.20.2-core.jar
Note: /tmp/sqoop-mediaadmin/compile/603b6e26b2fd1160693ba0a66786d12e/yesrdbms1.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
12/07/05 18:48:57 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-mediaadmin/compile/603b6e26b2fd1160693ba0a66786d12e/yesrdbms1.jar
12/07/05 18:48:57 WARN manager.MySQLManager: It looks like you are importing from mysql.
12/07/05 18:48:57 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
12/07/05 18:48:57 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
12/07/05 18:48:57 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
12/07/05 18:48:57 INFO mapreduce.ImportJobBase: Beginning import of yesrdbms1
12/07/05 18:48:58 INFO mapred.JobClient: Running job: job_201207051104_0011
12/07/05 18:48:59 INFO mapred.JobClient:  map 0% reduce 0%
12/07/05 18:49:09 INFO mapred.JobClient:  map 100% reduce 0%
12/07/05 18:49:11 INFO mapred.JobClient: Job complete: job_201207051104_0011
12/07/05 18:49:11 INFO mapred.JobClient: Counters: 5
12/07/05 18:49:11 INFO mapred.JobClient:   Job Counters
12/07/05 18:49:11 INFO mapred.JobClient:     Launched map tasks=1
12/07/05 18:49:11 INFO mapred.JobClient:   FileSystemCounters
12/07/05 18:49:11 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=9
12/07/05 18:49:11 INFO mapred.JobClient:   Map-Reduce Framework
12/07/05 18:49:11 INFO mapred.JobClient:     Map input records=1
12/07/05 18:49:11 INFO mapred.JobClient:     Spilled Records=0
12/07/05 18:49:11 INFO mapred.JobClient:     Map output records=1
12/07/05 18:49:11 INFO mapreduce.ImportJobBase: Transferred 9 bytes in 13.6291 seconds (0.6604 bytes/sec)
12/07/05 18:49:11 INFO mapreduce.ImportJobBase: Retrieved 1 records.
12/07/05 18:49:11 INFO hive.HiveImport: Removing temporary files from import process: /userdata/yogesh/sqoop/imports/yesrdbms1/_logs
12/07/05 18:49:11 INFO hive.HiveImport: Loading uploaded data into Hive
12/07/05 18:49:11 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `yesrdbms1` AS t LIMIT 1
12/07/05 18:49:13 INFO hive.HiveImport: Logging initialized using configuration in jar:file:/HADOOP/hive/lib/hive-common-0.8.1.jar!/hive-log4j.properties
12/07/05 18:49:13 INFO hive.HiveImport: Hive history file=/tmp/mediaadmin/hive_job_log_mediaadmin_201207051849_2011545354.txt
12/07/05 18:49:17 INFO hive.HiveImport: OK
12/07/05 18:49:17 INFO hive.HiveImport: Time taken: 3.864 seconds
12/07/05 18:49:17 INFO hive.HiveImport: Loading data to table default.yeshivee
12/07/05 18:49:17 INFO hive.HiveImport: OK
12/07/05 18:49:17 INFO hive.HiveImport: Time taken: 0.245 seconds
12/07/05 18:49:17 INFO hive.HiveImport: Hive import complete.

------------------------------------*****************************************------------------------------------
and also this one

sqoop import --connect jdbc:mysql://localhost:3306/Demo --username sqoop1 --password SQOOP1 --table oldtoy --verbose  -num-mappers 1 --warehouse-dir /userdata/yogesh/sqoop/imports --hive-import  --hive-table yeshive2 --create-hive-table

outcome is

Setting your password on the command-line is insecure. Consider using -P instead.
12/07/05 18:53:58 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
12/07/05 18:53:58 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
12/07/05 18:53:59 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
12/07/05 18:53:59 INFO tool.CodeGenTool: Beginning code generation
12/07/05 18:53:59 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `oldtoy` AS t LIMIT 1
12/07/05 18:53:59 INFO orm.CompilationManager: HADOOP_HOME is /HADOOP/hadoop-0.20.2/bin/..
12/07/05 18:53:59 INFO orm.CompilationManager: Found hadoop core jar at: /HADOOP/hadoop-0.20.2/bin/../hadoop-0.20.2-core.jar
Note: /tmp/sqoop-mediaadmin/compile/c5fab2a1adf9725e1c5d556d0cceefd6/oldtoy.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
12/07/05 18:54:00 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-mediaadmin/compile/c5fab2a1adf9725e1c5d556d0cceefd6/oldtoy.jar
12/07/05 18:54:00 WARN manager.MySQLManager: It looks like you are importing from mysql.
12/07/05 18:54:00 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
12/07/05 18:54:00 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
12/07/05 18:54:00 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
12/07/05 18:54:00 INFO mapreduce.ImportJobBase: Beginning import of oldtoy
12/07/05 18:54:01 INFO mapred.JobClient: Running job: job_201207051104_0013
12/07/05 18:54:02 INFO mapred.JobClient:  map 0% reduce 0%
12/07/05 18:54:16 INFO mapred.JobClient:  map 100% reduce 0%
12/07/05 18:54:18 INFO mapred.JobClient: Job complete: job_201207051104_0013
12/07/05
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB