Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # user - Creating compressed data with scooop


Copy link to this message
-
Re: Creating compressed data with scooop
Santosh Achhra 2013-01-08, 14:33
Hi Jarcec,
I appreciate your time on this. Here is the log captured from terminal
after I execute sqoop command

13/01/08 13:27:42 WARN tool.BaseSqoopTool: Setting your password on the
command-line is insecure. Consider using -P instead.
13/01/08 13:27:42 INFO tool.BaseSqoopTool: Using Hive-specific delimiters
for output. You can override
13/01/08 13:27:42 INFO tool.BaseSqoopTool: delimiters with
--fields-terminated-by, etc.
13/01/08 13:27:42 INFO manager.SqlManager: Using default fetchSize of 1000
13/01/08 13:27:42 INFO tool.CodeGenTool: Beginning code generation
13/01/08 13:27:42 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM TABLE AS t WHERE 1=0
13/01/08 13:27:42 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM TABLE AS t WHERE 1=0
13/01/08 13:27:42 INFO orm.CompilationManager: HADOOP_HOME is
/usr/lib/hadoop
Note:
/tmp/sqoop-saachhra/compile/d347fd96a56c490d589ea9522e92d7a0/optqc5d_TABLE.java
uses or overrides a deprecated API.
 Note: Recompile with -Xlint:deprecation for details.
13/01/08 13:27:44 INFO orm.CompilationManager: Writing jar file:
/tmp/sqoop-saachhra/compile/d347fd96a56c490d589ea9522e92d7a0/TABLE.jar
13/01/08 13:27:44 INFO mapreduce.ImportJobBase: Beginning import of TABLE
13/01/08 13:27:44 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM TABLE AS t WHERE 1=0
13/01/08 13:27:44 WARN snappy.LoadSnappy: Snappy native library is available
13/01/08 13:27:44 INFO snappy.LoadSnappy: Snappy native library loaded
13/01/08 13:27:46 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
13/01/08 13:27:46 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token
1148 for saachhra on 10.207.37.75:8020
13/01/08 13:27:46 INFO security.TokenCache: Got dt for
hdfs://sl73caeq02.visa.com:8020;uri=10.207.37.75:8020;t.service10.207.37.75:8020
13/01/08 13:27:46 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token
1149 for saachhra on 10.207.37.75:8020
13/01/08 13:27:46 INFO security.TokenCache: Got dt for hdfs://
sl73caeq02.visa.com:8020/user/saachhra/.staging/job_201301072354_0114/libjars/ant-eclipse-1.0-jvm1.2.jar;uri=10.207.37.75:8020;t.service=10.207.37.75:8020
13/01/08 13:27:47 INFO mapred.JobClient: Running job: job_201301072354_0114
13/01/08 13:27:48 INFO mapred.JobClient:  map 0% reduce 0%
13/01/08 13:27:59 INFO mapred.JobClient:  map 100% reduce 0%
13/01/08 13:28:00 INFO mapred.JobClient: Job complete: job_201301072354_0114
13/01/08 13:28:00 INFO mapred.JobClient: Counters: 23
13/01/08 13:28:00 INFO mapred.JobClient:   File System Counters
13/01/08 13:28:00 INFO mapred.JobClient:     FILE: Number of bytes read=0
13/01/08 13:28:00 INFO mapred.JobClient:     FILE: Number of bytes
written=170459
13/01/08 13:28:00 INFO mapred.JobClient:     FILE: Number of read
operations=0
13/01/08 13:28:00 INFO mapred.JobClient:     FILE: Number of large read
operations=0
13/01/08 13:28:00 INFO mapred.JobClient:     FILE: Number of write
operations=0
13/01/08 13:28:00 INFO mapred.JobClient:     HDFS: Number of bytes read=87
13/01/08 13:28:00 INFO mapred.JobClient:     HDFS: Number of bytes
written=7863800
13/01/08 13:28:00 INFO mapred.JobClient:     HDFS: Number of read
operations=1
13/01/08 13:28:00 INFO mapred.JobClient:     HDFS: Number of large read
operations=0
13/01/08 13:28:00 INFO mapred.JobClient:     HDFS: Number of write
operations=1
13/01/08 13:28:00 INFO mapred.JobClient:   Job Counters
13/01/08 13:28:00 INFO mapred.JobClient:     Launched map tasks=1
13/01/08 13:28:00 INFO mapred.JobClient:     Total time spent by all maps
in occupied slots (ms)=9118
13/01/08 13:28:00 INFO mapred.JobClient:     Total time spent by all
reduces in occupied slots (ms)=0
13/01/08 13:28:00 INFO mapred.JobClient:     Total time spent by all maps
waiting after reserving slots (ms)=0
13/01/08 13:28:00 INFO mapred.JobClient:     Total time spent by all
reduces waiting after reserving slots (ms)=0
13/01/08 13:28:00 INFO mapred.JobClient:   Map-Reduce Framework
13/01/08 13:28:00 INFO mapred.JobClient:     Map input records=14363
13/01/08 13:28:00 INFO mapred.JobClient:     Map output records=14363
13/01/08 13:28:00 INFO mapred.JobClient:     Input split bytes=87
13/01/08 13:28:00 INFO mapred.JobClient:     Spilled Records=0
13/01/08 13:28:00 INFO mapred.JobClient:     CPU time spent (ms)=4530
13/01/08 13:28:00 INFO mapred.JobClient:     Physical memory (bytes)
snapshot=303534080
13/01/08 13:28:00 INFO mapred.JobClient:     Virtual memory (bytes)
snapshot=1868890112
13/01/08 13:28:00 INFO mapred.JobClient:     Total committed heap usage
(bytes)=757792768
13/01/08 13:28:00 INFO mapreduce.ImportJobBase: Transferred 0 bytes in
15.6271 seconds (0 bytes/sec)
13/01/08 13:28:00 INFO mapreduce.ImportJobBase: Retrieved 14363 records.
13/01/08 13:28:00 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM TABLE AS t WHERE 1=0
13/01/08 13:28:00 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM TABLE AS t WHERE 1=0
13/01/08 13:28:00 INFO hive.HiveImport: Removing temporary files from
import process: hdfs://sl73caeq02.visa.com:8020/user/saachhra/TABLE/_logs
13/01/08 13:28:00 INFO hive.HiveImport: Loading uploaded data into Hive
13/01/08 13:28:02 INFO hive.HiveImport: Logging initialized using
configuration in file:/etc/hive/conf.dist/hive-log4j.properties
13/01/08 13:28:02 INFO hive.HiveImport: Hive history
file=/tmp/saachhra/hive_job_log_saachhra_201301081328_1669317999.txt
13/01/08 13:28:05 INFO hive.HiveImport: OK
13/01/08 13:28:05 INFO hive.HiveImport: Time taken: 2.487 seconds
13/01/08 13:28:06 INFO hive.HiveImport: Loading data to table TABLE
13/01/08 13:28:06 INFO hive.HiveImport: OK
13/01/08 13:28:06 INFO hive.HiveImport: Time taken: 0.918 seconds
13/01/08 13:28:06 INFO hive.HiveImport: Hive import complete.
13/01/08 13:28:06 INFO hive.HiveImport: Export directory is empty, removing
it.
sl73caeq02.visa.com:/export/home/saachhra:>

Good wishes,always !
Santosh
On Tue, Ja