Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> Creating compressed data with scooop


+
Santosh Achhra 2013-01-07, 11:34
+
Jarek Jarcec Cecho 2013-01-07, 12:31
+
Santosh Achhra 2013-01-08, 05:21
+
Jarek Jarcec Cecho 2013-01-08, 07:36
+
Santosh Achhra 2013-01-08, 08:01
+
Jarek Jarcec Cecho 2013-01-08, 11:09
Copy link to this message
-
Re: Creating compressed data with scooop
Hi Jarcec,
I appreciate your time on this. Here is the log captured from terminal
after I execute sqoop command

13/01/08 13:27:42 WARN tool.BaseSqoopTool: Setting your password on the
command-line is insecure. Consider using -P instead.
13/01/08 13:27:42 INFO tool.BaseSqoopTool: Using Hive-specific delimiters
for output. You can override
13/01/08 13:27:42 INFO tool.BaseSqoopTool: delimiters with
--fields-terminated-by, etc.
13/01/08 13:27:42 INFO manager.SqlManager: Using default fetchSize of 1000
13/01/08 13:27:42 INFO tool.CodeGenTool: Beginning code generation
13/01/08 13:27:42 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM TABLE AS t WHERE 1=0
13/01/08 13:27:42 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM TABLE AS t WHERE 1=0
13/01/08 13:27:42 INFO orm.CompilationManager: HADOOP_HOME is
/usr/lib/hadoop
Note:
/tmp/sqoop-saachhra/compile/d347fd96a56c490d589ea9522e92d7a0/optqc5d_TABLE.java
uses or overrides a deprecated API.
 Note: Recompile with -Xlint:deprecation for details.
13/01/08 13:27:44 INFO orm.CompilationManager: Writing jar file:
/tmp/sqoop-saachhra/compile/d347fd96a56c490d589ea9522e92d7a0/TABLE.jar
13/01/08 13:27:44 INFO mapreduce.ImportJobBase: Beginning import of TABLE
13/01/08 13:27:44 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM TABLE AS t WHERE 1=0
13/01/08 13:27:44 WARN snappy.LoadSnappy: Snappy native library is available
13/01/08 13:27:44 INFO snappy.LoadSnappy: Snappy native library loaded
13/01/08 13:27:46 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
13/01/08 13:27:46 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token
1148 for saachhra on 10.207.37.75:8020
13/01/08 13:27:46 INFO security.TokenCache: Got dt for
hdfs://sl73caeq02.visa.com:8020;uri=10.207.37.75:8020;t.service10.207.37.75:8020
13/01/08 13:27:46 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token
1149 for saachhra on 10.207.37.75:8020
13/01/08 13:27:46 INFO security.TokenCache: Got dt for hdfs://
sl73caeq02.visa.com:8020/user/saachhra/.staging/job_201301072354_0114/libjars/ant-eclipse-1.0-jvm1.2.jar;uri=10.207.37.75:8020;t.service=10.207.37.75:8020
13/01/08 13:27:47 INFO mapred.JobClient: Running job: job_201301072354_0114
13/01/08 13:27:48 INFO mapred.JobClient:  map 0% reduce 0%
13/01/08 13:27:59 INFO mapred.JobClient:  map 100% reduce 0%
13/01/08 13:28:00 INFO mapred.JobClient: Job complete: job_201301072354_0114
13/01/08 13:28:00 INFO mapred.JobClient: Counters: 23
13/01/08 13:28:00 INFO mapred.JobClient:   File System Counters
13/01/08 13:28:00 INFO mapred.JobClient:     FILE: Number of bytes read=0
13/01/08 13:28:00 INFO mapred.JobClient:     FILE: Number of bytes
written=170459
13/01/08 13:28:00 INFO mapred.JobClient:     FILE: Number of read
operations=0
13/01/08 13:28:00 INFO mapred.JobClient:     FILE: Number of large read
operations=0
13/01/08 13:28:00 INFO mapred.JobClient:     FILE: Number of write
operations=0
13/01/08 13:28:00 INFO mapred.JobClient:     HDFS: Number of bytes read=87
13/01/08 13:28:00 INFO mapred.JobClient:     HDFS: Number of bytes
written=7863800
13/01/08 13:28:00 INFO mapred.JobClient:     HDFS: Number of read
operations=1
13/01/08 13:28:00 INFO mapred.JobClient:     HDFS: Number of large read
operations=0
13/01/08 13:28:00 INFO mapred.JobClient:     HDFS: Number of write
operations=1
13/01/08 13:28:00 INFO mapred.JobClient:   Job Counters
13/01/08 13:28:00 INFO mapred.JobClient:     Launched map tasks=1
13/01/08 13:28:00 INFO mapred.JobClient:     Total time spent by all maps
in occupied slots (ms)=9118
13/01/08 13:28:00 INFO mapred.JobClient:     Total time spent by all
reduces in occupied slots (ms)=0
13/01/08 13:28:00 INFO mapred.JobClient:     Total time spent by all maps
waiting after reserving slots (ms)=0
13/01/08 13:28:00 INFO mapred.JobClient:     Total time spent by all
reduces waiting after reserving slots (ms)=0
13/01/08 13:28:00 INFO mapred.JobClient:   Map-Reduce Framework
13/01/08 13:28:00 INFO mapred.JobClient:     Map input records=14363
13/01/08 13:28:00 INFO mapred.JobClient:     Map output records=14363
13/01/08 13:28:00 INFO mapred.JobClient:     Input split bytes=87
13/01/08 13:28:00 INFO mapred.JobClient:     Spilled Records=0
13/01/08 13:28:00 INFO mapred.JobClient:     CPU time spent (ms)=4530
13/01/08 13:28:00 INFO mapred.JobClient:     Physical memory (bytes)
snapshot=303534080
13/01/08 13:28:00 INFO mapred.JobClient:     Virtual memory (bytes)
snapshot=1868890112
13/01/08 13:28:00 INFO mapred.JobClient:     Total committed heap usage
(bytes)=757792768
13/01/08 13:28:00 INFO mapreduce.ImportJobBase: Transferred 0 bytes in
15.6271 seconds (0 bytes/sec)
13/01/08 13:28:00 INFO mapreduce.ImportJobBase: Retrieved 14363 records.
13/01/08 13:28:00 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM TABLE AS t WHERE 1=0
13/01/08 13:28:00 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM TABLE AS t WHERE 1=0
13/01/08 13:28:00 INFO hive.HiveImport: Removing temporary files from
import process: hdfs://sl73caeq02.visa.com:8020/user/saachhra/TABLE/_logs
13/01/08 13:28:00 INFO hive.HiveImport: Loading uploaded data into Hive
13/01/08 13:28:02 INFO hive.HiveImport: Logging initialized using
configuration in file:/etc/hive/conf.dist/hive-log4j.properties
13/01/08 13:28:02 INFO hive.HiveImport: Hive history
file=/tmp/saachhra/hive_job_log_saachhra_201301081328_1669317999.txt
13/01/08 13:28:05 INFO hive.HiveImport: OK
13/01/08 13:28:05 INFO hive.HiveImport: Time taken: 2.487 seconds
13/01/08 13:28:06 INFO hive.HiveImport: Loading data to table TABLE
13/01/08 13:28:06 INFO hive.HiveImport: OK
13/01/08 13:28:06 INFO hive.HiveImport: Time taken: 0.918 seconds
13/01/08 13:28:06 INFO hive.HiveImport: Hive import complete.
13/01/08 13:28:06 INFO hive.HiveImport: Export directory is empty, removing
it.
sl73caeq02.visa.com:/export/home/saachhra:>

Good wishes,always !
Santosh
On Tue, Ja
+
Jarek Jarcec Cecho 2013-01-08, 15:43
+
Santosh Achhra 2013-01-08, 16:09
+
Jarek Jarcec Cecho 2013-01-09, 10:01
+
Santosh Achhra 2013-01-09, 11:45
+
Jarek Jarcec Cecho 2013-01-10, 09:40
+
Santosh Achhra 2013-01-10, 12:47
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB