Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop, mail # user - Creating compressed data with scooop


+
Santosh Achhra 2013-01-07, 11:34
+
Jarek Jarcec Cecho 2013-01-07, 12:31
+
Santosh Achhra 2013-01-08, 05:21
+
Jarek Jarcec Cecho 2013-01-08, 07:36
+
Santosh Achhra 2013-01-08, 08:01
+
Jarek Jarcec Cecho 2013-01-08, 11:09
+
Santosh Achhra 2013-01-08, 14:33
Copy link to this message
-
Re: Creating compressed data with scooop
Jarek Jarcec Cecho 2013-01-08, 15:43
Hi Santosh,
thank you for the log. I do not see anything suspicious, would you mind also providing log from one map task? In this particular case for job job_201301072354_0114 that Sqoop generates.

Jarcec

On Tue, Jan 08, 2013 at 10:33:58PM +0800, Santosh Achhra wrote:
> Hi Jarcec,
> I appreciate your time on this. Here is the log captured from terminal
> after I execute sqoop command
>
> 13/01/08 13:27:42 WARN tool.BaseSqoopTool: Setting your password on the
> command-line is insecure. Consider using -P instead.
> 13/01/08 13:27:42 INFO tool.BaseSqoopTool: Using Hive-specific delimiters
> for output. You can override
> 13/01/08 13:27:42 INFO tool.BaseSqoopTool: delimiters with
> --fields-terminated-by, etc.
> 13/01/08 13:27:42 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/01/08 13:27:42 INFO tool.CodeGenTool: Beginning code generation
> 13/01/08 13:27:42 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM TABLE AS t WHERE 1=0
> 13/01/08 13:27:42 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM TABLE AS t WHERE 1=0
> 13/01/08 13:27:42 INFO orm.CompilationManager: HADOOP_HOME is
> /usr/lib/hadoop
> Note:
> /tmp/sqoop-saachhra/compile/d347fd96a56c490d589ea9522e92d7a0/optqc5d_TABLE.java
> uses or overrides a deprecated API.
>  Note: Recompile with -Xlint:deprecation for details.
> 13/01/08 13:27:44 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-saachhra/compile/d347fd96a56c490d589ea9522e92d7a0/TABLE.jar
> 13/01/08 13:27:44 INFO mapreduce.ImportJobBase: Beginning import of TABLE
> 13/01/08 13:27:44 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM TABLE AS t WHERE 1=0
> 13/01/08 13:27:44 WARN snappy.LoadSnappy: Snappy native library is available
> 13/01/08 13:27:44 INFO snappy.LoadSnappy: Snappy native library loaded
> 13/01/08 13:27:46 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
> 13/01/08 13:27:46 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token
> 1148 for saachhra on 10.207.37.75:8020
> 13/01/08 13:27:46 INFO security.TokenCache: Got dt for
> hdfs://sl73caeq02.visa.com:8020;uri=10.207.37.75:8020;t.service> 10.207.37.75:8020
> 13/01/08 13:27:46 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token
> 1149 for saachhra on 10.207.37.75:8020
> 13/01/08 13:27:46 INFO security.TokenCache: Got dt for hdfs://
> sl73caeq02.visa.com:8020/user/saachhra/.staging/job_201301072354_0114/libjars/ant-eclipse-1.0-jvm1.2.jar;uri=10.207.37.75:8020;t.service=10.207.37.75:8020
> 13/01/08 13:27:47 INFO mapred.JobClient: Running job: job_201301072354_0114
> 13/01/08 13:27:48 INFO mapred.JobClient:  map 0% reduce 0%
> 13/01/08 13:27:59 INFO mapred.JobClient:  map 100% reduce 0%
> 13/01/08 13:28:00 INFO mapred.JobClient: Job complete: job_201301072354_0114
> 13/01/08 13:28:00 INFO mapred.JobClient: Counters: 23
> 13/01/08 13:28:00 INFO mapred.JobClient:   File System Counters
> 13/01/08 13:28:00 INFO mapred.JobClient:     FILE: Number of bytes read=0
> 13/01/08 13:28:00 INFO mapred.JobClient:     FILE: Number of bytes
> written=170459
> 13/01/08 13:28:00 INFO mapred.JobClient:     FILE: Number of read
> operations=0
> 13/01/08 13:28:00 INFO mapred.JobClient:     FILE: Number of large read
> operations=0
> 13/01/08 13:28:00 INFO mapred.JobClient:     FILE: Number of write
> operations=0
> 13/01/08 13:28:00 INFO mapred.JobClient:     HDFS: Number of bytes read=87
> 13/01/08 13:28:00 INFO mapred.JobClient:     HDFS: Number of bytes
> written=7863800
> 13/01/08 13:28:00 INFO mapred.JobClient:     HDFS: Number of read
> operations=1
> 13/01/08 13:28:00 INFO mapred.JobClient:     HDFS: Number of large read
> operations=0
> 13/01/08 13:28:00 INFO mapred.JobClient:     HDFS: Number of write
> operations=1
> 13/01/08 13:28:00 INFO mapred.JobClient:   Job Counters
> 13/01/08 13:28:00 INFO mapred.JobClient:     Launched map tasks=1
> 13/01/08 13:28:00 INFO mapred.JobClient:     Total time spent by all maps
+
Santosh Achhra 2013-01-08, 16:09
+
Jarek Jarcec Cecho 2013-01-09, 10:01
+
Santosh Achhra 2013-01-09, 11:45
+
Jarek Jarcec Cecho 2013-01-10, 09:40
+
Santosh Achhra 2013-01-10, 12:47