Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # user - Sqoop is at stuck


Copy link to this message
-
Re: Sqoop is at stuck
Jarek Jarcec Cecho 2013-04-26, 17:42
Also it seems that you're using LocalJobRunner instead of cluster mode. Whereas importing 12KB in local mode is perfectly fine, I would recommend to use real Hadoop cluster if you will start Sqoop bigger data sets.

Jarcec

On Fri, Apr 26, 2013 at 09:46:40AM -0400, Felix GV wrote:
> Well, that's probably because what you're sqooping is too small.
>
> 144 records for a total of 12 KB !
>
> A map reduce job is a pretty huge overhead for such a tiny task hehe...!
>
> --
> Félix
>
> On 2013-04-26, at 3:38, hakki buyukcengiz <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> > I have another problem. The sqoop output does not give the map reduce amounts, it did not print map %0, but it printed only map %100.
> >
> >
> > 13/04/26 10:10:17 WARN tool.BaseSqoopTool: Setting your password on the command-
> > 13/04/26 10:10:17 INFO manager.SqlManager: Using default fetchSize of 1000
> > 13/04/26 10:10:17 INFO tool.CodeGenTool: Beginning code generation
> > 13/04/26 10:10:24 INFO manager.OracleManager: Time zone has been set to GMT
> > 13/04/26 10:10:24 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM TEST_STG_HAKKI_4 t WHERE 1=0
> > 13/04/26 10:10:24 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/bin/../lib/hadoop-0.20-mapreduce
> > 13/04/26 10:10:24 INFO orm.CompilationManager: Found hadoop core jar at: /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/bin/../lib/hadoop-0.20-mapreduce/hadoop-2.0.0-mr1-cdh4.2.0-core.jar
> > Note: /tmp/sqoop-root/compile/7c2be3787d51d491a95963c2fd54134f/TEST_STG_HAKKI_4.java uses or overrides a deprecated API.
> > Note: Recompile with -Xlint:deprecation for details.
> > 13/04/26 10:10:26 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/7c2be3787d51d491a95963c2fd54134f/TEST_STG_HAKKI_4.jar
> > 13/04/26 10:10:26 INFO mapreduce.ExportJobBase: Beginning export of TEST_STG_HAKKI_4
> > 13/04/26 10:10:27 INFO manager.OracleManager: Time zone has been set to GMT
> > 13/04/26 10:10:28 WARN conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id
> > 13/04/26 10:10:28 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId> > 13/04/26 10:10:28 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> > 13/04/26 10:10:28 INFO input.FileInputFormat: Total input paths to process : 1
> > 13/04/26 10:10:28 INFO input.FileInputFormat: Total input paths to process : 1
> > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating mysql-connector-java-5.1.18-bin.jar in /tmp/hadoop-root/mapred/local/archive/3678350083127005779_1604051290_906865802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-1070016945669398427 with rwxr-xr-x
> > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar as /tmp/hadoop-root/mapred/local/archive/3678350083127005779_1604051290_906865802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar
> > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar as /tmp/hadoop-root/mapred/local/archive/3678350083127005779_1604051290_906865802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar
> > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating paranamer-2.3.jar in /tmp/hadoop-root/mapred/local/archive/1894696295733945198_1287407348_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-8994443185246229890 with rwxr-xr-x
> > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/paranamer-2.3.jar as /tmp/hadoop-root/mapred/local/archive/1894696295733945198_1287407348_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/paranamer-2.3.jar