Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Sqoop is at stuck


Copy link to this message
-
Re: Sqoop is at stuck
Also it seems that you're using LocalJobRunner instead of cluster mode. Whereas importing 12KB in local mode is perfectly fine, I would recommend to use real Hadoop cluster if you will start Sqoop bigger data sets.

Jarcec

On Fri, Apr 26, 2013 at 09:46:40AM -0400, Felix GV wrote:
> Well, that's probably because what you're sqooping is too small.
>
> 144 records for a total of 12 KB !
>
> A map reduce job is a pretty huge overhead for such a tiny task hehe...!
>
> --
> Félix
>
> On 2013-04-26, at 3:38, hakki buyukcengiz <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> > I have another problem. The sqoop output does not give the map reduce amounts, it did not print map %0, but it printed only map %100.
> >
> >
> > 13/04/26 10:10:17 WARN tool.BaseSqoopTool: Setting your password on the command-
> > 13/04/26 10:10:17 INFO manager.SqlManager: Using default fetchSize of 1000
> > 13/04/26 10:10:17 INFO tool.CodeGenTool: Beginning code generation
> > 13/04/26 10:10:24 INFO manager.OracleManager: Time zone has been set to GMT
> > 13/04/26 10:10:24 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM TEST_STG_HAKKI_4 t WHERE 1=0
> > 13/04/26 10:10:24 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/bin/../lib/hadoop-0.20-mapreduce
> > 13/04/26 10:10:24 INFO orm.CompilationManager: Found hadoop core jar at: /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/bin/../lib/hadoop-0.20-mapreduce/hadoop-2.0.0-mr1-cdh4.2.0-core.jar
> > Note: /tmp/sqoop-root/compile/7c2be3787d51d491a95963c2fd54134f/TEST_STG_HAKKI_4.java uses or overrides a deprecated API.
> > Note: Recompile with -Xlint:deprecation for details.
> > 13/04/26 10:10:26 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/7c2be3787d51d491a95963c2fd54134f/TEST_STG_HAKKI_4.jar
> > 13/04/26 10:10:26 INFO mapreduce.ExportJobBase: Beginning export of TEST_STG_HAKKI_4
> > 13/04/26 10:10:27 INFO manager.OracleManager: Time zone has been set to GMT
> > 13/04/26 10:10:28 WARN conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id
> > 13/04/26 10:10:28 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId> > 13/04/26 10:10:28 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> > 13/04/26 10:10:28 INFO input.FileInputFormat: Total input paths to process : 1
> > 13/04/26 10:10:28 INFO input.FileInputFormat: Total input paths to process : 1
> > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating mysql-connector-java-5.1.18-bin.jar in /tmp/hadoop-root/mapred/local/archive/3678350083127005779_1604051290_906865802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-1070016945669398427 with rwxr-xr-x
> > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar as /tmp/hadoop-root/mapred/local/archive/3678350083127005779_1604051290_906865802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar
> > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar as /tmp/hadoop-root/mapred/local/archive/3678350083127005779_1604051290_906865802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar
> > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Creating paranamer-2.3.jar in /tmp/hadoop-root/mapred/local/archive/1894696295733945198_1287407348_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-8994443185246229890 with rwxr-xr-x
> > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager: Cached file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/paranamer-2.3.jar as /tmp/hadoop-root/mapred/local/archive/1894696295733945198_1287407348_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/paranamer-2.3.jar
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB