Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Sqoop is at stuck


Copy link to this message
-
Re: Sqoop is at stuck
Hi,

How I switch to cluster mode?

Thanks
2013/4/26 Jarek Jarcec Cecho <[EMAIL PROTECTED]>

> Also it seems that you're using LocalJobRunner instead of cluster mode.
> Whereas importing 12KB in local mode is perfectly fine, I would recommend
> to use real Hadoop cluster if you will start Sqoop bigger data sets.
>
> Jarcec
>
> On Fri, Apr 26, 2013 at 09:46:40AM -0400, Felix GV wrote:
> > Well, that's probably because what you're sqooping is too small.
> >
> > 144 records for a total of 12 KB !
> >
> > A map reduce job is a pretty huge overhead for such a tiny task hehe...!
> >
> > --
> > Félix
> >
> > On 2013-04-26, at 3:38, hakki buyukcengiz <[EMAIL PROTECTED]>
> wrote:
> >
> > > Hi,
> > >
> > > I have another problem. The sqoop output does not give the map reduce
> amounts, it did not print map %0, but it printed only map %100.
> > >
> > >
> > > 13/04/26 10:10:17 WARN tool.BaseSqoopTool: Setting your password on
> the command-
> > > 13/04/26 10:10:17 INFO manager.SqlManager: Using default fetchSize of
> 1000
> > > 13/04/26 10:10:17 INFO tool.CodeGenTool: Beginning code generation
> > > 13/04/26 10:10:24 INFO manager.OracleManager: Time zone has been set
> to GMT
> > > 13/04/26 10:10:24 INFO manager.SqlManager: Executing SQL statement:
> SELECT t.* FROM TEST_STG_HAKKI_4 t WHERE 1=0
> > > 13/04/26 10:10:24 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
> /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/bin/../lib/hadoop-0.20-mapreduce
> > > 13/04/26 10:10:24 INFO orm.CompilationManager: Found hadoop core jar
> at:
> /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/bin/../lib/hadoop-0.20-mapreduce/hadoop-2.0.0-mr1-cdh4.2.0-core.jar
> > > Note:
> /tmp/sqoop-root/compile/7c2be3787d51d491a95963c2fd54134f/TEST_STG_HAKKI_4.java
> uses or overrides a deprecated API.
> > > Note: Recompile with -Xlint:deprecation for details.
> > > 13/04/26 10:10:26 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-root/compile/7c2be3787d51d491a95963c2fd54134f/TEST_STG_HAKKI_4.jar
> > > 13/04/26 10:10:26 INFO mapreduce.ExportJobBase: Beginning export of
> TEST_STG_HAKKI_4
> > > 13/04/26 10:10:27 INFO manager.OracleManager: Time zone has been set
> to GMT
> > > 13/04/26 10:10:28 WARN conf.Configuration: session.id is deprecated.
> Instead, use dfs.metrics.session-id
> > > 13/04/26 10:10:28 INFO jvm.JvmMetrics: Initializing JVM Metrics with
> processName=JobTracker, sessionId> > > 13/04/26 10:10:28 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
> > > 13/04/26 10:10:28 INFO input.FileInputFormat: Total input paths to
> process : 1
> > > 13/04/26 10:10:28 INFO input.FileInputFormat: Total input paths to
> process : 1
> > > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager:
> Creating mysql-connector-java-5.1.18-bin.jar in
> /tmp/hadoop-root/mapred/local/archive/3678350083127005779_1604051290_906865802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-1070016945669398427
> with rwxr-xr-x
> > > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager:
> Cached
> file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar
> as
> /tmp/hadoop-root/mapred/local/archive/3678350083127005779_1604051290_906865802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar
> > > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager:
> Cached
> file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar
> as
> /tmp/hadoop-root/mapred/local/archive/3678350083127005779_1604051290_906865802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar
> > > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager:
> Creating paranamer-2.3.jar in
> /tmp/hadoop-root/mapred/local/archive/1894696295733945198_1287407348_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-8994443185246229890

Hakki Buyukcengiz
TR