Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Sqoop is at stuck


Copy link to this message
-
Re: Sqoop is at stuck
Hi,

How I switch to cluster mode?

Thanks
2013/4/26 Jarek Jarcec Cecho <[EMAIL PROTECTED]>

> Also it seems that you're using LocalJobRunner instead of cluster mode.
> Whereas importing 12KB in local mode is perfectly fine, I would recommend
> to use real Hadoop cluster if you will start Sqoop bigger data sets.
>
> Jarcec
>
> On Fri, Apr 26, 2013 at 09:46:40AM -0400, Felix GV wrote:
> > Well, that's probably because what you're sqooping is too small.
> >
> > 144 records for a total of 12 KB !
> >
> > A map reduce job is a pretty huge overhead for such a tiny task hehe...!
> >
> > --
> > Félix
> >
> > On 2013-04-26, at 3:38, hakki buyukcengiz <[EMAIL PROTECTED]>
> wrote:
> >
> > > Hi,
> > >
> > > I have another problem. The sqoop output does not give the map reduce
> amounts, it did not print map %0, but it printed only map %100.
> > >
> > >
> > > 13/04/26 10:10:17 WARN tool.BaseSqoopTool: Setting your password on
> the command-
> > > 13/04/26 10:10:17 INFO manager.SqlManager: Using default fetchSize of
> 1000
> > > 13/04/26 10:10:17 INFO tool.CodeGenTool: Beginning code generation
> > > 13/04/26 10:10:24 INFO manager.OracleManager: Time zone has been set
> to GMT
> > > 13/04/26 10:10:24 INFO manager.SqlManager: Executing SQL statement:
> SELECT t.* FROM TEST_STG_HAKKI_4 t WHERE 1=0
> > > 13/04/26 10:10:24 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
> /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/bin/../lib/hadoop-0.20-mapreduce
> > > 13/04/26 10:10:24 INFO orm.CompilationManager: Found hadoop core jar
> at:
> /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/bin/../lib/hadoop-0.20-mapreduce/hadoop-2.0.0-mr1-cdh4.2.0-core.jar
> > > Note:
> /tmp/sqoop-root/compile/7c2be3787d51d491a95963c2fd54134f/TEST_STG_HAKKI_4.java
> uses or overrides a deprecated API.
> > > Note: Recompile with -Xlint:deprecation for details.
> > > 13/04/26 10:10:26 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-root/compile/7c2be3787d51d491a95963c2fd54134f/TEST_STG_HAKKI_4.jar
> > > 13/04/26 10:10:26 INFO mapreduce.ExportJobBase: Beginning export of
> TEST_STG_HAKKI_4
> > > 13/04/26 10:10:27 INFO manager.OracleManager: Time zone has been set
> to GMT
> > > 13/04/26 10:10:28 WARN conf.Configuration: session.id is deprecated.
> Instead, use dfs.metrics.session-id
> > > 13/04/26 10:10:28 INFO jvm.JvmMetrics: Initializing JVM Metrics with
> processName=JobTracker, sessionId> > > 13/04/26 10:10:28 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
> > > 13/04/26 10:10:28 INFO input.FileInputFormat: Total input paths to
> process : 1
> > > 13/04/26 10:10:28 INFO input.FileInputFormat: Total input paths to
> process : 1
> > > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager:
> Creating mysql-connector-java-5.1.18-bin.jar in
> /tmp/hadoop-root/mapred/local/archive/3678350083127005779_1604051290_906865802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-1070016945669398427
> with rwxr-xr-x
> > > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager:
> Cached
> file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar
> as
> /tmp/hadoop-root/mapred/local/archive/3678350083127005779_1604051290_906865802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar
> > > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager:
> Cached
> file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar
> as
> /tmp/hadoop-root/mapred/local/archive/3678350083127005779_1604051290_906865802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar
> > > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager:
> Creating paranamer-2.3.jar in
> /tmp/hadoop-root/mapred/local/archive/1894696295733945198_1287407348_259918802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-8994443185246229890

Hakki Buyukcengiz
TR
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB