Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # user - Sqoop is at stuck


Copy link to this message
-
Re: Sqoop is at stuck
Felix GV 2013-04-27, 20:28
I think you need to make sure Sqoop has the proper Hadoop config in its
class path.

So, for example, if you have config files in /etc/hadoop/conf/ that give
the hostname:port of your remote NameNode and JobTracker, then that
directory needs to be in Sqoop's class path in order for Sqoop to be able
to interact with your Hadoop cluster, and if it doesn't find such config
files, then it reverts to its default behavior, which is the local mode.

--
Felix
On Sat, Apr 27, 2013 at 2:54 PM, hakki buyukcengiz <
[EMAIL PROTECTED]> wrote:

> Hi,
>
> How I switch to cluster mode?
>
> Thanks
>
>
> 2013/4/26 Jarek Jarcec Cecho <[EMAIL PROTECTED]>
>
>> Also it seems that you're using LocalJobRunner instead of cluster mode.
>> Whereas importing 12KB in local mode is perfectly fine, I would recommend
>> to use real Hadoop cluster if you will start Sqoop bigger data sets.
>>
>> Jarcec
>>
>> On Fri, Apr 26, 2013 at 09:46:40AM -0400, Felix GV wrote:
>> > Well, that's probably because what you're sqooping is too small.
>> >
>> > 144 records for a total of 12 KB !
>> >
>> > A map reduce job is a pretty huge overhead for such a tiny task hehe...!
>> >
>> > --
>> > Félix
>> >
>> > On 2013-04-26, at 3:38, hakki buyukcengiz <[EMAIL PROTECTED]>
>> wrote:
>> >
>> > > Hi,
>> > >
>> > > I have another problem. The sqoop output does not give the map reduce
>> amounts, it did not print map %0, but it printed only map %100.
>> > >
>> > >
>> > > 13/04/26 10:10:17 WARN tool.BaseSqoopTool: Setting your password on
>> the command-
>> > > 13/04/26 10:10:17 INFO manager.SqlManager: Using default fetchSize of
>> 1000
>> > > 13/04/26 10:10:17 INFO tool.CodeGenTool: Beginning code generation
>> > > 13/04/26 10:10:24 INFO manager.OracleManager: Time zone has been set
>> to GMT
>> > > 13/04/26 10:10:24 INFO manager.SqlManager: Executing SQL statement:
>> SELECT t.* FROM TEST_STG_HAKKI_4 t WHERE 1=0
>> > > 13/04/26 10:10:24 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
>> /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/bin/../lib/hadoop-0.20-mapreduce
>> > > 13/04/26 10:10:24 INFO orm.CompilationManager: Found hadoop core jar
>> at:
>> /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/bin/../lib/hadoop-0.20-mapreduce/hadoop-2.0.0-mr1-cdh4.2.0-core.jar
>> > > Note:
>> /tmp/sqoop-root/compile/7c2be3787d51d491a95963c2fd54134f/TEST_STG_HAKKI_4.java
>> uses or overrides a deprecated API.
>> > > Note: Recompile with -Xlint:deprecation for details.
>> > > 13/04/26 10:10:26 INFO orm.CompilationManager: Writing jar file:
>> /tmp/sqoop-root/compile/7c2be3787d51d491a95963c2fd54134f/TEST_STG_HAKKI_4.jar
>> > > 13/04/26 10:10:26 INFO mapreduce.ExportJobBase: Beginning export of
>> TEST_STG_HAKKI_4
>> > > 13/04/26 10:10:27 INFO manager.OracleManager: Time zone has been set
>> to GMT
>> > > 13/04/26 10:10:28 WARN conf.Configuration: session.id is deprecated.
>> Instead, use dfs.metrics.session-id
>> > > 13/04/26 10:10:28 INFO jvm.JvmMetrics: Initializing JVM Metrics with
>> processName=JobTracker, sessionId>> > > 13/04/26 10:10:28 WARN mapred.JobClient: Use GenericOptionsParser for
>> parsing the arguments. Applications should implement Tool for the same.
>> > > 13/04/26 10:10:28 INFO input.FileInputFormat: Total input paths to
>> process : 1
>> > > 13/04/26 10:10:28 INFO input.FileInputFormat: Total input paths to
>> process : 1
>> > > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager:
>> Creating mysql-connector-java-5.1.18-bin.jar in
>> /tmp/hadoop-root/mapred/local/archive/3678350083127005779_1604051290_906865802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib-work-1070016945669398427
>> with rwxr-xr-x
>> > > 13/04/26 10:10:28 INFO filecache.TrackerDistributedCacheManager:
>> Cached
>> file:///opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar
>> as
>> /tmp/hadoop-root/mapred/local/archive/3678350083127005779_1604051290_906865802/file/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/sqoop/lib/mysql-connector-java-5.1.18-bin.jar