Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Re: Distributing the code to multiple nodes


Copy link to this message
-
Re: Distributing the code to multiple nodes
My execution is stuck at this position indefinitely:

[root@l1-dev06 bin]# ./hadoop jar /opt/ApacheHadoop/wordCount.jar
/opt/ApacheHadoop/temp/worker.log /opt/ApacheHadoop/OUT56
14/01/15 19:35:12 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
14/01/15 19:35:13 INFO client.RMProxy: Connecting to ResourceManager at /
10.12.11.210:1003
14/01/15 19:35:13 INFO client.RMProxy: Connecting to ResourceManager at /
10.12.11.210:1003
14/01/15 19:35:13 WARN mapreduce.JobSubmitter: Hadoop command-line option
parsing not performed. Implement the Tool interface and execute your
application with ToolRunner to remedy this.
14/01/15 19:35:14 INFO mapred.FileInputFormat: Total input paths to process
: 1
14/01/15 19:35:14 INFO mapreduce.JobSubmitter: number of splits:8
14/01/15 19:35:14 INFO Configuration.deprecation: user.name is deprecated.
Instead, use mapreduce.job.user.name
14/01/15 19:35:14 INFO Configuration.deprecation: mapred.jar is deprecated.
Instead, use mapreduce.job.jar
14/01/15 19:35:14 INFO Configuration.deprecation: mapred.output.value.class
is deprecated. Instead, use mapreduce.job.output.value.class
14/01/15 19:35:14 INFO Configuration.deprecation: mapred.job.name is
deprecated. Instead, use mapreduce.job.name
14/01/15 19:35:14 INFO Configuration.deprecation: mapred.input.dir is
deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
14/01/15 19:35:14 INFO Configuration.deprecation: mapred.output.dir is
deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
14/01/15 19:35:14 INFO Configuration.deprecation: mapred.map.tasks is
deprecated. Instead, use mapreduce.job.maps
14/01/15 19:35:14 INFO Configuration.deprecation: mapred.output.key.class
is deprecated. Instead, use mapreduce.job.output.key.class
14/01/15 19:35:14 INFO Configuration.deprecation: mapred.working.dir is
deprecated. Instead, use mapreduce.job.working.dir
14/01/15 19:35:14 INFO mapreduce.JobSubmitter: Submitting tokens for job:
job_1389794591210_0001
14/01/15 19:35:15 INFO impl.YarnClientImpl: Submitted application
application_1389794591210_0001 to ResourceManager at /10.12.11.210:1003
14/01/15 19:35:15 INFO mapreduce.Job: The url to track the job:
http://l1-dev06:8088/proxy/application_1389794591210_0001/
14/01/15 19:35:15 INFO mapreduce.Job: Running job: job_1389794591210_0001

On Wed, Jan 15, 2014 at 7:20 PM, Ashish Jain <[EMAIL PROTECTED]> wrote:

> I just now tried it again and I see following messages popping up in the
> log file:
>
> 2014-01-15 19:37:38,221 WARN
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue:
> Node : l1dev-211:1004 does not have sufficient resource for request :
> {Priority: 0, Capability: <memory:2048, vCores:1>, # Containers: 1,
> Location: *, Relax Locality: true} node total capability : <memory:1024,
> vCores:8>
> 2014-01-15 19:37:38,621 WARN
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue:
> Node : l1-dev06:1004 does not have sufficient resource for request :
> {Priority: 0, Capability: <memory:2048, vCores:1>, # Containers: 1,
> Location: *, Relax Locality: true} node total capability : <memory:1024,
> vCores:8>
>
> Do I need to increase the RAM allocation to slave nodes??
>
>
>
> On Wed, Jan 15, 2014 at 7:07 PM, Ashish Jain <[EMAIL PROTECTED]> wrote:
>
>> I tried that but somehow my map reduce jobs do not execute at all once I
>> set it to yarn
>>
>>
>> On Wed, Jan 15, 2014 at 7:00 PM, Nirmal Kumar <[EMAIL PROTECTED]
>> > wrote:
>>
>>>  Surely you don’t have to set **mapreduce.jobtracker.address** in
>>> mapred-site.xml
>>>
>>>
>>>
>>> In mapred-site.xml you just have to mention:
>>>
>>> <property>
>>>
>>> <name>mapreduce.framework.name</name>
>>>
>>> <value>yarn</value>
>>>
>>> </property>
>>>
>>>
>>>
>>> -Nirmal
>>>
>>> *From:* Ashish Jain [mailto:[EMAIL PROTECTED]]
>>> *Sent:* Wednesday, January 15, 2014 6:44 PM
>>>
>>> *To:* [EMAIL PROTECTED]
>>> *Subject:* Re: Distributing the code to multiple nodes