Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Re: Distributing the code to multiple nodes


Copy link to this message
-
Re: Distributing the code to multiple nodes
My execution is stuck at this position indefinitely:

[root@l1-dev06 bin]# ./hadoop jar /opt/ApacheHadoop/wordCount.jar
/opt/ApacheHadoop/temp/worker.log /opt/ApacheHadoop/OUT56
14/01/15 19:35:12 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
14/01/15 19:35:13 INFO client.RMProxy: Connecting to ResourceManager at /
10.12.11.210:1003
14/01/15 19:35:13 INFO client.RMProxy: Connecting to ResourceManager at /
10.12.11.210:1003
14/01/15 19:35:13 WARN mapreduce.JobSubmitter: Hadoop command-line option
parsing not performed. Implement the Tool interface and execute your
application with ToolRunner to remedy this.
14/01/15 19:35:14 INFO mapred.FileInputFormat: Total input paths to process
: 1
14/01/15 19:35:14 INFO mapreduce.JobSubmitter: number of splits:8
14/01/15 19:35:14 INFO Configuration.deprecation: user.name is deprecated.
Instead, use mapreduce.job.user.name
14/01/15 19:35:14 INFO Configuration.deprecation: mapred.jar is deprecated.
Instead, use mapreduce.job.jar
14/01/15 19:35:14 INFO Configuration.deprecation: mapred.output.value.class
is deprecated. Instead, use mapreduce.job.output.value.class
14/01/15 19:35:14 INFO Configuration.deprecation: mapred.job.name is
deprecated. Instead, use mapreduce.job.name
14/01/15 19:35:14 INFO Configuration.deprecation: mapred.input.dir is
deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
14/01/15 19:35:14 INFO Configuration.deprecation: mapred.output.dir is
deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
14/01/15 19:35:14 INFO Configuration.deprecation: mapred.map.tasks is
deprecated. Instead, use mapreduce.job.maps
14/01/15 19:35:14 INFO Configuration.deprecation: mapred.output.key.class
is deprecated. Instead, use mapreduce.job.output.key.class
14/01/15 19:35:14 INFO Configuration.deprecation: mapred.working.dir is
deprecated. Instead, use mapreduce.job.working.dir
14/01/15 19:35:14 INFO mapreduce.JobSubmitter: Submitting tokens for job:
job_1389794591210_0001
14/01/15 19:35:15 INFO impl.YarnClientImpl: Submitted application
application_1389794591210_0001 to ResourceManager at /10.12.11.210:1003
14/01/15 19:35:15 INFO mapreduce.Job: The url to track the job:
http://l1-dev06:8088/proxy/application_1389794591210_0001/
14/01/15 19:35:15 INFO mapreduce.Job: Running job: job_1389794591210_0001

On Wed, Jan 15, 2014 at 7:20 PM, Ashish Jain <[EMAIL PROTECTED]> wrote:

> I just now tried it again and I see following messages popping up in the
> log file:
>
> 2014-01-15 19:37:38,221 WARN
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue:
> Node : l1dev-211:1004 does not have sufficient resource for request :
> {Priority: 0, Capability: <memory:2048, vCores:1>, # Containers: 1,
> Location: *, Relax Locality: true} node total capability : <memory:1024,
> vCores:8>
> 2014-01-15 19:37:38,621 WARN
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue:
> Node : l1-dev06:1004 does not have sufficient resource for request :
> {Priority: 0, Capability: <memory:2048, vCores:1>, # Containers: 1,
> Location: *, Relax Locality: true} node total capability : <memory:1024,
> vCores:8>
>
> Do I need to increase the RAM allocation to slave nodes??
>
>
>
> On Wed, Jan 15, 2014 at 7:07 PM, Ashish Jain <[EMAIL PROTECTED]> wrote:
>
>> I tried that but somehow my map reduce jobs do not execute at all once I
>> set it to yarn
>>
>>
>> On Wed, Jan 15, 2014 at 7:00 PM, Nirmal Kumar <[EMAIL PROTECTED]
>> > wrote:
>>
>>>  Surely you don’t have to set **mapreduce.jobtracker.address** in
>>> mapred-site.xml
>>>
>>>
>>>
>>> In mapred-site.xml you just have to mention:
>>>
>>> <property>
>>>
>>> <name>mapreduce.framework.name</name>
>>>
>>> <value>yarn</value>
>>>
>>> </property>
>>>
>>>
>>>
>>> -Nirmal
>>>
>>> *From:* Ashish Jain [mailto:[EMAIL PROTECTED]]
>>> *Sent:* Wednesday, January 15, 2014 6:44 PM
>>>
>>> *To:* [EMAIL PROTECTED]
>>> *Subject:* Re: Distributing the code to multiple nodes
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB