|
Dibyendu Karmakar
2012-12-04, 10:55
Dibyendu Karmakar
2012-12-04, 11:01
Mohammad Tariq
2012-12-04, 11:29
Nitin Pawar
2012-12-04, 12:49
Robert Molina
2012-12-04, 20:58
Dibyendu Karmakar
2012-12-05, 07:32
Robert Molina
2012-12-05, 22:50
Dibyendu Karmakar
2012-12-06, 02:32
|
-
Re: Task Tracker Not Starting in Hadoop-1.0.3 Help PleaseDibyendu Karmakar 2012-12-04, 10:55
Yes, the jps command gives me the following...
on 172.16.4.120 --- 10160 SecondaryNameNode 9794 NameNode 9980 DataNode 9388 Jps 16267 JobTracker on 172.16.4.14 -- 6812 Jps 4523 DataNode On 12/4/12, Mohammad Tariq <[EMAIL PROTECTED]> wrote: > Hello Dibyendu, > > Are you able to see anything at http://172.16.4.120:50030?What does > JPS show on your terminal? Whatever logs you have posted here look > incomplete to me. Can you do a quick recheck? > > Regards, > Mohammad Tariq > > > > On Tue, Dec 4, 2012 at 3:55 PM, Dibyendu Karmakar > <[EMAIL PROTECTED]>wrote: > >> Hiii, I am new in hadoop. Trying to configure it in fully distributed >> mode. >> But after the command bin/start-all.sh or bin/start-mapred.sh or >> bin/hadoop-daemon.sh start tasktracker, TASKTRACKER IS GOING DOWN >> IMMEDIATELY WITHOUT ANY ERROR IN LOG FILE. >> HELP PLEASE... >> >> I'm using hadoop 1.0.3 version. >> I am using 2 computers - >> 172.16.4.12 (radix.itd.jusl.ac.in) - NAMENODE + JOBTRACKER + DATANODE >> 172.16.4.14 - DATANODE >> >> My conf is somewhat - >> < core-site.xml > >> fs.default.name ------ hdfs://172.16.4.120:9001 >> hadoop.tmp.dir ------ >> /home/usr/student/pg/yr11/dibyendu_karmakar/hadoop-1.0.3/temp >> >> < hdfs-site.xml > >> dfs.replication -- 2 >> dfs.name.dir --- >> /home/usr/student/pg/yr11/dibyendu_karmakar/hadoop-1.0.3/hadoop_name >> dfs.data.dir --- /tmp/hadoop_data >> >> < mapred-site.xml > >> mapred.job.tracker --- 172.16.4.120:9002 >> >> < slaves > >> 172.16.4.14 >> 172.16.4.120 >> >> < masters > >> 172.16.4.120 >> >> TASKTRACKER LOG FILE WITH NO ERROR -- ONLY INFO IS WRITTEN. >> < hadoop-dibyendu_karmakar-tasktracker-radix.itd.jusl.ac.in.log > >> 2012-12-04 15:40:22,831 INFO org.apache.hadoop.mapred.TaskTracker: >> STARTUP_MSG: >> /************************************************************ >> STARTUP_MSG: Starting TaskTracker >> STARTUP_MSG: host = radix.itd.jusl.ac.in/172.16.4.120 >> STARTUP_MSG: args = [] >> STARTUP_MSG: version = 1.0.3 >> STARTUP_MSG: build >> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r >> 1335192; compiled by 'hortonfo' on Tue May 8 20:31:25 UTC 2012 >> ************************************************************/ >> 2012-12-04 15:40:23,864 INFO >> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from >> hadoop-metrics2.properties2012-12-04 15:40:23,925 INFO >> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source >> MetricsSystem,sub=Stats registered. >> 2012-12-04 15:40:24,326 INFO >> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot >> period at 10 second(s). >> 2012-12-04 15:40:24,326 INFO >> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics >> system started >> 2012-12-04 15:40:24,559 INFO >> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source >> ugi registered. >> 2012-12-04 15:40:34,747 INFO org.mortbay.log: Logging to >> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via >> org.mortbay.log.Slf4jLog >> 2012-12-04 15:40:34,819 INFO org.apache.hadoop.http.HttpServer: Added >> global filtersafety >> (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) >> 2012-12-04 15:40:34,846 INFO >> org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' >> truncater with mapRetainSize=-1 and reduceRetainSize=-1 >> 2012-12-04 15:40:34,849 INFO org.apache.hadoop.mapred.TaskTracker: >> Starting tasktracker with owner as dibyendu_karmakar >> 2012-12-04 15:40:34,850 INFO org.apache.hadoop.mapred.TaskTracker: >> Good mapred local directories are: >> /home/usr/student/pg/yr11/dibyendu_karmakar/hadoop-1.0.3/temp/mapred/local >> >> PLEASE HELP ANYONE... >> IS THERE ANY CONFIGURATION NEEDED FOR TASKTRACKER? >> THANK YOU SO MUCH... >> -- >> Dibyendu Karmakar, >> < [EMAIL PROTECTED] > >> > -- Dibyendu Karmakar, < [EMAIL PROTECTED] >
-
Re: Task Tracker Not Starting in Hadoop-1.0.3 Help PleaseDibyendu Karmakar 2012-12-04, 11:01
THE ENTIRE CONTENT OF TASKTRACKER -- PASTED HERE WITH AN ATTACHMENT
ALSO --- THANK YOU FOR THE REPLY 2012-12-04 15:20:45,942 INFO org.apache.hadoop.mapred.TaskTracker: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting TaskTracker STARTUP_MSG: host = radix.itd.jusl.ac.in/172.16.4.120 STARTUP_MSG: args = [] STARTUP_MSG: version = 1.0.3 STARTUP_MSG: build https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1335192; compiled by 'hortonfo' on Tue May 8 20:31:25 UTC 2012 ************************************************************/ 2012-12-04 15:20:46,983 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2012-12-04 15:20:47,044 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered. 2012-12-04 15:20:47,445 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2012-12-04 15:20:47,445 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics system started 2012-12-04 15:20:48,478 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered. 2012-12-04 15:20:59,066 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2012-12-04 15:20:59,137 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) 2012-12-04 15:20:59,164 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1 2012-12-04 15:20:59,167 INFO org.apache.hadoop.mapred.TaskTracker: Starting tasktracker with owner as dibyendu_karmakar 2012-12-04 15:20:59,169 INFO org.apache.hadoop.mapred.TaskTracker: Good mapred local directories are: /home/usr/student/pg/yr11/dibyendu_karmakar/hadoop-1.0.3/temp/mapred/local 2012-12-04 15:21:11,190 INFO org.apache.hadoop.mapred.TaskTracker: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting TaskTracker STARTUP_MSG: host = radix.itd.jusl.ac.in/172.16.4.120 STARTUP_MSG: args = [] STARTUP_MSG: version = 1.0.3 STARTUP_MSG: build https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1335192; compiled by 'hortonfo' on Tue May 8 20:31:25 UTC 2012 ************************************************************/ 2012-12-04 15:21:11,971 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2012-12-04 15:21:12,031 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered. 2012-12-04 15:21:12,032 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2012-12-04 15:21:12,032 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics system started 2012-12-04 15:21:12,443 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered. 2012-12-04 15:21:22,630 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2012-12-04 15:21:22,701 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) 2012-12-04 15:21:22,728 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1 2012-12-04 15:21:22,732 INFO org.apache.hadoop.mapred.TaskTracker: Starting tasktracker with owner as dibyendu_karmakar 2012-12-04 15:21:22,733 INFO org.apache.hadoop.mapred.TaskTracker: Good mapred local directories are: /home/usr/student/pg/yr11/dibyendu_karmakar/hadoop-1.0.3/temp/mapred/local 2012-12-04 15:23:20,474 INFO org.apache.hadoop.mapred.TaskTracker: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting TaskTracker STARTUP_MSG: host = radix.itd.jusl.ac.in/172.16.4.120 STARTUP_MSG: args = [] STARTUP_MSG: version = 1.0.3 STARTUP_MSG: build https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1335192; compiled by 'hortonfo' on Tue May 8 20:31:25 UTC 2012 ************************************************************/ 2012-12-04 15:23:20,876 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2012-12-04 15:23:20,937 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered. 2012-12-04 15:23:20,939 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2012-12-04 15:23:20,939 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics system started 2012-12-04 15:23:21,969 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered. 2012-12-04 15:23:32,560 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2012-12-04 15:23:32,631 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) 2012-12-04 15:23:32,658 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1 2012-12-04 15:23:32,661 INFO org.apache.hadoop.mapred.TaskTracker: Starting tasktracker with owner as dibyendu_karmakar 2012-12-04 15:23:32,663 INFO org.apache.hadoop.mapred.TaskTracker: Good mapred local directories are: /home/usr/student/pg/yr11/dibyendu_karmakar/hadoop-1.0.3/temp/mapred/local 2012-12-04 15:24:11,542 INFO org.apache.hadoop.mapred.TaskTracker: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting TaskTracker STARTUP_MSG: host = radix.itd.jusl.ac.in/172.16.4.120 STARTUP_MSG: args = [] STARTUP_MSG: version = 1.0.3 STARTUP_MSG: build https://svn.apache.org/rep
-
Re: Task Tracker Not Starting in Hadoop-1.0.3 Help PleaseMohammad Tariq 2012-12-04, 11:29
I see. Could you please post your JT logs? Also, comment out "127.0.1.1" in
your "/etc/hosts" and try to restart the cluster. I am assuming rest of the things are working fine. (I am sorry as I am not able to post any direct answer, as it's very difficult to diagnose the actual problem without error logs). Regards, Mohammad Tariq On Tue, Dec 4, 2012 at 4:31 PM, Dibyendu Karmakar <[EMAIL PROTECTED]>wrote: > THE ENTIRE CONTENT OF TASKTRACKER -- PASTED HERE WITH AN ATTACHMENT > ALSO --- THANK YOU FOR THE REPLY > > > 2012-12-04 15:20:45,942 INFO org.apache.hadoop.mapred.TaskTracker: > STARTUP_MSG: > /************************************************************ > STARTUP_MSG: Starting TaskTracker > STARTUP_MSG: host = radix.itd.jusl.ac.in/172.16.4.120 > STARTUP_MSG: args = [] > STARTUP_MSG: version = 1.0.3 > STARTUP_MSG: build > https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r > 1335192; compiled by 'hortonfo' on Tue May 8 20:31:25 UTC 2012 > ************************************************************/ > 2012-12-04 15:20:46,983 INFO > org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from > hadoop-metrics2.properties > 2012-12-04 15:20:47,044 INFO > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > MetricsSystem,sub=Stats registered. > 2012-12-04 15:20:47,445 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot > period at 10 second(s). > 2012-12-04 15:20:47,445 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics > system started > 2012-12-04 15:20:48,478 INFO > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > ugi registered. > 2012-12-04 15:20:59,066 INFO org.mortbay.log: Logging to > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via > org.mortbay.log.Slf4jLog > 2012-12-04 15:20:59,137 INFO org.apache.hadoop.http.HttpServer: Added > global filtersafety > (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) > 2012-12-04 15:20:59,164 INFO > org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' > truncater with mapRetainSize=-1 and reduceRetainSize=-1 > 2012-12-04 15:20:59,167 INFO org.apache.hadoop.mapred.TaskTracker: > Starting tasktracker with owner as dibyendu_karmakar > 2012-12-04 15:20:59,169 INFO org.apache.hadoop.mapred.TaskTracker: > Good mapred local directories are: > /home/usr/student/pg/yr11/dibyendu_karmakar/hadoop-1.0.3/temp/mapred/local > 2012-12-04 15:21:11,190 INFO org.apache.hadoop.mapred.TaskTracker: > STARTUP_MSG: > /************************************************************ > STARTUP_MSG: Starting TaskTracker > STARTUP_MSG: host = radix.itd.jusl.ac.in/172.16.4.120 > STARTUP_MSG: args = [] > STARTUP_MSG: version = 1.0.3 > STARTUP_MSG: build > https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r > 1335192; compiled by 'hortonfo' on Tue May 8 20:31:25 UTC 2012 > ************************************************************/ > 2012-12-04 15:21:11,971 INFO > org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from > hadoop-metrics2.properties > 2012-12-04 15:21:12,031 INFO > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > MetricsSystem,sub=Stats registered. > 2012-12-04 15:21:12,032 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot > period at 10 second(s). > 2012-12-04 15:21:12,032 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics > system started > 2012-12-04 15:21:12,443 INFO > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > ugi registered. > 2012-12-04 15:21:22,630 INFO org.mortbay.log: Logging to > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via > org.mortbay.log.Slf4jLog > 2012-12-04 15:21:22,701 INFO org.apache.hadoop.http.HttpServer: Added > global filtersafety > (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) > 2012-12-04 15:21:22,728 INFO > org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs'
-
Re: Task Tracker Not Starting in Hadoop-1.0.3 Help PleaseNitin Pawar 2012-12-04, 12:49
surprising to see that TT is restarting without any error after it checks
for the directory ownership mapred local directories. can you try putting tasktracker startup in debug mode? (which version of JDK are you using? ) On Tue, Dec 4, 2012 at 4:59 PM, Mohammad Tariq <[EMAIL PROTECTED]> wrote: > I see. Could you please post your JT logs? Also, comment out "127.0.1.1" > in your "/etc/hosts" and try to restart the cluster. I am assuming rest of > the things are working fine. > > (I am sorry as I am not able to post any direct answer, as it's very > difficult to diagnose the actual problem without error logs). > > Regards, > Mohammad Tariq > > > > On Tue, Dec 4, 2012 at 4:31 PM, Dibyendu Karmakar <[EMAIL PROTECTED] > > wrote: > >> THE ENTIRE CONTENT OF TASKTRACKER -- PASTED HERE WITH AN ATTACHMENT >> ALSO --- THANK YOU FOR THE REPLY >> >> >> 2012-12-04 15:20:45,942 INFO org.apache.hadoop.mapred.TaskTracker: >> STARTUP_MSG: >> /************************************************************ >> STARTUP_MSG: Starting TaskTracker >> STARTUP_MSG: host = radix.itd.jusl.ac.in/172.16.4.120 >> STARTUP_MSG: args = [] >> STARTUP_MSG: version = 1.0.3 >> STARTUP_MSG: build >> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r >> 1335192; compiled by 'hortonfo' on Tue May 8 20:31:25 UTC 2012 >> ************************************************************/ >> 2012-12-04 15:20:46,983 INFO >> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from >> hadoop-metrics2.properties >> 2012-12-04 15:20:47,044 INFO >> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source >> MetricsSystem,sub=Stats registered. >> 2012-12-04 15:20:47,445 INFO >> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot >> period at 10 second(s). >> 2012-12-04 15:20:47,445 INFO >> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics >> system started >> 2012-12-04 15:20:48,478 INFO >> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source >> ugi registered. >> 2012-12-04 15:20:59,066 INFO org.mortbay.log: Logging to >> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via >> org.mortbay.log.Slf4jLog >> 2012-12-04 15:20:59,137 INFO org.apache.hadoop.http.HttpServer: Added >> global filtersafety >> (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) >> 2012-12-04 15:20:59,164 INFO >> org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' >> truncater with mapRetainSize=-1 and reduceRetainSize=-1 >> 2012-12-04 15:20:59,167 INFO org.apache.hadoop.mapred.TaskTracker: >> Starting tasktracker with owner as dibyendu_karmakar >> 2012-12-04 15:20:59,169 INFO org.apache.hadoop.mapred.TaskTracker: >> Good mapred local directories are: >> /home/usr/student/pg/yr11/dibyendu_karmakar/hadoop-1.0.3/temp/mapred/local >> 2012-12-04 15:21:11,190 INFO org.apache.hadoop.mapred.TaskTracker: >> STARTUP_MSG: >> /************************************************************ >> STARTUP_MSG: Starting TaskTracker >> STARTUP_MSG: host = radix.itd.jusl.ac.in/172.16.4.120 >> STARTUP_MSG: args = [] >> STARTUP_MSG: version = 1.0.3 >> STARTUP_MSG: build >> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r >> 1335192; compiled by 'hortonfo' on Tue May 8 20:31:25 UTC 2012 >> ************************************************************/ >> 2012-12-04 15:21:11,971 INFO >> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from >> hadoop-metrics2.properties >> 2012-12-04 15:21:12,031 INFO >> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source >> MetricsSystem,sub=Stats registered. >> 2012-12-04 15:21:12,032 INFO >> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot >> period at 10 second(s). >> 2012-12-04 15:21:12,032 INFO >> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics >> system started >> 2012-12-04 15:21:12,443 INFO >> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source Nitin Pawar
-
Re: Task Tracker Not Starting in Hadoop-1.0.3 Help PleaseRobert Molina 2012-12-04, 20:58
Hi Dibyendu,
I believe TT is trying to do some cleanup. What is currently in /home/usr/student/pg/yr11/dibyendu_karmakar/hadoop-1.0.3/temp/mapred/local Regards, Robert On Tue, Dec 4, 2012 at 4:49 AM, Nitin Pawar <[EMAIL PROTECTED]> wrote: > surprising to see that TT is restarting without any error after it checks > for the directory ownership mapred local directories. > > can you try putting tasktracker startup in debug mode? (which version of > JDK are you using? ) > > > On Tue, Dec 4, 2012 at 4:59 PM, Mohammad Tariq <[EMAIL PROTECTED]> wrote: > >> I see. Could you please post your JT logs? Also, comment out "127.0.1.1" >> in your "/etc/hosts" and try to restart the cluster. I am assuming rest of >> the things are working fine. >> >> (I am sorry as I am not able to post any direct answer, as it's very >> difficult to diagnose the actual problem without error logs). >> >> Regards, >> Mohammad Tariq >> >> >> >> On Tue, Dec 4, 2012 at 4:31 PM, Dibyendu Karmakar < >> [EMAIL PROTECTED]> wrote: >> >>> THE ENTIRE CONTENT OF TASKTRACKER -- PASTED HERE WITH AN ATTACHMENT >>> ALSO --- THANK YOU FOR THE REPLY >>> >>> >>> 2012-12-04 15:20:45,942 INFO org.apache.hadoop.mapred.TaskTracker: >>> STARTUP_MSG: >>> /************************************************************ >>> STARTUP_MSG: Starting TaskTracker >>> STARTUP_MSG: host = radix.itd.jusl.ac.in/172.16.4.120 >>> STARTUP_MSG: args = [] >>> STARTUP_MSG: version = 1.0.3 >>> STARTUP_MSG: build >>> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r >>> 1335192; compiled by 'hortonfo' on Tue May 8 20:31:25 UTC 2012 >>> ************************************************************/ >>> 2012-12-04 15:20:46,983 INFO >>> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from >>> hadoop-metrics2.properties >>> 2012-12-04 15:20:47,044 INFO >>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source >>> MetricsSystem,sub=Stats registered. >>> 2012-12-04 15:20:47,445 INFO >>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot >>> period at 10 second(s). >>> 2012-12-04 15:20:47,445 INFO >>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics >>> system started >>> 2012-12-04 15:20:48,478 INFO >>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source >>> ugi registered. >>> 2012-12-04 15:20:59,066 INFO org.mortbay.log: Logging to >>> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via >>> org.mortbay.log.Slf4jLog >>> 2012-12-04 15:20:59,137 INFO org.apache.hadoop.http.HttpServer: Added >>> global filtersafety >>> (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) >>> 2012-12-04 15:20:59,164 INFO >>> org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' >>> truncater with mapRetainSize=-1 and reduceRetainSize=-1 >>> 2012-12-04 15:20:59,167 INFO org.apache.hadoop.mapred.TaskTracker: >>> Starting tasktracker with owner as dibyendu_karmakar >>> 2012-12-04 15:20:59,169 INFO org.apache.hadoop.mapred.TaskTracker: >>> Good mapred local directories are: >>> >>> /home/usr/student/pg/yr11/dibyendu_karmakar/hadoop-1.0.3/temp/mapred/local >>> 2012-12-04 15:21:11,190 INFO org.apache.hadoop.mapred.TaskTracker: >>> STARTUP_MSG: >>> /************************************************************ >>> STARTUP_MSG: Starting TaskTracker >>> STARTUP_MSG: host = radix.itd.jusl.ac.in/172.16.4.120 >>> STARTUP_MSG: args = [] >>> STARTUP_MSG: version = 1.0.3 >>> STARTUP_MSG: build >>> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r >>> 1335192; compiled by 'hortonfo' on Tue May 8 20:31:25 UTC 2012 >>> ************************************************************/ >>> 2012-12-04 15:21:11,971 INFO >>> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from >>> hadoop-metrics2.properties >>> 2012-12-04 15:21:12,031 INFO >>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source >>> MetricsSystem,sub=Stats registered.
-
Re: Task Tracker Not Starting in Hadoop-1.0.3 Help PleaseDibyendu Karmakar 2012-12-05, 07:32
@ Nitin Power, I don't how to put Task Tracker in debug mode.
JDK VERSION - using "java -version" command ------------------------------------------------ java version "1.6.0" Java(TM) SE Runtime Environment (build 1.6.0-b105) Java HotSpot(TM) Client VM (build 1.6.0-b105, mixed mode, sharing) @ Robert Molina, doing "ls /home/usr/student/pg/yr11/dibyendu_karmakar/hadoop-1.0.3/temp/mapred/local" gives a directory named "taskTracker" But this directory is empty --- no file found after doing "ls" on this directory AND ALSO MY < hadoop.tmp.dir > IS /home/usr/student/pg/yr11/dibyendu_karmakar/hadoop-1.0.3/temp @ Mohammad Tariq, THE JOB TRACKER LOG FILE IS AS FOLLOWS ---- NO ERROR ALSO 2012-12-04 15:33:20,305 INFO org.apache.hadoop.mapred.JobTracker: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting JobTracker STARTUP_MSG: host = radix.itd.jusl.ac.in/172.16.4.120 STARTUP_MSG: args = [] STARTUP_MSG: version = 1.0.3 STARTUP_MSG: build https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1335192; compiled by 'hortonfo' on Tue May 8 20:31:25 UTC 2012 ************************************************************/ 2012-12-04 15:33:20,768 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2012-12-04 15:33:20,780 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered. 2012-12-04 15:33:20,880 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2012-12-04 15:33:20,880 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: JobTracker metrics system started 2012-12-04 15:33:21,277 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source QueueMetrics,q=default registered. 2012-12-04 15:33:22,527 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered. 2012-12-04 15:33:22,528 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens 2012-12-04 15:33:22,530 INFO org.apache.hadoop.mapred.JobTracker: Scheduler configured with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT, limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1) 2012-12-04 15:33:22,531 INFO org.apache.hadoop.util.HostsFileReader: Refreshing hosts (include/exclude) list 2012-12-04 15:33:22,537 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2012-12-04 15:33:22,537 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens 2012-12-04 15:33:22,868 INFO org.apache.hadoop.mapred.JobTracker: Starting jobtracker with owner as dibyendu_karmakar 2012-12-04 15:33:22,888 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcDetailedActivityForPort9002 registered. 2012-12-04 15:33:22,889 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcActivityForPort9002 registered. 2012-12-04 15:33:22,892 INFO org.apache.hadoop.ipc.Server: Starting SocketReader 2012-12-04 15:33:32,947 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2012-12-04 15:33:33,012 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) 2012-12-04 15:33:33,413 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 50030 2012-12-04 15:33:33,415 INFO org.apache.hadoop.http.HttpServer: listener.getLocalPort() returned 50030 webServer.getConnectors()[0].getLocalPort() returned 50030 2012-12-04 15:33:33,415 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 50030 2012-12-04 15:33:33,415 INFO org.mortbay.log: jetty-6.1.26 2012-12-04 15:33:34,274 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:50030 2012-12-04 15:33:34,279 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source jvm registered. 2012-12-04 15:33:34,280 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source JobTrackerMetrics registered. 2012-12-04 15:33:34,280 INFO org.apache.hadoop.mapred.JobTracker: JobTracker up at: 9002 2012-12-04 15:33:34,280 INFO org.apache.hadoop.mapred.JobTracker: JobTracker webserver: 50030 2012-12-04 15:33:34,704 INFO org.apache.hadoop.mapred.JobTracker: Cleaning up the system directory 2012-12-04 15:33:34,802 INFO org.apache.hadoop.mapred.JobTracker: History server being initialized in embedded mode 2012-12-04 15:33:34,806 INFO org.apache.hadoop.mapred.JobHistoryServer: Started job history server at: localhost:50030 2012-12-04 15:33:34,806 INFO org.apache.hadoop.mapred.JobTracker: Job History Server web address: localhost:50030 2012-12-04 15:33:34,810 INFO org.apache.hadoop.mapred.CompletedJobStatusStore: Completed job store is inactive 2012-12-04 15:33:35,426 INFO org.apache.hadoop.mapred.JobTracker: Refreshing hosts information 2012-12-04 15:33:35,765 INFO org.apache.hadoop.util.HostsFileReader: Setting the includes file to 2012-12-04 15:33:35,765 INFO org.apache.hadoop.util.HostsFileReader: Setting the excludes file to 2012-12-04 15:33:35,765 INFO org.apache.hadoop.util.HostsFileReader: Refreshing hosts (include/exclude) list 2012-12-04 15:33:35,765 INFO org.apache.hadoop.mapred.JobTracker: Decommissioning 0 nodes 2012-12-04 15:33:35,766 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2012-12-04 15:33:35,767 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 9002: starting 2012-12-04 15:33:35,767 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 9002: starting 2012-12-04 15:33:35,767 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1
-
Re: Task Tracker Not Starting in Hadoop-1.0.3 Help PleaseRobert Molina 2012-12-05, 22:50
Hi Dibyendu,
Looking back at your logs, you are trying to start TT on the 120 box where your the master processes along with datanode is running. I was wondering if you tried to launch TT on the 172.16.4.14 as well and get the same behavior. Regards, Robert On Tue, Dec 4, 2012 at 11:32 PM, Dibyendu Karmakar <[EMAIL PROTECTED]>wrote: > @ Nitin Power, I don't how to put Task Tracker in debug mode. > > JDK VERSION - using "java -version" command > ------------------------------------------------ > java version "1.6.0" > Java(TM) SE Runtime Environment (build 1.6.0-b105) > Java HotSpot(TM) Client VM (build 1.6.0-b105, mixed mode, sharing) > > @ Robert Molina, > doing "ls > /home/usr/student/pg/yr11/dibyendu_karmakar/hadoop-1.0.3/temp/mapred/local" > gives a directory named "taskTracker" > But this directory is empty --- no file found after doing "ls" on this > directory > > AND ALSO MY < hadoop.tmp.dir > IS > /home/usr/student/pg/yr11/dibyendu_karmakar/hadoop-1.0.3/temp > > > @ Mohammad Tariq, > > THE JOB TRACKER LOG FILE IS AS FOLLOWS ---- NO ERROR ALSO > > 2012-12-04 15:33:20,305 INFO org.apache.hadoop.mapred.JobTracker: > STARTUP_MSG: > /************************************************************ > STARTUP_MSG: Starting JobTracker > STARTUP_MSG: host = radix.itd.jusl.ac.in/172.16.4.120 > STARTUP_MSG: args = [] > STARTUP_MSG: version = 1.0.3 > STARTUP_MSG: build > https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r > 1335192; compiled by 'hortonfo' on Tue May 8 20:31:25 UTC 2012 > ************************************************************/ > 2012-12-04 15:33:20,768 INFO > org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from > hadoop-metrics2.properties > 2012-12-04 15:33:20,780 INFO > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > MetricsSystem,sub=Stats registered. > 2012-12-04 15:33:20,880 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot > period at 10 second(s). > 2012-12-04 15:33:20,880 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: JobTracker metrics > system started > 2012-12-04 15:33:21,277 INFO > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > QueueMetrics,q=default registered. > 2012-12-04 15:33:22,527 INFO > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > ugi registered. > 2012-12-04 15:33:22,528 INFO > > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: > Updating the current master key for generating delegation tokens > 2012-12-04 15:33:22,530 INFO org.apache.hadoop.mapred.JobTracker: > Scheduler configured with (memSizeForMapSlotOnJT, > memSizeForReduceSlotOnJT, limitMaxMemForMapTasks, > limitMaxMemForReduceTasks) (-1, -1, -1, -1) > 2012-12-04 15:33:22,531 INFO org.apache.hadoop.util.HostsFileReader: > Refreshing hosts (include/exclude) list > 2012-12-04 15:33:22,537 INFO > > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: > Starting expired delegation token remover thread, > tokenRemoverScanInterval=60 min(s) > 2012-12-04 15:33:22,537 INFO > > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: > Updating the current master key for generating delegation tokens > 2012-12-04 15:33:22,868 INFO org.apache.hadoop.mapred.JobTracker: > Starting jobtracker with owner as dibyendu_karmakar > 2012-12-04 15:33:22,888 INFO > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > RpcDetailedActivityForPort9002 registered. > 2012-12-04 15:33:22,889 INFO > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > RpcActivityForPort9002 registered. > 2012-12-04 15:33:22,892 INFO org.apache.hadoop.ipc.Server: Starting > SocketReader > 2012-12-04 15:33:32,947 INFO org.mortbay.log: Logging to > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via > org.mortbay.log.Slf4jLog > 2012-12-04 15:33:33,012 INFO org.apache.hadoop.http.HttpServer: Added
-
Re: Task Tracker Not Starting in Hadoop-1.0.3 Help PleaseDibyendu Karmakar 2012-12-06, 02:32
@ Robert,
I was trying the same in 172.16.4.14 and it was not running as well. Tasktracker is not running at all. I am using hadoop-1.0.3 But yesterday, I tried the old hadoop-0.22.0 and every daemon, including tasktracker is running quite beautifully.... Actually, I am trying to understand the code for hadoop... so the purpose is to run the WordCount program given in MapReduce Example... But I cann't complie this program in hadoop-0.22.0 because hadoop-core-version.jar is not there in hadoop-0.22.0 whereas it is in hadoop-1.0.3 and compilation is successful... THANKS FOR YOUR CONSTANT REPLY.. DON'T KNOW WHAT I'LL DO IN MY PROJECT ON HADOOP. THANKS AGAIN... On 12/6/12, Robert Molina <[EMAIL PROTECTED]> wrote: > Hi Dibyendu, > Looking back at your logs, you are trying to start TT on the 120 box where > your the master processes along with datanode is running. I was wondering > if you tried to launch TT on the 172.16.4.14 as well and get the same > behavior. > > Regards, > Robert > > On Tue, Dec 4, 2012 at 11:32 PM, Dibyendu Karmakar > <[EMAIL PROTECTED]>wrote: > >> @ Nitin Power, I don't how to put Task Tracker in debug mode. >> >> JDK VERSION - using "java -version" command >> ------------------------------------------------ >> java version "1.6.0" >> Java(TM) SE Runtime Environment (build 1.6.0-b105) >> Java HotSpot(TM) Client VM (build 1.6.0-b105, mixed mode, sharing) >> >> @ Robert Molina, >> doing "ls >> /home/usr/student/pg/yr11/dibyendu_karmakar/hadoop-1.0.3/temp/mapred/local" >> gives a directory named "taskTracker" >> But this directory is empty --- no file found after doing "ls" on this >> directory >> >> AND ALSO MY < hadoop.tmp.dir > IS >> /home/usr/student/pg/yr11/dibyendu_karmakar/hadoop-1.0.3/temp >> >> >> @ Mohammad Tariq, >> >> THE JOB TRACKER LOG FILE IS AS FOLLOWS ---- NO ERROR ALSO >> >> 2012-12-04 15:33:20,305 INFO org.apache.hadoop.mapred.JobTracker: >> STARTUP_MSG: >> /************************************************************ >> STARTUP_MSG: Starting JobTracker >> STARTUP_MSG: host = radix.itd.jusl.ac.in/172.16.4.120 >> STARTUP_MSG: args = [] >> STARTUP_MSG: version = 1.0.3 >> STARTUP_MSG: build >> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r >> 1335192; compiled by 'hortonfo' on Tue May 8 20:31:25 UTC 2012 >> ************************************************************/ >> 2012-12-04 15:33:20,768 INFO >> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from >> hadoop-metrics2.properties >> 2012-12-04 15:33:20,780 INFO >> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source >> MetricsSystem,sub=Stats registered. >> 2012-12-04 15:33:20,880 INFO >> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot >> period at 10 second(s). >> 2012-12-04 15:33:20,880 INFO >> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: JobTracker metrics >> system started >> 2012-12-04 15:33:21,277 INFO >> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source >> QueueMetrics,q=default registered. >> 2012-12-04 15:33:22,527 INFO >> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source >> ugi registered. >> 2012-12-04 15:33:22,528 INFO >> >> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: >> Updating the current master key for generating delegation tokens >> 2012-12-04 15:33:22,530 INFO org.apache.hadoop.mapred.JobTracker: >> Scheduler configured with (memSizeForMapSlotOnJT, >> memSizeForReduceSlotOnJT, limitMaxMemForMapTasks, >> limitMaxMemForReduceTasks) (-1, -1, -1, -1) >> 2012-12-04 15:33:22,531 INFO org.apache.hadoop.util.HostsFileReader: >> Refreshing hosts (include/exclude) list >> 2012-12-04 15:33:22,537 INFO >> >> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: >> Starting expired delegation token remover thread, >> tokenRemoverScanInterval=60 min(s) >> 2012-12-04 15:33:22,537 INFO >> >> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Dibyendu Karmakar, < [EMAIL PROTECTED] > |