Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> problem starting cdh3b2 jobtracker


Copy link to this message
-
problem starting cdh3b2 jobtracker
We installed cdh3b2 0.20.2+320 and saw some strange error in jobtracker log:

2010-07-02 01:49:31,977 INFO org.apache.hadoop.mapred.JobTracker: JobTracker
up at: 9001
2010-07-02 01:49:31,977 INFO org.apache.hadoop.mapred.JobTracker: JobTracker
webserver: 50030
2010-07-02 01:49:31,988 WARN org.apache.hadoop.mapred.JobTracker: Error
starting tracker: java.io.IOException: Cannot create toBeDeleted in
/data1/mapred/local
    at
org.apache.hadoop.util.MRAsyncDiskService.<init>(MRAsyncDiskService.java:85)
    at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:1688)
    at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:199)
    at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:191)
    at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:3765)

2010-07-02 01:49:32,990 INFO org.apache.hadoop.mapred.JobTracker: Scheduler
configured with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT,
limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1)
2010-07-02 01:49:32,991 FATAL org.apache.hadoop.mapred.JobTracker:
java.net.BindException: Problem binding to
sjc1-hadoop0.sjc1.ciq.com/10.201.8.204:9001<http://sjc1-hadoop0.sjc1.carrieriq.com/10.201.8.204:9001>:
Address already in use
    at org.apache.hadoop.ipc.Server.bind(Server.java:198)
    at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:261)
    at org.apache.hadoop.ipc.Server.<init>(Server.java:1043)
    at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:492)
    at org.apache.hadoop.ipc.RPC.getServer(RPC.java:454)
    at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:1628)
    at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:199)
    at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:191)
    at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:3765)
Caused by: java.net.BindException: Address already in use
    at sun.nio.ch.Net.bind(Native Method)
    at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:119)
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59)
    at org.apache.hadoop.ipc.Server.bind(Server.java:196)
    ... 8 more

2010-07-02 01:49:32,992 INFO org.apache.hadoop.mapred.JobTracker:
SHUTDOWN_MSG:

But 9001 wasn't used:
[sjc1-hadoop0.sjc1:hadoop 25618]netstat -nta | grep 9001
[sjc1-hadoop0.sjc1:hadoop 25619]netstat -nta | grep 9000
tcp        0      0 10.201.8.204:9000           0.0.0.0:*
LISTEN
tcp        0      0 10.201.8.204:9000           10.201.8.214:4223
ESTABLISHED
tcp        0      0 10.201.8.204:9000           10.201.8.212:49074
ESTABLISHED
tcp        0      0 10.201.8.204:9000           10.201.8.206:11910
ESTABLISHED
tcp        0      0 10.201.8.204:9000           10.201.8.210:62611
ESTABLISHED
tcp        0      0 10.201.8.204:9000           10.201.8.213:1299
ESTABLISHED
tcp        0      0 10.201.8.204:9000           10.201.8.205:9756
ESTABLISHED
tcp        0      0 10.201.8.204:9000           10.201.8.207:59207
ESTABLISHED

Here is output from ifconfig:
bond0     Link encap:Ethernet  HWaddr 00:30:48:60:53:94
          inet addr:10.201.8.204  Bcast:10.201.8.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:351496605 errors:0 dropped:1015 overruns:0 frame:0
          TX packets:178144953 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:119420730164 (111.2 GiB)  TX bytes:120002123131 (111.7
GiB)

eth0      Link encap:Ethernet  HWaddr 00:30:48:60:53:94
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:351496605 errors:0 dropped:1015 overruns:0 frame:0
          TX packets:178144953 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:119420730164 (111.2 GiB)  TX bytes:120002123131 (111.7
GiB)
          Interrupt:161

eth1      Link encap:Ethernet  HWaddr 00:30:48:60:53:94
          UP BROADCAST SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Interrupt:169

Has anyone encountered similar issue ?
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB