Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Can't initialize cluster


Copy link to this message
-
Re: Can't initialize cluster

  I made these changes. The log files in /var/log/hadoop-0.20-mapreduce
do not reflect todays date so they are not updated. The datanode log in
/var/log/hadoop-hdfs only shows this warning:

2013-05-01 08:16:12,678 WARN
org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to
server: devUbuntu05/172.16.26.68:9000

The entries in the log after that (and there are many) all are INFO
messages. Not more warnings or errors.

The namenode log is just INFO messages.

The secondarynamenode log (on the node that is acting as the secondary
namenode) contains the following error exception:

2013-05-01 08:34:16,431 ERROR
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Exception in
doCheckpoint
java.io.IOException: Inconsistent checkpoint fields.
LV = -40 namespaceID = 1097063765 cTime = 0 ; clusterId =
CID-23b9f9c7-2c25-411f-8bd2-4d5c9d7c25a1 ; blockpoolId =
BP-1306349046-172.16.26.68-1367256199559.
Expecting respectively: -40; 1666002936; 0;
CID-e3f6b811-c1b4-4778-a31e-14dea8b2cca8;
BP-1964515668-172.16.26.68-1366403021016.
         at
org.apache.hadoop.hdfs.server.namenode.CheckpointSignature.validateStorageInfo(CheckpointSignature.java:133)
         at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doCheckpoint(SecondaryNameNode.java:475)

The logs for the any of the nodes serving as the task tracker are out of
date.

When I try to attach to the master with a browser on port 50070 to look
at the HDFS the page responds but when I try to connect to the master at
port 50030 nothing is listening so it seems the JobTracker is down.

So it seems from what I can gather the reason that the JobTracker
doesn't come up is because of the error on the secondary name node. Do
you understand how I can get rid of this error?

Kevin

On Tue, Apr 30, 2013 at 8:55 PM, Mohammad Tariq wrote:

  Are you sure about the case in  devuBuntu06
devuBuntu07
devuBuntu08

Is 'B' actually in upper case?

Also,  mapred.local.dir  belongs to mapred-site.xml and  hadoop.tmp.dir
belongs to core-site.xml file.

Warm Regards, Tariq
https://mtariq.jux.com/ <https://mtariq.jux.com/>
  <https://mtariq.jux.com/>
cloudfront.blogspot.com <http://cloudfront.blogspot.com>
  <http://cloudfront.blogspot.com>

On Wed, May 1, 2013 at 7:09 AM, Kevin Burton < [EMAIL PROTECTED]
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
> wrote:
Here are what I consider the most applicable conf files.
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
 
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
Again thank you for your help.
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
 
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
Kevin
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
 
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
From: Mohammad Tariq [mailto: [EMAIL PROTECTED]
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')> ]
Sent: Tuesday, April 30, 2013 6:43 PM
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>

To: Kevin Burton
Subject: Re: Can't initialize cluster
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
  <javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
   <javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
Sorry for the late response Kevin. I was away for a while. I saw the
logs. There seems to be some problem with your configuration or name
resolution. Could you please show me your configuration files?
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>

  <javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
Warm Regards,
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
Tariq
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
https://mtariq.jux.com/ <https://mtariq.jux.com/>
<https://mtariq.jux.com/>
cloudfront.blogspot.com <http://cloudfront.blogspot.com>
<http://cloudfront.blogspot.com>
   <http://cloudfront.blogspot.com>
On Wed, May 1, 2013 at 4:27 AM, Kevin Burton < [EMAIL PROTECTED]
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
The JobTracker logs don’t seem to be updated:
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
 
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
kevin@devUbuntu05:/etc/hadoop/conf$ cd /var/log/hadoop-0.20-mapreduce/
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
kevin@devUbuntu05:/var/log/hadoop-0.20-mapreduce$
<javascript:parent.wgMail.openComposeWindow('kevin@devUbuntu05:/var/log/hadoop-0.20-mapreduce$')>
ls -l
<javascript:parent.wgMail.openComposeWindow('kevin@devUbuntu05:/var/log/hadoop-0.20-mapreduce$')>
total 3028
<javascript:parent.wgMail.openComposeWindow('kevin@devUbuntu05:/var/log/hadoop-0.20-mapreduce$')>
-rw-rw-r-- 1 mapred mapred  383838 Apr 30 15:11
hadoop-hadoop-jobtracker-devUbuntu05.log
<javascript:parent.wgMail.openComposeWindow('kevin@devUbuntu05:/var/log/hadoop-0.20-mapreduce$')>
 
<javascript:parent.wgMail.openComposeWindow('kevin@devUbuntu05:/var/log/hadoop-0.20-mapreduce$')>
But I have attached the last lines of the JT log along with the TT logs.
<javascript:parent.wgMail.openComposeWindow('kevin@devUbuntu05:/var/log/hadoop-0.20-mapreduce$')>
 
<javascript:parent.wgMail.openComposeWindow('kevin@devUbuntu05:/var/log/hadoop-0.20-mapreduce$')>
Kevin
<javascript:parent.wgMail.openComposeWindow('kevin@devUbuntu05:/var/log/hadoop-0.20-mapreduce$')>
 
<javascript:parent.wgMail.openComposeWindow('kevin@devUbuntu05:/var/log/hadoop-0.20-mapreduce$')>
From: Mohammad Tariq [mailto: [EMAIL PROTECTED]
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')> ]
Sent: Tuesday, April 30, 2013 5:38 PM
<javascript