Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Securing the Secondary Name Node


+
Christopher Penney 2013-09-10, 13:55
+
Christopher Penney 2013-09-12, 13:02
+
Christopher Penney 2013-09-20, 15:17
Copy link to this message
-
Re: Securing the Secondary Name Node
It does seem to be some kind of authentication issue as I can see "Login
failure for null from keytab" in the NN log, but I don't understand why I
get it.  The NN and 2NN are the same box.  Below are the logs from each.

On the 2NN I see:

2013-09-20 10:01:59,338 INFO
org.apache.hadoop.security.UserGroupInformation: Initiating logout for hdfs/
[EMAIL PROTECTED]
2013-09-20 10:01:59,338 INFO
org.apache.hadoop.security.UserGroupInformation: Initiating re-login for
hdfs/[EMAIL PROTECTED]
2013-09-20 10:01:59,410 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
transactions: 0 Total time for transactions(ms): 0Number of transactions
batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0
2013-09-20 10:01:59,410 INFO
org.apache.hadoop.hdfs.server.namenode.FSEditLog: closing edit log:
position=4,
editlog=/tmp/hadoop/tmp/hadoop-hdfs/dfs/namesecondary/current/edits
2013-09-20 10:01:59,410 INFO
org.apache.hadoop.hdfs.server.namenode.FSEditLog: close success: truncate
to 4, editlog=/tmp/hadoop/tmp/hadoop-hdfs/dfs/namesecondary/current/edits
2013-09-20 10:01:59,413 INFO
org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Opening connection
to http://hpctest3.realm.com:50070/getimage?getimage=1
2013-09-20 10:01:59,448 INFO
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Downloaded file
fsimage size 429 bytes.
2013-09-20 10:01:59,448 INFO
org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Opening connection
to http://hpctest3.realm.com:50070/getimage?getedit=1
2013-09-20 10:01:59,464 INFO
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Downloaded file
edits size 40 bytes.
2013-09-20 10:01:59,464 INFO org.apache.hadoop.hdfs.util.GSet: VM type
  = 64-bit
2013-09-20 10:01:59,464 INFO org.apache.hadoop.hdfs.util.GSet: 2% max
memory = 35.55625 MB
2013-09-20 10:01:59,464 INFO org.apache.hadoop.hdfs.util.GSet: capacity
 = 2^22 = 4194304 entries
2013-09-20 10:01:59,464 INFO org.apache.hadoop.hdfs.util.GSet:
recommended=4194304, actual=4194304
2013-09-20 10:01:59,473 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner=hdfs/
[EMAIL PROTECTED]
2013-09-20 10:01:59,473 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=supergroup
2013-09-20 10:01:59,473 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
isPermissionEnabled=true
2013-09-20 10:01:59,473 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
dfs.block.invalidate.limit=100
2013-09-20 10:01:59,473 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
isAccessTokenEnabled=true accessKeyUpdateInterval=600 min(s),
accessTokenLifetime=600 min(s)
2013-09-20 10:01:59,474 INFO
org.apache.hadoop.hdfs.server.namenode.NameNode: Caching file names
occuring more than 10 times
2013-09-20 10:01:59,475 INFO org.apache.hadoop.hdfs.server.common.Storage:
Number of files = 4
2013-09-20 10:01:59,475 INFO org.apache.hadoop.hdfs.server.common.Storage:
Number of files under construction = 0
2013-09-20 10:01:59,475 INFO org.apache.hadoop.hdfs.server.common.Storage:
Edits file /tmp/hadoop/tmp/hadoop-hdfs/dfs/namesecondary/current/edits of
size 40 edits # 2 loaded in 0 seconds.
2013-09-20 10:01:59,475 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
transactions: 0 Total time for transactions(ms): 0Number of transactions
batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0
2013-09-20 10:01:59,505 INFO
org.apache.hadoop.hdfs.server.namenode.FSEditLog: closing edit log:
position=40,
editlog=/tmp/hadoop/tmp/hadoop-hdfs/dfs/namesecondary/current/edits
2013-09-20 10:01:59,505 INFO
org.apache.hadoop.hdfs.server.namenode.FSEditLog: close success: truncate
to 40, editlog=/tmp/hadoop/tmp/hadoop-hdfs/dfs/namesecondary/current/edits
2013-09-20 10:01:59,506 INFO org.apache.hadoop.hdfs.server.common.Storage:
Image file of size 463 saved in 0 seconds.
2013-09-20 10:01:59,577 INFO
org.apache.hadoop.hdfs.server.namenode.FSEditLog: closing edit log:
position=4,
editlog=/tmp/hadoop/tmp/hadoop-hdfs/dfs/namesecondary/current/edits
2013-09-20 10:01:59,579 INFO
org.apache.hadoop.hdfs.server.namenode.FSEditLog: close success: truncate
to 4, editlog=/tmp/hadoop/tmp/hadoop-hdfs/dfs/namesecondary/current/edits
2013-09-20 10:01:59,688 INFO
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Posted URL
hpctest3.realm.com:50070putimage=1&port=50090&machine=hpctest3.realm.com
&token=-32:832852997:0:1379525282000:1379524942772&newChecksum=d11af5836a07cda4d46f91aadaba11b0
2013-09-20 10:01:59,689 INFO
org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Opening connection
to
http://hpctest3.realm.com:50070/getimage?putimage=1&port=50090&machine=hpctest3.realm.com&token=-32:832852997:0:1379525282000:1379524942772&newChecksum=d11af5836a07cda4d46f91aadaba11b0
2013-09-20 10:01:59,706 ERROR
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Exception in
doCheckpoint:
2013-09-20 10:01:59,706 ERROR
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode:
java.io.FileNotFoundException:
http://hpctest3.realm.com:50070/getimage?putimage=1&port=50090&machine=hpctest3.realm.com&token=-32:832852997:0:1379525282000:1379524942772&newChecksum=d11af5836a07cda4d46f91aadaba11b0
        at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1434)
        at
org.apache.hadoop.hdfs.server.namenode.TransferFsImage.getFileClient(TransferFsImage.java:172)
        at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.putFSImage(SecondaryNameNode.java:435)
        at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doCheckpoint(SecondaryNameNode.java:499)
        at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doWork(SecondaryNameNode.java:369)
        at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$3.run(SecondaryNameNode.java:327)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:337)
        at
org.apache.hadoop.security.UserGro
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB