Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Securing the Secondary Name Node


Copy link to this message
-
Re: Securing the Secondary Name Node
Christopher Penney 2013-09-20, 14:15
It does seem to be some kind of authentication issue as I can see "Login
failure for null from keytab" in the NN log, but I don't understand why I
get it.  The NN and 2NN are the same box.  Below are the logs from each.

On the 2NN I see:

2013-09-20 10:01:59,338 INFO
org.apache.hadoop.security.UserGroupInformation: Initiating logout for hdfs/
[EMAIL PROTECTED]
2013-09-20 10:01:59,338 INFO
org.apache.hadoop.security.UserGroupInformation: Initiating re-login for
hdfs/[EMAIL PROTECTED]
2013-09-20 10:01:59,410 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
transactions: 0 Total time for transactions(ms): 0Number of transactions
batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0
2013-09-20 10:01:59,410 INFO
org.apache.hadoop.hdfs.server.namenode.FSEditLog: closing edit log:
position=4,
editlog=/tmp/hadoop/tmp/hadoop-hdfs/dfs/namesecondary/current/edits
2013-09-20 10:01:59,410 INFO
org.apache.hadoop.hdfs.server.namenode.FSEditLog: close success: truncate
to 4, editlog=/tmp/hadoop/tmp/hadoop-hdfs/dfs/namesecondary/current/edits
2013-09-20 10:01:59,413 INFO
org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Opening connection
to http://hpctest3.realm.com:50070/getimage?getimage=1
2013-09-20 10:01:59,448 INFO
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Downloaded file
fsimage size 429 bytes.
2013-09-20 10:01:59,448 INFO
org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Opening connection
to http://hpctest3.realm.com:50070/getimage?getedit=1
2013-09-20 10:01:59,464 INFO
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Downloaded file
edits size 40 bytes.
2013-09-20 10:01:59,464 INFO org.apache.hadoop.hdfs.util.GSet: VM type
  = 64-bit
2013-09-20 10:01:59,464 INFO org.apache.hadoop.hdfs.util.GSet: 2% max
memory = 35.55625 MB
2013-09-20 10:01:59,464 INFO org.apache.hadoop.hdfs.util.GSet: capacity
 = 2^22 = 4194304 entries
2013-09-20 10:01:59,464 INFO org.apache.hadoop.hdfs.util.GSet:
recommended=4194304, actual=4194304
2013-09-20 10:01:59,473 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner=hdfs/
[EMAIL PROTECTED]
2013-09-20 10:01:59,473 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=supergroup
2013-09-20 10:01:59,473 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
isPermissionEnabled=true
2013-09-20 10:01:59,473 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
dfs.block.invalidate.limit=100
2013-09-20 10:01:59,473 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
isAccessTokenEnabled=true accessKeyUpdateInterval=600 min(s),
accessTokenLifetime=600 min(s)
2013-09-20 10:01:59,474 INFO
org.apache.hadoop.hdfs.server.namenode.NameNode: Caching file names
occuring more than 10 times
2013-09-20 10:01:59,475 INFO org.apache.hadoop.hdfs.server.common.Storage:
Number of files = 4
2013-09-20 10:01:59,475 INFO org.apache.hadoop.hdfs.server.common.Storage:
Number of files under construction = 0
2013-09-20 10:01:59,475 INFO org.apache.hadoop.hdfs.server.common.Storage:
Edits file /tmp/hadoop/tmp/hadoop-hdfs/dfs/namesecondary/current/edits of
size 40 edits # 2 loaded in 0 seconds.
2013-09-20 10:01:59,475 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
transactions: 0 Total time for transactions(ms): 0Number of transactions
batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0
2013-09-20 10:01:59,505 INFO
org.apache.hadoop.hdfs.server.namenode.FSEditLog: closing edit log:
position=40,
editlog=/tmp/hadoop/tmp/hadoop-hdfs/dfs/namesecondary/current/edits
2013-09-20 10:01:59,505 INFO
org.apache.hadoop.hdfs.server.namenode.FSEditLog: close success: truncate
to 40, editlog=/tmp/hadoop/tmp/hadoop-hdfs/dfs/namesecondary/current/edits
2013-09-20 10:01:59,506 INFO org.apache.hadoop.hdfs.server.common.Storage:
Image file of size 463 saved in 0 seconds.
2013-09-20 10:01:59,577 INFO
org.apache.hadoop.hdfs.server.namenode.FSEditLog: closing edit log:
position=4,
editlog=/tmp/hadoop/tmp/hadoop-hdfs/dfs/namesecondary/current/edits
2013-09-20 10:01:59,579 INFO
org.apache.hadoop.hdfs.server.namenode.FSEditLog: close success: truncate
to 4, editlog=/tmp/hadoop/tmp/hadoop-hdfs/dfs/namesecondary/current/edits
2013-09-20 10:01:59,688 INFO
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Posted URL
hpctest3.realm.com:50070putimage=1&port=50090&machine=hpctest3.realm.com
&token=-32:832852997:0:1379525282000:1379524942772&newChecksum=d11af5836a07cda4d46f91aadaba11b0
2013-09-20 10:01:59,689 INFO
org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Opening connection
to
http://hpctest3.realm.com:50070/getimage?putimage=1&port=50090&machine=hpctest3.realm.com&token=-32:832852997:0:1379525282000:1379524942772&newChecksum=d11af5836a07cda4d46f91aadaba11b0
2013-09-20 10:01:59,706 ERROR
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Exception in
doCheckpoint:
2013-09-20 10:01:59,706 ERROR
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode:
java.io.FileNotFoundException:
http://hpctest3.realm.com:50070/getimage?putimage=1&port=50090&machine=hpctest3.realm.com&token=-32:832852997:0:1379525282000:1379524942772&newChecksum=d11af5836a07cda4d46f91aadaba11b0
        at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1434)
        at
org.apache.hadoop.hdfs.server.namenode.TransferFsImage.getFileClient(TransferFsImage.java:172)
        at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.putFSImage(SecondaryNameNode.java:435)
        at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doCheckpoint(SecondaryNameNode.java:499)
        at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doWork(SecondaryNameNode.java:369)
        at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$3.run(SecondaryNameNode.java:327)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:337)
        at
org.apache.hadoop.security.UserGro