Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Securing the Secondary Name Node


Copy link to this message
-
Re: Securing the Secondary Name Node
Following up to my own post I tried upgrading to 1.2.1 and the problem
seemed to go away.  Now the 2NN is working fine.  I'm now seeing:

2013-09-20 11:13:31,318 INFO
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Posted URL
hpctest3.realm.com:50070putimage=1&port=50090&machine=hpctest3.realm.com
&token=-41:832852997:1379689519375:1379690010000:1379689644259&newChecksum=9c94140e6eb8c33a66a14f2e73206d29
2013-09-20 11:13:31,318 INFO
org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Opening connection
to
http://hpctest3.realm.com:50070/getimage?putimage=1&port=50090&machine=hpctest3.realm.com&token=-41:832852997:1379689519375:1379690010000:1379689644259&newChecksum=9c94140e6eb8c33a66a14f2e73206d29
2013-09-20 11:13:31,494 INFO
org.apache.hadoop.hdfs.server.namenode.GetImageServlet: GetImageServlet
allowing: hdfs/[EMAIL PROTECTED]
2013-09-20 11:13:31,631 INFO
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Checkpoint done.
New Image Size: 30906

   Chris
On Tue, Sep 10, 2013 at 9:55 AM, Christopher Penney <[EMAIL PROTECTED]>wrote:

>
> Hi,
>
> After hosting an insecure Hadoop environment for early testing I'm
> transitioning to something more secure that would (hopefully) more or less
> mirror what a production environment might look like.  I've integrated our
> Hadoop cluster into our Kerberos realm and everything is working ok except
> for our secondary name node.  When I invoke the secondarynamenode with
> "-checkpoint force" (when no other secondary name node process is running)
> I get:
>
> 13/09/10 09:44:25 INFO security.UserGroupInformation: Login successful for
> user hdfs/[EMAIL PROTECTED] using keytab file
> /etc/hadoop/hdfs.keytab
> 13/09/10 09:44:25 INFO mortbay.log: Logging to
> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
> org.mortbay.log.Slf4jLog
> 13/09/10 09:44:25 INFO http.HttpServer: Added global filtersafety
> (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
> 13/09/10 09:44:25 INFO http.HttpServer: Adding Kerberos (SPNEGO) filter to
> getimage
> 13/09/10 09:44:25 INFO http.HttpServer: Port returned by
> webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening
> the listener on 50090
> 13/09/10 09:44:25 INFO http.HttpServer: listener.getLocalPort() returned
> 50090 webServer.getConnectors()[0].getLocalPort() returned 50090
> 13/09/10 09:44:25 INFO http.HttpServer: Jetty bound to port 50090
> 13/09/10 09:44:25 INFO mortbay.log: jetty-6.1.26
> 13/09/10 09:44:26 INFO server.KerberosAuthenticationHandler: Login using
> keytab /etc/hadoop/hdfs.keytab, for principal HTTP/
> [EMAIL PROTECTED]
> 13/09/10 09:44:26 INFO server.KerberosAuthenticationHandler: Initialized,
> principal [HTTP/[EMAIL PROTECTED]] from keytab
> [/etc/hadoop/hdfs.keytab]
>  13/09/10 09:44:26 WARN server.AuthenticationFilter: 'signature.secret'
> configuration not set, using a random value as secret
> 13/09/10 09:44:26 INFO mortbay.log: Started
> SelectChannelConnector@0.0.0.0:50090
> 13/09/10 09:44:26 INFO namenode.SecondaryNameNode: Web server init done
> 13/09/10 09:44:26 INFO namenode.SecondaryNameNode: Secondary Web-server up
> at: 0.0.0.0:50090
>  13/09/10 09:44:26 WARN namenode.SecondaryNameNode: Checkpoint Period
> :3600 secs (60 min)
> 13/09/10 09:44:26 WARN namenode.SecondaryNameNode: Log Size Trigger
>  :67108864 bytes (65536 KB)
> 13/09/10 09:44:26 INFO namenode.TransferFsImage: Opening connection to
> http://hpctest3.realm.com:50070/getimage?getimage=1
> 13/09/10 09:44:26 INFO namenode.SecondaryNameNode: Downloaded file fsimage
> size 110 bytes.
> 13/09/10 09:44:26 INFO namenode.TransferFsImage: Opening connection to
> http://hpctest3.realm.com:50070/getimage?getedit=1
> 13/09/10 09:44:26 INFO namenode.SecondaryNameNode: Downloaded file edits
> size 40 bytes.
> 13/09/10 09:44:26 INFO util.GSet: VM type       = 64-bit
> 13/09/10 09:44:26 INFO util.GSet: 2% max memory = 35.55625 MB
> 13/09/10 09:44:26 INFO util.GSet: capacity      = 2^22 = 4194304 entries
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB