Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Re: Securing the Secondary Name Node


Copy link to this message
-
Re: Securing the Secondary Name Node
Chris ,

I think that the error occurs when NN tries to download the fsimage from
SNN.
You can check the NN logs to make sure whether this is true.

There could be different reasons for this.

1. NN fails to do SPNEGO with SNN.
2. NN's TGT expired. Unlikely in your test cluster.

Please post with any additional log info and I can help.

benoy
On Thu, Sep 12, 2013 at 6:02 AM, Christopher Penney <[EMAIL PROTECTED]>wrote:

> Does anyone have any suggestions or resources I might look at to resolve
> this?  The documentation on setting up Kerberos seems pretty light.
>
>    Chris
>
>
>
> On Tue, Sep 10, 2013 at 9:55 AM, Christopher Penney <[EMAIL PROTECTED]>wrote:
>
>>
>> Hi,
>>
>> After hosting an insecure Hadoop environment for early testing I'm
>> transitioning to something more secure that would (hopefully) more or less
>> mirror what a production environment might look like.  I've integrated our
>> Hadoop cluster into our Kerberos realm and everything is working ok except
>> for our secondary name node.  When I invoke the secondarynamenode with
>> "-checkpoint force" (when no other secondary name node process is running)
>> I get:
>>
>> 13/09/10 09:44:25 INFO security.UserGroupInformation: Login successful
>> for user hdfs/[EMAIL PROTECTED] using keytab file
>> /etc/hadoop/hdfs.keytab
>> 13/09/10 09:44:25 INFO mortbay.log: Logging to
>> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
>> org.mortbay.log.Slf4jLog
>> 13/09/10 09:44:25 INFO http.HttpServer: Added global filtersafety
>> (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
>> 13/09/10 09:44:25 INFO http.HttpServer: Adding Kerberos (SPNEGO) filter
>> to getimage
>> 13/09/10 09:44:25 INFO http.HttpServer: Port returned by
>> webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening
>> the listener on 50090
>> 13/09/10 09:44:25 INFO http.HttpServer: listener.getLocalPort() returned
>> 50090 webServer.getConnectors()[0].getLocalPort() returned 50090
>> 13/09/10 09:44:25 INFO http.HttpServer: Jetty bound to port 50090
>> 13/09/10 09:44:25 INFO mortbay.log: jetty-6.1.26
>> 13/09/10 09:44:26 INFO server.KerberosAuthenticationHandler: Login using
>> keytab /etc/hadoop/hdfs.keytab, for principal HTTP/
>> [EMAIL PROTECTED]
>> 13/09/10 09:44:26 INFO server.KerberosAuthenticationHandler: Initialized,
>> principal [HTTP/[EMAIL PROTECTED]] from keytab
>> [/etc/hadoop/hdfs.keytab]
>>  13/09/10 09:44:26 WARN server.AuthenticationFilter: 'signature.secret'
>> configuration not set, using a random value as secret
>> 13/09/10 09:44:26 INFO mortbay.log: Started
>> SelectChannelConnector@0.0.0.0:50090
>> 13/09/10 09:44:26 INFO namenode.SecondaryNameNode: Web server init done
>> 13/09/10 09:44:26 INFO namenode.SecondaryNameNode: Secondary Web-server
>> up at: 0.0.0.0:50090
>>  13/09/10 09:44:26 WARN namenode.SecondaryNameNode: Checkpoint Period
>> :3600 secs (60 min)
>> 13/09/10 09:44:26 WARN namenode.SecondaryNameNode: Log Size Trigger
>>  :67108864 bytes (65536 KB)
>> 13/09/10 09:44:26 INFO namenode.TransferFsImage: Opening connection to
>> http://hpctest3.realm.com:50070/getimage?getimage=1
>> 13/09/10 09:44:26 INFO namenode.SecondaryNameNode: Downloaded file
>> fsimage size 110 bytes.
>> 13/09/10 09:44:26 INFO namenode.TransferFsImage: Opening connection to
>> http://hpctest3.realm.com:50070/getimage?getedit=1
>> 13/09/10 09:44:26 INFO namenode.SecondaryNameNode: Downloaded file edits
>> size 40 bytes.
>> 13/09/10 09:44:26 INFO util.GSet: VM type       = 64-bit
>> 13/09/10 09:44:26 INFO util.GSet: 2% max memory = 35.55625 MB
>> 13/09/10 09:44:26 INFO util.GSet: capacity      = 2^22 = 4194304 entries
>> 13/09/10 09:44:26 INFO util.GSet: recommended=4194304, actual=4194304
>> 13/09/10 09:44:26 INFO namenode.FSNamesystem: fsOwner=hdfs/
>> [EMAIL PROTECTED]
>> 13/09/10 09:44:26 INFO namenode.FSNamesystem: supergroup=supergroup
>> 13/09/10 09:44:26 INFO namenode.FSNamesystem: isPermissionEnabled=true
>> 13/09/10 09:44:26 INFO namenode.FSNamesystem:
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB