Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Secure Hadoop and non-secure HBase


Copy link to this message
-
Re: Secure Hadoop and non-secure HBase
Hi Eric,

Glad that worked for you.  We need to get some information on the security
configuration into the HBase online docs.  Let us know if you run into
further problems.

Gary
On Sun, Sep 11, 2011 at 6:00 PM, Eric Yang <[EMAIL PROTECTED]> wrote:

> Hi Gary,
>
> This is exactly what I missed.  Without those settings in my
> hbase-site.xml, I saw these messages in regional server log file:
>
> 2011-09-09 17:28:14,172 WARN org.apache.hadoop.ipc.Client: Couldn't setup
> connection for [EMAIL PROTECTED] tonn/
> [EMAIL PROTECTED]
> 2011-09-09 17:28:14,172 WARN org.apache.hadoop.hdfs.DFSClient: Problem
> renewing lease for DFSClient_hb_rs_hrt8n38.cc1.ygridcore.net
> ,60020,1315520835373_1315520835752
> java.io.IOException: Call to hrt8n37.cc1.ygridcore.net/98.137.233.217:8020failed on local exception: java.io.IOException: Couldn't setup connection
> [EMAIL PROTECTED] to nn/[EMAIL PROTECTED]
>       at org.apache.hadoop.ipc.Client.wrapException(Client.java:1104)
>       at org.apache.hadoop.ipc.Client.call(Client.java:1072)
>       at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
>       at $Proxy9.renewLease(Unknown Source)
>       at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
>       at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>       at java.lang.reflect.Method.invoke(Method.java:597)
>       at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>       at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>       at $Proxy9.renewLease(Unknown Source)
>       at
> org.apache.hadoop.hdfs.DFSClient$LeaseChecker.renew(DFSClient.java:1176)
>       at
> org.apache.hadoop.hdfs.DFSClient$LeaseChecker.run(DFSClient.java:1188)
>       at java.lang.Thread.run(Thread.java:619)
>
> The region server can respond query but data is not committed to HDFS.  I
> manually run kinit on the command line, and the log shows:
>
> 2011-09-09 17:28:50,466 INFO
> org.apache.hadoop.security.UserGroupInformation: Initiating logout for
> [EMAIL PROTECTED]
> 2011-09-09 17:28:50,466 INFO
> org.apache.hadoop.security.UserGroupInformation: Initiating re-login for
> [EMAIL PROTECTED]
>
> I will put the keytab and kerberos.principal in the configuration file to
> verify token renewal in HBase.  Thank you.
>
> regards,
> Eric
>
> On Sep 11, 2011, at 3:57 PM, Gary Helmling wrote:
>
> > Hi Eric,
> >
> > If you configure
> >
> > hbase.master.keytab.file
> > hbase.master.kerberos.principal
> > hbase.regionserver.keytab.file
> > hbase.regionserver.kerberos.principal
> >
> > in your hbase-site.xml, then the master and region server processes
> should
> > login from the keytab files on startup, as Todd mentions.  It's also my
> > understanding that they don't need a renewal thread in that case.  The
> RPC
> > client just tries a relogin from the keytab in the case of a connection
> > error.
> >
> > Can you describe a bit more what you're seeing so that we can understand
> the
> > context?
> >
> > Gary
> >
> >
> > On Sun, Sep 11, 2011 at 3:13 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
> >
> >> Hi Eric,
> >>
> >> Could you please explain more fully what you mean by this? The daemons
> >> generally run using keytabs, not user credentials, and thus shouldn't
> >> need the explicit TGT Renewer, right?
> >>
> >> -Todd
> >>
> >> On Sun, Sep 11, 2011 at 11:04 AM, Eric Yang <[EMAIL PROTECTED]> wrote:
> >>> Hi all,
> >>>
> >>> Hortonworks has a patch for secure append for Apache Hadoop 0.20.205 to
> >> work with HBase 0.90.x.  However, secure Hadoop and HBase would work
> until
> >> kerberos token expires.  There is currently no code that renews kerberos
> >> token in HBase.  Hence, it is possible to add a cron job to periodically
> >> renew the HBase user token to keep the system running.  What does the
> >> community think about having a setup script for cron job as part of
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB