Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> sqoop import into secure Hbase with kerberos


Copy link to this message
-
Re: sqoop import into secure Hbase with kerberos
Does this mean that sqoop tries to read  hbase-site.xml and then expectes
hbase to pass the  delegation token to it thru hbase.security.user class ?
I am using hbase 94.9
Hbase complains with the following msg -
2013-08-05 11:59:33,121 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer:
org.apache.hadoop.hbase.security.AccessDeniedException: Token generation
only allowed for Kerberos authenticated clients
at
org.apache.hadoop.hbase.security.token.TokenProvider.getAuthenticationToken(TokenProvider.java:87)

What am I missing here? Should I specify anything in sqoop-site.xml
 related to kerberos?

Cheers,
Suhas.
On Tue, Aug 6, 2013 at 11:23 AM, Abraham Elmahrek <[EMAIL PROTECTED]> wrote:

> Sorry, apparently this is an HBase specific token. See here
> http://wiki.apache.org/hadoop/Hbase/HBaseTokenAuthentication.
>
>
> On Tue, Aug 6, 2013 at 11:13 AM, Abraham Elmahrek <[EMAIL PROTECTED]>wrote:
>
>> Suhas,
>>
>> Sqoop 1.4.3 simply fetches the authenticated user from credentials cache
>> and fetches a delegation token for HBase. See
>> https://issues.apache.org/jira/browse/SQOOP-599 for more information.
>>
>> -Abe
>>
>>
>> On Tue, Aug 6, 2013 at 11:09 AM, Suhas Satish <[EMAIL PROTECTED]>wrote:
>>
>>> I was able to isolate this problem to the Sqoop side not picking up
>>> correct kerberos credentials. Hbase is picking up the correct kerberos
>>> credentials when Hbase put and scan are done in isolation without using
>>> Sqoop.
>>>
>>> A direct map-reduce put into HBase uses the following 2 methods -
>>> HBaseConfiguration.merge(conf, HBaseConfiguration.create(conf));
>>> TableMapReduceUtil.initCredentials(job);
>>>
>>> I was looking at how sqoop 1.4.3 does HBase puts to see if it converts
>>> sqoop import arguments into map-reduce jobs and uses the above methods
>>> somewhere. This is what I found -
>>> HBasePutProcessor.java  - SqoopRecordProcessor that performs a HBase
>>> "put" operation - has a method to get hadoop configuration, but none to
>>> merge any kerberos specific configurations specified  in sqoop-site.xml-
>>>
>>>   public Configuration getConf() {
>>>     return this.conf;
>>>
>>>
>>>
>>> HBaseUtil.java   - makes sure hbase jars are present on class path
>>> PutTransformer.java  - converts jdbc statements in the form of K-V map
>>> into hbase put commands and returns a list
>>> ToStringPutTransformer.java - extends the above class
>>>
>>>  Does anyone know sqoop internals of how to specify kerberos
>>> configurations and get sqoop to read them?
>>>
>>> Cheers,
>>> Suhas.
>>>
>>>
>>> On Tue, Aug 6, 2013 at 10:31 AM, Suhas Satish <[EMAIL PROTECTED]>wrote:
>>>
>>>> Ataching the logs here at the time of authentication, I do not see any
>>>> error msges here.
>>>>
>>>> /var/log/kadmind.log
>>>> /var/log/krb5kdc.log
>>>>
>>>> Please let me know if there is any other places I can find other log
>>>> files
>>>>
>>>> Cheers,
>>>> Suhas.
>>>>
>>>>
>>>> On Mon, Aug 5, 2013 at 4:48 PM, Abraham Elmahrek <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> User,
>>>>>
>>>>> Could you please provide your KDC logs around the time you tried to
>>>>> authenticate?
>>>>>
>>>>> Note: A kerberos client will negotiate the encryption algorithm it
>>>>> can/will use with the KDC. It may choose AES-256.
>>>>>
>>>>> -Abe
>>>>>
>>>>>
>>>>> On Mon, Aug 5, 2013 at 3:55 PM, Suhas Satish <[EMAIL PROTECTED]>wrote:
>>>>>
>>>>>> I generated a keytab with the following cmd and it supports multiple
>>>>>> encryption types other than aes256 as listed below.
>>>>>> But I still get the same error from sqoop import tool because the
>>>>>> sqoop.keytab is not being read (sqoop being the hbase client in this case).
>>>>>>
>>>>>> kadmin:  ktadd -k sqoop.keytab kuser1
>>>>>> Entry for principal kuser1 with kvno 2, encryption type
>>>>>> aes256-cts-hmac-sha1-96 added to keytab WRFILE:sqoop.keytab.
>>>>>> Entry for principal kuser1 with kvno 2, encryption type
>>>>>> aes128-cts-hmac-sha1-96 added to keytab WRFILE:sqoop.keytab.
>>>>>> Entry for principal kuser1 with kvno 2, encryption type des3-cbc-sha1