No reponse from the user list. Reposting in the dev list:

Hi,

It seems like the HiveMetaStoreClient is the only way to programmatically
access (and retreive)  the metastore for DDL and metadata queries as well as
retrieving delegation tokens. It isn't documented anywhere, so I'd like to
know if it is advisable to have code make use of this directly or is this an
internal API?  If the latter what would be the recommended way of
programmatically doing the previously mentioned operations?

-Francis

Ideally you want to do everything through the hiveQL language. Calling
the read only methods are safe and would be easier/better then Calling
'show tables' and attempting to parse. But I would consider the rest
of the interface for "experts" only. Also this interface can change
slightly between versions making it harder to build code against.

Edward

On Thu, Mar 29, 2012 at 2:50 PM, Francis Liu <[EMAIL PROTECTED]> wrote:
> No reponse from the user list. Reposting in the dev list:
>
> Hi,
>
> It seems like the HiveMetaStoreClient is the only way to programmatically
> access (and retreive)  the metastore for DDL and metadata queries as well as
> retrieving delegation tokens. It isn't documented anywhere, so I'd like to
> know if it is advisable to have code make use of this directly or is this an
> internal API?  If the latter what would be the recommended way of
> programmatically doing the previously mentioned operations?
>
> -Francis
>
>

Hi Edward,

We would also need other apis such as dropping partitions and tables as
well as retrieving delegation tokens. "Experts only" is doable. The main
concern of it being public is wether in future releases, Hive developers
will do their best to make sure the interface is backwards compatible.

-Francis

On 3/29/12 1:04 PM, "Edward Capriolo" <[EMAIL PROTECTED]> wrote:

>Ideally you want to do everything through the hiveQL language. Calling
>the read only methods are safe and would be easier/better then Calling
>'show tables' and attempting to parse. But I would consider the rest
>of the interface for "experts" only. Also this interface can change
>slightly between versions making it harder to build code against.
>
>Edward
>
>On Thu, Mar 29, 2012 at 2:50 PM, Francis Liu <[EMAIL PROTECTED]> wrote:
>> No reponse from the user list. Reposting in the dev list:
>>
>> Hi,
>>
>> It seems like the HiveMetaStoreClient is the only way to
>>programmatically
>> access (and retreive)  the metastore for DDL and metadata queries as
>>well as
>> retrieving delegation tokens. It isn't documented anywhere, so I'd like
>>to
>> know if it is advisable to have code make use of this directly or is
>>this an
>> internal API?  If the latter what would be the recommended way of
>> programmatically doing the previously mentioned operations?
>>
>> -Francis
>>
>>

Hi Francis,

The HiveMetaStoreClient library is a thin wrapper layer written around the
Thrift generated client code. It is not a public API and is subject to
change in the future. If "experts only" is doable, then I recommend
integrating directly at the Thrift API level (which is a public API), e.g.
call the Thrift generated code directly instead of accessing it through the
HiveMetaStoreClient library.

Thanks.

Carl

On Thu, Mar 29, 2012 at 2:05 PM, Francis Liu <[EMAIL PROTECTED]> wrote:

> Hi Edward,
>
> We would also need other apis such as dropping partitions and tables as
> well as retrieving delegation tokens. "Experts only" is doable. The main
> concern of it being public is wether in future releases, Hive developers
> will do their best to make sure the interface is backwards compatible.
>
> -Francis
>
> On 3/29/12 1:04 PM, "Edward Capriolo" <[EMAIL PROTECTED]> wrote:
>
> >Ideally you want to do everything through the hiveQL language. Calling
> >the read only methods are safe and would be easier/better then Calling
> >'show tables' and attempting to parse. But I would consider the rest
> >of the interface for "experts" only. Also this interface can change
> >slightly between versions making it harder to build code against.
> >
> >Edward
> >
> >On Thu, Mar 29, 2012 at 2:50 PM, Francis Liu <[EMAIL PROTECTED]> wrote:
> >> No reponse from the user list. Reposting in the dev list:
> >>
> >> Hi,
> >>
> >> It seems like the HiveMetaStoreClient is the only way to
> >>programmatically
> >> access (and retreive)  the metastore for DDL and metadata queries as
> >>well as
> >> retrieving delegation tokens. It isn't documented anywhere, so I'd like
> >>to
> >> know if it is advisable to have code make use of this directly or is
> >>this an
> >> internal API?  If the latter what would be the recommended way of
> >> programmatically doing the previously mentioned operations?
> >>
> >> -Francis
> >>
> >>
>
>
>

Hi Carl,

I see, thanks for clarifying that. It'd be brittle to write our own client
as HiveMetaStoreClient is not really that "thin" since it is already
performing callbacks for HiveMetaHook and someone might decide to add more
(or remove) logic into it.

The java API is an ask from one of our users for Hcatalog. Given that
HiveMetaStoreClient is a thin wrapper around public apis, then change
shouldn't be drastic? It seems the cleaner way to go about it is to create
an hcat client wrapper around HiveMetaStoreClient and have HCatClient as
public?

-Francis
On 3/29/12 3:34 PM, "Carl Steinbach" <[EMAIL PROTECTED]> wrote:

>Hi Francis,
>
>The HiveMetaStoreClient library is a thin wrapper layer written around the
>Thrift generated client code. It is not a public API and is subject to
>change in the future. If "experts only" is doable, then I recommend
>integrating directly at the Thrift API level (which is a public API), e.g.
>call the Thrift generated code directly instead of accessing it through
>the
>HiveMetaStoreClient library.
>
>Thanks.
>
>Carl
>
>On Thu, Mar 29, 2012 at 2:05 PM, Francis Liu <[EMAIL PROTECTED]> wrote:
>
>> Hi Edward,
>>
>> We would also need other apis such as dropping partitions and tables as
>> well as retrieving delegation tokens. "Experts only" is doable. The main
>> concern of it being public is wether in future releases, Hive developers
>> will do their best to make sure the interface is backwards compatible.
>>
>> -Francis
>>
>> On 3/29/12 1:04 PM, "Edward Capriolo" <[EMAIL PROTECTED]> wrote:
>>
>> >Ideally you want to do everything through the hiveQL language. Calling
>> >the read only methods are safe and would be easier/better then Calling
>> >'show tables' and attempting to parse. But I would consider the rest
>> >of the interface for "experts" only. Also this interface can change
>> >slightly between versions making it harder to build code against.
>> >
>> >Edward
>> >
>> >On Thu, Mar 29, 2012 at 2:50 PM, Francis Liu <[EMAIL PROTECTED]> wrote:
>> >> No reponse from the user list. Reposting in the dev list:
>> >>
>> >> Hi,
>> >>
>> >> It seems like the HiveMetaStoreClient is the only way to
>> >>programmatically
>> >> access (and retreive)  the metastore for DDL and metadata queries as
>> >>well as
>> >> retrieving delegation tokens. It isn't documented anywhere, so I'd
>>like
>> >>to
>> >> know if it is advisable to have code make use of this directly or is
>> >>this an
>> >> internal API?  If the latter what would be the recommended way of
>> >> programmatically doing the previously mentioned operations?
>> >>
>> >> -Francis
>> >>
>> >>
>>
>>
>>

Sorry for the delay in responding.
> The java API is an ask from one of our users for Hcatalog. Given that
> HiveMetaStoreClient is a thin wrapper around public apis, then change
> shouldn't be drastic? It seems the cleaner way to go about it is to create
> an hcat client wrapper around HiveMetaStoreClient and have HCatClient as
> public?
>

The problem with this proposal is that until HiveMetaStoreClient is a
public API, HCatalog has just as little right to integrate with it directly
as any other third-party. I think the best solution is to invest the time
required to make HiveMetaStoreClient a public API, or to wrap
HiveMetaStoreClient with HiveMetaStoreClient2 and make that a public API
(we'll probably need to take the latter approach since HiveMetastoreClient
currently exposes raw Thrift types).

I filed HIVE-3280 to track this task.

Thanks.

Carl