Hi,

What is a good way to copy the entire content of a Hive metastore and insert it into another Hive metastore? The second metastore contains existing metadata that needs to be preserved. Both metastores are in MySQL, not fronted by any Hive metastore server.

My guess is Hive has some metastore API that may be appropriate here, but I don't know if that is true because I know zilch about that API - I don't even know where the API documentation is.

Thanks.
Steven

On Fri, Sep 30, 2011 at 5:05 PM, Steven Wong <[EMAIL PROTECTED]> wrote:

> Hi,
>
>
>
> What is a good way to copy the entire content of a Hive metastore and
> insert it into another Hive metastore? The second metastore contains
> existing metadata that needs to be preserved. Both metastores are in MySQL,
> not fronted by any Hive metastore server.
>
>
>
> My guess is Hive has some metastore API that may be appropriate here, but I
> don’t know if that is true because I know zilch about that API — I don’t
> even know where the API documentation is.
>
>
>
> Thanks.
>
> Steven
>

I would say a simple way is to take mysql dumps and then append the dump
files together. As long as you do not have two tables with the same name I
do not see any challenges with that. This portion of the API is not very
well documented because most do not interact with the metastore directly,
they only interact with HQL.

I think going to MySQL directly will have the problem of colliding internal ids (e.g. TBLS.TBL_ID).  I don't know if the API will have the same problem or not.

If there is no API documentation, which code (classes, methods) should I look at?
From: Edward Capriolo [mailto:[EMAIL PROTECTED]]
Sent: Friday, September 30, 2011 3:16 PM
To: [EMAIL PROTECTED]
Subject: Re: Copying a Hive metastore
On Fri, Sep 30, 2011 at 5:05 PM, Steven Wong <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Hi,

What is a good way to copy the entire content of a Hive metastore and insert it into another Hive metastore? The second metastore contains existing metadata that needs to be preserved. Both metastores are in MySQL, not fronted by any Hive metastore server.

My guess is Hive has some metastore API that may be appropriate here, but I don't know if that is true because I know zilch about that API - I don't even know where the API documentation is.

Thanks.
Steven

I would say a simple way is to take mysql dumps and then append the dump files together. As long as you do not have two tables with the same name I do not see any challenges with that. This portion of the API is not very well documented because most do not interact with the metastore directly, they only interact with HQL.