How can you add a partition to a table programmatically? I have a mapreducejob that needs to perform a "ALTER TABLE x ADD PARTITION IF NOT EXIST (foo=3D bar) LOCATION '<Mapreduce output>' after it completes. Is there an example of this anwhere?

The most programmatic way is to use external tables and then use
HiveService (thrift) to add the partition after your MR job has ran
successfully.
On Mon, Feb 27, 2012 at 12:17 PM, Hans Uhlig <[EMAIL PROTECTED]> wrote:
> How can you add a partition to a table programmatically? I have a
> mapreduce=  job that needs to perform a "ALTER TABLE x ADD PARTITION IF NOT
> EXIST (foo=  =3D bar) LOCATION '<Mapreduce output>' after it completes. Is
> there an exa= mple of this anwhere?

I am using external tables already. Is the Metastore API not a usable API
from inside another Java Application?

On Mon, Feb 27, 2012 at 9:19 AM, Edward Capriolo <[EMAIL PROTECTED]>wrote:

> The most programmatic way is to use external tables and then use
> HiveService (thrift) to add the partition after your MR job has ran
> successfully.
>
>
> On Mon, Feb 27, 2012 at 12:17 PM, Hans Uhlig <[EMAIL PROTECTED]> wrote:
> > How can you add a partition to a table programmatically? I have a
> > mapreduce=  job that needs to perform a "ALTER TABLE x ADD PARTITION IF
> NOT
> > EXIST (foo=  =3D bar) LOCATION '<Mapreduce output>' after it completes.
> Is
> > there an exa= mple of this anwhere?
>

You can use the metastore API directly if you source all the correct
hadoop and hive xml files, have all of hives/lib in your classpath and
network access.  It is much more graceful to correct through the
thrift service.
On Mon, Feb 27, 2012 at 12:53 PM, Hans Uhlig <[EMAIL PROTECTED]> wrote:
> I am using external tables already. Is the Metastore API not a usable API
> from inside another Java Application?
>
> On Mon, Feb 27, 2012 at 9:19 AM, Edward Capriolo <[EMAIL PROTECTED]>
> wrote:
>>
>> The most programmatic way is to use external tables and then use
>> HiveService (thrift) to add the partition after your MR job has ran
>> successfully.
>>
>>
>> On Mon, Feb 27, 2012 at 12:17 PM, Hans Uhlig <[EMAIL PROTECTED]> wrote:
>> > How can you add a partition to a table programmatically? I have a
>> > mapreduce=  job that needs to perform a "ALTER TABLE x ADD PARTITION IF
>> > NOT
>> > EXIST (foo=  =3D bar) LOCATION '<Mapreduce output>' after it completes.
>> > Is
>> > there an exa= mple of this anwhere?
>
>

If you have hive-server running somewhere you can do following-

HiveConf hiveConf = new HiveConf(MyClass.class);

hiveConf.set("hive.metastore.local", "false");

hiveConf.set(ConfVars.METASTOREURIS.varname, url);

HiveMetaStoreClient client = new HiveMetaStoreClient(hiveConf);

and then do-
client.add_partition(part);

Thanks,
Aniket

On Mon, Feb 27, 2012 at 9:58 AM, Edward Capriolo <[EMAIL PROTECTED]>wrote:

> You can use the metastore API directly if you source all the correct
> hadoop and hive xml files, have all of hives/lib in your classpath and
> network access.  It is much more graceful to correct through the
> thrift service.
>
>
> On Mon, Feb 27, 2012 at 12:53 PM, Hans Uhlig <[EMAIL PROTECTED]> wrote:
> > I am using external tables already. Is the Metastore API not a usable API
> > from inside another Java Application?
> >
> > On Mon, Feb 27, 2012 at 9:19 AM, Edward Capriolo <[EMAIL PROTECTED]>
> > wrote:
> >>
> >> The most programmatic way is to use external tables and then use
> >> HiveService (thrift) to add the partition after your MR job has ran
> >> successfully.
> >>
> >>
> >> On Mon, Feb 27, 2012 at 12:17 PM, Hans Uhlig <[EMAIL PROTECTED]>
> wrote:
> >> > How can you add a partition to a table programmatically? I have a
> >> > mapreduce=  job that needs to perform a "ALTER TABLE x ADD PARTITION
> IF
> >> > NOT
> >> > EXIST (foo=  =3D bar) LOCATION '<Mapreduce output>' after it
> completes.
> >> > Is
> >> > there an exa= mple of this anwhere?
> >
> >
>

--
"...:::Aniket:::... Quetzalco@tl"

Also consider that if you're using the metastore API, you'll most likely
need to recompile  whenever you upgrade hive, while you most likely won't
need to do that if you use thrift.

R.

On Mon, Feb 27, 2012 at 9:58 AM, Edward Capriolo <[EMAIL PROTECTED]>wrote:

> You can use the metastore API directly if you source all the correct
> hadoop and hive xml files, have all of hives/lib in your classpath and
> network access.  It is much more graceful to correct through the
> thrift service.
>
>
> On Mon, Feb 27, 2012 at 12:53 PM, Hans Uhlig <[EMAIL PROTECTED]> wrote:
> > I am using external tables already. Is the Metastore API not a usable API
> > from inside another Java Application?
> >
> > On Mon, Feb 27, 2012 at 9:19 AM, Edward Capriolo <[EMAIL PROTECTED]>
> > wrote:
> >>
> >> The most programmatic way is to use external tables and then use
> >> HiveService (thrift) to add the partition after your MR job has ran
> >> successfully.
> >>
> >>
> >> On Mon, Feb 27, 2012 at 12:17 PM, Hans Uhlig <[EMAIL PROTECTED]>
> wrote:
> >> > How can you add a partition to a table programmatically? I have a
> >> > mapreduce=  job that needs to perform a "ALTER TABLE x ADD PARTITION
> IF
> >> > NOT
> >> > EXIST (foo=  =3D bar) LOCATION '<Mapreduce output>' after it
> completes.
> >> > Is
> >> > there an exa= mple of this anwhere?
> >
> >
>