Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> alter table add partition error

Copy link to this message
RE: alter table add partition error
Hi Paul,

  Yes I don't mind working on a patch though I have no knowledge of how
thrift code works. I was also confused by your suggestion for a fix. I
thought the fix would be in
ThriftHiveMetaStore.Client.recv_get_partition() - to first check
if(result.o1 != null) throw result.o1, then to just return
result.success and not throw the "unknown result" exception. I tried
something along these lines and the alter table worked. Am I missing






From: Paul Yang [mailto:[EMAIL PROTECTED]]
Sent: Monday, June 21, 2010 2:35 PM
Subject: RE: alter table add partition error


Ah okay. So what's going on is that getPartition() is supposed to return
null if the specified partition doesn't exist. But because Thrift can't
handle null return values, we are seeing this exception. This isn't a
problem with a local metastore because Thrift isn't used.


One solution to this problem would be modify ObjectStore.getPartition()
to throw a NoSuchObjectException instead of returning null for
non-existent partitions. Then Hive.getPartition() can catch the
exception and return null to retain the original behavior.


Pradeep, are you interested in working on a patch for this problem?


From: Pradeep Kamath [mailto:[EMAIL PROTECTED]]
Sent: Monday, June 21, 2010 2:11 PM
Subject: RE: alter table add partition error


I tried debugging in code a little more. Here is what I found:

The code in ThriftHiveMetaStore eventually makes a call -
get_partition() passing the partition key values for the partition I am
trying to add using alter table. I assume this is to check that the
partition doesn't already exist.

I added a debug line in the following code:

  public Partition recv_get_partition() throws MetaException, TException


      TMessage msg = iprot_.readMessageBegin();

      if (msg.type == TMessageType.EXCEPTION) {

        TApplicationException x = TApplicationException.read(iprot_);


        throw x;


      get_partition_result result = new get_partition_result();



      System.err.println("XXX: result:" + result);


      if (result.isSetSuccess()) {

        return result.success;


      if (result.o1 != null) {

        throw result.o1;


      throw new
"get_partition failed: unknown result");



I also put a debug statements in the read() method:

    public void read(TProtocol iprot) throws TException {

      TField field;


      System.err.println("XXX: Reading TProtocol object:");

      while (true)


        field = iprot.readFieldBegin();

        System.err.println("XXX: field just read:" + field);

        if (field.type == TType.STOP) {



I got

XXX: Reading TProtocol object:

XXX: field just read:<TField name:'' type:0 field-id:0>

XXX: result:get_partition_result(success:null, o1:null)


The field read in the thrift response message is of type "STOP" and with
id of type SUCCESS. This seems right since there are no existing
partitions. But the way the rest of the code handles this, results in
the exception.


Any pointers?





From: Pradeep Kamath [mailto:[EMAIL PROTECTED]]
Sent: Friday, June 18, 2010 2:51 PM
Subject: RE: alter table add partition error


Looks like the standalone script works fine against the existing

./ThriftHiveMetastore-remote -h localhost:9080 get_partition_by_name
default dummy datestamp=20100602/srcid=100/action=view/testid=10

Partition(parameters={'transient_lastDdlTime': '1276881277'},
tableName='dummy', createTime=1276881277, lastAccessTime=0,
values=['20100602', '100', 'view', '10'], dbName='default',
reKeyTextOutputFormat', sortCols=[],
cols=[FieldSchema(comment=None, type='string', name='partition_name'),
FieldSchema(comment=None, type='int', name='partition_id')],
compressed=False, bucketCols=[], numBuckets=-1, parameters={},
.LazySimpleSerDe', name=None, parameters={'serialization.format': '1'}),



However when I tried to add another partition with the hive cli using

hive  -e "ALTER TABLE dummy add partition(datestamp = '20100602', srcid
= '100',action='click',testid='10') location

10/06/18 14:49:13 WARN conf.Configuration: DEPRECATED: hadoop-site.xml
found in the classpath. Usage of hadoop-site.xml is deprecated. Instead
use core-site.xml, mapred-site.xml and hdfs-site.xml to override
properties of core-default.xml, mapred-default.xml and hdfs-default.xml

Hive history

FAILED: Error in metadata: org.apache.thrift.TApplicationException:
get_partition failed: unknown result

FAILED: Execution Error, return code 1 from


tail -30 /tmp/pradeepk/hive.log


        at java.lang.reflect.Method.invoke(Method.java:597)

        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)


2010-06-18 14:49:14,124 ERROR exec.DDLTask
(SessionState.java:printError(277)) - FAILED: Error in metadata:
org.apache.thrift.TApplicationException: get_partition failed: unknown