Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> alter table add partition error


Copy link to this message
-
RE: alter table add partition error
Here are the create table and alter table statements:

CREATE external TABLE if not exists mytable (

 

  bc string

  ,src_spaceid string

  ,srcpvid string

  ,dstpvid string

  ,dst_spaceid string

  ,page_params map<string, string>

  ,clickinfo map<string, string>

  ,viewinfo array<map<string, string>>

 

)

PARTITIONED BY ( datestamp string, srcid string, action string, testid
string )

row format serde 'com.yahoo.mySerde'

stored as inputformat 'org.apache.hadoop.mapred.SequenceFileInputFormat'
outputformat 'org.apache.hadoop.mapred.SequenceFileOutputFormat'

location '/user/pradeepk/mytable';

 

hive --auxpath ult-serde.jar -e "ALTER TABLE mytable add
partition(datestamp = '20091101', srcid '19174',action='click',testid='NOTESTID') location
'/user/pradeepk/mytable/20091101/19174/click/NOTESTID';"

 

I get the following error:

Hive history
file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt

FAILED: Error in metadata: org.apache.thrift.TApplicationException:
get_partition failed: unknown result

FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask

If I don't use thrift and use a hive-site.xml to directly talk to the
db, the alter table seems to succeed:

hive --auxpath ult-serde.jar --config hive_conf_without_thrift -e "ALTER
TABLE mytable add partition(datestamp = '20091101', srcid '19174',action='click',testid='NOTESTID') location
'/user/pradeepk/mytable/20091101/19174/click/NOTESTID';"

 

However I get errors when I try to run a query:

[pradeepk@chargesize:~/dev]hive --auxpath ult-serde.jar --config
hive_conf_without_thrift -e "select src_spaceid from
ult_search_austria_ult where datestamp='20091101';"

10/06/17 13:22:34 WARN conf.Configuration: DEPRECATED: hadoop-site.xml
found in the classpath. Usage of hadoop-site.xml is deprecated. Instead
use core-site.xml, mapred-site.xml and hdfs-site.xml to override
properties of core-default.xml, mapred-default.xml and hdfs-default.xml
respectively

Hive history
file=/tmp/pradeepk/hive_job_log_pradeepk_201006171322_1913647383.txt

Total MapReduce jobs = 1

Launching Job 1 out of 1

Number of reduce tasks is set to 0 since there's no reduce operator

java.lang.IllegalArgumentException: Can not create a Path from an empty
string

        at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)

        at org.apache.hadoop.fs.Path.<init>(Path.java:90)

        at org.apache.hadoop.fs.Path.<init>(Path.java:50)

        at
org.apache.hadoop.mapred.JobClient.copyRemoteFiles(JobClient.java:523)

        at
org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient
.java:603)

        at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)

        at
org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:730)

        at
org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:684)

        at
org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)

        at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:
55)

        at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:631)

        at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:504)

 

Any help is much appreciated.

 

Pradeep

 

________________________________

From: Ashish Thusoo [mailto:[EMAIL PROTECTED]]
Sent: Thursday, June 17, 2010 11:15 AM
To: [EMAIL PROTECTED]
Subject: RE: alter table add partition error

 

hmm... Can you send the exact command and also the create table command
for this table.

 

Ashish

 

________________________________

From: Pradeep Kamath [mailto:[EMAIL PROTECTED]]
Sent: Thursday, June 17, 2010 9:09 AM
To: [EMAIL PROTECTED]
Subject: RE: alter table add partition error

Sorry - that was a cut-paste error - I don't have the action part - so I
am specifying key-value pairs. Since what I am trying to do seems like a
basic operation, I am wondering if it's something to do with my Serde -
unfortunately the error I see gives me no clue of what could be wrong -
any help would be greatly appreciated!

 

Thanks,

Pradeep

 

________________________________

From: yq he [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, June 16, 2010 5:54 PM
To: [EMAIL PROTECTED]
Subject: Re: alter table add partition error

 

Hi Pradeep,

 

partition definition need to be key-value pairs. partition key `action`
seems missed the value.

 

Thanks

Yongqiang

On Wed, Jun 16, 2010 at 5:22 PM, Pradeep Kamath <[EMAIL PROTECTED]>
wrote:

Hi,

    I am trying to create an external table against already existing
data in sequencefile format. However I have written a custom Serde to
interpret the data. I am able to create the table fine but get the
exception shown in the session output below when I try to add partition
- any help would be greatly appreciated.

 

Thanks,

Pradeep

 

== session output ==
 

[pradeepk@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER
TABLE mytable add partition(datestamp = '20091101', srcid = '10',action)
location '/user/pradeepk/mytable/20091101/10';"

10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml
found in the classpath. Usage of hadoop-site.xml is deprecated. Instead
use core-site.xml, mapred-site.xml and hdfs-site.xml to override
properties of core-default.xml, mapred-default.xml and hdfs-default.xml
respectively

Hive history
file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt

FAILED: Error in metadata: org.apache.thrift.TApplicationException:
get_partition failed: unknown result

FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask

[pradeepk@chargesize:~/dev/howl]

 

== session output ==
 

/tmp/pradeepk/hive.log has:

2010-06-16 17:09:00,841 ERROR exec.DDLTask
(SessionState.java:printError(269)) - FAILED: Error in metadata:
org.apache.thrift.TApplicationException: get_partition failed: unknown
result

org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.thrift.TApplic
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB