Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> alter table add partition error


+
Pradeep Kamath 2010-06-17, 00:22
+
yq he 2010-06-17, 00:54
+
Pradeep Kamath 2010-06-17, 16:09
+
Ashish Thusoo 2010-06-17, 18:15
Copy link to this message
-
RE: alter table add partition error
Here are the create table and alter table statements:

CREATE external TABLE if not exists mytable (

 

  bc string

  ,src_spaceid string

  ,srcpvid string

  ,dstpvid string

  ,dst_spaceid string

  ,page_params map<string, string>

  ,clickinfo map<string, string>

  ,viewinfo array<map<string, string>>

 

)

PARTITIONED BY ( datestamp string, srcid string, action string, testid
string )

row format serde 'com.yahoo.mySerde'

stored as inputformat 'org.apache.hadoop.mapred.SequenceFileInputFormat'
outputformat 'org.apache.hadoop.mapred.SequenceFileOutputFormat'

location '/user/pradeepk/mytable';

 

hive --auxpath ult-serde.jar -e "ALTER TABLE mytable add
partition(datestamp = '20091101', srcid '19174',action='click',testid='NOTESTID') location
'/user/pradeepk/mytable/20091101/19174/click/NOTESTID';"

 

I get the following error:

Hive history
file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt

FAILED: Error in metadata: org.apache.thrift.TApplicationException:
get_partition failed: unknown result

FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask

If I don't use thrift and use a hive-site.xml to directly talk to the
db, the alter table seems to succeed:

hive --auxpath ult-serde.jar --config hive_conf_without_thrift -e "ALTER
TABLE mytable add partition(datestamp = '20091101', srcid '19174',action='click',testid='NOTESTID') location
'/user/pradeepk/mytable/20091101/19174/click/NOTESTID';"

 

However I get errors when I try to run a query:

[pradeepk@chargesize:~/dev]hive --auxpath ult-serde.jar --config
hive_conf_without_thrift -e "select src_spaceid from
ult_search_austria_ult where datestamp='20091101';"

10/06/17 13:22:34 WARN conf.Configuration: DEPRECATED: hadoop-site.xml
found in the classpath. Usage of hadoop-site.xml is deprecated. Instead
use core-site.xml, mapred-site.xml and hdfs-site.xml to override
properties of core-default.xml, mapred-default.xml and hdfs-default.xml
respectively

Hive history
file=/tmp/pradeepk/hive_job_log_pradeepk_201006171322_1913647383.txt

Total MapReduce jobs = 1

Launching Job 1 out of 1

Number of reduce tasks is set to 0 since there's no reduce operator

java.lang.IllegalArgumentException: Can not create a Path from an empty
string

        at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)

        at org.apache.hadoop.fs.Path.<init>(Path.java:90)

        at org.apache.hadoop.fs.Path.<init>(Path.java:50)

        at
org.apache.hadoop.mapred.JobClient.copyRemoteFiles(JobClient.java:523)

        at
org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient
.java:603)

        at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)

        at
org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:730)

        at
org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:684)

        at
org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)

        at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:
55)

        at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:631)

        at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:504)

 

Any help is much appreciated.

 

Pradeep

 

________________________________

From: Ashish Thusoo [mailto:[EMAIL PROTECTED]]
Sent: Thursday, June 17, 2010 11:15 AM
To: [EMAIL PROTECTED]
Subject: RE: alter table add partition error

 

hmm... Can you send the exact command and also the create table command
for this table.

 

Ashish

 

________________________________

From: Pradeep Kamath [mailto:[EMAIL PROTECTED]]
Sent: Thursday, June 17, 2010 9:09 AM
To: [EMAIL PROTECTED]
Subject: RE: alter table add partition error

Sorry - that was a cut-paste error - I don't have the action part - so I
am specifying key-value pairs. Since what I am trying to do seems like a
basic operation, I am wondering if it's something to do with my Serde -
unfortunately the error I see gives me no clue of what could be wrong -
any help would be greatly appreciated!

 

Thanks,

Pradeep

 

________________________________

From: yq he [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, June 16, 2010 5:54 PM
To: [EMAIL PROTECTED]
Subject: Re: alter table add partition error

 

Hi Pradeep,

 

partition definition need to be key-value pairs. partition key `action`
seems missed the value.

 

Thanks

Yongqiang

On Wed, Jun 16, 2010 at 5:22 PM, Pradeep Kamath <[EMAIL PROTECTED]>
wrote:

Hi,

    I am trying to create an external table against already existing
data in sequencefile format. However I have written a custom Serde to
interpret the data. I am able to create the table fine but get the
exception shown in the session output below when I try to add partition
- any help would be greatly appreciated.

 

Thanks,

Pradeep

 

== session output ==
 

[pradeepk@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER
TABLE mytable add partition(datestamp = '20091101', srcid = '10',action)
location '/user/pradeepk/mytable/20091101/10';"

10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml
found in the classpath. Usage of hadoop-site.xml is deprecated. Instead
use core-site.xml, mapred-site.xml and hdfs-site.xml to override
properties of core-default.xml, mapred-default.xml and hdfs-default.xml
respectively

Hive history
file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt

FAILED: Error in metadata: org.apache.thrift.TApplicationException:
get_partition failed: unknown result

FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask

[pradeepk@chargesize:~/dev/howl]

 

== session output ==
 

/tmp/pradeepk/hive.log has:

2010-06-16 17:09:00,841 ERROR exec.DDLTask
(SessionState.java:printError(269)) - FAILED: Error in metadata:
org.apache.thrift.TApplicationException: get_partition failed: unknown
result

org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.thrift.TApplic
+
Pradeep Kamath 2010-06-18, 17:19
+
Ning Zhang 2010-06-18, 17:49
+
Edward Capriolo 2010-06-18, 18:22
+
Pradeep Kamath 2010-06-18, 18:25
+
Ning Zhang 2010-06-18, 18:51
+
Paul Yang 2010-06-18, 18:59
+
Pradeep Kamath 2010-06-18, 19:18
+
Paul Yang 2010-06-18, 19:10
+
Pradeep Kamath 2010-06-18, 20:37
+
Paul Yang 2010-06-18, 21:19
+
Pradeep Kamath 2010-06-18, 21:51
+
Pradeep Kamath 2010-06-21, 21:11
+
Paul Yang 2010-06-21, 21:34
+
Pradeep Kamath 2010-06-22, 00:46
+
Paul Yang 2010-06-22, 01:25
+
Pradeep Kamath 2010-06-22, 17:21
+
Paul Yang 2010-06-23, 01:06
+
Pradeep Kamath 2010-06-23, 16:03
+
John Sichi 2010-06-23, 16:44
+
Pradeep Kamath 2010-06-25, 20:36