I have created a partitioned HIVE external table as follows
create external table test_part (key int, val int) partitioned by (part int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' STORED AS TEXTFILE LOCATION '/test/';
I have the following folders and files in /test
Now I try to automatically add the partitions into the table using the 'msck repair' command
hive> msck repair table test_part;
Partitions not in metastore: test_part:part=3 test_part:part=4
Repair: Added partition to metastore test_part:part=3
Repair: Added partition to metastore test_part:part=4
Time taken: 0.685 seconds
As you can see only partitions which do not contain any data have been added. part=1 and part=2 folders have been ignored.
Is this by design? Am I using this correctly?
The only alternative I found is to explicitly add partitions using 'alter table add partition' once every subfolder.
Is there a simpler way to achieve this?
Thanks in advance
Mark Grover 2013-02-07, 19:53
Krishnappa, Suresh 2013-02-07, 21:17