Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> hive and hdfs location


Copy link to this message
-
Re: hive and hdfs location
Programmatically,
listPartitionsByFilter on
HiveMetaStoreClient returns list of partitions for a given filter criteria
(which supports only string based partitions, we need to open a jira for
that).
For each of these partitions, you can

ptn.getSd().getLocation() to get its location.
Or you can use mysql directly (if thats okay with you, I m sure its not
recommended).
something like,
select LOCATION from PARTITIONS ptn, SDS sd where sd.SD_ID=ptn.SD_ID and
PART_NAME='part=1'... you may also want to join this with TBLS for things
like TBL_NAME. You will have to consider non partitioned tables separately.
All the information should be available in schema of metastore.

Thanks,
Aniket
On Mon, Jan 9, 2012 at 9:24 PM, Chris Kudelka <[EMAIL PROTECTED]> wrote:

> Is there an easy/elegant way to query hive for a table's (and a
> partition's) location in HDFS?
>
> I'm aware that you can get the location using "describe extended
> table_name" but it like to be able to query on just the location key.
>
> If not, is there a way to do so using the mysql metastore db?
>
> Ideally, I'd come out with something like:
>
> hdfs://namenode:/user/hive/warehouse/database.db/table/partition_1/datafile
>
> Thanks!
>

--
"...:::Aniket:::... Quetzalco@tl"
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB