Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - hive issue with sub-directories


Copy link to this message
-
Re: hive issue with sub-directories
bejoy_ks@... 2013-03-11, 06:36
Hi Suresh

AFAIK as of now a partition cannot contain sub directories, it can contain only files.

You may have to move the sub dirs out of the parent dir 'a' and create separate partitions for those.

Regards
Bejoy KS

Sent from remote device, Please excuse typos

-----Original Message-----
From: Suresh Krishnappa <[EMAIL PROTECTED]>
Date: Mon, 11 Mar 2013 10:58:05
To: <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: hive issue with sub-directories

Hi Mark,
I am using external table in HIVE.

This is how I am adding the partition

> alter table <mytable> add partition (pt=1) location '/test/a/';

I am able to run HIVE queries only if '/test/a/b' folder is deleted.

How can I retain this folder structure and still issue queries?

Thanks
Suresh

On Sun, Mar 10, 2013 at 12:48 AM, Mark Grover
<[EMAIL PROTECTED]>wrote:

> Suresh,
> By default, the partition column name has to be appear in HDFS
> directory structure.
>
> e.g.
> /user/hive/warehouse/<table name>/<partition col name>=<partition col
> value>/data1.txt
> /user/hive/warehouse/<table name>/<partition col name>=<partition col
> value>/data2.txt
>
>
> On Thu, Mar 7, 2013 at 7:20 AM, Suresh Krishnappa
> <[EMAIL PROTECTED]> wrote:
> > Hi All,
> > I have the following directory structure in hdfs
> >
> > /test/a/
> > /test/a/1.avro
> > /test/a/2.avro
> > /test/a/b/
> > /test/a/b/3.avro
> >
> > I created an external HIVE table using Avro Serde and added /test/a as a
> > partition to this table.
> >
> > I am not able to run a select query. Always getting the error 'not a
> file'
> > on '/test/a/b'
> >
> > Is this by design, a bug or am I missing some configuration?
> > I am using HIVE 0.10
> >
> > Thanks
> > Suresh
> >
>