Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Hive External Table issue


Copy link to this message
-
Re: Hive External Table issue
Ramki Palle 2013-06-20, 18:34
Nitin,

Can you go through the thread with subject "S3/EMR Hive: Load contents of a
single file"  on Tue, 26 Mar, 17:11> at
http://mail-archives.apache.org/mod_mbox/hive-user/201303.mbox/thread?1

 This gives the whole discussion about the topic of table location pointing
to a filename vs. directory.

Can you give your insight from this discussion and the discussion you
mentioned at stackoverflow link?

Regards,
Ramki.

On Thu, Jun 20, 2013 at 11:14 AM, Nitin Pawar <[EMAIL PROTECTED]>wrote:

> Also see this JIRA
> https://issues.apache.org/jira/browse/HIVE-951
>
> I think issue you are facing is due to the JIRA
>
>
> On Thu, Jun 20, 2013 at 11:41 PM, Nitin Pawar <[EMAIL PROTECTED]>wrote:
>
>> Mark has answered this before
>>
>> http://stackoverflow.com/questions/11269203/when-creating-an-external-table-in-hive-can-i-point-the-location-to-specific-fil
>>
>> If this link does not answer your question, do let us know
>>
>>
>> On Thu, Jun 20, 2013 at 11:33 PM, sanjeev sagar <[EMAIL PROTECTED]>wrote:
>>
>>> Two issues:
>>>
>>> 1. I've created external tables in hive based on file location before
>>> and it work without any issue. It don't have to be a directory.
>>>
>>> 2. If there are more than one file in the directory, and you create
>>> external table based on directory then how the table knows that which file
>>> it need to look for the data?
>>>
>>> I tried to create the table based on directory, it created the table but
>>> all the rows were NULL.
>>>
>>> -Sanjeev
>>>
>>>
>>> On Thu, Jun 20, 2013 at 10:30 AM, Nitin Pawar <[EMAIL PROTECTED]>wrote:
>>>
>>>> in hive when you create table and use the location to refer hdfs path,
>>>> that path is supposed to be a directory.
>>>> If the directory is not existing it will try to create it and if its a
>>>> file it will throw an error as its not a directory
>>>>
>>>> thats the error you are getting that location you referred is a file.
>>>> Change it to the directory and see if that works for you
>>>>
>>>>
>>>> On Thu, Jun 20, 2013 at 10:57 PM, sanjeev sagar <
>>>> [EMAIL PROTECTED]> wrote:
>>>>
>>>>> I did mention in my mail the hdfs file exists in that location. See
>>>>> below
>>>>>
>>>>> In HDFS: file exists
>>>>>
>>>>>
>>>>>
>>>>> hadoop fs -ls
>>>>>
>>>>> /user/flume/events/request_logs/
>>>>> ar1.vgs.mypoints.com/13-06-13/FlumeData.1371144648033
>>>>>
>>>>> Found 1 items
>>>>>
>>>>> -rw-r--r--   3 hdfs supergroup 2242037226 2013-06-13 11:14
>>>>>
>>>>> /user/flume/events/request_logs/
>>>>> ar1.vgs.mypoints.com/13-06-13/FlumeData.1371144648033
>>>>>
>>>>> so the directory and file both exists.
>>>>>
>>>>>
>>>>> On Thu, Jun 20, 2013 at 10:24 AM, Nitin Pawar <[EMAIL PROTECTED]
>>>>> > wrote:
>>>>>
>>>>>> MetaException(message:hdfs://
>>>>>> h1.vgs.mypoints.com:8020/user/flume/events/request_logs/ar1.vgs.mypoints.com/13-06-13/FlumeData.1371144648033
>>>>>>
>>>>>> is not a directory or unable to create one)
>>>>>>
>>>>>>
>>>>>> it clearly says its not a directory. Point to the dictory and it will
>>>>>> work
>>>>>>
>>>>>>
>>>>>> On Thu, Jun 20, 2013 at 10:52 PM, sanjeev sagar <
>>>>>> [EMAIL PROTECTED]> wrote:
>>>>>>
>>>>>>> Hello Everyone, I'm running into the following Hive external table
>>>>>>> issue.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> hive> CREATE EXTERNAL TABLE access(
>>>>>>>
>>>>>>>      >       host STRING,
>>>>>>>
>>>>>>>      >       identity STRING,
>>>>>>>
>>>>>>>      >       user STRING,
>>>>>>>
>>>>>>>      >       time STRING,
>>>>>>>
>>>>>>>      >       request STRING,
>>>>>>>
>>>>>>>      >       status STRING,
>>>>>>>
>>>>>>>      >       size STRING,
>>>>>>>
>>>>>>>      >       referer STRING,
>>>>>>>
>>>>>>>      >       agent STRING)
>>>>>>>
>>>>>>>      >       ROW FORMAT SERDE
>>>>>>>
>>>>>>> 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
>>>>>>>
>>>>>>>      >       WITH SERDEPROPERTIES (
>>>>>>>
>>>>>>>      >      "input.regex" = "([^ ]*) ([^ ]*) ([^ ]*)
>>>>>>> (-|\\[[^\\]]*\\])
>>>>>>>
>>>>>>> ([^ \"]*|\"[^\"]*\") (-|[0-9]*) (-|[0-9]*)(?: ([^ \"]*|\"[^\"]*\")