Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> S3/EMR Hive: Load contents of a single file


+
Tony Burton 2013-03-26, 17:11
+
Ramki Palle 2013-03-26, 17:41
+
Sanjay Subramanian 2013-03-26, 17:21
+
Tony Burton 2013-03-26, 17:39
+
Sanjay Subramanian 2013-03-26, 17:41
+
Tony Burton 2013-03-26, 17:45
+
Keith Wiley 2013-03-26, 19:39
+
Tony Burton 2013-03-27, 08:46
+
Tony Burton 2013-03-27, 09:58
+
Keith Wiley 2013-03-27, 17:02
Copy link to this message
-
RE: S3/EMR Hive: Load contents of a single file

No problem Keith - it was a worthwhile exercise for me to go back and double check everything was working as expected.
-----Original Message-----
From: Keith Wiley [mailto:[EMAIL PROTECTED]]
Sent: 27 March 2013 17:03
To: [EMAIL PROTECTED]
Subject: Re: S3/EMR Hive: Load contents of a single file

Okay, I also saw your previous response which analyzed queries into two tables built around two files in the same directory.  I guess I was simply wrong in my understanding that a Hive table is fundamentally associated with a directory instead of a file.  Turns out, it be can either one.  A directory table uses all files in the directory while a file table uses one specific file and properly avoids sibling files.  My bad.

Thanks for the careful analysis and clarification.  TIL!

Cheers!

On Mar 27, 2013, at 02:58 , Tony Burton wrote:

> A bit more info - do an extended description of the table:
>  
> $ desc extended gsrc1;
>  
> And the "location" field is "location:s3://mybucket/path/to/data/src1.txt"
>  
> Do the same on a table created with a location pointing at the directory and the same info gives (not surprisingly) "location:s3://mybucket/path/to/data/"
>

________________________________________________________________________________
Keith Wiley     [EMAIL PROTECTED]     keithwiley.com    music.keithwiley.com

"I used to be with it, but then they changed what it was.  Now, what I'm with isn't it, and what's it seems weird and scary to me."
                                           --  Abe (Grandpa) Simpson ________________________________________________________________________________

Please consider the environment before printing this email

Inbound Email has been scanned for viruses and SPAM

**********************************************************************
Please consider the environment before printing this email or attachments

This email and any attachments are confidential, protected by copyright and may be legally privileged.  If you are not the intended recipient, then the dissemination or copying of this email is prohibited. If you have received this in error, please notify the sender by replying by email and then delete the email completely from your system.  Neither Sporting Index nor the sender accepts responsibility for any virus, or any other defect which might affect any computer or IT system into which the email is received and/or opened.  It is the responsibility of the recipient to scan the email and no responsibility is accepted for any loss or damage arising in any way from receipt or use of this email.  Sporting Index Ltd is a company registered in England and Wales with company number 2636842, whose registered office is at Gateway House, Milverton Street, London, SE11 4AP.  Sporting Index Ltd is authorised and regulated by the UK Financial Services Authority (reg. no. 150404) and Gambling Commission (reg. no. 000-027343-R-308898-001).  Any financial promotion contained herein has been issued
and approved by Sporting Index Ltd.

Outbound email has been scanned for viruses and SPAM
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB