Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> How to load a web log file (text format) to Hive with compression

Copy link to this message
Re: How to load a web log file (text format) to Hive with compression
Hive has built-in support for gzipped text files so you don't need to do
anything special with them (meaning you just load them like regular files).
However, there is one important limitation with this approach these files
are not splittable. Depending on the size of your log files and how many of
them you have perday that may not be a problem. The other, slightly more
complicated approach is to use sequence files, which support splittable
compression. You can find more details here:

On Mon, Nov 11, 2013 at 5:09 PM, Raj Hadoop <[EMAIL PROTECTED]> wrote:

> Hi,
> I have a web log files (text format). I want to load these files to a Hive
> table in compressed format. How do I do it ?
> Should I compress the text file (using any Linux utilities) and then
> create the Hive table?
> Can any one provide me the Hive syntax for loading the compressed file?
> Thanks,
> Raj