Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Best format to use


Copy link to this message
-
Re: Best format to use
impala can work with compressed files, but it's sequence file, not
compressed directly.
On Tue, Apr 9, 2013 at 7:48 AM, Mark <[EMAIL PROTECTED]> wrote:

> Trying to determine what the best format to use for storing daily logs. We
> recently switch from snappy (.snappy) to gzip (.deflate) but I'm wondering
> if there is something better? Our main clients for these daily logs are pig
> and hive using an external table. We were thinking about testing out impala
> but we see that it doesn't work with compressed text files. Any suggestions?
>
> Thanks
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB