Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> hive.exec.compress.intermediate does not work?


Copy link to this message
-
hive.exec.compress.intermediate does not work?
I have a query that runs as 2 MR jobs:

set hive.exec.compress.intermediate = true;
set hive.exec.compress.output = true;
set mapred.output.compression.codec = org.apache.hadoop.io.compress.GzipCodec;
set mapred.output.compression.type = BLOCK;
from (
    select c
    from t1
    blah blah
) x
insert overwrite table t2
partition (part = 0)
select *
blah blah;

The output of the 1st MR job is not compressed, even though I've set hive.exec.compress.intermediate = true.

Am I missing something? Using Hive 0.5.

Thanks.
Steven

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB