Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Unable to disable compression of output


Copy link to this message
-
Unable to disable compression of output
Hello, I'm running a pretty simple pig job but despite my best efforts to
disable compression, the output parts are written in in .lzo_deflate form
like

"/user/jkebinger/pigresults3.gz/part-r-00000.lzo_deflate"

I'm launching the job like this JAVA_OPTS=-Dmapred.output.compress=false
PIG_OPTS=-Dmapred.output.compress=false ../pig-0.10.0/bin/pig test.pig

in a vain effort to turn that off.

Can someone please point me in the right direction?
The store statement looks like this:

STORE bothcountedp into '/user/jkebinger/pigresults3.gz' USING
PigStorage();

(the filename there has varied from no extension to .gz with no effect on
how the files inside it are treated)

thanks!

-James
+
James Kebinger 2012-07-30, 19:24
+
souri datta 2012-07-30, 18:52
+
James Kebinger 2012-07-30, 19:13
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB