Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Unable to disable compression of output


Copy link to this message
-
Unable to disable compression of output
Hello, I'm running a pretty simple pig job but despite my best efforts to
disable compression, the output parts are written in in .lzo_deflate form
like

"/user/jkebinger/pigresults3.gz/part-r-00000.lzo_deflate"

I'm launching the job like this JAVA_OPTS=-Dmapred.output.compress=false
PIG_OPTS=-Dmapred.output.compress=false ../pig-0.10.0/bin/pig test.pig

in a vain effort to turn that off.

Can someone please point me in the right direction?
The store statement looks like this:

STORE bothcountedp into '/user/jkebinger/pigresults3.gz' USING
PigStorage();

(the filename there has varied from no extension to .gz with no effect on
how the files inside it are treated)

thanks!

-James
+
James Kebinger 2012-07-30, 19:24
+
souri datta 2012-07-30, 18:52
+
James Kebinger 2012-07-30, 19:13