Hello, I'm running a pretty simple pig job but despite my best efforts to
disable compression, the output parts are written in in .lzo_deflate form
like

"/user/jkebinger/pigresults3.gz/part-r-00000.lzo_deflate"

I'm launching the job like this JAVA_OPTS=-Dmapred.output.compress=false
PIG_OPTS=-Dmapred.output.compress=false ../pig-0.10.0/bin/pig test.pig

in a vain effort to turn that off.

Can someone please point me in the right direction?
The store statement looks like this:

STORE bothcountedp into '/user/jkebinger/pigresults3.gz' USING
PigStorage();

(the filename there has varied from no extension to .gz with no effect on
how the files inside it are treated)

thanks!

-James

James,
 I may not have understood your question fully,but did you try renaming the
file w/o the '.gz' ? In hadoop/pig, if you have bz2 extn, files get
compressed.Hence,..

On Mon, Jul 30, 2012 at 11:51 PM, James Kebinger <[EMAIL PROTECTED]>wrote:

> Hello, I'm running a pretty simple pig job but despite my best efforts to
> disable compression, the output parts are written in in .lzo_deflate form
> like
>
> "/user/jkebinger/pigresults3.gz/part-r-00000.lzo_deflate"
>
> I'm launching the job like this JAVA_OPTS=-Dmapred.output.compress=false
> PIG_OPTS=-Dmapred.output.compress=false ../pig-0.10.0/bin/pig test.pig
>
> in a vain effort to turn that off.
>
> Can someone please point me in the right direction?
>
>
> The store statement looks like this:
>
> STORE bothcountedp into '/user/jkebinger/pigresults3.gz' USING
> PigStorage();
>
> (the filename there has varied from no extension to .gz with no effect on
> how the files inside it are treated)
>
> thanks!
>
> -James
>

Yes, whatever I name the output directory, the files inside are compressed
with the lzo codec.

On Mon, Jul 30, 2012 at 2:52 PM, souri datta <[EMAIL PROTECTED]>wrote:

> James,
>  I may not have understood your question fully,but did you try renaming the
> file w/o the '.gz' ? In hadoop/pig, if you have bz2 extn, files get
> compressed.Hence,..
>
> On Mon, Jul 30, 2012 at 11:51 PM, James Kebinger <[EMAIL PROTECTED]
> >wrote:
>
> > Hello, I'm running a pretty simple pig job but despite my best efforts to
> > disable compression, the output parts are written in in .lzo_deflate form
> > like
> >
> > "/user/jkebinger/pigresults3.gz/part-r-00000.lzo_deflate"
> >
> > I'm launching the job like this JAVA_OPTS=-Dmapred.output.compress=false
> > PIG_OPTS=-Dmapred.output.compress=false ../pig-0.10.0/bin/pig test.pig
> >
> > in a vain effort to turn that off.
> >
> > Can someone please point me in the right direction?
> >
> >
> > The store statement looks like this:
> >
> > STORE bothcountedp into '/user/jkebinger/pigresults3.gz' USING
> > PigStorage();
> >
> > (the filename there has varied from no extension to .gz with no effect on
> > how the files inside it are treated)
> >
> > thanks!
> >
> > -James
> >
>

Turns out I needed to set the option inside my pig script, as in:

SET mapred.output.compress false;

On Mon, Jul 30, 2012 at 2:21 PM, James Kebinger <[EMAIL PROTECTED]> wrote:

> Hello, I'm running a pretty simple pig job but despite my best efforts to
> disable compression, the output parts are written in in .lzo_deflate form
> like
>
> "/user/jkebinger/pigresults3.gz/part-r-00000.lzo_deflate"
>
> I'm launching the job like this JAVA_OPTS=-Dmapred.output.compress=false
> PIG_OPTS=-Dmapred.output.compress=false ../pig-0.10.0/bin/pig test.pig
>
> in a vain effort to turn that off.
>
> Can someone please point me in the right direction?
>
>
> The store statement looks like this:
>
> STORE bothcountedp into '/user/jkebinger/pigresults3.gz' USING
> PigStorage();
>
> (the filename there has varied from no extension to .gz with no effect on
> how the files inside it are treated)
>
> thanks!
>
> -James
>