Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Unable to disable compression of output


Copy link to this message
-
Re: Unable to disable compression of output
Yes, whatever I name the output directory, the files inside are compressed
with the lzo codec.

On Mon, Jul 30, 2012 at 2:52 PM, souri datta <[EMAIL PROTECTED]>wrote:

> James,
>  I may not have understood your question fully,but did you try renaming the
> file w/o the '.gz' ? In hadoop/pig, if you have bz2 extn, files get
> compressed.Hence,..
>
> On Mon, Jul 30, 2012 at 11:51 PM, James Kebinger <[EMAIL PROTECTED]
> >wrote:
>
> > Hello, I'm running a pretty simple pig job but despite my best efforts to
> > disable compression, the output parts are written in in .lzo_deflate form
> > like
> >
> > "/user/jkebinger/pigresults3.gz/part-r-00000.lzo_deflate"
> >
> > I'm launching the job like this JAVA_OPTS=-Dmapred.output.compress=false
> > PIG_OPTS=-Dmapred.output.compress=false ../pig-0.10.0/bin/pig test.pig
> >
> > in a vain effort to turn that off.
> >
> > Can someone please point me in the right direction?
> >
> >
> > The store statement looks like this:
> >
> > STORE bothcountedp into '/user/jkebinger/pigresults3.gz' USING
> > PigStorage();
> >
> > (the filename there has varied from no extension to .gz with no effect on
> > how the files inside it are treated)
> >
> > thanks!
> >
> > -James
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB