Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # dev >> ChukwaStorer


Copy link to this message
-
Re: ChukwaStorer
I would think adding another constructor that takes these options in
the signature would be the best approach.
On Wed, Nov 10, 2010 at 6:31 PM, Corbin Hoenes <[EMAIL PROTECTED]> wrote:
> I'm using the chukwa-pig7 contrib in my pig scripts and noticed it trys to compress the chukwa output when I do a store.
>
>
> @Override
>  public void setStoreLocation(String location, Job job) throws IOException {
>    FileOutputFormat.setOutputPath(job, new Path(location));
>    FileOutputFormat.setCompressOutput(job, true);
>    FileOutputFormat.setOutputCompressorClass(job, DefaultCodec.class);
>    job.setOutputKeyClass(ChukwaRecordKey.class);
>    job.setOutputValueClass(ChukwaRecord.class);
>  }
>
> What is the best way to make this configurable? I was hoping I could do something like this:
>
> @Override
>  public void setStoreLocation(String location, Job job) throws IOException {
>    FileOutputFormat.setOutputPath(job, new Path(location));
>    FileOutputFormat.setCompressOutput(job, this.compress);
>    if(this.compress)
>      FileOutputFormat.setOutputCompressorClass(job, DefaultCodec.class);
>    job.setOutputKeyClass(ChukwaRecordKey.class);
>    job.setOutputValueClass(ChukwaRecord.class);
>  }
>
> How to make the codec class pluggable/configurable as well--for example if one wanted to use LZO?
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB