Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # dev >> About CHUKWA-581,Demux do not support user custom reducer package name.


Copy link to this message
-
Re: About CHUKWA-581,Demux do not support user custom reducer package name.
The recommended change can improve usability to implement parsers outside
of Chukwa packages.  +1 on the change.  Do you mind open a jira for this?
 Thanks

regards,
Eric
On Mon, Jul 8, 2013 at 6:54 AM, 唐 颖 <[EMAIL PROTECTED]> wrote:

> We can write our own mapper ,because MapProcessorFactory uses
> Class.forName(parserClass).getConstructor().newInstance();
> since we config the parserClass in config file.
>
> But the ReduceProcessorFactory inits  the reducer class using the
> classname "org.apache.hadoop.chukwa.extraction.demux.processor.reducer."+
> reduceType.
> So the reducer class must be put in package
> org.apache.hadoop.chukwa.extraction.demux.processor.reducer and named
> reduceType.
>
> Why not we put the reducer  class name in config file either?
> For exam, the {customdatatype_mapper} means mapper class
> ,{customdatatype_reducer} means reducer class.
>
> This can give chukka more flexibility.
>
> --
> Best regards,
>
> Ivy Tang
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB