Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa, mail # dev - About CHUKWA-581,Demux do not support user custom reducer package name.


Copy link to this message
-
Re: About CHUKWA-581,Demux do not support user custom reducer package name.
Eric Yang 2013-07-09, 05:22
The recommended change can improve usability to implement parsers outside
of Chukwa packages.  +1 on the change.  Do you mind open a jira for this?
 Thanks

regards,
Eric
On Mon, Jul 8, 2013 at 6:54 AM, 唐 颖 <[EMAIL PROTECTED]> wrote:

> We can write our own mapper ,because MapProcessorFactory uses
> Class.forName(parserClass).getConstructor().newInstance();
> since we config the parserClass in config file.
>
> But the ReduceProcessorFactory inits  the reducer class using the
> classname "org.apache.hadoop.chukwa.extraction.demux.processor.reducer."+
> reduceType.
> So the reducer class must be put in package
> org.apache.hadoop.chukwa.extraction.demux.processor.reducer and named
> reduceType.
>
> Why not we put the reducer  class name in config file either?
> For exam, the {customdatatype_mapper} means mapper class
> ,{customdatatype_reducer} means reducer class.
>
> This can give chukka more flexibility.
>
> --
> Best regards,
>
> Ivy Tang
>
>