Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa, mail # dev - About CHUKWA-581,Demux do not support user custom reducer package name.


Copy link to this message
-
Re: About CHUKWA-581,Demux do not support user custom reducer package name.
IvyTang 2013-07-09, 05:42
This issue has been reported  https://issues.apache.org/jira/browse/CHUKWA-581.

And i think i can submit a patch later .

Should i open a new jira  or just assign this CHUKWA-581 to myself?

This can implement reducer outside chukwa.

--
Best regards,

Ivy Tang

On 2013-7-9, at 下午1:22, Eric Yang <[EMAIL PROTECTED]> wrote:

> The recommended change can improve usability to implement parsers outside
> of Chukwa packages.  +1 on the change.  Do you mind open a jira for this?
> Thanks
>
> regards,
> Eric
>
>
> On Mon, Jul 8, 2013 at 6:54 AM, 唐 颖 <[EMAIL PROTECTED]> wrote:
>
>> We can write our own mapper ,because MapProcessorFactory uses
>> Class.forName(parserClass).getConstructor().newInstance();
>> since we config the parserClass in config file.
>>
>> But the ReduceProcessorFactory inits  the reducer class using the
>> classname "org.apache.hadoop.chukwa.extraction.demux.processor.reducer."+
>> reduceType.
>> So the reducer class must be put in package
>> org.apache.hadoop.chukwa.extraction.demux.processor.reducer and named
>> reduceType.
>>
>> Why not we put the reducer  class name in config file either?
>> For exam, the {customdatatype_mapper} means mapper class
>> ,{customdatatype_reducer} means reducer class.
>>
>> This can give chukka more flexibility.
>>
>> --
>> Best regards,
>>
>> Ivy Tang
>>
>>