Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa, mail # dev - About CHUKWA-581,Demux do not support user custom reducer package name.


Copy link to this message
-
About CHUKWA-581,Demux do not support user custom reducer package name.
唐 颖 2013-07-08, 13:54
We can write our own mapper ,because MapProcessorFactory uses
Class.forName(parserClass).getConstructor().newInstance();
since we config the parserClass in config file.

But the ReduceProcessorFactory inits  the reducer class using the classname "org.apache.hadoop.chukwa.extraction.demux.processor.reducer."+ reduceType.
So the reducer class must be put in package org.apache.hadoop.chukwa.extraction.demux.processor.reducer and named reduceType.

Why not we put the reducer  class name in config file either?
For exam, the {customdatatype_mapper} means mapper class ,{customdatatype_reducer} means reducer class.

This can give chukka more flexibility.

--
Best regards,

Ivy Tang

+
Eric Yang 2013-07-09, 05:22
+
IvyTang 2013-07-09, 05:42
+
Eric Yang 2013-07-10, 04:01
+
IvyTang 2013-07-12, 07:04
+
IvyTang 2013-07-12, 09:01
+
Eric Yang 2013-07-13, 20:35