Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa >> mail # dev >> About CHUKWA-581,Demux do not support user custom reducer package name.


Copy link to this message
-
About CHUKWA-581,Demux do not support user custom reducer package name.
We can write our own mapper ,because MapProcessorFactory uses
Class.forName(parserClass).getConstructor().newInstance();
since we config the parserClass in config file.

But the ReduceProcessorFactory inits  the reducer class using the classname "org.apache.hadoop.chukwa.extraction.demux.processor.reducer."+ reduceType.
So the reducer class must be put in package org.apache.hadoop.chukwa.extraction.demux.processor.reducer and named reduceType.

Why not we put the reducer  class name in config file either?
For exam, the {customdatatype_mapper} means mapper class ,{customdatatype_reducer} means reducer class.

This can give chukka more flexibility.

--
Best regards,

Ivy Tang

+
Eric Yang 2013-07-09, 05:22
+
IvyTang 2013-07-09, 05:42
+
Eric Yang 2013-07-10, 04:01
+
IvyTang 2013-07-12, 07:04
+
IvyTang 2013-07-12, 09:01
+
Eric Yang 2013-07-13, 20:35
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB