Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa >> mail # dev >> About CHUKWA-581,Demux do not support user custom reducer package name.


+
唐 颖 2013-07-08, 13:54
+
Eric Yang 2013-07-09, 05:22
+
IvyTang 2013-07-09, 05:42
Copy link to this message
-
Re: About CHUKWA-581,Demux do not support user custom reducer package name.
Yes that will help.  Please include test case as well.  Thanks Ivy.

regards,
Eric
On Mon, Jul 8, 2013 at 10:42 PM, IvyTang <[EMAIL PROTECTED]> wrote:

> This issue has been reported
> https://issues.apache.org/jira/browse/CHUKWA-581.
>
> And i think i can submit a patch later .
>
> Should i open a new jira  or just assign this CHUKWA-581 to myself?
>
> This can implement reducer outside chukwa.
>
> --
> Best regards,
>
> Ivy Tang
>
> On 2013-7-9, at 下午1:22, Eric Yang <[EMAIL PROTECTED]> wrote:
>
> > The recommended change can improve usability to implement parsers outside
> > of Chukwa packages.  +1 on the change.  Do you mind open a jira for this?
> > Thanks
> >
> > regards,
> > Eric
> >
> >
> > On Mon, Jul 8, 2013 at 6:54 AM, 唐 颖 <[EMAIL PROTECTED]> wrote:
> >
> >> We can write our own mapper ,because MapProcessorFactory uses
> >> Class.forName(parserClass).getConstructor().newInstance();
> >> since we config the parserClass in config file.
> >>
> >> But the ReduceProcessorFactory inits  the reducer class using the
> >> classname
> "org.apache.hadoop.chukwa.extraction.demux.processor.reducer."+
> >> reduceType.
> >> So the reducer class must be put in package
> >> org.apache.hadoop.chukwa.extraction.demux.processor.reducer and named
> >> reduceType.
> >>
> >> Why not we put the reducer  class name in config file either?
> >> For exam, the {customdatatype_mapper} means mapper class
> >> ,{customdatatype_reducer} means reducer class.
> >>
> >> This can give chukka more flexibility.
> >>
> >> --
> >> Best regards,
> >>
> >> Ivy Tang
> >>
> >>
>
>
+
IvyTang 2013-07-12, 07:04
+
IvyTang 2013-07-12, 09:01
+
Eric Yang 2013-07-13, 20:35
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB