Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Equivalent of Decorators in Flume NG


Copy link to this message
-
Re: Equivalent of Decorators in Flume NG
Hi Harish,
Why do you want to do that? Can you describe your use case?

Regards,
Mike

On Tue, Oct 2, 2012 at 1:28 PM, Harish Mandala <[EMAIL PROTECTED]>wrote:

> Hello,
>
> Alright, so maybe interceptors were not exactly what I wanted.
>
> It seems the number of events going into an interceptor must equal the
> number coming out. However, what if I need to take out the data from a
> certain event, and create multiple events from subsets of the data which
> would then be multiplexed using the selector to different locations. Would
> the job of splitting one event into many best be done in a Source or Sink?
>
> I was contemplating modifying the AvroSource or AvroSink for my purposes.
> However, it seems the sink also tallies output event counts and input event
> counts, and makes sure they're the same. That leaves me the option of
> writing a custom source based off the AvroSource. Is my thinking correct?
>
> Thanks,
> Harish
>
> On Mon, Oct 1, 2012 at 6:45 PM, Harish Mandala <[EMAIL PROTECTED]
> >wrote:
>
> > Hi Percy,
> >
> > Thanks! Interceptors seem good enough.
> >
> > Regards,
> > Harish
> >
> >
> > On Mon, Oct 1, 2012 at 6:32 PM, Mike Percy <[EMAIL PROTECTED]> wrote:
> >
> >> Hi Harish,
> >> At this time Flume NG doesn't support unbatching or sink-side plugins.
> >> Interceptors provide source-side tagging, filtering, and transformation
> >> capability, however.
> >>
> >> Regards,
> >> Mike
> >>
> >>
> >> On Mon, Oct 1, 2012 at 3:23 PM, Harish Mandala <[EMAIL PROTECTED]
> >> >wrote:
> >>
> >> > Hello,
> >> >
> >> > Am I right in thinking Flume NG no longer has the concept of Sink
> >> > Decorators? I wanted to do some custom deserialization on incoming
> event
> >> > data, and split one event into several (De-batching and re-routing).
> >> What's
> >> > the best way to implement this in Flume NG?
> >> >
> >> > Thanks,
> >> > Harish
> >> >
> >>
> >
> >
>