-Re: Flume custom decorator for Rolling FileSink output bucketing
Thanks Mike for the suggestion. The reason I am thinking of usual file
system for log storage is to avoid latency issues for file retrieval as
well as to allow users to scrape log files using grep / awk and multitude
of other powerful commands available in conventional storage.
I am now thinking of coming up with my own decorator classes for
RollingFile sink. Any pointers on how I can get started on writing my
Another quick question: Can you, Mike or somebody from flume community tell
me how to use the commands documented here at:
Is this available for flume-ng distributed with Cloudera solution i.e.
Best and thanks a lot again,
On Fri, Mar 15, 2013 at 12:38 PM, Mike Percy <[EMAIL PROTECTED]> wrote:
> Dib, you could use the HDFS sink with a file:// URL as an option.
> On Fri, Mar 15, 2013 at 12:16 PM, Dibyajyoti Ghosh <
> [EMAIL PROTECTED]> wrote:
> > Dear flume team,
> > I am using flume 1.3.0 bundled with Cloudera 4.2.0 distribution for log
> > local file system. But current implementation of FileSink doesn't have
> > inline decorators like in HDFS Sink where output can be stored to
> > directories based on event meta data e.g. hostname of the event or
> > timestamp or some other attribute in the message object.
> > How can I do the same for FileSink?
> > Thanks a lot,
> > - dib