Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - Multi-line Events


Copy link to this message
-
Re: Multi-line Events
Cameron Gandevia 2012-10-14, 23:53
Thanks for the reply, my plan was to write a simple python script to handle
this if there wasn't already a mechanism.

I am also going to look at modifying FLUME-1425 to support pluggable
readers but couldn't get the patch to apply to trunk. I will take a look
again later.
On Oct 13, 2012 7:27 AM, "Brock Noland" <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I don't think there is anyway handle multi-line events with exec and
> tail -F source. One could be written in python/perl/etc fairly easily
> but would have to be written.
>
> Brock
>
> On Fri, Oct 12, 2012 at 9:44 PM, Cameron Gandevia <[EMAIL PROTECTED]>
> wrote:
> > The plan is to eventually move to using the log4j appender but for now we
> > don't want to introduce any dependencies in our application.
> >
> > On Oct 12, 2012 6:30 PM, "Brock Noland" <[EMAIL PROTECTED]> wrote:
> >>
> >> Hi,
> >>
> >> As opposed to using tail, why not use the API or log4j to send events
> >> directly to a flume agent?
> >>
> >> Brock
> >>
> >> On Friday, October 12, 2012 at 8:21 PM, Cameron Gandevia wrote:
> >>
> >> Hi
> >>
> >> We are currently using the original flume and are considering switching
> to
> >> flume ng using the exec "tail -F" source and eventually the spool
> directory
> >> source
> >>
> https://issues.apache.org/jira/browse/FLUME-1425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> .
> >> My only question is I don't see how to handle multi-line log entries
> such
> >> as stack traces using either of these methods. Is this possible?
> >>
> >> Thanks
> >>
> >>
> >
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce -
> http://incubator.apache.org/mrunit/
>