Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> How to configure Regex in file names flume 1.3


Copy link to this message
-
Re: How to configure Regex in file names flume 1.3
Deepak,
   which version of flume are u using ? regex support is in 1.4, but you
will need to use the 'shell' configuration in Exec Source for that.. see
the User Guide for an example

-roshan
On Tue, Jul 23, 2013 at 10:50 AM, Kumar, Deepak8 <[EMAIL PROTECTED]>wrote:

>  Could anyone assist on this?****
>
> ** **
>
> Could we have a regex in tail –F****
>
> ** **
>
> Regards,****
>
> Deepak****
>
> ** **
>
> *From:* Kumar, Deepak8 [CCC-OT_IT NE]
> *Sent:* Sunday, July 21, 2013 12:29 PM
> *To:* Kumar, Deepak8 [CCC-OT_IT NE]; '[EMAIL PROTECTED]'; '
> [EMAIL PROTECTED]'
>
> *Subject:* RE: How to configure Regex in file names flume 1.3****
>
>  ** **
>
> A simple tail works if I give the exact file name. But if I give a regex
> as file name like****
>
> agent.sources.source1.command = tail -F /var/log/flume-ng/template.*****
>
> ** **
>
> then it is not working.****
>
> ** **
>
> Regards,****
>
> Deepak****
>
> ** **
>
> *From:* Kumar, Deepak8 [CCC-OT_IT NE]
> *Sent:* Sunday, July 21, 2013 12:27 PM
> *To:* '[EMAIL PROTECTED]'; '[EMAIL PROTECTED]'; Kumar, Deepak8
> [CCC-OT_IT NE]
> *Subject:* RE: How to configure Regex in file names flume 1.3****
>
> ** **
>
> Hi,****
>
> I have checked it but it is not working. I verified from JConsole but it
> is not sending the log events.****
>
> ** **
>
> Do we have any other way to cross check?****
>
> ** **
>
> Regards,****
>
> Deepak****
>
> ** **
>
> *From:* Israel Ekpo [mailto:[EMAIL PROTECTED] <[EMAIL PROTECTED]>]
> *Sent:* Saturday, July 20, 2013 4:46 PM
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: How to configure Regex in file names flume 1.3****
>
> ** **
>
> Deepak,****
>
> ** **
>
> The sample configuration you have looks valid****
>
> ** **
>
> Try it out and let us know if you are experiencing any problems****
>
> ** **
>
> More information on how to configure the Exec Source is available here.***
> *
>
> ** **
>
> http://flume.apache.org/FlumeUserGuide.html#exec-source****
>
> ** **
>
>
> ****
>
> *Author and Instructor for the Upcoming Book and Lecture Series*****
>
> *Massive Log Data Aggregation, Processing, Searching and Visualization
> with Open Source Software*****
>
> *http://massivelogdata.com*****
>
> ** **
>
> On 19 July 2013 14:00, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:****
>
> Hi,****
>
> Could anyone please assist on this?****
>
>  ****
>
> Regards,****
>
> Deepak****
>
>  ****
>
> *From:* Kumar, Deepak8 [CCC-OT_IT NE]
> *Sent:* Monday, July 15, 2013 8:00 AM
> *To:* '[EMAIL PROTECTED]'
> *Cc:* Kumar, Deepak8 [CCC-OT_IT NE]
> *Subject:* How to configure Regex in file names flume 1.3****
>
>  ****
>
> Hi,****
>
> I am using flume 1.3. Could we configure files names as regular
> expressions in flume-conf.properties.****
>
>  ****
>
> e.g.****
>
> agent.sources.source1.type = exec****
>
> agent.sources.source1.command = tail -F /var/log/flume-ng/template.log****
>
> agent.sources.source1.channels = fileChannel****
>
> agent.sources.source1.interceptors = logIntercept****
>
>  ****
>
> Could we have it something like this in flume 1.3****
>
> agent.sources.source1.command = tail -F /var/log/flume-ng/template.*****
>
>  ****
>
> Regards,****
>
> Deepak****
>
> ** **
>