Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> How to configure Regex in file names flume 1.3


+
Kumar, Deepak8 2013-07-15, 11:59
+
Kumar, Deepak8 2013-07-19, 18:00
+
Israel Ekpo 2013-07-20, 20:46
+
Kumar, Deepak8 2013-07-21, 16:26
+
Kumar, Deepak8 2013-07-21, 16:29
+
Kumar, Deepak8 2013-07-23, 17:50
Copy link to this message
-
Re: How to configure Regex in file names flume 1.3
Deepak,
   which version of flume are u using ? regex support is in 1.4, but you
will need to use the 'shell' configuration in Exec Source for that.. see
the User Guide for an example

-roshan
On Tue, Jul 23, 2013 at 10:50 AM, Kumar, Deepak8 <[EMAIL PROTECTED]>wrote:

>  Could anyone assist on this?****
>
> ** **
>
> Could we have a regex in tail –F****
>
> ** **
>
> Regards,****
>
> Deepak****
>
> ** **
>
> *From:* Kumar, Deepak8 [CCC-OT_IT NE]
> *Sent:* Sunday, July 21, 2013 12:29 PM
> *To:* Kumar, Deepak8 [CCC-OT_IT NE]; '[EMAIL PROTECTED]'; '
> [EMAIL PROTECTED]'
>
> *Subject:* RE: How to configure Regex in file names flume 1.3****
>
>  ** **
>
> A simple tail works if I give the exact file name. But if I give a regex
> as file name like****
>
> agent.sources.source1.command = tail -F /var/log/flume-ng/template.*****
>
> ** **
>
> then it is not working.****
>
> ** **
>
> Regards,****
>
> Deepak****
>
> ** **
>
> *From:* Kumar, Deepak8 [CCC-OT_IT NE]
> *Sent:* Sunday, July 21, 2013 12:27 PM
> *To:* '[EMAIL PROTECTED]'; '[EMAIL PROTECTED]'; Kumar, Deepak8
> [CCC-OT_IT NE]
> *Subject:* RE: How to configure Regex in file names flume 1.3****
>
> ** **
>
> Hi,****
>
> I have checked it but it is not working. I verified from JConsole but it
> is not sending the log events.****
>
> ** **
>
> Do we have any other way to cross check?****
>
> ** **
>
> Regards,****
>
> Deepak****
>
> ** **
>
> *From:* Israel Ekpo [mailto:[EMAIL PROTECTED] <[EMAIL PROTECTED]>]
> *Sent:* Saturday, July 20, 2013 4:46 PM
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: How to configure Regex in file names flume 1.3****
>
> ** **
>
> Deepak,****
>
> ** **
>
> The sample configuration you have looks valid****
>
> ** **
>
> Try it out and let us know if you are experiencing any problems****
>
> ** **
>
> More information on how to configure the Exec Source is available here.***
> *
>
> ** **
>
> http://flume.apache.org/FlumeUserGuide.html#exec-source****
>
> ** **
>
>
> ****
>
> *Author and Instructor for the Upcoming Book and Lecture Series*****
>
> *Massive Log Data Aggregation, Processing, Searching and Visualization
> with Open Source Software*****
>
> *http://massivelogdata.com*****
>
> ** **
>
> On 19 July 2013 14:00, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:****
>
> Hi,****
>
> Could anyone please assist on this?****
>
>  ****
>
> Regards,****
>
> Deepak****
>
>  ****
>
> *From:* Kumar, Deepak8 [CCC-OT_IT NE]
> *Sent:* Monday, July 15, 2013 8:00 AM
> *To:* '[EMAIL PROTECTED]'
> *Cc:* Kumar, Deepak8 [CCC-OT_IT NE]
> *Subject:* How to configure Regex in file names flume 1.3****
>
>  ****
>
> Hi,****
>
> I am using flume 1.3. Could we configure files names as regular
> expressions in flume-conf.properties.****
>
>  ****
>
> e.g.****
>
> agent.sources.source1.type = exec****
>
> agent.sources.source1.command = tail -F /var/log/flume-ng/template.log****
>
> agent.sources.source1.channels = fileChannel****
>
> agent.sources.source1.interceptors = logIntercept****
>
>  ****
>
> Could we have it something like this in flume 1.3****
>
> agent.sources.source1.command = tail -F /var/log/flume-ng/template.*****
>
>  ****
>
> Regards,****
>
> Deepak****
>
> ** **
>
+
Kumar, Deepak8 2013-07-24, 06:54
+
Roshan Naik 2013-07-26, 00:36
+
Lenin Raj 2013-07-20, 09:58
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB