Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Roll based on date


+
Martinus m 2013-10-17, 01:56
+
David Sinclair 2013-10-17, 13:17
Copy link to this message
-
Re: Roll based on date
Hi David,

It's syslogtcp.

Thanks.

Martinus
On Thu, Oct 17, 2013 at 9:17 PM, David Sinclair <
[EMAIL PROTECTED]> wrote:

> What type of source are you using?
>
>
> On Wed, Oct 16, 2013 at 9:56 PM, Martinus m <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> Is there any option in HDFS sink that I can start rolling a new file
>> whenever the date in the log change? For example, I got below logs :
>>
>> Oct 16 23:58:56 test-host : just test
>> Oct 16 23:59:51 test-host : test again
>> Oct 17 00:00:56 test-host : just test
>> Oct 17 00:00:56 test-host : test again
>>
>> Then I want it to make a file on S3 bucket with result like this :
>>
>> FlumeData.2013-10-16.1381916293017 <-- all the logs with Oct 16 from this
>> year 2013 will goes to here and when it's reach Oct 17 year 2013, then it
>> will start to sink into a new file below :
>>
>> FlumeData.2013-10-17.1381940047117
>>
>> Thanks.
>>
>
>
+
David Sinclair 2013-10-18, 14:35
+
Martinus m 2013-10-22, 06:15
+
David Sinclair 2013-10-22, 14:07
+
DSuiter RDX 2013-10-22, 15:16
+
Martinus m 2013-10-23, 02:17
+
David Sinclair 2013-10-23, 12:48
+
Martinus m 2013-10-24, 06:21
+
David Sinclair 2013-10-24, 13:49
+
Martinus m 2013-10-25, 02:09
+
David Sinclair 2013-10-25, 14:20
+
Martinus m 2013-10-28, 10:54
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB