Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - Roll based on date


+
Martinus m 2013-10-17, 01:56
+
David Sinclair 2013-10-17, 13:17
Copy link to this message
-
Re: Roll based on date
Martinus m 2013-10-18, 02:23
Hi David,

It's syslogtcp.

Thanks.

Martinus
On Thu, Oct 17, 2013 at 9:17 PM, David Sinclair <
[EMAIL PROTECTED]> wrote:

> What type of source are you using?
>
>
> On Wed, Oct 16, 2013 at 9:56 PM, Martinus m <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> Is there any option in HDFS sink that I can start rolling a new file
>> whenever the date in the log change? For example, I got below logs :
>>
>> Oct 16 23:58:56 test-host : just test
>> Oct 16 23:59:51 test-host : test again
>> Oct 17 00:00:56 test-host : just test
>> Oct 17 00:00:56 test-host : test again
>>
>> Then I want it to make a file on S3 bucket with result like this :
>>
>> FlumeData.2013-10-16.1381916293017 <-- all the logs with Oct 16 from this
>> year 2013 will goes to here and when it's reach Oct 17 year 2013, then it
>> will start to sink into a new file below :
>>
>> FlumeData.2013-10-17.1381940047117
>>
>> Thanks.
>>
>
>
+
David Sinclair 2013-10-18, 14:35
+
Martinus m 2013-10-22, 06:15
+
David Sinclair 2013-10-22, 14:07
+
DSuiter RDX 2013-10-22, 15:16
+
Martinus m 2013-10-23, 02:17
+
David Sinclair 2013-10-23, 12:48
+
Martinus m 2013-10-24, 06:21
+
David Sinclair 2013-10-24, 13:49
+
Martinus m 2013-10-25, 02:09
+
David Sinclair 2013-10-25, 14:20
+
Martinus m 2013-10-28, 10:54