Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - "Standard" HTTP logging stack? (0.9.3 => 1.2 Upgrade)


+
Robert Slifka 2012-11-29, 00:33
+
Brock Noland 2012-11-29, 17:19
Copy link to this message
-
Re: "Standard" HTTP logging stack? (0.9.3 => 1.2 Upgrade)
Robert Slifka 2012-11-29, 17:52
Hi Brock,

Thanks for the heads up, wasn't aware that was incoming.

If you don't mind me asking, what's the current recommended approach with
1.2?

Rob

On Thu, Nov 29, 2012 at 9:19 AM, Brock Noland <[EMAIL PROTECTED]> wrote:

> I would use the Spool Directory source in 1.3 to upload the logs files
> once apache rolls the log.
>
>
> On Wed, Nov 28, 2012 at 6:33 PM, Robert Slifka <[EMAIL PROTECTED]>wrote:
>
>> Hi all,
>>
>> Simple HTTP access.log aggregation case here.  Front ends running Apache
>> needing to aggregate/sink all access.log data to S3.
>>
>> Right now we're on OG 0.9.3:
>>
>> nginx => access.log => tailSource => Collector => S3
>>
>>
>> For 1.2, I was under the impression we would simplify and move to:
>>
>> Apache => Syslog => Agent…Agent => S3.
>>
>>
>> However it looks like Apache can only Syslog the error.log.  Am I missing
>> something?
>>
>> What's the recommended 'standard' setup?
>>
>> Thanks guys,
>>
>> Rob
>>
>>
>
>
> --
> Apache MRUnit - Unit testing MapReduce -
> http://incubator.apache.org/mrunit/
>
+
Brock Noland 2012-11-29, 20:24