Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> "Standard" HTTP logging stack? (0.9.3 => 1.2 Upgrade)


+
Robert Slifka 2012-11-29, 00:33
+
Brock Noland 2012-11-29, 17:19
+
Robert Slifka 2012-11-29, 17:52
Copy link to this message
-
Re: "Standard" HTTP logging stack? (0.9.3 => 1.2 Upgrade)
It's possible you could use the exec source and tail -F but personally I
would use either the Flume 1.3 RC (until it's released).

On Thu, Nov 29, 2012 at 11:52 AM, Robert Slifka <[EMAIL PROTECTED]>wrote:

> Hi Brock,
>
> Thanks for the heads up, wasn't aware that was incoming.
>
> If you don't mind me asking, what's the current recommended approach with
> 1.2?
>
> Rob
>
>
> On Thu, Nov 29, 2012 at 9:19 AM, Brock Noland <[EMAIL PROTECTED]> wrote:
>
>> I would use the Spool Directory source in 1.3 to upload the logs files
>> once apache rolls the log.
>>
>>
>> On Wed, Nov 28, 2012 at 6:33 PM, Robert Slifka <[EMAIL PROTECTED]>wrote:
>>
>>> Hi all,
>>>
>>> Simple HTTP access.log aggregation case here.  Front ends running Apache
>>> needing to aggregate/sink all access.log data to S3.
>>>
>>> Right now we're on OG 0.9.3:
>>>
>>> nginx => access.log => tailSource => Collector => S3
>>>
>>>
>>> For 1.2, I was under the impression we would simplify and move to:
>>>
>>> Apache => Syslog => Agent…Agent => S3.
>>>
>>>
>>> However it looks like Apache can only Syslog the error.log.  Am I
>>> missing something?
>>>
>>> What's the recommended 'standard' setup?
>>>
>>> Thanks guys,
>>>
>>> Rob
>>>
>>>
>>
>>
>> --
>> Apache MRUnit - Unit testing MapReduce -
>> http://incubator.apache.org/mrunit/
>>
>
>
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB