Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> about flume-ng agent


Copy link to this message
-
Re: about flume-ng agent
The HTTPSource will be a part of Apache Flume-1.3.0.  

Thanks,
Hari

--
Hari Shreedharan
On Thursday, October 25, 2012 at 4:59 PM, Paul Chavez wrote:

> I was unaware of an HTTPSource, after reviewing the FLUME-1199 issue it may well be the best use case for us.
>
> That said, while I am manually building and testing the latest snapshot for the Windows side, our actual Hadoop machines are running CDH4.1.1 which has flume-ng 1.2.0.
>
> When a version of flume-ng containing HTTPSource is packaged along with the rest of the hadoop distribution I will look at it. As a 'windoze guy' ;-) I do not manage the hadoop systems.
>
> Thank you,
> Paul Chavez
>
> -----Original Message-----
> From: Will McQueen [mailto:[EMAIL PROTECTED]]
>
> Would the new HttpSource work for you?
>
> On Oct 25, 2012, at 2:47 PM, Brock Noland <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
>
> > On Thu, Oct 25, 2012 at 1:16 PM, Paul Chavez
> > <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
> > >
> > > I am no expert either but the various syslog related RFC and RFC-type documentation I can find recommends that messeages be kept small in order to avoid fragmentation.
> >
> > Ahh OK.
> >
> > >
> > > Testing with the syslogTCP source did not show any issues with fragmentation, but the tool we are using to send syslog messages over TCP (LogParser) does not separate messages with a carriage return so messages weren't parsed correctly by the flume source.
> >
> > Yes TCP/IP will deliver the events in ordert.
> >
> > > I would hope so but I am not in a position to make any guarantees on behalf of my employer.
> >
> > No worries, I understand!
> >
> > Brock
> >
> > --
> > Apache MRUnit - Unit testing MapReduce -
> > http://incubator.apache.org/mrunit/
> >
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB