Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - Flume to stream logs live

Copy link to this message
Re: Flume to stream logs live
Juhani Connolly 2012-12-17, 01:33
If you can't get flume running on windows your alternatives look like
syslog, scribed and fluentd. We used scribed before moving to flume, but
tbh, it can be a pain to compile and the hdfs support on it is not the
best, we had data loss if there was an hdfs restart etc. I can't say
much about the other two

On 12/14/2012 11:47 PM, Kartashov, Andy wrote:
> Flummers,
> Loved working with Flume 1.2 -- very easy and simple configuration, it
> was a pleasure to work with. Managed to "tail --F" logs from unix
> server and into a hdfs cluster. The problem started when I also needed
> to push logs from a Windows application server.  Spent three days
> researching on how to install flume on Windows and run  a deamon/agent
> that will push the logs to the Avro source I successfully configured
> and ran on Unix. No luck. So I am looking t alternative. Is there
> other framework available out there to help me with my issue. What
> about scribe?
> Andy Kartashov
> *MPAC*
> IT Architecture, Co-op
> 1340 Pickering Parkway, Pickering, L1V 0C4
> (Phone : (905) 837 6269
> (Mobile: (416) 722 1787
> NOTICE: This e-mail message and any attachments are confidential,
> subject to copyright and may be privileged. Any unauthorized use,
> copying or disclosure is prohibited. If you are not the intended
> recipient, please delete and contact the sender immediately. Please
> consider the environment before printing this e-mail. AVIS : le
> pr�sent courriel et toute pi�ce jointe qui l'accompagne sont
> confidentiels, prot�g�s par le droit d'auteur et peuvent �tre couverts
> par le secret professionnel. Toute utilisation, copie ou divulgation
> non autoris�e est interdite. Si vous n'�tes pas le destinataire pr�vu
> de ce courriel, supprimez-le et contactez imm�diatement l'exp�diteur.
> Veuillez penser � l'environnement avant d'imprimer le pr�sent courriel