Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Flume to stream logs live

Copy link to this message
Re: Flume to stream logs live
If you can't get flume running on windows your alternatives look like
syslog, scribed and fluentd. We used scribed before moving to flume, but
tbh, it can be a pain to compile and the hdfs support on it is not the
best, we had data loss if there was an hdfs restart etc. I can't say
much about the other two

On 12/14/2012 11:47 PM, Kartashov, Andy wrote:
> Flummers,
> Loved working with Flume 1.2 -- very easy and simple configuration, it
> was a pleasure to work with. Managed to "tail --F" logs from unix
> server and into a hdfs cluster. The problem started when I also needed
> to push logs from a Windows application server.  Spent three days
> researching on how to install flume on Windows and run  a deamon/agent
> that will push the logs to the Avro source I successfully configured
> and ran on Unix. No luck. So I am looking t alternative. Is there
> other framework available out there to help me with my issue. What
> about scribe?
> Andy Kartashov
> *MPAC*
> IT Architecture, Co-op
> 1340 Pickering Parkway, Pickering, L1V 0C4
> (Phone : (905) 837 6269
> (Mobile: (416) 722 1787
> NOTICE: This e-mail message and any attachments are confidential,
> subject to copyright and may be privileged. Any unauthorized use,
> copying or disclosure is prohibited. If you are not the intended
> recipient, please delete and contact the sender immediately. Please
> consider the environment before printing this e-mail. AVIS : le
> pr�sent courriel et toute pi�ce jointe qui l'accompagne sont
> confidentiels, prot�g�s par le droit d'auteur et peuvent �tre couverts
> par le secret professionnel. Toute utilisation, copie ou divulgation
> non autoris�e est interdite. Si vous n'�tes pas le destinataire pr�vu
> de ce courriel, supprimez-le et contactez imm�diatement l'exp�diteur.
> Veuillez penser � l'environnement avant d'imprimer le pr�sent courriel