Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - Source and Sink on different machine


Copy link to this message
-
Re: Source and Sink on different machine
Nitin Pawar 2012-12-28, 07:58
yes thats doable

what you are trying to do is agnet1 ->agenet2|agent3 -> hbase

second portion of  the flume part you can have it on same host where you
have logs being generated.
you can look at flume failover configuration at
http://flume.apache.org/FlumeUserGuide.html#flume-sink-processors

also your question on stackoverflow is answered by Dmitry in Detail
On Fri, Dec 28, 2012 at 1:23 PM, Abhijeet Pathak <
[EMAIL PROTECTED]> wrote:

>  I want to setup a failover configuration.****
>
> ** **
>
> On Machine1 I’ve a source.****
>
> On Machine2 & Machine3, I have (hbase) sinks.****
>
> I want to configure them in failover mode. That is, by default events
> should go to sink on Machine2’s sink. But if Machine2 fails then they
> should automatically go to Machine3’s sink.****
>
> ** **
>
> Is that possible in Flume-NG? How?****
>
> ** **
>
> Thanks,****
>
> Abhijeet****
>
> ** **
>
> *From:* Nitin Pawar [mailto:[EMAIL PROTECTED]]
> *Sent:* Friday, December 28, 2012 1:18 PM
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: Source and Sink on different machine****
>
> ** **
>
> if I understood it correctly, you want to put your log data into hbase
> which is hosted remotely? ****
>
> ** **
>
> yes thats easily doable. ****
>
> ** **
>
> Flume-NG does everything alone instead of older master-slave architecture.
> It has currently source channel and sink. you can collect logs on a machine
> via an agent and using same agent you can sink them into hbase  on a remote
> host ****
>
> ** **
>
> On Fri, Dec 28, 2012 at 12:06 PM, Abhijeet Pathak <
> [EMAIL PROTECTED]> wrote:****
>
> Hi,****
>
>  ****
>
> It was possible to have source and sinks on different machines using Flume
> 0.9x.****
>
>  ****
>
> Is it possible to have source and sinks on different machines in Flume-NG
> (1.3.0)?****
>
>  ****
>
> I want to use tail source on one machine and hbase sink on another machine.
> ****
>
> How should I go about it?****
>
>  ****
>
> Thanks,****
>
> Abhijeet****
>
>
> This message contains information that may be privileged or confidential
> and is the property of the KPIT Cummins Infosystems Ltd. It is intended
> only for the person to whom it is addressed. If you are not the intended
> recipient, you are not authorized to read, print, retain copy, disseminate,
> distribute, or use this message or any part thereof. If you receive this
> message in error, please notify the sender immediately and delete all
> copies of this message. KPIT Cummins Infosystems Ltd. does not accept any
> liability for virus infected mails. ****
>
>
>
> ****
>
> ** **
>
> --
> Nitin Pawar****
>
> This message contains information that may be privileged or confidential
> and is the property of the KPIT Cummins Infosystems Ltd. It is intended
> only for the person to whom it is addressed. If you are not the intended
> recipient, you are not authorized to read, print, retain copy, disseminate,
> distribute, or use this message or any part thereof. If you receive this
> message in error, please notify the sender immediately and delete all
> copies of this message. KPIT Cummins Infosystems Ltd. does not accept any
> liability for virus infected mails.
>

--
Nitin Pawar