Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Configuring failover sync in Flume-ng


+
Abhijeet Pathak 2012-12-19, 08:23
+
Alexander Lorenz 2012-12-19, 08:29
+
Abhijeet Pathak 2012-12-19, 09:01
+
Alexander Alten-Lorenz 2012-12-19, 09:21
+
Abhijeet Pathak 2012-12-19, 10:51
+
Alexander Alten-Lorenz 2012-12-19, 11:03
Copy link to this message
-
RE: Configuring failover sync in Flume-ng
Oh okay.

I'll upgrade and try.

Thanks! :)

-----Original Message-----
From: Alexander Alten-Lorenz [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, December 19, 2012 4:33 PM
To: [EMAIL PROTECTED]
Subject: Re: Configuring failover sync in Flume-ng

Oh, yes. You should upgrade into 1.3.0, which was released last week. Wasn't clear for me that you're running on 1.1.0

Do you have a change to upgrade into the newest release?

Best,
 Alex

On Dec 19, 2012, at 11:51 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:

> It's still not working.
>
> I read somewhere that failover support was implemented after Jan 2012.
> I've flume 1.1.0+121-1.cdh4.0.1.p0.1~precise-cdh4.0.1 installed.
>
> Can that be a reason for it not working?
>
> Thanks,
> Abhijeet
>
> -----Original Message-----
> From: Alexander Alten-Lorenz [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, December 19, 2012 2:52 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Configuring failover sync in Flume-ng
>
> YOUR_NAME.sinks = HOSTSINK-1 HOSTSINK-2
>
> HOSTSINK-1.type> HOSTSINK-1.bind=HOSTNAME-1
> HOSTSINK-1.port=needed by JDBC?
> HOSTSINK-1.channels=YOUR_CHANNELS
>
> the same for the other HOSTSINK and deploy the config to the hosts and startup. Should work.
>
> Best,
> Alex
>
> On Dec 19, 2012, at 10:01 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>
>> Yes, I saw that.
>>
>> But I'm not sure how to tell flume that the failover sink is running on "another" machine.
>>
>> -----Original Message-----
>> From: Alexander Lorenz [mailto:[EMAIL PROTECTED]]
>> Sent: Wednesday, December 19, 2012 1:59 PM
>> To: [EMAIL PROTECTED]
>> Subject: Re: Configuring failover sync in Flume-ng
>>
>> Hi,
>>
>> Failover:
>> http://flume.apache.org/FlumeUserGuide.html#failover-sink-processor
>>
>> Just define more as one JDBC channel and add them to the failover config (see the examples).
>>
>> Best
>> - Alex
>>
>> --
>> Alexander Alten-Lorenz
>> http://mapredit.blogspot.com
>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>
>> On Dec 19, 2012, at 9:23 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>>
>>> Hi,
>>>
>>> I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc], 1 sink [AsyncHbase]) on one server.
>>>
>>> However, I am not able to find documentation on how to configure failover sinks for such scenario.
>>>
>>> I want to run same sink as backup/failover sink on different server.
>>> How can that be configured?
>>>
>>> Thanks,
>>> Abhijeet
>>>
>>
>
+
Abhijeet Pathak 2012-12-19, 13:55
+
Alexander Alten-Lorenz 2012-12-19, 14:05
+
Brock Noland 2012-12-19, 14:40
+
Abhijeet Pathak 2012-12-20, 07:48
+
Nitin Pawar 2012-12-19, 11:19
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB