Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Configuring failover sync in Flume-ng


Copy link to this message
-
RE: Configuring failover sync in Flume-ng
Okay.

-----Original Message-----
From: Brock Noland [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, December 19, 2012 8:10 PM
To: [EMAIL PROTECTED]
Subject: Re: Configuring failover sync in Flume-ng

Just as a side note: File Channel has received significantly more testing than JDBC Channel.

On Wed, Dec 19, 2012 at 8:05 AM, Alexander Alten-Lorenz <[EMAIL PROTECTED]> wrote:
> Yes, why not luring the wiki :)
> https://cwiki.apache.org/confluence/display/FLUME/Home
>
> Stable: http://www.apache.org/dyn/closer.cgi/flume/1.3.0
> Just grab the tar.gz - or use 1.2.0 as deb-package, I suspect the balancing stuff is inside too.
>
> Best,
>  Alex
>
> On Dec 19, 2012, at 2:55 PM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>
>> I am running Ubuntu 12.04 server edition.
>> The repo has flume 1.2 release available.
>> How should I install 1.3? Using source?
>> Is there a stable release available?
>>
>> Thanks,
>> Abhijeet
>>
>> -----Original Message-----
>> From: Alexander Alten-Lorenz [mailto:[EMAIL PROTECTED]]
>> Sent: Wednesday, December 19, 2012 4:33 PM
>> To: [EMAIL PROTECTED]
>> Subject: Re: Configuring failover sync in Flume-ng
>>
>> Oh, yes. You should upgrade into 1.3.0, which was released last week.
>> Wasn't clear for me that you're running on 1.1.0
>>
>> Do you have a change to upgrade into the newest release?
>>
>> Best,
>> Alex
>>
>> On Dec 19, 2012, at 11:51 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>>
>>> It's still not working.
>>>
>>> I read somewhere that failover support was implemented after Jan 2012.
>>> I've flume 1.1.0+121-1.cdh4.0.1.p0.1~precise-cdh4.0.1 installed.
>>>
>>> Can that be a reason for it not working?
>>>
>>> Thanks,
>>> Abhijeet
>>>
>>> -----Original Message-----
>>> From: Alexander Alten-Lorenz [mailto:[EMAIL PROTECTED]]
>>> Sent: Wednesday, December 19, 2012 2:52 PM
>>> To: [EMAIL PROTECTED]
>>> Subject: Re: Configuring failover sync in Flume-ng
>>>
>>> YOUR_NAME.sinks = HOSTSINK-1 HOSTSINK-2
>>>
>>> HOSTSINK-1.type>>> HOSTSINK-1.bind=HOSTNAME-1
>>> HOSTSINK-1.port=needed by JDBC?
>>> HOSTSINK-1.channels=YOUR_CHANNELS
>>>
>>> the same for the other HOSTSINK and deploy the config to the hosts and startup. Should work.
>>>
>>> Best,
>>> Alex
>>>
>>> On Dec 19, 2012, at 10:01 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>>>
>>>> Yes, I saw that.
>>>>
>>>> But I'm not sure how to tell flume that the failover sink is running on "another" machine.
>>>>
>>>> -----Original Message-----
>>>> From: Alexander Lorenz [mailto:[EMAIL PROTECTED]]
>>>> Sent: Wednesday, December 19, 2012 1:59 PM
>>>> To: [EMAIL PROTECTED]
>>>> Subject: Re: Configuring failover sync in Flume-ng
>>>>
>>>> Hi,
>>>>
>>>> Failover:
>>>> http://flume.apache.org/FlumeUserGuide.html#failover-sink-processor
>>>>
>>>> Just define more as one JDBC channel and add them to the failover config (see the examples).
>>>>
>>>> Best
>>>> - Alex
>>>>
>>>> --
>>>> Alexander Alten-Lorenz
>>>> http://mapredit.blogspot.com
>>>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>>>
>>>> On Dec 19, 2012, at 9:23 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc], 1 sink [AsyncHbase]) on one server.
>>>>>
>>>>> However, I am not able to find documentation on how to configure failover sinks for such scenario.
>>>>>
>>>>> I want to run same sink as backup/failover sink on different server.
>>>>> How can that be configured?
>>>>>
>>>>> Thanks,
>>>>> Abhijeet
>>>>>
>>>>
>>>
>>
>

--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB