Hi,

I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc], 1 sink [AsyncHbase]) on one server.

However, I am not able to find documentation on how to configure failover sinks for such scenario.

I want to run same sink as backup/failover sink on different server.
How can that be configured?

Thanks,
Abhijeet

Hi,

Failover:
http://flume.apache.org/FlumeUserGuide.html#failover-sink-processor

Just define more as one JDBC channel and add them to the failover config (see the examples).

Best
- Alex

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF

On Dec 19, 2012, at 9:23 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc], 1 sink [AsyncHbase]) on one server.
>
> However, I am not able to find documentation on how to configure failover sinks for such scenario.
>
> I want to run same sink as backup/failover sink on different server.
> How can that be configured?
>
> Thanks,
> Abhijeet
>

Yes, I saw that.

But I'm not sure how to tell flume that the failover sink is running on "another" machine.

-----Original Message-----
From: Alexander Lorenz [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, December 19, 2012 1:59 PM
To: [EMAIL PROTECTED]
Subject: Re: Configuring failover sync in Flume-ng

Hi,

Failover:
http://flume.apache.org/FlumeUserGuide.html#failover-sink-processor

Just define more as one JDBC channel and add them to the failover config (see the examples).

Best
- Alex

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF

On Dec 19, 2012, at 9:23 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc], 1 sink [AsyncHbase]) on one server.
>
> However, I am not able to find documentation on how to configure failover sinks for such scenario.
>
> I want to run same sink as backup/failover sink on different server.
> How can that be configured?
>
> Thanks,
> Abhijeet
>

YOUR_NAME.sinks = HOSTSINK-1 HOSTSINK-2

HOSTSINK-1.typeHOSTSINK-1.bind=HOSTNAME-1
HOSTSINK-1.port=needed by JDBC?
HOSTSINK-1.channels=YOUR_CHANNELS

the same for the other HOSTSINK and deploy the config to the hosts and startup. Should work.

Best,
 Alex

On Dec 19, 2012, at 10:01 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:

> Yes, I saw that.
>
> But I'm not sure how to tell flume that the failover sink is running on "another" machine.
>
> -----Original Message-----
> From: Alexander Lorenz [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, December 19, 2012 1:59 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Configuring failover sync in Flume-ng
>
> Hi,
>
> Failover:
> http://flume.apache.org/FlumeUserGuide.html#failover-sink-processor
>
> Just define more as one JDBC channel and add them to the failover config (see the examples).
>
> Best
> - Alex
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
> On Dec 19, 2012, at 9:23 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc], 1 sink [AsyncHbase]) on one server.
>>
>> However, I am not able to find documentation on how to configure failover sinks for such scenario.
>>
>> I want to run same sink as backup/failover sink on different server.
>> How can that be configured?
>>
>> Thanks,
>> Abhijeet
>>
>

It's still not working.

I read somewhere that failover support was implemented after Jan 2012.
I've flume 1.1.0+121-1.cdh4.0.1.p0.1~precise-cdh4.0.1 installed.

Can that be a reason for it not working?

Thanks,
Abhijeet

-----Original Message-----
From: Alexander Alten-Lorenz [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, December 19, 2012 2:52 PM
To: [EMAIL PROTECTED]
Subject: Re: Configuring failover sync in Flume-ng

YOUR_NAME.sinks = HOSTSINK-1 HOSTSINK-2

HOSTSINK-1.typeHOSTSINK-1.bind=HOSTNAME-1
HOSTSINK-1.port=needed by JDBC?
HOSTSINK-1.channels=YOUR_CHANNELS

the same for the other HOSTSINK and deploy the config to the hosts and startup. Should work.

Best,
 Alex

On Dec 19, 2012, at 10:01 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:

> Yes, I saw that.
>
> But I'm not sure how to tell flume that the failover sink is running on "another" machine.
>
> -----Original Message-----
> From: Alexander Lorenz [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, December 19, 2012 1:59 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Configuring failover sync in Flume-ng
>
> Hi,
>
> Failover:
> http://flume.apache.org/FlumeUserGuide.html#failover-sink-processor
>
> Just define more as one JDBC channel and add them to the failover config (see the examples).
>
> Best
> - Alex
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
> On Dec 19, 2012, at 9:23 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc], 1 sink [AsyncHbase]) on one server.
>>
>> However, I am not able to find documentation on how to configure failover sinks for such scenario.
>>
>> I want to run same sink as backup/failover sink on different server.
>> How can that be configured?
>>
>> Thanks,
>> Abhijeet
>>
>

Oh, yes. You should upgrade into 1.3.0, which was released last week. Wasn't clear for me that you're running on 1.1.0

Do you have a change to upgrade into the newest release?

Best,
 Alex

On Dec 19, 2012, at 11:51 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:

> It's still not working.
>
> I read somewhere that failover support was implemented after Jan 2012.
> I've flume 1.1.0+121-1.cdh4.0.1.p0.1~precise-cdh4.0.1 installed.
>
> Can that be a reason for it not working?
>
> Thanks,
> Abhijeet
>
> -----Original Message-----
> From: Alexander Alten-Lorenz [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, December 19, 2012 2:52 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Configuring failover sync in Flume-ng
>
> YOUR_NAME.sinks = HOSTSINK-1 HOSTSINK-2
>
> HOSTSINK-1.type> HOSTSINK-1.bind=HOSTNAME-1
> HOSTSINK-1.port=needed by JDBC?
> HOSTSINK-1.channels=YOUR_CHANNELS
>
> the same for the other HOSTSINK and deploy the config to the hosts and startup. Should work.
>
> Best,
> Alex
>
> On Dec 19, 2012, at 10:01 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>
>> Yes, I saw that.
>>
>> But I'm not sure how to tell flume that the failover sink is running on "another" machine.
>>
>> -----Original Message-----
>> From: Alexander Lorenz [mailto:[EMAIL PROTECTED]]
>> Sent: Wednesday, December 19, 2012 1:59 PM
>> To: [EMAIL PROTECTED]
>> Subject: Re: Configuring failover sync in Flume-ng
>>
>> Hi,
>>
>> Failover:
>> http://flume.apache.org/FlumeUserGuide.html#failover-sink-processor
>>
>> Just define more as one JDBC channel and add them to the failover config (see the examples).
>>
>> Best
>> - Alex
>>
>> --
>> Alexander Alten-Lorenz
>> http://mapredit.blogspot.com
>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>
>> On Dec 19, 2012, at 9:23 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>>
>>> Hi,
>>>
>>> I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc], 1 sink [AsyncHbase]) on one server.
>>>
>>> However, I am not able to find documentation on how to configure failover sinks for such scenario.
>>>
>>> I want to run same sink as backup/failover sink on different server.
>>> How can that be configured?
>>>
>>> Thanks,
>>> Abhijeet
>>>
>>
>

Oh okay.

I'll upgrade and try.

Thanks! :)

-----Original Message-----
From: Alexander Alten-Lorenz [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, December 19, 2012 4:33 PM
To: [EMAIL PROTECTED]
Subject: Re: Configuring failover sync in Flume-ng

Oh, yes. You should upgrade into 1.3.0, which was released last week. Wasn't clear for me that you're running on 1.1.0

Do you have a change to upgrade into the newest release?

Best,
 Alex

On Dec 19, 2012, at 11:51 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:

> It's still not working.
>
> I read somewhere that failover support was implemented after Jan 2012.
> I've flume 1.1.0+121-1.cdh4.0.1.p0.1~precise-cdh4.0.1 installed.
>
> Can that be a reason for it not working?
>
> Thanks,
> Abhijeet
>
> -----Original Message-----
> From: Alexander Alten-Lorenz [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, December 19, 2012 2:52 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Configuring failover sync in Flume-ng
>
> YOUR_NAME.sinks = HOSTSINK-1 HOSTSINK-2
>
> HOSTSINK-1.type> HOSTSINK-1.bind=HOSTNAME-1
> HOSTSINK-1.port=needed by JDBC?
> HOSTSINK-1.channels=YOUR_CHANNELS
>
> the same for the other HOSTSINK and deploy the config to the hosts and startup. Should work.
>
> Best,
> Alex
>
> On Dec 19, 2012, at 10:01 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>
>> Yes, I saw that.
>>
>> But I'm not sure how to tell flume that the failover sink is running on "another" machine.
>>
>> -----Original Message-----
>> From: Alexander Lorenz [mailto:[EMAIL PROTECTED]]
>> Sent: Wednesday, December 19, 2012 1:59 PM
>> To: [EMAIL PROTECTED]
>> Subject: Re: Configuring failover sync in Flume-ng
>>
>> Hi,
>>
>> Failover:
>> http://flume.apache.org/FlumeUserGuide.html#failover-sink-processor
>>
>> Just define more as one JDBC channel and add them to the failover config (see the examples).
>>
>> Best
>> - Alex
>>
>> --
>> Alexander Alten-Lorenz
>> http://mapredit.blogspot.com
>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>
>> On Dec 19, 2012, at 9:23 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>>
>>> Hi,
>>>
>>> I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc], 1 sink [AsyncHbase]) on one server.
>>>
>>> However, I am not able to find documentation on how to configure failover sinks for such scenario.
>>>
>>> I want to run same sink as backup/failover sink on different server.
>>> How can that be configured?
>>>
>>> Thanks,
>>> Abhijeet
>>>
>>
>

I am running Ubuntu 12.04 server edition.
The repo has flume 1.2 release available.
How should I install 1.3? Using source?
Is there a stable release available?

Thanks,
Abhijeet

-----Original Message-----
From: Alexander Alten-Lorenz [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, December 19, 2012 4:33 PM
To: [EMAIL PROTECTED]
Subject: Re: Configuring failover sync in Flume-ng

Oh, yes. You should upgrade into 1.3.0, which was released last week. Wasn't clear for me that you're running on 1.1.0

Do you have a change to upgrade into the newest release?

Best,
 Alex

On Dec 19, 2012, at 11:51 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:

> It's still not working.
>
> I read somewhere that failover support was implemented after Jan 2012.
> I've flume 1.1.0+121-1.cdh4.0.1.p0.1~precise-cdh4.0.1 installed.
>
> Can that be a reason for it not working?
>
> Thanks,
> Abhijeet
>
> -----Original Message-----
> From: Alexander Alten-Lorenz [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, December 19, 2012 2:52 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Configuring failover sync in Flume-ng
>
> YOUR_NAME.sinks = HOSTSINK-1 HOSTSINK-2
>
> HOSTSINK-1.type> HOSTSINK-1.bind=HOSTNAME-1
> HOSTSINK-1.port=needed by JDBC?
> HOSTSINK-1.channels=YOUR_CHANNELS
>
> the same for the other HOSTSINK and deploy the config to the hosts and startup. Should work.
>
> Best,
> Alex
>
> On Dec 19, 2012, at 10:01 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>
>> Yes, I saw that.
>>
>> But I'm not sure how to tell flume that the failover sink is running on "another" machine.
>>
>> -----Original Message-----
>> From: Alexander Lorenz [mailto:[EMAIL PROTECTED]]
>> Sent: Wednesday, December 19, 2012 1:59 PM
>> To: [EMAIL PROTECTED]
>> Subject: Re: Configuring failover sync in Flume-ng
>>
>> Hi,
>>
>> Failover:
>> http://flume.apache.org/FlumeUserGuide.html#failover-sink-processor
>>
>> Just define more as one JDBC channel and add them to the failover config (see the examples).
>>
>> Best
>> - Alex
>>
>> --
>> Alexander Alten-Lorenz
>> http://mapredit.blogspot.com
>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>
>> On Dec 19, 2012, at 9:23 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>>
>>> Hi,
>>>
>>> I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc], 1 sink [AsyncHbase]) on one server.
>>>
>>> However, I am not able to find documentation on how to configure failover sinks for such scenario.
>>>
>>> I want to run same sink as backup/failover sink on different server.
>>> How can that be configured?
>>>
>>> Thanks,
>>> Abhijeet
>>>
>>
>

Yes, why not luring the wiki :)
https://cwiki.apache.org/confluence/display/FLUME/Home

Stable: http://www.apache.org/dyn/closer.cgi/flume/1.3.0
Just grab the tar.gz - or use 1.2.0 as deb-package, I suspect the balancing stuff is inside too.

Best,
 Alex

On Dec 19, 2012, at 2:55 PM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:

> I am running Ubuntu 12.04 server edition.
> The repo has flume 1.2 release available.
> How should I install 1.3? Using source?
> Is there a stable release available?
>
> Thanks,
> Abhijeet
>
> -----Original Message-----
> From: Alexander Alten-Lorenz [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, December 19, 2012 4:33 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Configuring failover sync in Flume-ng
>
> Oh, yes. You should upgrade into 1.3.0, which was released last week. Wasn't clear for me that you're running on 1.1.0
>
> Do you have a change to upgrade into the newest release?
>
> Best,
> Alex
>
> On Dec 19, 2012, at 11:51 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>
>> It's still not working.
>>
>> I read somewhere that failover support was implemented after Jan 2012.
>> I've flume 1.1.0+121-1.cdh4.0.1.p0.1~precise-cdh4.0.1 installed.
>>
>> Can that be a reason for it not working?
>>
>> Thanks,
>> Abhijeet
>>
>> -----Original Message-----
>> From: Alexander Alten-Lorenz [mailto:[EMAIL PROTECTED]]
>> Sent: Wednesday, December 19, 2012 2:52 PM
>> To: [EMAIL PROTECTED]
>> Subject: Re: Configuring failover sync in Flume-ng
>>
>> YOUR_NAME.sinks = HOSTSINK-1 HOSTSINK-2
>>
>> HOSTSINK-1.type>> HOSTSINK-1.bind=HOSTNAME-1
>> HOSTSINK-1.port=needed by JDBC?
>> HOSTSINK-1.channels=YOUR_CHANNELS
>>
>> the same for the other HOSTSINK and deploy the config to the hosts and startup. Should work.
>>
>> Best,
>> Alex
>>
>> On Dec 19, 2012, at 10:01 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>>
>>> Yes, I saw that.
>>>
>>> But I'm not sure how to tell flume that the failover sink is running on "another" machine.
>>>
>>> -----Original Message-----
>>> From: Alexander Lorenz [mailto:[EMAIL PROTECTED]]
>>> Sent: Wednesday, December 19, 2012 1:59 PM
>>> To: [EMAIL PROTECTED]
>>> Subject: Re: Configuring failover sync in Flume-ng
>>>
>>> Hi,
>>>
>>> Failover:
>>> http://flume.apache.org/FlumeUserGuide.html#failover-sink-processor
>>>
>>> Just define more as one JDBC channel and add them to the failover config (see the examples).
>>>
>>> Best
>>> - Alex
>>>
>>> --
>>> Alexander Alten-Lorenz
>>> http://mapredit.blogspot.com
>>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>>
>>> On Dec 19, 2012, at 9:23 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>>>
>>>> Hi,
>>>>
>>>> I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc], 1 sink [AsyncHbase]) on one server.
>>>>
>>>> However, I am not able to find documentation on how to configure failover sinks for such scenario.
>>>>
>>>> I want to run same sink as backup/failover sink on different server.
>>>> How can that be configured?
>>>>
>>>> Thanks,
>>>> Abhijeet
>>>>
>>>
>>
>

Just as a side note: File Channel has received significantly more
testing than JDBC Channel.

On Wed, Dec 19, 2012 at 8:05 AM, Alexander Alten-Lorenz
<[EMAIL PROTECTED]> wrote:
> Yes, why not luring the wiki :)
> https://cwiki.apache.org/confluence/display/FLUME/Home
>
> Stable: http://www.apache.org/dyn/closer.cgi/flume/1.3.0
> Just grab the tar.gz - or use 1.2.0 as deb-package, I suspect the balancing stuff is inside too.
>
> Best,
>  Alex
>
> On Dec 19, 2012, at 2:55 PM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>
>> I am running Ubuntu 12.04 server edition.
>> The repo has flume 1.2 release available.
>> How should I install 1.3? Using source?
>> Is there a stable release available?
>>
>> Thanks,
>> Abhijeet
>>
>> -----Original Message-----
>> From: Alexander Alten-Lorenz [mailto:[EMAIL PROTECTED]]
>> Sent: Wednesday, December 19, 2012 4:33 PM
>> To: [EMAIL PROTECTED]
>> Subject: Re: Configuring failover sync in Flume-ng
>>
>> Oh, yes. You should upgrade into 1.3.0, which was released last week. Wasn't clear for me that you're running on 1.1.0
>>
>> Do you have a change to upgrade into the newest release?
>>
>> Best,
>> Alex
>>
>> On Dec 19, 2012, at 11:51 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>>
>>> It's still not working.
>>>
>>> I read somewhere that failover support was implemented after Jan 2012.
>>> I've flume 1.1.0+121-1.cdh4.0.1.p0.1~precise-cdh4.0.1 installed.
>>>
>>> Can that be a reason for it not working?
>>>
>>> Thanks,
>>> Abhijeet
>>>
>>> -----Original Message-----
>>> From: Alexander Alten-Lorenz [mailto:[EMAIL PROTECTED]]
>>> Sent: Wednesday, December 19, 2012 2:52 PM
>>> To: [EMAIL PROTECTED]
>>> Subject: Re: Configuring failover sync in Flume-ng
>>>
>>> YOUR_NAME.sinks = HOSTSINK-1 HOSTSINK-2
>>>
>>> HOSTSINK-1.type>>> HOSTSINK-1.bind=HOSTNAME-1
>>> HOSTSINK-1.port=needed by JDBC?
>>> HOSTSINK-1.channels=YOUR_CHANNELS
>>>
>>> the same for the other HOSTSINK and deploy the config to the hosts and startup. Should work.
>>>
>>> Best,
>>> Alex
>>>
>>> On Dec 19, 2012, at 10:01 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>>>
>>>> Yes, I saw that.
>>>>
>>>> But I'm not sure how to tell flume that the failover sink is running on "another" machine.
>>>>
>>>> -----Original Message-----
>>>> From: Alexander Lorenz [mailto:[EMAIL PROTECTED]]
>>>> Sent: Wednesday, December 19, 2012 1:59 PM
>>>> To: [EMAIL PROTECTED]
>>>> Subject: Re: Configuring failover sync in Flume-ng
>>>>
>>>> Hi,
>>>>
>>>> Failover:
>>>> http://flume.apache.org/FlumeUserGuide.html#failover-sink-processor
>>>>
>>>> Just define more as one JDBC channel and add them to the failover config (see the examples).
>>>>
>>>> Best
>>>> - Alex
>>>>
>>>> --
>>>> Alexander Alten-Lorenz
>>>> http://mapredit.blogspot.com
>>>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>>>
>>>> On Dec 19, 2012, at 9:23 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc], 1 sink [AsyncHbase]) on one server.
>>>>>
>>>>> However, I am not able to find documentation on how to configure failover sinks for such scenario.
>>>>>
>>>>> I want to run same sink as backup/failover sink on different server.
>>>>> How can that be configured?
>>>>>
>>>>> Thanks,
>>>>> Abhijeet
>>>>>
>>>>
>>>
>>
>

--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

Okay.

-----Original Message-----
From: Brock Noland [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, December 19, 2012 8:10 PM
To: [EMAIL PROTECTED]
Subject: Re: Configuring failover sync in Flume-ng

Just as a side note: File Channel has received significantly more testing than JDBC Channel.

On Wed, Dec 19, 2012 at 8:05 AM, Alexander Alten-Lorenz <[EMAIL PROTECTED]> wrote:
> Yes, why not luring the wiki :)
> https://cwiki.apache.org/confluence/display/FLUME/Home
>
> Stable: http://www.apache.org/dyn/closer.cgi/flume/1.3.0
> Just grab the tar.gz - or use 1.2.0 as deb-package, I suspect the balancing stuff is inside too.
>
> Best,
>  Alex
>
> On Dec 19, 2012, at 2:55 PM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>
>> I am running Ubuntu 12.04 server edition.
>> The repo has flume 1.2 release available.
>> How should I install 1.3? Using source?
>> Is there a stable release available?
>>
>> Thanks,
>> Abhijeet
>>
>> -----Original Message-----
>> From: Alexander Alten-Lorenz [mailto:[EMAIL PROTECTED]]
>> Sent: Wednesday, December 19, 2012 4:33 PM
>> To: [EMAIL PROTECTED]
>> Subject: Re: Configuring failover sync in Flume-ng
>>
>> Oh, yes. You should upgrade into 1.3.0, which was released last week.
>> Wasn't clear for me that you're running on 1.1.0
>>
>> Do you have a change to upgrade into the newest release?
>>
>> Best,
>> Alex
>>
>> On Dec 19, 2012, at 11:51 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>>
>>> It's still not working.
>>>
>>> I read somewhere that failover support was implemented after Jan 2012.
>>> I've flume 1.1.0+121-1.cdh4.0.1.p0.1~precise-cdh4.0.1 installed.
>>>
>>> Can that be a reason for it not working?
>>>
>>> Thanks,
>>> Abhijeet
>>>
>>> -----Original Message-----
>>> From: Alexander Alten-Lorenz [mailto:[EMAIL PROTECTED]]
>>> Sent: Wednesday, December 19, 2012 2:52 PM
>>> To: [EMAIL PROTECTED]
>>> Subject: Re: Configuring failover sync in Flume-ng
>>>
>>> YOUR_NAME.sinks = HOSTSINK-1 HOSTSINK-2
>>>
>>> HOSTSINK-1.type>>> HOSTSINK-1.bind=HOSTNAME-1
>>> HOSTSINK-1.port=needed by JDBC?
>>> HOSTSINK-1.channels=YOUR_CHANNELS
>>>
>>> the same for the other HOSTSINK and deploy the config to the hosts and startup. Should work.
>>>
>>> Best,
>>> Alex
>>>
>>> On Dec 19, 2012, at 10:01 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>>>
>>>> Yes, I saw that.
>>>>
>>>> But I'm not sure how to tell flume that the failover sink is running on "another" machine.
>>>>
>>>> -----Original Message-----
>>>> From: Alexander Lorenz [mailto:[EMAIL PROTECTED]]
>>>> Sent: Wednesday, December 19, 2012 1:59 PM
>>>> To: [EMAIL PROTECTED]
>>>> Subject: Re: Configuring failover sync in Flume-ng
>>>>
>>>> Hi,
>>>>
>>>> Failover:
>>>> http://flume.apache.org/FlumeUserGuide.html#failover-sink-processor
>>>>
>>>> Just define more as one JDBC channel and add them to the failover config (see the examples).
>>>>
>>>> Best
>>>> - Alex
>>>>
>>>> --
>>>> Alexander Alten-Lorenz
>>>> http://mapredit.blogspot.com
>>>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>>>
>>>> On Dec 19, 2012, at 9:23 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc], 1 sink [AsyncHbase]) on one server.
>>>>>
>>>>> However, I am not able to find documentation on how to configure failover sinks for such scenario.
>>>>>
>>>>> I want to run same sink as backup/failover sink on different server.
>>>>> How can that be configured?
>>>>>
>>>>> Thanks,
>>>>> Abhijeet
>>>>>
>>>>
>>>
>>
>

--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

1.1.0 to 1.3.0 should be fairly easy job
On Dec 19, 2012 4:24 PM, "Abhijeet Pathak" <[EMAIL PROTECTED]>
wrote:

> It's still not working.
>
> I read somewhere that failover support was implemented after Jan 2012.
> I've flume 1.1.0+121-1.cdh4.0.1.p0.1~precise-cdh4.0.1 installed.
>
> Can that be a reason for it not working?
>
> Thanks,
> Abhijeet
>
> -----Original Message-----
> From: Alexander Alten-Lorenz [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, December 19, 2012 2:52 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Configuring failover sync in Flume-ng
>
> YOUR_NAME.sinks = HOSTSINK-1 HOSTSINK-2
>
> HOSTSINK-1.type> HOSTSINK-1.bind=HOSTNAME-1
> HOSTSINK-1.port=needed by JDBC?
> HOSTSINK-1.channels=YOUR_CHANNELS
>
> the same for the other HOSTSINK and deploy the config to the hosts and
> startup. Should work.
>
> Best,
>  Alex
>
> On Dec 19, 2012, at 10:01 AM, Abhijeet Pathak <
> [EMAIL PROTECTED]> wrote:
>
> > Yes, I saw that.
> >
> > But I'm not sure how to tell flume that the failover sink is running on
> "another" machine.
> >
> > -----Original Message-----
> > From: Alexander Lorenz [mailto:[EMAIL PROTECTED]]
> > Sent: Wednesday, December 19, 2012 1:59 PM
> > To: [EMAIL PROTECTED]
> > Subject: Re: Configuring failover sync in Flume-ng
> >
> > Hi,
> >
> > Failover:
> > http://flume.apache.org/FlumeUserGuide.html#failover-sink-processor
> >
> > Just define more as one JDBC channel and add them to the failover config
> (see the examples).
> >
> > Best
> > - Alex
> >
> > --
> > Alexander Alten-Lorenz
> > http://mapredit.blogspot.com
> > German Hadoop LinkedIn Group: http://goo.gl/N8pCF
> >
> > On Dec 19, 2012, at 9:23 AM, Abhijeet Pathak <
> [EMAIL PROTECTED]> wrote:
> >
> >> Hi,
> >>
> >> I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc],
> 1 sink [AsyncHbase]) on one server.
> >>
> >> However, I am not able to find documentation on how to configure
> failover sinks for such scenario.
> >>
> >> I want to run same sink as backup/failover sink on different server.
> >> How can that be configured?
> >>
> >> Thanks,
> >> Abhijeet
> >>
> >
>
>