Hi,

i want to use flume in my logging-system:

log4j->flume-Agent-> |
log4j->flume-Agent-> |->flume-collector->elasticsearch-node
log4j->flume-Agent-> |->flume-collector->elasticsearch-node
log4j->flume-Agent-> |

in this case using the load balancing sink processor is no problem. But now
i want to increase the number of server nodes (flume-collector->
elasticsearch-node). How can i tell my clients there is a new server? i
think i have to reconfigure my load balancing sink processor, but i can't
restart the client flume agent, because i would lose messages.
And i use the log4j 2 embedded flume appender. i cant restart the whole
application

Has anybody a solution? :/

Regards,
Simon

Hi,

Sounds like OG (collector). In that case you can do this per flume-shell:
http://archive.cloudera.com/cdh/3/flume/UserGuide/#_the_flume_command_shell

Btw, please check Flume 1.x, version 1.3.0 will be released very soon (I think this week). OG will be not actively developed anymore, so I suggest a move to Flume 1.x is the best thing you can do.
Check our wiki as well the UserGuide too:
Wiki: https://cwiki.apache.org/confluence/display/FLUME/Home
http://flume.apache.org/FlumeUserGuide.html (for the 1.3. release online)
Cheers,
 Alex

On Nov 29, 2012, at 9:43 AM, Simon Monecke <[EMAIL PROTECTED]> wrote:

> Hi,
>
> i want to use flume in my logging-system:
>
> log4j->flume-Agent-> |
> log4j->flume-Agent-> |->flume-collector->elasticsearch-node
> log4j->flume-Agent-> |->flume-collector->elasticsearch-node
> log4j->flume-Agent-> |
>
> in this case using the load balancing sink processor is no problem. But now
> i want to increase the number of server nodes (flume-collector->
> elasticsearch-node). How can i tell my clients there is a new server? i
> think i have to reconfigure my load balancing sink processor, but i can't
> restart the client flume agent, because i would lose messages.
> And i use the log4j 2 embedded flume appender. i cant restart the whole
> application
>
> Has anybody a solution? :/
>
> Regards,
> Simon

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF

Hi Simon,

Assuming you are using flume ng you can just modify the config add another
collector and save the file. No need for a restart. The agent will check in
periodically for changes. AFAIK every 30 seconds.

-Jeff
On Thu, Nov 29, 2012 at 12:43 AM, Simon Monecke <[EMAIL PROTECTED]>wrote:

> Hi,
>
> i want to use flume in my logging-system:
>
> log4j->flume-Agent-> |
> log4j->flume-Agent-> |->flume-collector->elasticsearch-node
> log4j->flume-Agent-> |->flume-collector->elasticsearch-node
> log4j->flume-Agent-> |
>
> in this case using the load balancing sink processor is no problem. But
> now i want to increase the number of server nodes (flume-collector->
> elasticsearch-node). How can i tell my clients there is a new server? i
> think i have to reconfigure my load balancing sink processor, but i can't
> restart the client flume agent, because i would lose messages.
> And i use the log4j 2 embedded flume appender. i cant restart the whole
> application
>
> Has anybody a solution? :/
>
> Regards,
> Simon
>