-how to know which line the agent have sent ?
周梦想 2013-02-06, 02:45
I'm using tailDirs('mydir') source of the agent to gather logs to hadoop
hdfs. I notice some documents advise that if the agent collapsed, I have
to remove files in 'mydir' and clear flume.agent.logdir. Thus I will lose
some data or have duplicate data. And I don't know which line the agent
have sent to.
I'm worrying about the agent failure and resend or miss-send the content to
collector. I want to know how to check which line of log file the agent
have sent if the agent exit suddenly. The files in flute log dir, such as
sending,sent can't be read.
Please give some advise to process such situation.
Alexander Alten-Lorenz 2013-02-06, 07:50
周梦想 2013-02-07, 03:03
周梦想 2013-02-07, 03:24
Jeong-shik Jang 2013-02-07, 04:18
周梦想 2013-02-07, 08:33
hoo.smth 2013-02-16, 09:00