-Re: File Channel Capacity issue
Hari Shreedharan 2012-11-23, 21:14
The FileChannel actually uses a fixed size checkpoint file -- so it is not possible to set it to unlimited size (the checkpoint file is mmap-ed to a fixed size buffer). To change the capacity of the channel, the easiest way off the top of my head is:
* Shutdown the agent.
* Delete all files in the file channel's checkpoint directory. (not the data directories. Also you might want to move them out, rather than delete to be safe)
* Change your configuration to increase the capacity of the channel.
* Restart the agent - this will cause full replay, so the agent might take sometime to start up if there are a lot of events in the channel (to avoid this - shutdown the source before shutting the agent down - so the sink can drain out the channel completely, wait for about 1-2 mins after the channel is empty so that the data files get deleted (this happens only immediately after a checkpoint - you can verify this by making sure each data dir has only 2 files each), since all events have been sent out - so during restart the channel will be quite empty, with very little to replay).
Hope this helps.
On Friday, November 23, 2012 at 11:58 AM, Camp, Roy wrote:
> I am having issue with a slow sink (not flume related) but it is causing my file channel to overflow. When trying to increase it I get the following error but can’t seem to find the setting it refers to in any of the documentation. Additionally, is there a way to set the file channel to unlimited? It doesn’t seem to like 0.
> java.lang.IllegalStateException: Channel closed [channel=collectorfile]. Due to java.lang.IllegalStateException: Configured capacity is 100000000 but the checkpoint file capacity is 1000000. See FileChannel documentation on how to change a channels capacity.