A circular file on hdfs is not possible.
Some of the ways around this limitation:
- Create a series of files and delete the oldest file when you have too
- Put the data into an hbase table and do something similar.
- Use completely different technology like mongodb which has built in
support for a circular buffer (capped collection).
Is there any way to use a hdfs file as a Circular buffer? I mean,
if I set a quotas to a directory on hdfs, and writting data to a file
in that directory continuously. Once the quotas exceeded, I can
redirect the writter and write the data from the beginning of the file