Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Is there any way to use a hdfs file as a Circular buffer?


Copy link to this message
-
Re: Is there any way to use a hdfs file as a Circular buffer?
A circular file on hdfs is not possible.

Some of the ways around this limitation:
- Create a series of files and delete the oldest file when you have too
much.
- Put the data into an hbase table and do something similar.
- Use completely different technology like mongodb which has built in
support for a circular buffer (capped collection).

Niels

Hi all,
   Is there any way to use a hdfs file as a Circular buffer? I mean,
if I set a quotas to a directory on hdfs, and writting data to a file
in that directory continuously. Once the quotas exceeded, I can
redirect the writter and write the data from the beginning of the file
automatically .
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB