Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> concurrent write [ is it defined and consistent ? ]


Copy link to this message
-
Re: concurrent write [ is it defined and consistent ? ]
Atomic append is useful in contexts like transactional logging and  
producer-consumer workload sharing systems. Such usage cases are not  
covered by map-reduce programming model.

On Dec 1, 2009, at 8:02 AM, Owen O'Malley wrote:

>
> On Dec 1, 2009, at 6:30 AM, Brian Bockelman wrote:
>
>> The upcoming 0.21.x release will be the first to support appends  
>> after the file is closed (still single-writer though).  I'd  
>> speculate that this lays out some ground work that is necessary for  
>> multi-writer atomic appends, but I don't even know if atomic  
>> appends are even on the roadmap.
>
> The short answer is that I don't think anyone has even thought  
> deeply about it. I haven't heard anyone talking about doing it  
> anytime soon.
>
> My personal inclination is that atomic append does very bad things  
> to both the design of the file system and the user interface to the  
> file system. Clearly they added atomic append to GFS before they had  
> MapReduce. It seems like most applications would be better served by  
> implementing in MapReduce rather than using atomic append anyways...
>
> -- Owen
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB