Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> concurrent write [ is it defined and consistent ? ]


Copy link to this message
-
Re: concurrent write [ is it defined and consistent ? ]
Atomic append is useful in contexts like transactional logging and  
producer-consumer workload sharing systems. Such usage cases are not  
covered by map-reduce programming model.

On Dec 1, 2009, at 8:02 AM, Owen O'Malley wrote:

>
> On Dec 1, 2009, at 6:30 AM, Brian Bockelman wrote:
>
>> The upcoming 0.21.x release will be the first to support appends  
>> after the file is closed (still single-writer though).  I'd  
>> speculate that this lays out some ground work that is necessary for  
>> multi-writer atomic appends, but I don't even know if atomic  
>> appends are even on the roadmap.
>
> The short answer is that I don't think anyone has even thought  
> deeply about it. I haven't heard anyone talking about doing it  
> anytime soon.
>
> My personal inclination is that atomic append does very bad things  
> to both the design of the file system and the user interface to the  
> file system. Clearly they added atomic append to GFS before they had  
> MapReduce. It seems like most applications would be better served by  
> implementing in MapReduce rather than using atomic append anyways...
>
> -- Owen