Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> race condition in hadoop 0.20.2 (cdh3u1)

Stan Rosenberg 2012-01-17, 21:21
Brock Noland 2012-01-17, 23:38
Copy link to this message
Re: race condition in hadoop 0.20.2 (cdh3u1)
On Tue, Jan 17, 2012 at 6:38 PM, Brock Noland <[EMAIL PROTECTED]> wrote:
> This class is invalid. A single thread will be executing your mapper
> or reducer but there will be multiple threads (background threads such
> as the SpillThread) creating MyKey instances which is exactly what you
> are seeing. This is by design.

Could you please refer me to where this design decision/assumption
is/was documented?  Imho, this assumption clashes with the overall
object re-use methodology. I would have at least considered
making 'readFields' and 'write' synchronized, even if it is to
indicate that there are multiple threads executing
serialization/de-serialization.  (As only a few threads are competing
in this case, the performance penalty would have been negligible.)