Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> race condition in hadoop 0.20.2 (cdh3u1)


+
Stan Rosenberg 2012-01-17, 21:21
+
Brock Noland 2012-01-17, 23:38
Copy link to this message
-
Re: race condition in hadoop 0.20.2 (cdh3u1)
On Tue, Jan 17, 2012 at 6:38 PM, Brock Noland <[EMAIL PROTECTED]> wrote:
> This class is invalid. A single thread will be executing your mapper
> or reducer but there will be multiple threads (background threads such
> as the SpillThread) creating MyKey instances which is exactly what you
> are seeing. This is by design.
>

Could you please refer me to where this design decision/assumption
is/was documented?  Imho, this assumption clashes with the overall
object re-use methodology. I would have at least considered
making 'readFields' and 'write' synchronized, even if it is to
indicate that there are multiple threads executing
serialization/de-serialization.  (As only a few threads are competing
in this case, the performance penalty would have been negligible.)

Thanks,

stan
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB