Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Re: Is hadoop tread safe?


+
Harsh J 2013-08-29, 15:19
+
Dinkar Sitaram 2013-08-30, 11:54
+
Pavan Sudheendra 2013-08-29, 14:33
Copy link to this message
-
Re: Is hadoop tread safe?
Adam Muise 2013-08-29, 14:40
Mappers don't communicate with each other in traditional MapReduce. If you
need something more MPI-ish then look to MPI over YARN or write your own
YARN app.

If you need multi-threading within the mapper then it is up to you as the
java developer to make it thread safe. Use the concurrent libraries like
anything else and Bob's your uncle. Having overly-complicated mappers can
be difficult to manage however and it kind of misses the mark for MapReduce
problems.

Maybe if you expand on your use case a bit someone here can provide
specific advice.
On Thu, Aug 29, 2013 at 10:33 AM, Pavan Sudheendra <[EMAIL PROTECTED]>wrote:

> Hi all,
>
> Is hadoop thread safe? Do mappers make use of threads in any chance? A
> little bit of information on the way they execute in parallel would help me
> out. Thanks.
>
> Regards,
> Pavan
>

--
*
*
*
*
*Adam Muise*
Solution Engineer
*Hortonworks*
[EMAIL PROTECTED]
416-417-4037

Hortonworks - Develops, Distributes and Supports Enterprise Apache
Hadoop.<http://hortonworks.com/>

Hortonworks Virtual Sandbox <http://hortonworks.com/sandbox>

Hadoop: Disruptive Possibilities by Jeff
Needham<http://hortonworks.com/resources/?did=72&cat=1>

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.