Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: Is hadoop tread safe?


Copy link to this message
-
Re: Is hadoop tread safe?
No, I had written a huge Map Reduce program which talks with hbase and does
a lot of computing using it as a source as well as sink.. One of my
colleague saw my code and saw that I had used a lot of static function
instead of making use of proper OOP concepts.. He was telling me that it
shouldn't be the way I should go about doing it.. But my code works fine..
So, was wondering will I face any problem in the future because of this..
That's all.

Regards,
Pavan
On Aug 29, 2013 8:11 PM, "Adam Muise" <[EMAIL PROTECTED]> wrote:

> Mappers don't communicate with each other in traditional MapReduce. If you
> need something more MPI-ish then look to MPI over YARN or write your own
> YARN app.
>
> If you need multi-threading within the mapper then it is up to you as the
> java developer to make it thread safe. Use the concurrent libraries like
> anything else and Bob's your uncle. Having overly-complicated mappers can
> be difficult to manage however and it kind of misses the mark for MapReduce
> problems.
>
> Maybe if you expand on your use case a bit someone here can provide
> specific advice.
>
>
> On Thu, Aug 29, 2013 at 10:33 AM, Pavan Sudheendra <[EMAIL PROTECTED]>wrote:
>
>> Hi all,
>>
>> Is hadoop thread safe? Do mappers make use of threads in any chance? A
>> little bit of information on the way they execute in parallel would help me
>> out. Thanks.
>>
>> Regards,
>> Pavan
>>
>
>
>
> --
> *
> *
> *
> *
> *Adam Muise*
> Solution Engineer
> *Hortonworks*
> [EMAIL PROTECTED]
> 416-417-4037
>
> Hortonworks - Develops, Distributes and Supports Enterprise Apache Hadoop.<http://hortonworks.com/>
>
> Hortonworks Virtual Sandbox <http://hortonworks.com/sandbox>
>
> Hadoop: Disruptive Possibilities by Jeff Needham<http://hortonworks.com/resources/?did=72&cat=1>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.