Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: Is hadoop tread safe?


Copy link to this message
-
Re: Is hadoop tread safe?
No, I had written a huge Map Reduce program which talks with hbase and does
a lot of computing using it as a source as well as sink.. One of my
colleague saw my code and saw that I had used a lot of static function
instead of making use of proper OOP concepts.. He was telling me that it
shouldn't be the way I should go about doing it.. But my code works fine..
So, was wondering will I face any problem in the future because of this..
That's all.

Regards,
Pavan
On Aug 29, 2013 8:11 PM, "Adam Muise" <[EMAIL PROTECTED]> wrote:

> Mappers don't communicate with each other in traditional MapReduce. If you
> need something more MPI-ish then look to MPI over YARN or write your own
> YARN app.
>
> If you need multi-threading within the mapper then it is up to you as the
> java developer to make it thread safe. Use the concurrent libraries like
> anything else and Bob's your uncle. Having overly-complicated mappers can
> be difficult to manage however and it kind of misses the mark for MapReduce
> problems.
>
> Maybe if you expand on your use case a bit someone here can provide
> specific advice.
>
>
> On Thu, Aug 29, 2013 at 10:33 AM, Pavan Sudheendra <[EMAIL PROTECTED]>wrote:
>
>> Hi all,
>>
>> Is hadoop thread safe? Do mappers make use of threads in any chance? A
>> little bit of information on the way they execute in parallel would help me
>> out. Thanks.
>>
>> Regards,
>> Pavan
>>
>
>
>
> --
> *
> *
> *
> *
> *Adam Muise*
> Solution Engineer
> *Hortonworks*
> [EMAIL PROTECTED]
> 416-417-4037
>
> Hortonworks - Develops, Distributes and Supports Enterprise Apache Hadoop.<http://hortonworks.com/>
>
> Hortonworks Virtual Sandbox <http://hortonworks.com/sandbox>
>
> Hadoop: Disruptive Possibilities by Jeff Needham<http://hortonworks.com/resources/?did=72&cat=1>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB