Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Most efficient way to support shared content among all mappers


Copy link to this message
-
Re: Most efficient way to support shared content among all mappers
Hey,

For a more detailed discussion of how to use memcached for this purpose, see
the paper "Low-Latency, High-Throughput Access to Static Global Resources
within the Hadoop Framework":
http://www.umiacs.umd.edu/~jimmylin/publications/Lin_etal_TR2009.pdf.

Regards,
Jeff

On Fri, May 8, 2009 at 2:49 PM, jason hadoop <[EMAIL PROTECTED]> wrote:

> Most of the people with this need are using some variant of memcached, or
> other distributed hash table.
>
> On Fri, May 8, 2009 at 10:07 AM, Joe <[EMAIL PROTECTED]> wrote:
>
> >
> > Hi,
> > As a newcomer to Hadoop, I wonder any efficient way to support shared
> > content among all mappers. For example, to implement an neural network
> > algorithm, I want the NN data structure accessible by all mappers.
> > Thanks for your comments!
> > - Joe
> >
> >
> >
> >
>
>
> --
> Alpha Chapters of my book on Hadoop are available
> http://www.apress.com/book/view/9781430219422
> www.prohadoopbook.com a community for Hadoop Professionals
>