Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Most efficient way to support shared content among all mappers


Copy link to this message
-
Re: Most efficient way to support shared content among all mappers
Hey,

For a more detailed discussion of how to use memcached for this purpose, see
the paper "Low-Latency, High-Throughput Access to Static Global Resources
within the Hadoop Framework":
http://www.umiacs.umd.edu/~jimmylin/publications/Lin_etal_TR2009.pdf.

Regards,
Jeff

On Fri, May 8, 2009 at 2:49 PM, jason hadoop <[EMAIL PROTECTED]> wrote:

> Most of the people with this need are using some variant of memcached, or
> other distributed hash table.
>
> On Fri, May 8, 2009 at 10:07 AM, Joe <[EMAIL PROTECTED]> wrote:
>
> >
> > Hi,
> > As a newcomer to Hadoop, I wonder any efficient way to support shared
> > content among all mappers. For example, to implement an neural network
> > algorithm, I want the NN data structure accessible by all mappers.
> > Thanks for your comments!
> > - Joe
> >
> >
> >
> >
>
>
> --
> Alpha Chapters of my book on Hadoop are available
> http://www.apress.com/book/view/9781430219422
> www.prohadoopbook.com a community for Hadoop Professionals
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB