Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: Yarn communication between containers


Copy link to this message
-
Re: Yarn communication between containers
Hi Sandy

I noticed that Yarn uses Jetty, so I thought maybe there is some API hidden
to provide such a functionality (for MapReduce implementation). My current
idea is to start netty server/client on each container. I will definitely
look up the classes you mentioned. Thanks for code guidance (Yarn is a big
project and sometimes grep is not enough ;) )

regards
tmp

2013/3/27 Sandy Ryza <[EMAIL PROTECTED]>

> Hi tmp,
>
> YARN doesn't provide an explicit protocol for doing this.  Applications
> are expected to have their own mechanism for communication between task
> containers, other task containers, and app masters.  If you want to see how
> this is done in MapReduce, I would suggest looking at the
> TaskUmbilicalProtocol, which is implemented on the AM side by
> TaskAttemptListenerImpl and created on the Container side in YarnChild.
>  Through such a protocol, you could give containers the locations of other
> containers that would allow them to communicate correctly.
>
> -Sandy
>
>
> On Wed, Mar 27, 2013 at 5:08 AM, blah blah <[EMAIL PROTECTED]> wrote:
>
>> Hi
>>
>> I would like the containers to communicate with each other (also some
>> heart beats to AppMaster). How can I achieve this? Is there some code
>> snippet in the Yarn project or tutorial, that could guide me on
>> implementing this functionality?
>>
>> regards
>> tmp
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB