Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: Yarn communication between containers


Copy link to this message
-
Re: Yarn communication between containers
Hi Sandy

I noticed that Yarn uses Jetty, so I thought maybe there is some API hidden
to provide such a functionality (for MapReduce implementation). My current
idea is to start netty server/client on each container. I will definitely
look up the classes you mentioned. Thanks for code guidance (Yarn is a big
project and sometimes grep is not enough ;) )

regards
tmp

2013/3/27 Sandy Ryza <[EMAIL PROTECTED]>

> Hi tmp,
>
> YARN doesn't provide an explicit protocol for doing this.  Applications
> are expected to have their own mechanism for communication between task
> containers, other task containers, and app masters.  If you want to see how
> this is done in MapReduce, I would suggest looking at the
> TaskUmbilicalProtocol, which is implemented on the AM side by
> TaskAttemptListenerImpl and created on the Container side in YarnChild.
>  Through such a protocol, you could give containers the locations of other
> containers that would allow them to communicate correctly.
>
> -Sandy
>
>
> On Wed, Mar 27, 2013 at 5:08 AM, blah blah <[EMAIL PROTECTED]> wrote:
>
>> Hi
>>
>> I would like the containers to communicate with each other (also some
>> heart beats to AppMaster). How can I achieve this? Is there some code
>> snippet in the Yarn project or tutorial, that could guide me on
>> implementing this functionality?
>>
>> regards
>> tmp
>>
>
>