Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> connection overload strategies


Copy link to this message
-
connection overload strategies
If my YARN application tasks are all reading/writing HDFS simultaneously and some node is unable to honor a connection request because it is overloaded, what happens?  I've seen HDFS attempt to retry connections.
For that matter, how does MR under YARN deal with connection overload during the shuffle phase, where it seems that such cases would be likely?
Thanks
John