Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Hive parallel execution deadlocks, need restart of yarn-nodemanager


Copy link to this message
-
Hive parallel execution deadlocks, need restart of yarn-nodemanager
Is there a known deadlock issue or bug when using Hive parallel execution with more parallel hive threads than there are computing nodemanagers ?

On my test cluster, i have set Hive parallel excution to 2 or 3 threads, and have only 1 computing nodemanager with 5 cpu cores.

When i run a hive request with a lot of unions that decomposes in a lot of jobs to be executed in parallel, after a few jobs done, it always endup deadlocking on 0% at mapping for all parallel jobs (from Hive0server2 logs). If i restart hadoop-yarn-nodemanager on the nodemanager server, Hive gets out of its deadlock and continues, until getting deadlocked a bit later again.

Alex