Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> flume master down, flume node still has multiple orphan processes


Copy link to this message
-
flume master down, flume node still has multiple orphan processes
Hi,

I noticed that when my flume master is down for a quite while. My flume
agent is retrying to connect to the master, i can tell that from the flume
logs as below;

2012-12-19 11:11:42,126 WARN com.cloudera.flume.agent.MultiMasterRPC: Could
not connect to any master nodes (tried 1: [flume-master-01:35872])
2012-12-19 11:11:52,140 WARN com.cloudera.flume.agent.MultiMasterRPC: Could
not connect to any master nodes (tried 1: [flume-master-01:35872])
2012-12-19 11:12:02,152 WARN com.cloudera.flume.agent.MultiMasterRPC: Could
not connect to any master nodes (tried 1: [flume-master-01:35872])
2012-12-19 11:12:12,162 WARN com.cloudera.flume.agent.MultiMasterRPC: Could
not connect to any master nodes (tried 1: [flume-master-01:35872])

from "top", I've seen a lot of java processes owned by flume. followed my
instinct, I stopped flume agent, after a while, I run "top" on it, I still
have a lot of processes owned by flume user.

12735 flume     20   0 1487m  26m  128 S  0.3  1.3   0:05.41 java
13828 flume     20   0 1487m   24   24 S  0.3  0.0   8:11.92 java
14442 flume     20   0 1487m    0    0 S  0.3  0.0   8:06.81 java
15033 flume     20   0 1487m  26m  164 S  0.3  1.3   0:04.29 java
15453 flume     20   0 1487m    4    4 S  0.3  0.0   8:07.55 java
15964 flume     20   0 1487m  27m  164 S  0.3  1.4   0:04.09 java
16098 flume     20   0 1487m    0    0 S  0.3  0.0   8:12.67 java
......................

Why so many processes are generated here and they don't go away even when
flume is stopped.

--
All the best,
Shengjie Min
+
Nitin Pawar 2012-12-19, 11:54
+
Shengjie Min 2012-12-19, 12:08
+
Alexander Alten-Lorenz 2012-12-19, 12:20