Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> flume master down, flume node still has multiple orphan processes


Copy link to this message
-
flume master down, flume node still has multiple orphan processes
Hi,

I noticed that when my flume master is down for a quite while. My flume
agent is retrying to connect to the master, i can tell that from the flume
logs as below;

2012-12-19 11:11:42,126 WARN com.cloudera.flume.agent.MultiMasterRPC: Could
not connect to any master nodes (tried 1: [flume-master-01:35872])
2012-12-19 11:11:52,140 WARN com.cloudera.flume.agent.MultiMasterRPC: Could
not connect to any master nodes (tried 1: [flume-master-01:35872])
2012-12-19 11:12:02,152 WARN com.cloudera.flume.agent.MultiMasterRPC: Could
not connect to any master nodes (tried 1: [flume-master-01:35872])
2012-12-19 11:12:12,162 WARN com.cloudera.flume.agent.MultiMasterRPC: Could
not connect to any master nodes (tried 1: [flume-master-01:35872])

from "top", I've seen a lot of java processes owned by flume. followed my
instinct, I stopped flume agent, after a while, I run "top" on it, I still
have a lot of processes owned by flume user.

12735 flume     20   0 1487m  26m  128 S  0.3  1.3   0:05.41 java
13828 flume     20   0 1487m   24   24 S  0.3  0.0   8:11.92 java
14442 flume     20   0 1487m    0    0 S  0.3  0.0   8:06.81 java
15033 flume     20   0 1487m  26m  164 S  0.3  1.3   0:04.29 java
15453 flume     20   0 1487m    4    4 S  0.3  0.0   8:07.55 java
15964 flume     20   0 1487m  27m  164 S  0.3  1.4   0:04.09 java
16098 flume     20   0 1487m    0    0 S  0.3  0.0   8:12.67 java
......................

Why so many processes are generated here and they don't go away even when
flume is stopped.

--
All the best,
Shengjie Min
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB