Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - flume master down, flume node still has multiple orphan processes


+
Shengjie Min 2012-12-19, 11:34
+
Nitin Pawar 2012-12-19, 11:54
+
Shengjie Min 2012-12-19, 12:08
Copy link to this message
-
Re: flume master down, flume node still has multiple orphan processes
Alexander Alten-Lorenz 2012-12-19, 12:20
You have to kill -9 them, when you let run a jstack on a JVM pid you'll see WAITING threads.

Best,
 Alex
On Dec 19, 2012, at 12:34 PM, Shengjie Min <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I noticed that when my flume master is down for a quite while. My flume
> agent is retrying to connect to the master, i can tell that from the flume
> logs as below;
>
> 2012-12-19 11:11:42,126 WARN com.cloudera.flume.agent.MultiMasterRPC: Could
> not connect to any master nodes (tried 1: [flume-master-01:35872])
> 2012-12-19 11:11:52,140 WARN com.cloudera.flume.agent.MultiMasterRPC: Could
> not connect to any master nodes (tried 1: [flume-master-01:35872])
> 2012-12-19 11:12:02,152 WARN com.cloudera.flume.agent.MultiMasterRPC: Could
> not connect to any master nodes (tried 1: [flume-master-01:35872])
> 2012-12-19 11:12:12,162 WARN com.cloudera.flume.agent.MultiMasterRPC: Could
> not connect to any master nodes (tried 1: [flume-master-01:35872])
>
> from "top", I've seen a lot of java processes owned by flume. followed my
> instinct, I stopped flume agent, after a while, I run "top" on it, I still
> have a lot of processes owned by flume user.
>
> 12735 flume     20   0 1487m  26m  128 S  0.3  1.3   0:05.41 java
>
>
>
>
> 13828 flume     20   0 1487m   24   24 S  0.3  0.0   8:11.92 java
>
>
>
>
> 14442 flume     20   0 1487m    0    0 S  0.3  0.0   8:06.81 java
>
>
>
>
> 15033 flume     20   0 1487m  26m  164 S  0.3  1.3   0:04.29 java
>
>
>
>
> 15453 flume     20   0 1487m    4    4 S  0.3  0.0   8:07.55 java
>
>
>
>
> 15964 flume     20   0 1487m  27m  164 S  0.3  1.4   0:04.09 java
>
>
>
>
> 16098 flume     20   0 1487m    0    0 S  0.3  0.0   8:12.67 java
> ......................
>
> Why so many processes are generated here and they don't go away even when
> flume is stopped.
>
> --
> All the best,
> Shengjie Min