Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - runit and Kafka


Copy link to this message
-
Re: runit and Kafka
Guy Doulberg 2012-07-08, 07:12
Acttully, I found out that there are 3 process that are created when
running:

/opt/kafka/bin/kafka-server-**start.sh /etc/kafka/server.properties

guyd      4883  4658  0 09:36 pts/5    00:00:00 /bin/bash ./kafka-server-start.sh ../config/server.properties
guyd      4885  4883  0 09:36 pts/5    00:00:00 /bin/bash ./kafka-run-class.sh kafka.Kafka ../config/server.properties
guyd      4887  4885  2 09:36 pts/5    00:00:00 java -Xmx512M -server -Dlog4j.configuration=file:./../config/log4j.properties -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=9999 -cp :./../project/boot/scala-2.8.0/lib/scala-compiler.jar:./../project/boot/scala-2.8.0/lib/scala-library.jar:./../core/target/scala_2.8.0/kafka-0.7.0.jar:./../core/lib/zkclient-20110412.jar:./../core/lib_managed/scala_2.8.0/compile/jopt-simple-3.2.jar:./../core/lib_managed/scala_2.8.0/compile/log4j-1.2.15.jar:./../core/lib_managed/scala_2.8.0/compile/zookeeper-3.3.3.jar kafka.Kafka ../config/server.properties
Why do we need these 3 processes?

On 07/08/2012 09:35 AM, Guy Doulberg wrote:
> Hi Jay,
> Thanks for your replay.
>
> Not exactly,
>
> I can see that the pid the runit save, is deferent from the pid that I
> acctually have running on the machine , could it be that the launcher of
> the kafka broker forks another process with a different PID?
>
> Thanks.
>
> The only way I could kill the process is by using kill -9 , I suspect
> that runit doesn't use the
> On 07/05/2012 10:19 PM, Jay Kreps wrote:
>> I think what you are saying is that you don't have the process id for the
>> broker so you don't have the right thing to kill?
>>
>> Two solutions I know:
>> 1. Change your script to store the process id somewhere, and use that for
>> the kill
>> 2. Use pkill and just kill all kafka instances (only works if there is only
>> one broker per node)
>>
>> -Jay
>>
>> On Thu, Jul 5, 2012 at 2:24 AM, Guy Doulberg <[EMAIL PROTECTED]>wrote:
>>
>>> Hi guys,
>>>
>>> In my deployment of Kafka I am using runit to control the service of the
>>> kafka broker.
>>>
>>> In the run script I have this line:
>>> #!/bin/sh
>>> exec 2>&1
>>>
>>> exec /opt/kafka/bin/kafka-server-**start.sh /etc/kafka/server.properties
>>>
>>>
>>>
>>> It works fine, if I want to start the broker, but if I want to stop or
>>> restart the process, the kafka broker doesn't get kill...
>>>
>>> So eventually the old kafka broker prevents from the new broker to start
>>> (Because the ports are already binded).
>>>
>>>
>>> Of course the running the same line of code in the shell and then ctrl-c
>>> ing works good (as you know kafka broker is not a daemon)
>>>
>>> I was wondering if someone has encountered this phenomena and solved it.
>>>
>>> Thanks,
>>>
>>> Guy Doulberg
>>>
>