Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Copy kafka data between servers?


Copy link to this message
-
Re: Copy kafka data between servers?
Sure I can collect the logs. However, the strange thing in my case is
that the zookeeper-server-stop.sh script (kill -SIGINT) didn't
actually kill the zookeeper process in my server. When you tried
shutting down zookeeper in your step, did you double check to see if
the zookeeper process had been killed or not? (ps aux | grep
"zookeeper")

thanks,

Jason

On Fri, Mar 1, 2013 at 5:40 PM, Neha Narkhede <[EMAIL PROTECTED]> wrote:
> I tried doing the following, but couldn't reproduce your issue -
>
> 1. Start zookeeper and kafka
> 2. Send some messages
> 3. Shutdown kafka and zookeeper
> 4. Start zookeeper and Kafka
> 5. Consume all messages
>
> Do you mind sending around the entire Kafka and Consumer logs ?
>
> Thanks,
> Neha
>
>
> On Thu, Feb 28, 2013 at 5:10 PM, Jason Huang <[EMAIL PROTECTED]> wrote:
>
>> Hello,
>>
>> I actually tried to load the data back with the same instance of kafka
>> on server A so the broker id must be the same. The reason I brought
>> this up at the first place is because we've had some issues
>> recognizing the messages on a server stop/restart. I was able to
>> reproduce our issue with following steps:
>>
>> (1) servers start:
>> nohup sudo /opt/kafka-0.8.0/bin/zookeeper-server-start.sh
>> /opt/kafka-0.8.0/config/zookeeper.properties >
>> /opt/kafka-0.8.0/data/kafka-logs/zook.out 2>&1 &
>> nohup sudo /opt/kafka-0.8.0/kafka-server-start.sh
>> /opt/kafka-0.8.0/config/server.properties >
>> /opt/kafka-0.8.0/data/kafka-logs/kafka.out 2>&1 &
>>
>> (2) create some messages
>>
>> (3) stop server
>> sudo /opt/kafka-0.8.0/bin/kafka-server-stop.sh
>> sudo /opt/kafka-0.8.0/bin/zookeeper-server-start.sh
>>
>> Notice that kafka-server-stop.sh uses kill -SIGTERM and
>> zookeeper-server-start.sh uses kill -SIGINT. My observation is that on
>> our server kill -SIGINT doesn't actually kill the zookeeper process.
>> (I can still see that running when I check the processes).
>>
>> Start from this state (running kill -SIGTERM for kafka server and kill
>> -SIGINT for zookeeper server), we restart the zookeeper and kafka
>> services:
>> nohup sudo /opt/kafka-0.8.0/bin/zookeeper-server-start.sh
>> /opt/kafka-0.8.0/config/zookeeper.properties >
>> /opt/kafka-0.8.0/data/kafka-logs/zook.out 2>&1 &
>> nohup sudo /opt/kafka-0.8.0/kafka-server-start.sh
>> /opt/kafka-0.8.0/config/server.properties >
>> /opt/kafka-0.8.0/data/kafka-logs/kafka.out 2>&1 &
>>
>> Then when we tried to fetch the messages from existing topics and
>> partitions, we get the following error:
>> WARN [KafkaApi-1] Error while responding to offset request
>> (kafka.server.KafkaApis)
>> kafka.common.UnknownTopicOrPartitionException: Topic topic_general
>> partition 0 doesn't exist on 1
>> at
>> kafka.server.ReplicaManager.getLeaderReplicaIfLocal(ReplicaManager.scala:163)
>>
>> I am not sure if anyone has experienced this before. It appears to me
>> that because kill -SIGINT didn't actually kill the previous zookeeper
>> process, running from that state messes up the partition/topic
>> information with zookeeper? And maybe because of that, copying the log
>> files and trying to reload them won't work (because somehow
>> information were corrupted)?
>>
>> thanks,
>>
>> Jason
>>
>>
>>
>> On Thu, Feb 28, 2013 at 12:10 PM, Neha Narkhede <[EMAIL PROTECTED]>
>> wrote:
>> > I'm guessing the  brokerid of the new broker is not the same as the old
>> one
>> > maybe ? This will work only if you copy the data over and maintain the
>> same
>> > broker id.
>> > If not, then this could be a bug.
>> >
>> > Thanks,
>> > Neha
>> >
>> >
>> > On Thu, Feb 28, 2013 at 3:21 AM, Jason Huang <[EMAIL PROTECTED]>
>> wrote:
>> >
>> >> I've started by only coping $log.dir from server A to server B. Both
>> >> server A and server B ran same version of kafka 0.8 with same
>> >> configuration files.
>> >>
>> >> However, after running kafka 0.8 on server B I get the following
>> >> exception when I tried to fetch the message:
>> >> 2013-02-28 05:56:35,851] WARN [KafkaApi-1] Error while responding to