Kafka, mail # user - Re: Replacing brokers in a cluster (0.8) - 2013-07-22, 17:50
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
Re: Replacing brokers in a cluster (0.8)
Here's a ruby cli that you can use to replace brokers...it shells out to
the kafka-reassign-partitions.sh tool after figuring out broker lists from
zk. Hope its useful.
#!/usr/bin/env ruby

require 'excon'
require 'json'
require 'zookeeper'

def replace(arr, o, n)
  arr.map{|v| v == o ? n : v }

if ARGV.length != 4
  puts "Usage: bundle exec bin/replace-instance zkstr topic-name
old-broker-id new-broker-id"
  zkstr = ARGV[0]
  zk = Zookeeper.new(zkstr)
  topic = ARGV[1]
  old = ARGV[2].to_i
  new = ARGV[3].to_i
  puts "Replacing broker #{old} with #{new} on all partitions of topic

  current = JSON.parse(zk.get(:path => "/brokers/topics/#{topic}")[:data])
  replacements_array = []
  replacements = {"partitions" => replacements_array}
  current["partitions"].each { |partition, brokers|
replacements_array.push({"topic" => topic, "partition" => partition.to_i,
"replicas" => replace(brokers, old, new)}) }

  replacement_json = JSON.generate(replacements)
  file = "/tmp/replace-#{topic}-#{old}-#{new}"
  if File.exist?(file)
    File.delete file
  File.open(file, 'w') { |f| f.write(replacement_json) }

  puts "./bin/kafka-reassign-partitions.sh --zookeeper #{zkstr}
  system "./bin/kafka-reassign-partitions.sh --zookeeper #{zkstr}

On Mon, Jul 22, 2013 at 10:40 AM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB