Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> kafka and ruby client, beginner questions


Copy link to this message
-
kafka and ruby client, beginner questions
I have some messages already send to kafka using the producer perf test
tool, but when I fire up irb I don't see any messages:

>irb
require 'rubygems'
require 'kafka'

c = Kafka::Consumer.new(:topic => "test")
m1 = c.consume
[]

So it says it is empty.

If I create a loop like:

c.loop do |msg|
  puts "rec"
  puts msg
end

And then I run the producer perf test, I see the messages outputted.

So it seems I am missing something, why does't it return any messages when
I simply connect and their are messages already in the system, or does
connecting to a specific topic push the offset to the end of the queue?

Now if I create a consumer, and then create a producer and send a message,
then when I call consumer.consume I get a message.
When I cat the /tmp/kafka-logs/test-0/000000...000.kafka file, I can see my
message and it seems to be in string format like:

6??helloworld

Shouldn't it be encoded?
+
Sebastian Eichner 2012-05-07, 08:07
+
S Ahmed 2012-05-07, 13:47
+
Sebastian Eichner 2012-05-07, 13:52
+
S Ahmed 2012-05-07, 13:57
+
Felix GV 2012-05-08, 18:24