Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> kafka and ruby client, beginner questions


Copy link to this message
-
kafka and ruby client, beginner questions
I have some messages already send to kafka using the producer perf test
tool, but when I fire up irb I don't see any messages:

>irb
require 'rubygems'
require 'kafka'

c = Kafka::Consumer.new(:topic => "test")
m1 = c.consume
[]

So it says it is empty.

If I create a loop like:

c.loop do |msg|
  puts "rec"
  puts msg
end

And then I run the producer perf test, I see the messages outputted.

So it seems I am missing something, why does't it return any messages when
I simply connect and their are messages already in the system, or does
connecting to a specific topic push the offset to the end of the queue?

Now if I create a consumer, and then create a producer and send a message,
then when I call consumer.consume I get a message.
When I cat the /tmp/kafka-logs/test-0/000000...000.kafka file, I can see my
message and it seems to be in string format like:

6??helloworld

Shouldn't it be encoded?
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB