Can we see count of messages or bytes of a topic present in kafka using jconsole or any other way ? Jmx metric "BytesInperSec" doesn't show data stored in kafka. As per my understanding: "BytesInperSec" is count of bytes in a topic that have been received by the broker since the broker process started. It is just counter value that get incremented each time a new byte is received by broker. The value is reset to 0 when broker is restarted. Also the Byte count will not be decremented if any bytes are deleted from broker.
Is there any way to check count of bytes stored in kafka even after kafka is restarted?
Not a direct way to see this, but we use Kafka 0.7.2 and monitor the disk space on it, so that's one way to see *roughly* what is happening. If you use something like SPM to monitor your Kafka nodes (see http://blog.sematext.com/2013/10/16/announcement-spm-performance-monitoring-for-kafka/) and if broker stores data on the disk partition that nothing else writes to, then you can look at disk usage and read/write rates for that specific partition and thus get the more precise numbers.
If there's a more direct way to see/get this number, I'm all eyeballs!
Otis Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Thu, Dec 26, 2013 at 11:37 PM, Barto Rael <[EMAIL PROTECTED]> wrote:
We are using kafka 0.8.0. I was looking for some jmx metrix that can give count of messages/bytes stored in kafka.
If we look into data stores like hbase we can see how many messages are stored in hbase or if we look into search engine like elasticsearch then also we can see how many messages are stored/indexed in elasticsearch. In similar way i was expecting that there should be direct way to see count of messages or bytes stored in kafka without using any external tool.
Is there any direct way to see count of messages or bytes stored in kafka ? On Sat, Dec 28, 2013 at 3:32 AM, Otis Gospodnetic < [EMAIL PROTECTED]> wrote: