Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> [jira] [Updated] (KAFKA-685) ConsumerOffsetChecker does not work with 0.8


Copy link to this message
-
[jira] [Updated] (KAFKA-685) ConsumerOffsetChecker does not work with 0.8

     [ https://issues.apache.org/jira/browse/KAFKA-685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Maxime Brugidou updated KAFKA-685:
----------------------------------

    Description:
The ConsumerOffsetChecker does not work anymore with 0.8, this tool is very useful when used with the MirrorMaker.

Here is a patch to make it work with some cosmetic changes:

* script-friendly formatting (one line per partition)
* offsets do not correspond to bytes anymore (so the lag is in number of messages, not GiB)
* --broker-info optional option to print the broker list at the end (like the previous version)

Example:

bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group KafkaMirror --zkconnect zoo.example.org:2181
Group           Topic           Pid Offset          logSize         Lag             Owner
KafkaMirror     test       0  215385          215385          0               Some(KafkaMirror_broker01-1379350-71cf9117-0)
KafkaMirror     test       1  683564          683564          0               Some(KafkaMirror_broker03-1379351-71cf9117-0)
KafkaMirror     test2     0  176943          176943          0               Some(KafkaMirror_broker05-1379353-71cf91
  was:
The ConsoleOffsetChecker does not work anymore with 0.8, this tool is very useful when used with the MirrorMaker.

Here is a patch to make it work with some cosmetic changes:

* script-friendly formatting (one line per partition)
* offsets do not correspond to bytes anymore (so the lag is in number of messages, not GiB)
* --broker-info optional option to print the broker list at the end (like the previous version)

Example:

bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group KafkaMirror --zkconnect zoo.example.org:2181
Group           Topic           Pid Offset          logSize         Lag             Owner
KafkaMirror     test       0  215385          215385          0               Some(KafkaMirror_broker01-1379350-71cf9117-0)
KafkaMirror     test       1  683564          683564          0               Some(KafkaMirror_broker03-1379351-71cf9117-0)
KafkaMirror     test2     0  176943          176943          0               Some(KafkaMirror_broker05-1379353-71cf91
    
> ConsumerOffsetChecker does not work with 0.8
> --------------------------------------------
>
>                 Key: KAFKA-685
>                 URL: https://issues.apache.org/jira/browse/KAFKA-685
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 0.8
>            Reporter: Maxime Brugidou
>         Attachments: KAFKA-685-2.patch, KAFKA-685.patch
>
>
> The ConsumerOffsetChecker does not work anymore with 0.8, this tool is very useful when used with the MirrorMaker.
> Here is a patch to make it work with some cosmetic changes:
> * script-friendly formatting (one line per partition)
> * offsets do not correspond to bytes anymore (so the lag is in number of messages, not GiB)
> * --broker-info optional option to print the broker list at the end (like the previous version)
> Example:
> bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group KafkaMirror --zkconnect zoo.example.org:2181
> Group           Topic           Pid Offset          logSize         Lag             Owner
> KafkaMirror     test       0  215385          215385          0               Some(KafkaMirror_broker01-1379350-71cf9117-0)
> KafkaMirror     test       1  683564          683564          0               Some(KafkaMirror_broker03-1379351-71cf9117-0)
> KafkaMirror     test2     0  176943          176943          0               Some(KafkaMirror_broker05-1379353-71cf91

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB