Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # dev >> Re: Review Request 15201: address more review comments


+
Jun Rao 2013-11-12, 16:34
+
Joel Koshy 2013-11-13, 02:09
Copy link to this message
-
Re: Review Request 15201: address more review comments


> On Nov. 13, 2013, 2:09 a.m., Joel Koshy wrote:
> > Looks good overall - this is definitely a useful tool to have. I'm posting comments so far but I will come back to it tomorrow to finish up.
> >
> > So what would be the recommended procedure when the tool reports an inconsistency? Would operations have to go in, stop the brokers and truncate the log segments to the last segments that are verified to be consistent? If so, it would be convenient if the tool could issue another OffsetRequest to report the offset boundaries of the last consistent segments so you don't have to work too hard to determine the segment file.
> >
> > Also, unless I'm reading it incorrect additional caveats are that the tool can only keep working so long as: partitions are not being added (or reassigned); logs are not being compacted.
> >

The tool is meant for detecting inconsistent replicas during testing so that we can fix the bugs. Fixing inconsistent replicas in operation will be hard and will require manual work.

The limitation of the tools is that it won't pick up newly created partitions or partitions moved by the reassignment tool. For those cases, we can just restart the tool.
> On Nov. 13, 2013, 2:09 a.m., Joel Koshy wrote:
> > config/tools-log4j.properties, line 16
> > <https://reviews.apache.org/r/15201/diff/3/?file=382689#file382689line16>
> >
> >     Is this needed?

removed.
> On Nov. 13, 2013, 2:09 a.m., Joel Koshy wrote:
> > core/src/main/scala/kafka/tools/ReplicaVerificationTool.scala, line 98
> > <https://reviews.apache.org/r/15201/diff/3/?file=382691#file382691line98>
> >
> >     This is becoming my most frequent review comment :)
> >     Can you use CommandLineUtils.checkRequiredArgs?

done.
> On Nov. 13, 2013, 2:09 a.m., Joel Koshy wrote:
> > core/src/main/scala/kafka/tools/ReplicaVerificationTool.scala, line 106
> > <https://reviews.apache.org/r/15201/diff/3/?file=382691#file382691line106>
> >
> >     Any reason to not use kafka.consumer.Whitelist and the isTopicAllowed API that it provides?

Yes, it can be reused.
- Jun
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15201/#review28744
-----------------------------------------------------------
On Nov. 12, 2013, 4:34 p.m., Jun Rao wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15201/
> -----------------------------------------------------------
>
> (Updated Nov. 12, 2013, 4:34 p.m.)
>
>
> Review request for kafka.
>
>
> Bugs: KAFKA-1117
>     https://issues.apache.org/jira/browse/KAFKA-1117
>
>
> Repository: kafka
>
>
> Description
> -------
>
> kafka-1117; fix 3
>
>
> kafka-1117; fix 2
>
>
> kafka-1117; fix 1
>
>
> kafka-1117
>
>
> Diffs
> -----
>
>   config/tools-log4j.properties 79240490149835656e2a013a9702c5aa41c104f1
>   core/src/main/scala/kafka/api/OffsetResponse.scala 08dc3cd3d166efba6b2b43f6e148f636b175affe
>   core/src/main/scala/kafka/tools/ReplicaVerificationTool.scala PRE-CREATION
>
> Diff: https://reviews.apache.org/r/15201/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jun Rao
>
>
 
+
Jun Rao 2013-11-14, 16:23