Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # dev - Re: Review Request 15674: add unit test for aborting during cleaning


Copy link to this message
-
Re: Review Request 15674: add unit test for aborting during cleaning
Jun Rao 2014-01-03, 20:33


> On Jan. 3, 2014, 4:22 p.m., Jay Kreps wrote:
> > core/src/main/scala/kafka/log/LogCleaner.scala, line 262
> > <https://reviews.apache.org/r/15674/diff/4/?file=413443#file413443line262>
> >
> >     Would these be better as just InProgress, Aborted, and Paused?

My feeling is that in the future there could be similar states in other components such as the controller. So, having a longer name will make it clear which state it really is.
> On Jan. 3, 2014, 4:22 p.m., Jay Kreps wrote:
> > core/src/main/scala/kafka/log/LogCleaner.scala, line 298
> > <https://reviews.apache.org/r/15674/diff/4/?file=413443#file413443line298>
> >
> >     It seems like the logic for chosing logs is not part of the state management, is there any way to seperate this out?

That's a bit hard. grabFilthestLog() accesses checkpoints and inProgress, both of which need to be protected by the lock.
> On Jan. 3, 2014, 4:22 p.m., Jay Kreps wrote:
> > core/src/test/scala/unit/kafka/log/CleanerTest.scala, line 98
> > <https://reviews.apache.org/r/15674/diff/4/?file=413448#file413448line98>
> >
> >     Do we have coverage for cleaning + truncation simultaneously?

Added a unit test for aborting during log cleaning.
- Jun
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15674/#review31134
-----------------------------------------------------------
On Jan. 3, 2014, 8:27 p.m., Jun Rao wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15674/
> -----------------------------------------------------------
>
> (Updated Jan. 3, 2014, 8:27 p.m.)
>
>
> Review request for kafka.
>
>
> Bugs: KAFKA-1074
>     https://issues.apache.org/jira/browse/KAFKA-1074
>
>
> Repository: kafka
>
>
> Description
> -------
>
> add test for aborting during cleaning
>
>
> checkpoint recovery points in truncateFullyAndStartAt()
>
>
> move cleaning states and locks to a separate class
>
>
> minor fix 2
>
>
> minor fix
>
>
> remove unused var and exception
>
>
> support pause/resume log clean and remove OptimisticLockFailureException
>
>
> kafka-1074; minor fix3
>
>
> kafka-1074; minor fix2
>
>
> kafka-1074; minor fix
>
>
> kafka-1074; better synchronization with log cleaner
>
>
> kafka-1074; fix 4
>
>
> kafka-1074; fix 3
>
>
> kafka-1074; fix 2
>
>
> kafka-1074
>
>
> Diffs
> -----
>
>   core/src/main/scala/kafka/cluster/Partition.scala 5c9307d71641ccc6c09a54b69d5aa2b4bc2a4cde
>   core/src/main/scala/kafka/common/OptimisticLockFailureException.scala 0e69110e71667ad75a460534f4422a7b6ec1cdc6
>   core/src/main/scala/kafka/common/ThreadShutdownException.scala PRE-CREATION
>   core/src/main/scala/kafka/log/Log.scala beda421b558544196379bd9ab7855cea7614e8e3
>   core/src/main/scala/kafka/log/LogCleaner.scala ccde2abd99d2204775e4d3e9836aca34eb6747a3
>   core/src/main/scala/kafka/log/LogManager.scala 81be88aa618ed5614703d45a0556b77c97290085
>   core/src/main/scala/kafka/server/ReplicaFetcherThread.scala 715845b167c44268bd7e4b76dfb69199bfb2fad4
>   core/src/main/scala/kafka/server/ReplicaManager.scala 242c18d47828b7c5e6b1fc219a0f1199fb1f9512
>   core/src/test/scala/unit/kafka/admin/AdminTest.scala 52d35a3ccc86a66acb57c1138d4f2fea8f4ca4b0
>   core/src/test/scala/unit/kafka/log/CleanerTest.scala 5a312bf0803c1df4636e2e64ba83036a4e8e92dd
>
> Diff: https://reviews.apache.org/r/15674/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jun Rao
>
>