Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # dev - kafka.utils.Utils


Copy link to this message
-
Re: kafka.utils.Utils
Chris Burroughs 2012-03-26, 01:28
It would also be nice if the truly generic ZKUtils
(createEphemeralPathExpectConflict) and the kafka specific
(getConsumersPerTopic) were in separate classes.

On 03/25/2012 06:23 PM, Jay Kreps wrote:
> Hey guys,
>
> The Utils helpers are meant to be generic non-kafka-specific utility
> functions. Ideally it should not have dependencies outside java.* and
> scala.*. I notice a lot of code in there that is zookeeper-specific (my
> fault I think), that should really be a ZkUtils class. I also notice a lot
> of code that isn't properly helper code at all but actually very
> kafka-specific stuff.  This isn't a big deal, just a little hacky.
>
> Here are some suspicious methods:
>
>   def getCompressionCodec(props: Properties, codec: String):
> CompressionCodec
>   def getTopicRentionHours(retentionHours: String) : Map[String, Int]
>   def getTopicFlushIntervals(allIntervals: String) : Map[String, Int]
>   def getTopicPartitions(allPartitions: String) : Map[String, Int]
>   def getConsumerTopicMap(consumerTopicString: String) : Map[String, Int]
>   class SnapshotStats
>   class Stats
>   ...
>
> In general if we are putting domain-specific logic into generic static
> helpers like this it probably means we are missing some domain object to
> stick these on.
>
> I will file a ticket to clean these up, but I wanted to email around to try
> to explain the motivation.
>
> -Jay
>