Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> kafka.utils.Utils


Copy link to this message
-
Re: kafka.utils.Utils
It would also be nice if the truly generic ZKUtils
(createEphemeralPathExpectConflict) and the kafka specific
(getConsumersPerTopic) were in separate classes.

On 03/25/2012 06:23 PM, Jay Kreps wrote:
> Hey guys,
>
> The Utils helpers are meant to be generic non-kafka-specific utility
> functions. Ideally it should not have dependencies outside java.* and
> scala.*. I notice a lot of code in there that is zookeeper-specific (my
> fault I think), that should really be a ZkUtils class. I also notice a lot
> of code that isn't properly helper code at all but actually very
> kafka-specific stuff.  This isn't a big deal, just a little hacky.
>
> Here are some suspicious methods:
>
>   def getCompressionCodec(props: Properties, codec: String):
> CompressionCodec
>   def getTopicRentionHours(retentionHours: String) : Map[String, Int]
>   def getTopicFlushIntervals(allIntervals: String) : Map[String, Int]
>   def getTopicPartitions(allPartitions: String) : Map[String, Int]
>   def getConsumerTopicMap(consumerTopicString: String) : Map[String, Int]
>   class SnapshotStats
>   class Stats
>   ...
>
> In general if we are putting domain-specific logic into generic static
> helpers like this it probably means we are missing some domain object to
> stick these on.
>
> I will file a ticket to clean these up, but I wanted to email around to try
> to explain the motivation.
>
> -Jay
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB