Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Re: Consumer re-design proposal


Copy link to this message
-
Re: Consumer re-design proposal
Chris Burroughs 2012-06-20, 02:30
+list

The sentiment makes sense.  I just know with the people I talked to the
(arguably deceptive) simplicity of the SimpleConsumer was a significant
selling point.

On 2012-06-19 01:18, Jay Kreps wrote:
> I think the hope was to have a single API and make each piece of
> functionality optional. E.g. you could have manual offset management OR you
> could have manual partition assignment OR both. You are right that if you
> disable everything you are left with something not much better than the
> current simple consumer except that it would support leadership changes (a
> la replication). But somehow having a single public API seems like a good
> practice just to have a single place where monitoring, testing, etc.
>
> -Jay
>
> On Mon, Jun 18, 2012 at 7:00 PM, Chris Burroughs
> <[EMAIL PROTECTED]>wrote:
>
>> On 06/12/2012 12:59 PM, Jay Kreps wrote:
>>>    2. Try to replace the "simple consumer" and "high level consumer"
>> with a
>>>    single, general interface that has all the advantages of both.
>>
>> I've read through the wiki pages but think I'm missing the forrest for
>> the trees.
>>
>> For a consumer that wants "Manual partition assignment" and "Manual
>> offset management", what does the proposed offer over the existing
>> SimpleConsumer?
>>
>