Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - question about broker


Copy link to this message
-
Re: question about broker
Chris Burroughs 2012-06-27, 21:43
This is pretty fundamental to the contract the the broker provides.
Without changing code you could use some LD_PRELOAD hackery to disable
fsync and then set log.flush.interval=1 to one.  But that's really not
recommended.

On 06/20/2012 01:11 PM, R S wrote:
> Thank you Neha ,
>
> Just to clarify , when you say high watermark , my understanding of that is
> the maximum offset which is available to consumer .
> The reason iam asking about this is , we want to use kafka for ultra fast
> messaging to do budget updates in shortest possible time , hence we just
> want to send messages to our aggregation service as fast as possible so was
> curious if there is a way we can just skip write to the disk and  just
> support Pub-Sub.
>
> regards,
> rks
>
> On Wed, Jun 20, 2012 at 10:34 PM, Neha Narkhede <[EMAIL PROTECTED]>wrote:
>
>> The broker maintains a high watermark for each topic partition log.
>> This high watermark moves only after the messages are written to disk.
>> And, the consumer can only consume data exposed by this high
>> watermark.
>>
>> Thanks,
>> Neha
>>
>> On Wed, Jun 20, 2012 at 10:02 AM, R S <[EMAIL PROTECTED]> wrote:
>>> Hi,
>>>
>>> When producer sends messages to broker on a particular topic and at the
>>> sametime  suppose consumer is fetching messages from the same topic. Will
>>> broker be sending message to consumer only after writing the message to
>>> disk or would writing to disk will happen in a asynchronous way.
>>>
>>>
>>> regards,
>>> rks
>>
>