Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> can producers(from same system) send messages to separate broker systems?


Copy link to this message
-
Re: can producers(from same system) send messages to separate broker systems?
The feature I mentioned is only available on 0.8. In 0.7.2, you can tweak
producer batch size and the flush interval on the broker for the high
priority topics. Note that setting those too low will have performance
implications.

Thanks,
Neha
On May 17, 2013 2:25 PM, "Neha Narkhede" <[EMAIL PROTECTED]> wrote:

> Do you have any tests that measure that your high priority data is being
> delayed ? Assuming you are using 0.8, the end to end latency can be reduced
> by tuning some configs on the consumer (fetch.min.bytes, fetch.wait.max.ms
> ).
> The defaults for these configs are already tuned for low latency though.
>
> Thanks,
> Neha
>
>
> On Tue, May 14, 2013 at 11:46 AM, Chitra Raveendran <
> [EMAIL PROTECTED]> wrote:
>
>> Thanks for the reply,
>>
>> I have a 3 node kafka cluster, and i have 2 topics( one of very high
>> priority and other normal data), i need to transmit the normal data to two
>> brokers in the cluster , and the high priority data directly to the 3rd
>> broker.
>>
>> This is so that my high priority data has a clear path and can be
>> transmitted without any delay at all.
>>
>> What should I do to achieve this ? Just specifying appropriate broker.list
>> for each producer , is enough?
>>
>>
>> On Tue, May 14, 2013 at 9:11 PM, Neha Narkhede <[EMAIL PROTECTED]
>> >wrote:
>>
>> > Yes there can be. You just need to make sure they are configured with
>> > separate broker.list.
>> >
>> > Thanks,
>> > Neha
>> > On May 14, 2013 5:28 AM, "Andrea Gazzarini" <[EMAIL PROTECTED]
>> >
>> > wrote:
>> >
>> > > Of course...what is the problem? Or maybe you're missing some other
>> > > constraint?
>> > >
>> > > On 05/14/2013 02:20 PM, Chitra Raveendran wrote:
>> > >
>> > >> Hi
>> > >>
>> > >> Can  there be 2 producers in the same server, both sending their own
>> > >> separate topics to separate broker systems?
>> > >>
>> > >>
>> > >
>> >
>>
>>
>>
>> --
>> Chitra Raveendran
>> Data Scientist
>> *Flutura Business Solutions Pvt. Ltd*
>> Tel     : +918197563660
>> email  : [EMAIL PROTECTED]
>>
>
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB