| clear query|facets|time |
Search criteria: .
Results from 51 to 60 from
92 (0.103s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: filter before flush to disk - Kafka - [mail # user]
|
|
...One downside is if my logic was messed up, I don't have a timeframe of rolling the logic back (which was one of the benefits of kafka's design choice of having messages kept around for x day...
|
|
|
Author: S Ahmed,
2012-05-15, 15:43
|
|
|
Re: filter before flush to disk - Kafka - [mail # user]
|
|
...What do you mean? " I think the direction we are going is instead to just let you co-locate this processing on the same box. This gives the isolation of separate processes and th...
|
|
|
Author: S Ahmed,
2012-05-15, 15:42
|
|
|
filter before flush to disk - Kafka - [mail # user]
|
|
...Would it be possible to filter the collection before it gets flush to disk? Say I am tracking page views per user, and I could perform a rollup before it gets flushed to disk (using a ...
|
|
|
Author: S Ahmed,
2012-05-15, 13:38
|
|
|
Re: since flushes are batched, is it still io intensive? - Kafka - [mail # user]
|
|
...then why have an operations page? j/k thanks Jay! Just a note, and I hope nobody takes it the wrong way, but I was lookign at the flume project and I really appreciated how much ...
|
|
|
Author: S Ahmed,
2012-05-12, 01:21
|
|
|
Re: since flushes are batched, is it still io intensive? - Kafka - [mail # user]
|
|
...Do you tune the o/s dedicated memory for page cache? Or that's all automatic.... It would be cool if linkedin posted some of their server level tweaks if that is critical to gett...
|
|
|
Author: S Ahmed,
2012-05-11, 16:16
|
|
|
Re: since flushes are batched, is it still io intensive? - Kafka - [mail # user]
|
|
...What about memory? I know you guys have 24GB of ram per server? Basically I'm juggling between going with a dedicated box (which has faster IO), or ec2 which has slower IO but ch...
|
|
|
Author: S Ahmed,
2012-05-11, 15:03
|
|
|
since flushes are batched, is it still io intensive? - Kafka - [mail # user]
|
|
...I was thinking (and after doing some tests on dedicated and ec2), would you still say kafka is io intensive? Considering writes are batched every x seconds, and you have a single kafka...
|
|
|
Author: S Ahmed,
2012-05-11, 14:13
|
|
|
Re: where does the in-memory log to file persistence take place - Kafka - [mail # dev]
|
|
...couldn't this be written in a way that it doesn't block? or limits the time, like it makes a copy of the reference, then replaces it when a newed up channel (synchronized). Oh, this is...
|
|
|
Author: S Ahmed,
2012-05-10, 04:35
|
|
|
kafka file persistance blocks, ramifications on payload sizes and flush timings - Kafka - [mail # user]
|
|
...On the dev list I was curious as to how kafka persists the in-memory to disk, and Jay responded with: "filechannel.force() always fully syncs the file to disk. This is done irrespectiv...
|
|
|
Author: S Ahmed,
2012-05-09, 15:19
|
|
|
Re: where does the in-memory log to file persistence take place - Kafka - [mail # dev]
|
|
...Slowly trying to understand it, have to wramp up on my scala. When the flush/sink occurrs, does it pull items of the collection 1 by 1 or does it do this in bulk somehow while locking ...
|
|
|
Author: S Ahmed,
2012-05-08, 20:44
|
|
|
|
