| clear
query|
facets|
time |
Search criteria: .
Results from 1 to 10 from
49212 (0.0s).
|
|
|
Loading phrases to help you
refine your search...
|
|
[KAFKA-5337] Partition assignment strategy that distributes lag evenly across consumers in each group -
Kafka - [issue]
|
| ...Existing partition assignment strategies (RangeAssignor and RoundRobinAssignor) do not account for the current consumer group lag on each partition. This can result in sub-optimal assi... |
|
|
|
|
|
|
|
[KAFKA-4304] Extend Interactive Queries for return latest update timestamp per key -
Kafka - [issue]
|
| ...Currently, when querying state store, it is not clear when the key was updated last. The ides of this JIRA is to make the latest update timestamp for each key-value-pair of the state store a... |
|
|
|
|
|
|
|
[KAFKA-5251] Producer should drop queued sends when transaction is aborted -
Kafka - [issue]
|
| ...As an optimization, if a transaction is aborted, we can drop any records which have not yet been sent to the brokers. However, to avoid the sequence number getting out of sync, we need to co... |
|
|
|
|
|
|
|
[expand - 1 more]
[collapse]
-
Kafka Streams - possible to know partition? -
Kafka - [mail # user]
|
| ...I just added this FAQ:https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-HowcanIaccessrecordmetadata?-MatthiasOn 5/28/17 12:36 PM, Srimanth G wrote:> Hello,> When using Kafka St... |
|
|
|
|
| |
|
Kafka Streams - possible to know partition? - Kafka - [mail # user]
|
| ...Hello,When using Kafka Streams, is it possible to know which partition the byte[]are coming from?Either in the Serde classes, or any other user accessible functions I wouldlike to know the p... |
|
|
|
|
|
|
|
[expand - 4 more]
[collapse]
-
Producer Async Issue -
Kafka - [mail # user]
|
| ...I think setting max.block.ms=0 does what you want.-JayOn Sat, May 27, 2017 at 12:40 PM, Abhimanyu Nagrath <[EMAIL PROTECTED]> wrote:> HI Hans,>> What exactly I meant by asynch... |
|
|
|
|
| |
|
Producer Async Issue - Kafka - [mail # user]
|
| ...So you just want to not block and just silently throw away the messages and lose them forever? Kafka's persistence is all in the broker so there is no client side storing of data on disk. Th... |
|
|
|
|
|
Producer Async Issue - Kafka - [mail # user]
|
| ...HI Hans,What exactly I meant by asynchronous is that when my Kafka broker is downand I am trying to produce the message . It is getting stuck till theconfigured max.block.ms and after that f... |
|
|
|
|
|
Producer Async Issue - Kafka - [mail # user]
|
| ...The producer is asynchronous (assuming you mean the Java Producer)https://kafka.apache.org/0102/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html-hans> On May 27, 20... |
|
|
|
|
|
Producer Async Issue - Kafka - [mail # user]
|
| ...Hi,I am using Kafka 0.10.2 single node cluster and I want to knowKafka producer completely asynchronous. So what configuration I need tochange in order to make producer completely asynchrono... |
|
|
|
|
|
|
|
[KAFKA-5343] MockProducer should not handle send/ProducerRecord after close -
Kafka - [issue]
|
| ...Currently MockProducer implements close with void, i.e. it just does nothing on close.Assuming production code built for "max.in.flight.requests.per.connection=1" which transforms a list of ... |
|
|
|
|
|
|
|
[expand - 16 more]
[collapse]
-
[DISCUSS]: KIP-161: streams record processing exception handlers -
Kafka - [mail # user]
|
| ...+1On 26.05.2017 18:36, Damian Guy wrote:> In that case, though, every access to that key is doomed to failure as the> database is corrupted. So i think it should probably die in a stea... |
|
|
|
|
| |
|
[DISCUSS]: KIP-161: streams record processing exception handlers - Kafka - [mail # user]
|
| ...Hi Eno,that does make a lot more sense to me. when you pop stuff out of a topic you can at least put the coordinates (topicpartition,offset) additionally into the log wich is probably kinda ... |
|
|
|
|
|
[DISCUSS]: KIP-161: streams record processing exception handlers - Kafka - [mail # user]
|
| ...In that case, though, every access to that key is doomed to failure as thedatabase is corrupted. So i think it should probably die in a steaming heapat that point!On Fri, 26 May 2017 at 17:3... |
|
|
|
|
|
[DISCUSS]: KIP-161: streams record processing exception handlers - Kafka - [mail # user]
|
| ...Hi Damian,I was thinking of cases when there is bit-rot on the storage itself and we get a malformed record that cannot be de-serialized. There is an interesting intersection here with CRCs ... |
|
|
|
|
|
[DISCUSS]: KIP-161: streams record processing exception handlers - Kafka - [mail # user]
|
| ...Eno,Under what circumstances would you get a deserialization exception from thestate store? I can only think of the case where someone has provided a baddeserializer to a method that creates... |
|
|
|
|
|
[DISCUSS]: KIP-161: streams record processing exception handlers - Kafka - [mail # user]
|
| ...See latest reply to Jan's note. I think I unnecessarily broadened the scope of this KIP to the point where it sounded like it handles all sorts of exceptions. The scope should be strictly li... |
|
|
|
|
|
[DISCUSS]: KIP-161: streams record processing exception handlers - Kafka - [mail # user]
|
| ...Hi Jan,You're right. I think I got carried away and broadened the scope of this KIP beyond it's original purpose. This handler will only be there for deserialization errors, i.e., "poison pi... |
|
|
|
|
|
[DISCUSS]: KIP-161: streams record processing exception handlers - Kafka - [mail # user]
|
| ..."bad" for this case would mean, that we got an`DeserializationException`. I am not sure if any other processing errorshould be covered?@Eno: this raises one one question. Might it be better ... |
|
|
|
|
|
[DISCUSS]: KIP-161: streams record processing exception handlers - Kafka - [mail # dev]
|
| ...> On May 26, 2017, at 5:13 AM, Eno Thereska wrote:> > >> >> >> With regard to `DeserializationException`, do you thing it might make>> sense to have a ... |
|
|
|
|
|
[DISCUSS]: KIP-161: streams record processing exception handlers - Kafka - [mail # user]
|
| ...About `LogAndThresholdExceptionHandler`:If the handler needs to keep track of number of failed messages, than itbecomes stateful -- not sure if we should do that. But maybe we canintroduce 2... |
|
|
|
|
|
|
|
[KAFKA-5342] Distinguish abortable failures in transactional producer -
Kafka - [issue]
|
| ...The transactional producer distinguishes two classes of user-visible errors:1. Abortable errors: these are errors which are fatal to the ongoing transaction, but which can be successfully ab... |
|
|
|
|
|
|
|
[expand - 8 more]
[collapse]
-
[DISCUSS]: KIP-159: Introducing Rich functions to Streams -
Kafka - [mail # dev]
|
| ...After your response on KIP-149 related with ValueTransformerSupplier,everythingyou mentioned now makes complete sense. Thanks for clarification.Just a note: We will have additional (to KIP-1... |
|
|
|
|
| |
|
[DISCUSS]: KIP-159: Introducing Rich functions to Streams - Kafka - [mail # dev]
|
| ...Thanks for you comments Jeyhun,I agree about the disadvantages. Only the punctuation part is somethingI don't buy. IMHO, RichFunctions should not allow to register and usepunctuation. If you... |
|
|
|
|
|
[DISCUSS]: KIP-159: Introducing Rich functions to Streams - Kafka - [mail # dev]
|
| ...Hi,Thanks for your comments. I will refer the overall approach as richfunctions until we find a better name.I think there are some pros and cons of the approach you described.Pros is that it... |
|
|
|
|
|
[DISCUSS]: KIP-159: Introducing Rich functions to Streams - Kafka - [mail # dev]
|
| ...I confess, the term is borrowed from Flink :)Personally, I never thought about it, but I tend to agree with Michal. Ialso want to clarify, that the main purpose is the ability to accessrecor... |
|
|
|
|
|
[DISCUSS]: KIP-159: Introducing Rich functions to Streams - Kafka - [mail # dev]
|
| ...Hi Michal,Thanks for your comments. I see your point and I agree with it. However, Idon't have a better idea for naming. I checked MR source code. There it isused JobConfigurable and Closabl... |
|
|
|
|
|
[DISCUSS]: KIP-159: Introducing Rich functions to Streams - Kafka - [mail # dev]
|
| ...Hi Jeyhun,I understand your argument about "Rich" in RichFunctions. Perhaps I'm just being too puritan here, but let me ask this anyway:What is it that makes something a function? To me a fu... |
|
|
|
|
|
[DISCUSS]: KIP-159: Introducing Rich functions to Streams - Kafka - [mail # dev]
|
| ...Hi Michal,Thanks for your comments.To me at least it feels strange that something is called a function yet> doesn't follow the functional interface definition of having just one> abstr... |
|
|
|
|
|
[DISCUSS]: KIP-159: Introducing Rich functions to Streams - Kafka - [mail # dev]
|
| ...Hi Jeyhun,I'd like to understand better the premise of RichFunctions and why |init(Some params)|,|close() |are said to be needed.To me at least it feels strange that something is called a fu... |
|
|
|
|
|
[DISCUSS]: KIP-159: Introducing Rich functions to Streams - Kafka - [mail # dev]
|
| ...Dear community,As we discussed in KIP-149 [DISCUSS] thread [1], I would like to initiateKIP for rich functions (interfaces) [2].I would like to get your comments.[1]http://search-hadoop.com/... |
|
|
|
|
|
|
|
[expand - 41 more]
[collapse]
-
[DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner -
Kafka - [mail # dev]
|
| ...Thanks for clarification Matthias, now everything is clear.On Sun, May 28, 2017 at 6:21 PM Matthias J. Sax wrote:> I don't think we can drop ValueTransformerSupplier. If you don't have an... |
|
|
|
|
| |
|
[DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner - Kafka - [mail # dev]
|
| ...I don't think we can drop ValueTransformerSupplier. If you don't have ansupplier, you only get a single instance of your function. But for astateful transformation, we need multiple instance... |
|
|
|
|
|
[DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner - Kafka - [mail # dev]
|
| ...Hi,I updated KIP.Just to avoid misunderstanding, I meant deprecating ValueTransformerSupplierand I am ok with ValueTransformer.So instead of using ValueTransformerSupplier can't we dir... |
|
|
|
|
|
[DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner - Kafka - [mail # dev]
|
| ...Thanks Jeyhun.About ValueTransformer: I don't think we can remove it. Note,ValueTransformer allows to attach a state and also allows to registerpunctuations. Both those features will not be ... |
|
|
|
|
|
[DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner - Kafka - [mail # dev]
|
| ...Hi Matthias,Thanks for your comments.I tested the deep copy approach. It has significant overhead. Especiallyfor "light" and stateless operators it slows down the topologysignificantly (>... |
|
|
|
|
|
[DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner - Kafka - [mail # dev]
|
| ...One more question:Should we add any of - InitizialierWithKey - ReducerWithKey - ValueTransformerWithKeyTo get consistent/complete API, it might be a good idea. Any thoughts?-M... |
|
|
|
|
|
[DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner - Kafka - [mail # dev]
|
| ...Jeyhun,I was just wondering if you did look into the key-deep-copy idea wediscussed. I am curious to see what the impact might be.-MatthiasOn 5/20/17 2:03 AM, Jeyhun Karimov wrote:> Hi,&g... |
|
|
|
|
|
[DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner - Kafka - [mail # dev]
|
| ...Hi,Thanks for your comments. I rethink about including rich functions intothis KIP.I think once we include rich functions in this KIP and then fixProcessorContext in another KIP and incorpor... |
|
|
|
|
|
[DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner - Kafka - [mail # dev]
|
| ...With the current KIP, using ValueMapper and ValueMapperWithKeyinterfaces, RichFunction seems to be an independent add-on. To fix theoriginal issue to allow key access, RichFunctions are not ... |
|
|
|
|
|
[DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner - Kafka - [mail # dev]
|
| ...Hi Damian,Thanks for your comments. I think providing to users *interface* ratherthan *abstract class* should be preferred (Matthias also raised this issue), anyway I changed the correspondi... |
|
|
|
|
|
|
|
|