[ https://issues.apache.org/jira/browse/KAFKA-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Neha Narkhede updated KAFKA-671:
--------------------------------

    Labels: bugs p1  (was: )
    
> DelayedProduce requests should not hold full producer request data
> ------------------------------------------------------------------
>
>                 Key: KAFKA-671
>                 URL: https://issues.apache.org/jira/browse/KAFKA-671
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.8
>            Reporter: Joel Koshy
>            Assignee: Sriram Subramanian
>            Priority: Blocker
>              Labels: bugs, p1
>             Fix For: 0.8.1
>
>         Attachments: outOfMemFix-v1.patch
>
>
> Per summary, this leads to unnecessary memory usage.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

 

     [ https://issues.apache.org/jira/browse/KAFKA-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sriram Subramanian updated KAFKA-671:
-------------------------------------

    Attachment: outOfMemFix-v2.patch

- Ignore the changes in bin and system tests
- this change is cleaner and a lot safer.
  - the data is now an mutable map
  - we just cache the topicpartition - message size in producer request
  - we clear the map after handling the request
  - the toString implementation of produceRequest uses the cached map instead of the data
  - the byteBuffer in RequestChannel.Request is now made private and is set to null after deserialization.
  - update test cases to work with these changes
  - I will be updating the thread with how the heap characteristics look before and after this change.
                
> DelayedProduce requests should not hold full producer request data
> ------------------------------------------------------------------
>
>                 Key: KAFKA-671
>                 URL: https://issues.apache.org/jira/browse/KAFKA-671
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.8
>            Reporter: Joel Koshy
>            Assignee: Sriram Subramanian
>            Priority: Blocker
>              Labels: bugs, p1
>             Fix For: 0.8.1
>
>         Attachments: outOfMemFix-v1.patch, outOfMemFix-v2.patch
>
>
> Per summary, this leads to unnecessary memory usage.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

 

     [ https://issues.apache.org/jira/browse/KAFKA-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Neha Narkhede updated KAFKA-671:
--------------------------------

    Status: Patch Available  (was: Open)
    
> DelayedProduce requests should not hold full producer request data
> ------------------------------------------------------------------
>
>                 Key: KAFKA-671
>                 URL: https://issues.apache.org/jira/browse/KAFKA-671
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.8
>            Reporter: Joel Koshy
>            Assignee: Sriram Subramanian
>            Priority: Blocker
>              Labels: bugs, p1
>             Fix For: 0.8.1
>
>         Attachments: outOfMemFix-v1.patch, outOfMemFix-v2.patch, outOfMemFix-v2-rebase.patch, outOfMemFix-v3.patch
>
>
> Per summary, this leads to unnecessary memory usage.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira