Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # dev - kafka pull request: Unmap before resizing

lizziew 2013-07-25, 15:05
Copy link to this message
Re: kafka pull request: Unmap before resizing
Jay Kreps 2013-07-25, 16:42
Hi, can you file a JIRA and attach the patch there? This does the Apache
copyright stuff...

Instructions here:

This seems like a good thing to have. Is there a more portable way to do
this? Obviously this would not work on a non-sun JVM. I think the ideal
would be to add a tryUnmap() method that attempts the case but swallows any

Also I would like to fully understand the Sun's objection to allowing user
code to unmap buffers. In that ticket on the previous thread on
Windows+mmap there was a mention that forcing the unmap was somehow a
security hole. I don't understand this. Unix obviously supports unmap and
there is no security issue with that as normal process-level memory
isolation protects processes from reading each others memory or files.  I
suspect maybe this would only be true in cases where you are trying to
isolate different code in the same JVM which is not a valid use case. But I
want to fully understand the issue before we add something someone claims
is unsafe.

On Thu, Jul 25, 2013 at 12:07 AM, lizziew <[EMAIL PROTECTED]> wrote:

> GitHub user lizziew opened a pull request:
>     https://github.com/apache/kafka/pull/6
>     Unmap before resizing
>     While I was studying how MappedByteBuffer works, I saw a sharing
> runtime exception on Windows. I applied what I learned to generate a patch
> which uses an internal open JDK API to solve this problem.
>     ---
>     Caused by: java.io.IOException: The requested operation cannot be
> performed
>     on a
>      file with a user-mapped section open
>             at java.io.RandomAccessFile.setLength(Native Method)
>             at kafka.log.OffsetIndex.liftedTree2$1(OffsetIndex.scala:263)
>             at kafka.log.OffsetIndex.resize(OffsetIndex.scala:262)
>             at kafka.log.OffsetIndex.trimToValidSize(OffsetIndex.scala:247)
>             at kafka.log.Log.rollToOffset(Log.scala:518)
>             at kafka.log.Log.roll(Log.scala:502)
>             at kafka.log.Log.maybeRoll(Log.scala:484)
>             at kafka.log.Log.append(Log.scala:297)
> You can merge this pull request into a Git repository by running:
>     $ git pull https://github.com/lizziew/kafka 0.8
> Alternatively you can review and apply these changes as the patch at:
>     https://github.com/apache/kafka/pull/6.patch
> ----
> ----