Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> Unmap before resizing


Copy link to this message
-
Re: Unmap before resizing
Awesome, thanks for the contribution! I posted a few follow-up items on the
ticket.

-Jay
On Mon, Aug 12, 2013 at 10:07 PM, elizabeth <[EMAIL PROTECTED]>wrote:

> Hi, I just filed the JIRA as instructed.
>
> https://issues.apache.org/jira/browse/KAFKA-1008
>
> Thanks!
>
> =====
>
> Hi, can you file a JIRA and attach the patch there? This does the Apache
> copyright stuff...
>
> Instructions here:http://kafka.apache.org/contributing.html
>
> This seems like a good thing to have. Is there a more portable way to do
> this? Obviously this would not work on a non-sun JVM. I think the ideal
> would be to add a tryUnmap() method that attempts the case but swallows any
> exception.
>
> Also I would like to fully understand the Sun's objection to allowing user
> code to unmap buffers. In that ticket on the previous thread on
> Windows+mmap there was a mention that forcing the unmap was somehow a
> security hole. I don't understand this. Unix obviously supports unmap and
> there is no security issue with that as normal process-level memory
> isolation protects processes from reading each others memory or files.  I
> suspect maybe this would only be true in cases where you are trying to
> isolate different code in the same JVM which is not a valid use case. But I
> want to fully understand the issue before we add something someone claims
> is unsafe.
>
> -Jay
>
>
> On Thu, Jul 25, 2013 at 12:07 AM, lizziew <[EMAIL PROTECTED]> wrote:
>
> > GitHub user lizziew opened a pull request:
> >
> >     https://github.com/apache/kafka/pull/6
> >
> >     Unmap before resizing
> >
> >     While I was studying how MappedByteBuffer works, I saw a sharing
> > runtime exception on Windows. I applied what I learned to generate a
> patch
> > which uses an internal open JDK API to solve this problem.
> >
> >     ---
> >
> >     Caused by: java.io.IOException: The requested operation cannot be
> > performed
> >     on a
> >      file with a user-mapped section open
> >             at java.io.RandomAccessFile.setLength(Native Method)
> >             at kafka.log.OffsetIndex.liftedTree2$1(OffsetIndex.scala:263)
> >             at kafka.log.OffsetIndex.resize(OffsetIndex.scala:262)
> >             at
> kafka.log.OffsetIndex.trimToValidSize(OffsetIndex.scala:247)
> >             at kafka.log.Log.rollToOffset(Log.scala:518)
> >             at kafka.log.Log.roll(Log.scala:502)
> >             at kafka.log.Log.maybeRoll(Log.scala:484)
> >             at kafka.log.Log.append(Log.scala:297)
> >
> > You can merge this pull request into a Git repository by running:
> >
> >     $ git pull https://github.com/lizziew/kafka 0.8
> >
> > Alternatively you can review and apply these changes as the patch at:
> >
> >     https://github.com/apache/kafka/pull/6.patch
> >
> > ----
> >
> > ----
> >
> >
>