Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Drill >> mail # dev >> Out of Memory Error on tests


+
Tommy Guy 2013-11-16, 06:44
+
Timothy Chen 2013-11-16, 06:55
+
Tommy Guy 2013-11-16, 07:37
+
Timothy Chen 2013-11-16, 07:56
Copy link to this message
-
Re: Out of Memory Error on tests
Ah - I tried it on a new, 64-bit VM and the tests passed.

Tommy
On Fri, Nov 15, 2013 at 11:56 PM, Timothy Chen <[EMAIL PROTECTED]> wrote:

> That's what I thought took
>
> Of course it's possible there is a bug in your reader allocating too large
> or too many times memory.
>
> Tim
>
> Sent from my iPhone
>
> > On Nov 15, 2013, at 9:37 PM, Tommy Guy <[EMAIL PROTECTED]> wrote:
> >
> > Odd, aren't we setting this in pom.xml:
> >
> >        <plugin>
> >          <artifactId>maven-surefire-plugin</artifactId>
> >          <version>2.15</version>
> >          <configuration>
> >            <argLine>-XX:MaxDirectMemorySize=4096M </argLine>
> >          </configuration>
> >        </plugin>
> >
> > Thanks, Tim.
> >
> > Tommy
> >
> >
> >
> >> On Fri, Nov 15, 2013 at 10:55 PM, Timothy Chen <[EMAIL PROTECTED]>
> wrote:
> >>
> >> I forgot what the flag is but you also want to set your direct memory to
> >> allocate at a larger size too.
> >>
> >> Tim
> >>
> >> Sent from my iPhone
> >>
> >>> On Nov 15, 2013, at 8:44 PM, Tommy Guy <[EMAIL PROTECTED]>
> wrote:
> >>>
> >>> Hello everyone,
> >>>
> >>> I'm working on a CSV parser for Drill, and I'm running into trouble
> with
> >>> the tests. I've set -Xmx=2048m and error persists. I've tried this on
> >>> Ubuntu 11.10 and 13.10 (13.10 running in Vagrant)
> >>>
> >>> Anyone seen this before?
> >>>
> >>> Tommy
> >>>
> >>> Here's an example stack trace
> >>> from
> >>
> org.apache.drill.exec.physical.impl.TestSimpleFunctions.testByteSubstring
> >>>
> >>> java.lang.UnsupportedOperationException: null
> >>> at
> io.netty.buffer.PoolArenaL$DirectArena.newByteBuf(PoolArenaL.java:400)
> >>> at io.netty.buffer.PoolArenaL.allocate(PoolArenaL.java:94)
> >>> at
> >>
> io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:236)
> >>> at
> >>
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:132)
> >>> at
> >>
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:123)
> >>> at
> >>
> org.apache.drill.exec.memory.DirectBufferAllocator.buffer(DirectBufferAllocator.java:35)
> >>> at
> org.apache.drill.exec.vector.IntVector.allocateNew(IntVector.java:100)
> >>> at
> >>
> org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:29)
> >>> at
> >>
> org.apache.drill.exec.store.mock.MockRecordReader.getVector(MockRecordReader.java:63)
> >>> at
> >>
> org.apache.drill.exec.store.mock.MockRecordReader.setup(MockRecordReader.java:78)
> >>> at
> >> org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:65)
> >>> at
> >>
> org.apache.drill.exec.store.mock.MockScanBatchCreator.getBatch(MockScanBatchCreator.java:44)
> >>> at
> >>
> org.apache.drill.exec.physical.impl.ImplCreator.visitSubScan(ImplCreator.java:105)
> >>> at
> >>
> org.apache.drill.exec.physical.impl.ImplCreator.visitSubScan(ImplCreator.java:60)
> >>> at
> >>
> org.apache.drill.exec.store.mock.MockSubScanPOP.accept(MockSubScanPOP.java:94)
> >>> at
> >>
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195)
> >>> at
> >>
> org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:202)
> >>> at
> >>
> org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:60)
> >>> at
> >>
> org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:40)
> >>> at
> >>
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195)
> >>> at
> >>
> org.apache.drill.exec.physical.impl.ImplCreator.visitProject(ImplCreator.java:92)
> >>> at
> >>
> org.apache.drill.exec.physical.impl.ImplCreator.visitProject(ImplCreator.java:60)
> >>> at
> org.apache.drill.exec.physical.config.Project.accept(Project.java:51)
> >>> at
> >>
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195)
> >>> at
> >>
> org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:202)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB