Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill >> mail # dev >> Out of Memory Error on tests


Copy link to this message
-
Re: Out of Memory Error on tests
Odd, aren't we setting this in pom.xml:

        <plugin>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>2.15</version>
          <configuration>
            <argLine>-XX:MaxDirectMemorySize=4096M </argLine>
          </configuration>
        </plugin>

Thanks, Tim.

Tommy

On Fri, Nov 15, 2013 at 10:55 PM, Timothy Chen <[EMAIL PROTECTED]> wrote:

> I forgot what the flag is but you also want to set your direct memory to
> allocate at a larger size too.
>
> Tim
>
> Sent from my iPhone
>
> > On Nov 15, 2013, at 8:44 PM, Tommy Guy <[EMAIL PROTECTED]> wrote:
> >
> > Hello everyone,
> >
> > I'm working on a CSV parser for Drill, and I'm running into trouble with
> > the tests. I've set -Xmx=2048m and error persists. I've tried this on
> > Ubuntu 11.10 and 13.10 (13.10 running in Vagrant)
> >
> > Anyone seen this before?
> >
> > Tommy
> >
> > Here's an example stack trace
> > from
> org.apache.drill.exec.physical.impl.TestSimpleFunctions.testByteSubstring
> >
> > java.lang.UnsupportedOperationException: null
> > at io.netty.buffer.PoolArenaL$DirectArena.newByteBuf(PoolArenaL.java:400)
> > at io.netty.buffer.PoolArenaL.allocate(PoolArenaL.java:94)
> > at
> >
> io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:236)
> > at
> >
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:132)
> > at
> >
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:123)
> > at
> >
> org.apache.drill.exec.memory.DirectBufferAllocator.buffer(DirectBufferAllocator.java:35)
> > at org.apache.drill.exec.vector.IntVector.allocateNew(IntVector.java:100)
> > at
> >
> org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:29)
> > at
> >
> org.apache.drill.exec.store.mock.MockRecordReader.getVector(MockRecordReader.java:63)
> > at
> >
> org.apache.drill.exec.store.mock.MockRecordReader.setup(MockRecordReader.java:78)
> > at
> org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:65)
> > at
> >
> org.apache.drill.exec.store.mock.MockScanBatchCreator.getBatch(MockScanBatchCreator.java:44)
> > at
> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitSubScan(ImplCreator.java:105)
> > at
> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitSubScan(ImplCreator.java:60)
> > at
> >
> org.apache.drill.exec.store.mock.MockSubScanPOP.accept(MockSubScanPOP.java:94)
> > at
> >
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195)
> > at
> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:202)
> > at
> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:60)
> > at
> >
> org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:40)
> > at
> >
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195)
> > at
> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitProject(ImplCreator.java:92)
> > at
> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitProject(ImplCreator.java:60)
> > at org.apache.drill.exec.physical.config.Project.accept(Project.java:51)
> > at
> >
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195)
> > at
> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:202)
> > at
> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:60)
> > at
> >
> org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:40)
> > at
> >
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195)
> > at
> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitScreen(ImplCreator.java:145)
> > at
> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitScreen(ImplCreator.java:60)
> > at org.apache.drill.exec.physical.config.Screen.accept(Screen.java:102)
> > at
> >
> org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:212)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB