Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill >> mail # dev >> Out of Memory Error on tests


Copy link to this message
-
Re: Out of Memory Error on tests
That's what I thought took

Of course it's possible there is a bug in your reader allocating too large or too many times memory.

Tim

Sent from my iPhone

> On Nov 15, 2013, at 9:37 PM, Tommy Guy <[EMAIL PROTECTED]> wrote:
>
> Odd, aren't we setting this in pom.xml:
>
>        <plugin>
>          <artifactId>maven-surefire-plugin</artifactId>
>          <version>2.15</version>
>          <configuration>
>            <argLine>-XX:MaxDirectMemorySize=4096M </argLine>
>          </configuration>
>        </plugin>
>
> Thanks, Tim.
>
> Tommy
>
>
>
>> On Fri, Nov 15, 2013 at 10:55 PM, Timothy Chen <[EMAIL PROTECTED]> wrote:
>>
>> I forgot what the flag is but you also want to set your direct memory to
>> allocate at a larger size too.
>>
>> Tim
>>
>> Sent from my iPhone
>>
>>> On Nov 15, 2013, at 8:44 PM, Tommy Guy <[EMAIL PROTECTED]> wrote:
>>>
>>> Hello everyone,
>>>
>>> I'm working on a CSV parser for Drill, and I'm running into trouble with
>>> the tests. I've set -Xmx=2048m and error persists. I've tried this on
>>> Ubuntu 11.10 and 13.10 (13.10 running in Vagrant)
>>>
>>> Anyone seen this before?
>>>
>>> Tommy
>>>
>>> Here's an example stack trace
>>> from
>> org.apache.drill.exec.physical.impl.TestSimpleFunctions.testByteSubstring
>>>
>>> java.lang.UnsupportedOperationException: null
>>> at io.netty.buffer.PoolArenaL$DirectArena.newByteBuf(PoolArenaL.java:400)
>>> at io.netty.buffer.PoolArenaL.allocate(PoolArenaL.java:94)
>>> at
>> io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:236)
>>> at
>> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:132)
>>> at
>> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:123)
>>> at
>> org.apache.drill.exec.memory.DirectBufferAllocator.buffer(DirectBufferAllocator.java:35)
>>> at org.apache.drill.exec.vector.IntVector.allocateNew(IntVector.java:100)
>>> at
>> org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:29)
>>> at
>> org.apache.drill.exec.store.mock.MockRecordReader.getVector(MockRecordReader.java:63)
>>> at
>> org.apache.drill.exec.store.mock.MockRecordReader.setup(MockRecordReader.java:78)
>>> at
>> org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:65)
>>> at
>> org.apache.drill.exec.store.mock.MockScanBatchCreator.getBatch(MockScanBatchCreator.java:44)
>>> at
>> org.apache.drill.exec.physical.impl.ImplCreator.visitSubScan(ImplCreator.java:105)
>>> at
>> org.apache.drill.exec.physical.impl.ImplCreator.visitSubScan(ImplCreator.java:60)
>>> at
>> org.apache.drill.exec.store.mock.MockSubScanPOP.accept(MockSubScanPOP.java:94)
>>> at
>> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195)
>>> at
>> org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:202)
>>> at
>> org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:60)
>>> at
>> org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:40)
>>> at
>> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195)
>>> at
>> org.apache.drill.exec.physical.impl.ImplCreator.visitProject(ImplCreator.java:92)
>>> at
>> org.apache.drill.exec.physical.impl.ImplCreator.visitProject(ImplCreator.java:60)
>>> at org.apache.drill.exec.physical.config.Project.accept(Project.java:51)
>>> at
>> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195)
>>> at
>> org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:202)
>>> at
>> org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:60)
>>> at
>> org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:40)
>>> at
>> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195)
>>> at
>> org.apache.drill.exec.physical.impl.ImplCreator.visitScreen(ImplCreator.java:145)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB