Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Drill >> mail # dev >> Out of Memory Error on tests


Copy link to this message
-
Out of Memory Error on tests
Hello everyone,

I'm working on a CSV parser for Drill, and I'm running into trouble with
the tests. I've set -Xmx=2048m and error persists. I've tried this on
Ubuntu 11.10 and 13.10 (13.10 running in Vagrant)

Anyone seen this before?

Tommy

Here's an example stack trace
from org.apache.drill.exec.physical.impl.TestSimpleFunctions.testByteSubstring

java.lang.UnsupportedOperationException: null
at io.netty.buffer.PoolArenaL$DirectArena.newByteBuf(PoolArenaL.java:400)
at io.netty.buffer.PoolArenaL.allocate(PoolArenaL.java:94)
at
io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:236)
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:132)
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:123)
at
org.apache.drill.exec.memory.DirectBufferAllocator.buffer(DirectBufferAllocator.java:35)
at org.apache.drill.exec.vector.IntVector.allocateNew(IntVector.java:100)
at
org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:29)
at
org.apache.drill.exec.store.mock.MockRecordReader.getVector(MockRecordReader.java:63)
at
org.apache.drill.exec.store.mock.MockRecordReader.setup(MockRecordReader.java:78)
at org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:65)
at
org.apache.drill.exec.store.mock.MockScanBatchCreator.getBatch(MockScanBatchCreator.java:44)
at
org.apache.drill.exec.physical.impl.ImplCreator.visitSubScan(ImplCreator.java:105)
at
org.apache.drill.exec.physical.impl.ImplCreator.visitSubScan(ImplCreator.java:60)
at
org.apache.drill.exec.store.mock.MockSubScanPOP.accept(MockSubScanPOP.java:94)
at
org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195)
at
org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:202)
at
org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:60)
at
org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:40)
at
org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195)
at
org.apache.drill.exec.physical.impl.ImplCreator.visitProject(ImplCreator.java:92)
at
org.apache.drill.exec.physical.impl.ImplCreator.visitProject(ImplCreator.java:60)
at org.apache.drill.exec.physical.config.Project.accept(Project.java:51)
at
org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195)
at
org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:202)
at
org.apache.drill.exec.physical.impl.ImplCreator.visitIteratorValidator(ImplCreator.java:60)
at
org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:40)
at
org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:195)
at
org.apache.drill.exec.physical.impl.ImplCreator.visitScreen(ImplCreator.java:145)
at
org.apache.drill.exec.physical.impl.ImplCreator.visitScreen(ImplCreator.java:60)
at org.apache.drill.exec.physical.config.Screen.accept(Screen.java:102)
at
org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:212)
at
org.apache.drill.exec.physical.impl.TestSimpleFunctions.testByteSubstring(TestSimpleFunctions.java:200)

I'm also seeing the error in Lillth:
Exception in pipeline. Closing channel between local /127.0.1.1:31010 and
remote /127.0.0.1:43946

Lillith shows:
java.lang.OutOfMemoryError

at
org.apache.drill.exec.rpc.RpcExceptionHandler.exceptionCaught(RpcExceptionHandler.java:33)
at
io.netty.channel.DefaultChannelHandlerContext.invokeExceptionCaught(DefaultChannelHandlerContext.java:272)
at
io.netty.channel.DefaultChannelHandlerContext.fireExceptionCaught(DefaultChannelHandlerContext.java:250)
at
io.netty.channel.ChannelHandlerAdapter.exceptionCaught(ChannelHandlerAdapter.java:60)
at
io.netty.channel.DefaultChannelHandlerContext.invokeExceptionCaught(DefaultChannelHandlerContext.java:272)
at
io.netty.channel.DefaultChannelHandlerContext.fireExceptionCaught(DefaultChannelHandlerContext.java:250)
at
io.netty.channel.ChannelHandlerAdapter.exceptionCaught(ChannelHandlerAdapter.java:60)
at
io.netty.channel.DefaultChannelHandlerContext.invokeExceptionCaught(DefaultChannelHandlerContext.java:272)
+
Timothy Chen 2013-11-16, 06:55
+
Tommy Guy 2013-11-16, 07:37
+
Timothy Chen 2013-11-16, 07:56
+
Tommy Guy 2013-11-16, 23:27