Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> More heap space for unit tests


Copy link to this message
-
Re: More heap space for unit tests
It also looks like TestMasterFailover isn't cleaning up after itself.
Each test method just does TEST_UTIL.shutdownMiniCluster() at the end,
without use of a finally block.  So it wouldn't get called in the case
of an exception.  Testing a patch for that now.

On Wed, Nov 23, 2011 at 9:52 AM, Gary Helmling <[EMAIL PROTECTED]> wrote:
> I think this might be caused by:
> https://issues.apache.org/jira/browse/HBASE-4857
>
> There is a recursive loop on the leader
> AuthenticationTokenSecretManger when a KeeperException is encountered.
>
>
>
> On Wed, Nov 23, 2011 at 8:53 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> The heap space error happened again for build #9
>>
>> TestMasterFailover hung in build 9 too.
>>
>> On Tue, Nov 22, 2011 at 2:07 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>>> The interesting thing is that their counterparts in HBase-0.92 build
>>> didn't run out of heap :-)
>>>
>>>
>>> On Tue, Nov 22, 2011 at 2:05 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
>>>
>>>> Usually IME this means that some test is generating many GB of logs,
>>>> etc... before raising heap let's look and see if some of the tests are
>>>> being excessively noisy
>>>>
>>>> -Todd
>>>>
>>>> On Tue, Nov 22, 2011 at 2:01 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>>> > Hi,
>>>> > I found the following in
>>>> >
>>>> https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.92-security/lastCompletedBuild/console
>>>> > :
>>>> >
>>>> > Running org.apache.hadoop.hbase.master.TestDistributedLogSplitting
>>>> > Exception in thread "ThreadedStreamConsumer"
>>>> > java.lang.OutOfMemoryError: Java heap space
>>>> >        at java.util.Arrays.copyOf(Arrays.java:2882)
>>>> >        at
>>>> java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
>>>> >        at
>>>> java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
>>>> >        at java.lang.StringBuffer.append(StringBuffer.java:224)
>>>> >        at
>>>> org.apache.maven.surefire.report.TestSetRunListener.getAsString(TestSetRunListener.java:201)
>>>> >        at
>>>> org.apache.maven.surefire.report.TestSetRunListener.testError(TestSetRunListener.java:139)
>>>> >        at
>>>> org.apache.maven.plugin.surefire.booterclient.output.ForkClient.consumeLine(ForkClient.java:112)
>>>> > Running org.apache.hadoop.hbase.master.TestMasterFailover
>>>> > Exception in thread "ThreadedStreamConsumer"
>>>> > java.lang.OutOfMemoryError: Java heap space
>>>> >        at java.util.Arrays.copyOf(Arrays.java:2882)
>>>> >        at
>>>> java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
>>>> >        at
>>>> java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
>>>> >        at java.lang.StringBuffer.append(StringBuffer.java:224)
>>>> >        at
>>>> org.apache.maven.surefire.report.TestSetRunListener.getAsString(TestSetRunListener.java:201)
>>>> >        at
>>>> org.apache.maven.surefire.report.TestSetRunListener.testError(TestSetRunListener.java:139)
>>>> >
>>>> > Currently each unit test is given 1.9GB of heap.
>>>> > I think we should give more heap to the tests.
>>>> >
>>>> > Cheers
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> Todd Lipcon
>>>> Software Engineer, Cloudera
>>>>
>>>
>>>
>>
>