Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> More heap space for unit tests


Copy link to this message
-
Re: More heap space for unit tests
It also looks like TestMasterFailover isn't cleaning up after itself.
Each test method just does TEST_UTIL.shutdownMiniCluster() at the end,
without use of a finally block.  So it wouldn't get called in the case
of an exception.  Testing a patch for that now.

On Wed, Nov 23, 2011 at 9:52 AM, Gary Helmling <[EMAIL PROTECTED]> wrote:
> I think this might be caused by:
> https://issues.apache.org/jira/browse/HBASE-4857
>
> There is a recursive loop on the leader
> AuthenticationTokenSecretManger when a KeeperException is encountered.
>
>
>
> On Wed, Nov 23, 2011 at 8:53 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> The heap space error happened again for build #9
>>
>> TestMasterFailover hung in build 9 too.
>>
>> On Tue, Nov 22, 2011 at 2:07 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>>> The interesting thing is that their counterparts in HBase-0.92 build
>>> didn't run out of heap :-)
>>>
>>>
>>> On Tue, Nov 22, 2011 at 2:05 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
>>>
>>>> Usually IME this means that some test is generating many GB of logs,
>>>> etc... before raising heap let's look and see if some of the tests are
>>>> being excessively noisy
>>>>
>>>> -Todd
>>>>
>>>> On Tue, Nov 22, 2011 at 2:01 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>>> > Hi,
>>>> > I found the following in
>>>> >
>>>> https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.92-security/lastCompletedBuild/console
>>>> > :
>>>> >
>>>> > Running org.apache.hadoop.hbase.master.TestDistributedLogSplitting
>>>> > Exception in thread "ThreadedStreamConsumer"
>>>> > java.lang.OutOfMemoryError: Java heap space
>>>> >        at java.util.Arrays.copyOf(Arrays.java:2882)
>>>> >        at
>>>> java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
>>>> >        at
>>>> java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
>>>> >        at java.lang.StringBuffer.append(StringBuffer.java:224)
>>>> >        at
>>>> org.apache.maven.surefire.report.TestSetRunListener.getAsString(TestSetRunListener.java:201)
>>>> >        at
>>>> org.apache.maven.surefire.report.TestSetRunListener.testError(TestSetRunListener.java:139)
>>>> >        at
>>>> org.apache.maven.plugin.surefire.booterclient.output.ForkClient.consumeLine(ForkClient.java:112)
>>>> > Running org.apache.hadoop.hbase.master.TestMasterFailover
>>>> > Exception in thread "ThreadedStreamConsumer"
>>>> > java.lang.OutOfMemoryError: Java heap space
>>>> >        at java.util.Arrays.copyOf(Arrays.java:2882)
>>>> >        at
>>>> java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
>>>> >        at
>>>> java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
>>>> >        at java.lang.StringBuffer.append(StringBuffer.java:224)
>>>> >        at
>>>> org.apache.maven.surefire.report.TestSetRunListener.getAsString(TestSetRunListener.java:201)
>>>> >        at
>>>> org.apache.maven.surefire.report.TestSetRunListener.testError(TestSetRunListener.java:139)
>>>> >
>>>> > Currently each unit test is given 1.9GB of heap.
>>>> > I think we should give more heap to the tests.
>>>> >
>>>> > Cheers
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> Todd Lipcon
>>>> Software Engineer, Cloudera
>>>>
>>>
>>>
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB