Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper, mail # user - OOM/no more threads?


Copy link to this message
-
Re: OOM/no more threads?
Jordan Zimmerman 2012-07-09, 03:13
Good idea - I'll try that.

-JZ

On Jul 8, 2012, at 8:11 PM, Camille Fournier wrote:

> What does a jstack on another long-running instance show? Anything out of
> the norm?
>
> C
>
> On Sun, Jul 8, 2012 at 11:08 PM, Jordan Zimmerman <
> [EMAIL PROTECTED]> wrote:
>
>> I'm not sure which of the instances went down, but it could be a few weeks.
>>
>> -JZ
>>
>> On Jul 8, 2012, at 8:06 PM, Camille Fournier wrote:
>>
>>> Yeah a tickle in the back of my mind says that this is a JVM bug. When
>> was
>>> the last time you restarted these instances?
>>>
>>> C
>>>
>>> On Sun, Jul 8, 2012 at 11:04 PM, Jordan Zimmerman <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> It's not really out of memory. It's "unable to create new native
>> thread".
>>>> It appears to have to do with JMX (notice the RMI).
>>>>
>>>> -JZ
>>>>
>>>> On Jul 8, 2012, at 7:43 PM, Camille Fournier wrote:
>>>>
>>>>> Not known specifically that I know of. Can you give us more details?
>> Did
>>>>> you actually run out of memory? File descriptors? What does a heap dump
>>>>> tell you?
>>>>>
>>>>> C
>>>>>
>>>>> On Sun, Jul 8, 2012 at 10:29 PM, Jordan Zimmerman <
>>>>> [EMAIL PROTECTED]> wrote:
>>>>>
>>>>>> Is this a known issue in 3.3.3? If so, is it fixed in 3.3.5?
>>>>>>
>>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: WARNING: RMI TCP
>>>>>> Accept-0: accept loop for ServerSocket[addr>>>>>> 0.0.0.0/0.0.0.0,port=0,localport=59462] throws
>>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: Jul 8, 2012
>>>> 11:01:28
>>>>>> PM sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
>>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: at
>>>>>> java.lang.Thread.run(Thread.java:636)
>>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: at
>>>>>>
>> sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:359)
>>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: at
>>>>>>
>>>>
>> sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:402)
>>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: at
>>>>>>
>>>>
>> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1336)
>>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: at
>>>>>>
>>>>
>> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:943)
>>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: at
>>>>>> java.lang.Thread.start(Thread.java:614)
>>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server: at
>>>>>> java.lang.Thread.start0(Native Method)
>>>>>> Sun Jul 08 23:01:28 GMT 2012 ERROR ZooKeeper Server:
>>>>>> java.lang.OutOfMemoryError: unable to create new native thread
>>>>>>
>>>>>>
>>>>
>>>>
>>
>>