Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - A "Java heap space" question


Copy link to this message
-
Re: A "Java heap space" question
Jun Rao 2012-12-27, 06:25
400KB messages are relatively large. How many messages are you sending per
sec? How big is your JVM heap? You may need a bigger heap size.

Thanks,

Jun

On Wed, Dec 26, 2012 at 5:41 PM, xingcan <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I run 'kafka-0.7.2' on a single node and today I got some error logs below.
>
> ...
>  [2012-12-26 08:51:18,496] INFO End registering broker topic
> /brokers/topics/jnits/1 (kafka.server.KafkaZooKeeper)
> 161557 [2012-12-26 08:51:19,103] INFO Closing socket connection to /
> 172.18.116.52. (kafka.network.Processor)
> 161558 [2012-12-26 08:51:18,416] ERROR OOME with size 313705
> (kafka.network.BoundedByteBufferReceive)
> 161559 java.lang.OutOfMemoryError: Java heap space
> 161560 [2012-12-26 08:51:18,341] ERROR OOME with size 317477
> (kafka.network.BoundedByteBufferReceive)
> 161561 java.lang.OutOfMemoryError: Java heap space
> 161562 [2012-12-26 08:51:19,178] ERROR Closing socket for /172.18.0.34because
> of error (kafka.network.Processor)
> 161563 java.lang.OutOfMemoryError: Java heap space
> 161564 [2012-12-26 08:51:19,178] ERROR Closing socket for
> /172.18.116.46 because
> of error (kafka.network.Processor)
> 161565 java.lang.OutOfMemoryError: Java heap space
> 161566 [2012-12-26 08:51:19,375] ERROR OOME with size 269336
> (kafka.network.BoundedByteBufferReceive)
> 161567 java.lang.OutOfMemoryError: Java heap space
> 161568 [2012-12-26 08:51:19,375] ERROR Closing socket for
> /172.18.113.38 because
> of error (kafka.network.Processor)
> 161569 java.lang.OutOfMemoryError: Java heap space
> ...
>
> The size of each message is about 400KB. Maybe the messages are produced
> too fast or the disk can not be written at such a rate(about 33MB/s/disk)?
> Can anyone help me explain it? Thanks a lot.
> --
> *Xingcan*
>