Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> consumer null pointer exception


+
Kane Kane 2013-11-01, 08:29
+
Neha Narkhede 2013-11-01, 10:51
+
Kane Kane 2013-11-01, 14:47
+
Neha Narkhede 2013-11-01, 16:25
+
Kane Kane 2013-11-01, 18:36
Copy link to this message
-
Re: consumer null pointer exception
Yes, could you send around the log4j file of the consumer around the time
of the error in question.

Thanks,
Neha
On Fri, Nov 1, 2013 at 11:35 AM, Kane Kane <[EMAIL PROTECTED]> wrote:

> Hello Neha, rest of the log goes to my consumer code, are you interested
> in? It's a little bit modified version of ConsoleConsumer.
>
> Thanks.
>
>
> On Fri, Nov 1, 2013 at 9:24 AM, Neha Narkhede <[EMAIL PROTECTED]
> >wrote:
>
> > Could you send the entire consumer log?
> >
> >
> > On Fri, Nov 1, 2013 at 7:45 AM, Kane Kane <[EMAIL PROTECTED]> wrote:
> >
> > > Hello Neha, I think i might be hitting this. I didn't shutdown the
> > consumer
> > > (at least intentionally). Basically it was just an attempt to pipe ~1T
> > > through kafka, i would wild guess it's related to log expansion.
> Because
> > > around the time it happened i saw messages about expanding log, is it
> > > possible to be related?
> > >
> > > Thanks.
> > >
> > >
> > > On Fri, Nov 1, 2013 at 3:50 AM, Neha Narkhede <[EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > I think you are hitting
> > https://issues.apache.org/jira/browse/KAFKA-824.
> > > > Was the consumer being shutdown at that time?
> > > >
> > > >
> > > > On Fri, Nov 1, 2013 at 1:28 AM, Kane Kane <[EMAIL PROTECTED]>
> > wrote:
> > > >
> > > > > I've got the following exception running produce/consume loop for
> > > several
> > > > > hours,
> > > > > that was just single exception, but during that time both producers
> > and
> > > > > consumers slowed down a lot. After that looks like everything works
> > > fine,
> > > > > though i have
> > > > > suspicion some messages were lost.
> > > > > Can anyone explain what it might be?
> > > > >
> > > > > WARN  consumer.ZookeeperConsumerConnector -
> > > > > [group2_kafka-001.sendgrid.net-1383289
> > > > > 653031-b02bae92], exception during commitOffsets
> > > > > java.lang.NullPointerException
> > > > >         at
> > > > >
> > > >
> > >
> >
> org.I0Itec.zkclient.ZkConnection.writeDataReturnStat(ZkConnection.java:115)
> > > > >         at org.I0Itec.zkclient.ZkClient$10.call(ZkClient.java:817)
> > > > >         at
> > > > > org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
> > > > >         at
> > > > > org.I0Itec.zkclient.ZkClient.writeDataReturnStat(ZkClient.java:813)
> > > > >         at
> org.I0Itec.zkclient.ZkClient.writeData(ZkClient.java:808)
> > > > >         at
> org.I0Itec.zkclient.ZkClient.writeData(ZkClient.java:777)
> > > > >         at kafka.utils.ZkUtils$.updatePersistentPath(Unknown
> Source)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$3$$anonfun$apply$5.apply(Unknown
> > > > > Source)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$3$$anonfun$apply$5.apply(Unknown
> > > > > Source)
> > > > >         at
> > scala.collection.Iterator$class.foreach(Iterator.scala:727)
> > > > >         at
> > > scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
> > > > >         at
> > > > > scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> > > > >         at
> > scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$3.apply(Unknown
> > > > > Source)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$3.apply(Unknown
> > > > > Source)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772)
> > > > >         at
> > scala.collection.Iterator$class.foreach(Iterator.scala:727)
> > > > >         at kafka.utils.Pool$$anon$1.foreach(Unknown Source)
> > > > >         at
> > > > > scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> > > > >         at kafka.utils.Pool.foreach(Unknown Source)

 
+
Kane Kane 2013-11-02, 06:39
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB