Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> consumer null pointer exception


Copy link to this message
-
Re: consumer null pointer exception
Unfortunately I've recompiled and installed kafka from scratch today,
deleting all data. I'm still doing these tests, will send you
stacktrace if I would be able to repro it.

Thanks for help!

On Fri, Nov 1, 2013 at 6:13 PM, Neha Narkhede <[EMAIL PROTECTED]> wrote:
> Yes, could you send around the log4j file of the consumer around the time
> of the error in question.
>
> Thanks,
> Neha
>
>
> On Fri, Nov 1, 2013 at 11:35 AM, Kane Kane <[EMAIL PROTECTED]> wrote:
>
>> Hello Neha, rest of the log goes to my consumer code, are you interested
>> in? It's a little bit modified version of ConsoleConsumer.
>>
>> Thanks.
>>
>>
>> On Fri, Nov 1, 2013 at 9:24 AM, Neha Narkhede <[EMAIL PROTECTED]
>> >wrote:
>>
>> > Could you send the entire consumer log?
>> >
>> >
>> > On Fri, Nov 1, 2013 at 7:45 AM, Kane Kane <[EMAIL PROTECTED]> wrote:
>> >
>> > > Hello Neha, I think i might be hitting this. I didn't shutdown the
>> > consumer
>> > > (at least intentionally). Basically it was just an attempt to pipe ~1T
>> > > through kafka, i would wild guess it's related to log expansion.
>> Because
>> > > around the time it happened i saw messages about expanding log, is it
>> > > possible to be related?
>> > >
>> > > Thanks.
>> > >
>> > >
>> > > On Fri, Nov 1, 2013 at 3:50 AM, Neha Narkhede <[EMAIL PROTECTED]
>> > > >wrote:
>> > >
>> > > > I think you are hitting
>> > https://issues.apache.org/jira/browse/KAFKA-824.
>> > > > Was the consumer being shutdown at that time?
>> > > >
>> > > >
>> > > > On Fri, Nov 1, 2013 at 1:28 AM, Kane Kane <[EMAIL PROTECTED]>
>> > wrote:
>> > > >
>> > > > > I've got the following exception running produce/consume loop for
>> > > several
>> > > > > hours,
>> > > > > that was just single exception, but during that time both producers
>> > and
>> > > > > consumers slowed down a lot. After that looks like everything works
>> > > fine,
>> > > > > though i have
>> > > > > suspicion some messages were lost.
>> > > > > Can anyone explain what it might be?
>> > > > >
>> > > > > WARN  consumer.ZookeeperConsumerConnector -
>> > > > > [group2_kafka-001.sendgrid.net-1383289
>> > > > > 653031-b02bae92], exception during commitOffsets
>> > > > > java.lang.NullPointerException
>> > > > >         at
>> > > > >
>> > > >
>> > >
>> >
>> org.I0Itec.zkclient.ZkConnection.writeDataReturnStat(ZkConnection.java:115)
>> > > > >         at org.I0Itec.zkclient.ZkClient$10.call(ZkClient.java:817)
>> > > > >         at
>> > > > > org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
>> > > > >         at
>> > > > > org.I0Itec.zkclient.ZkClient.writeDataReturnStat(ZkClient.java:813)
>> > > > >         at
>> org.I0Itec.zkclient.ZkClient.writeData(ZkClient.java:808)
>> > > > >         at
>> org.I0Itec.zkclient.ZkClient.writeData(ZkClient.java:777)
>> > > > >         at kafka.utils.ZkUtils$.updatePersistentPath(Unknown
>> Source)
>> > > > >         at
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$3$$anonfun$apply$5.apply(Unknown
>> > > > > Source)
>> > > > >         at
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$3$$anonfun$apply$5.apply(Unknown
>> > > > > Source)
>> > > > >         at
>> > scala.collection.Iterator$class.foreach(Iterator.scala:727)
>> > > > >         at
>> > > scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
>> > > > >         at
>> > > > > scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
>> > > > >         at
>> > scala.collection.AbstractIterable.foreach(Iterable.scala:54)
>> > > > >         at
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$3.apply(Unknown
>> > > > > Source)
>> > > > >         at
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$3.apply(Unknown
>> > > > > Source)
>> > > > >         at
>> >
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB