Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - consumer offset not saved in zk


+
rk vishu 2013-05-23, 06:22
+
Neha Narkhede 2013-05-23, 13:48
+
rk vishu 2013-05-23, 21:31
+
Jun Rao 2013-05-23, 14:08
+
rk vishu 2013-05-23, 21:32
+
Neha Narkhede 2013-05-23, 21:40
+
rk vishu 2013-05-23, 22:16
Copy link to this message
-
Re: consumer offset not saved in zk
Neha Narkhede 2013-05-23, 22:24
Can you please  try the following -

./sbt clean assembly-package-dependency package

Thanks,
Neha
On Thu, May 23, 2013 at 3:16 PM, rk vishu <[EMAIL PROTECTED]> wrote:

> Neha,
>
> Thanks for pointing out the log4j. I turned on logs at INFO level. Now i
> see some warnings as below.
>
> WARN [Kafka-consumer-autocommit-1] (Logging.scala:88) -
> [1_BELC02K41GGDKQ4.sea.corp.expecn.com-1369346576173-22148419], exception
> during commitOffsets
> java.lang.NoSuchMethodError:
>
> org.I0Itec.zkclient.ZkClient.writeData(Ljava/lang/String;Ljava/lang/Object;)Lorg/apache/zookeeper/data/Stat;
>     at kafka.utils.ZkUtils$.updatePersistentPath(ZkUtils.scala:304)
>     at
>
> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2$$anonfun$apply$5.apply(ZookeeperConsumerConnector.scala:253)
>     at
>
> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2$$anonfun$apply$5.apply(ZookeeperConsumerConnector.scala:250)
>     at scala.collection.Iterator$class.foreach(Iterator.scala:631)
>     at
>
> scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:549)
>     at scala.collection.IterableLike$class.foreach(IterableLike.scala:79)
>     at
>
> scala.collection.JavaConversions$JCollectionWrapper.foreach(JavaConversions.scala:570)
>     at
>
> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2.apply(ZookeeperConsumerConnector.scala:250)
>     at
>
> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2.apply(ZookeeperConsumerConnector.scala:248)
>     at scala.collection.Iterator$class.foreach(Iterator.scala:631)
>     at kafka.utils.Pool$$anon$1.foreach(Pool.scala:83)
>     at scala.collection.IterableLike$class.foreach(IterableLike.scala:79)
>     at kafka.utils.Pool.foreach(Pool.scala:27)
>     at
>
> kafka.consumer.ZookeeperConsumerConnector.commitOffsets(ZookeeperConsumerConnector.scala:248)
>     at
>
> kafka.consumer.ZookeeperConsumerConnector.autoCommit(ZookeeperConsumerConnector.scala:234)
>     at
>
> kafka.consumer.ZookeeperConsumerConnector$$anonfun$1.apply$mcV$sp(ZookeeperConsumerConnector.scala:117)
>     at kafka.utils.Utils$$anon$2.run(Utils.scala:67)
>     at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>     at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>     at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>     at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>     at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
>     at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
>     at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>     at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>     at java.lang.Thread.run(Thread.java:680)
>
> below are is what i am using in my maven dependencies for ZK. Looks like
> zkclient 0.1 has method with void return type for
> org.I0Itec.zkclient.ZkClient.writeData. Do i need to use different
> dependency?
>
>         <dependency>
>             <groupId>org.scala-lang</groupId>
>             <artifactId>scala-library</artifactId>
>             <version>2.8.1</version>
>         </dependency>
>         <dependency>
>             <groupId>com.github.sgroschupf</groupId>
>             <artifactId>zkclient</artifactId>
>             <version>0.1</version>
>         </dependency>
>         <dependency>
>             <groupId>org.apache.zookeeper</groupId>
>             <artifactId>zookeeper</artifactId>
>             <version>3.4.5</version>
>             <exclusions>
>                 <exclusion>
>                     <groupId>com.sun.jmx</groupId>
>                     <artifactId>jmxri</artifactId>
>                 </exclusion>
>                 <exclusion>

 
+
rk vishu 2013-05-23, 22:54