Hi,

Installed 0.8.0 yesterday, 3 physical machines, 9 brokers running (3 per
machine). 2 topics, with 3 replicas each

the console producer/consumer examples work fine.

When I run my producer logic I get the following error whether on the
cluster or in my dev environment:

Exception in thread "main" kafka.common.FailedToSendMessageException:
Failed to send messages after 3 tries.
at
kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:70)
at kafka.producer.Producer.send(Producer.scala:75)
at kafka.javaapi.producer.Producer.send(Producer.scala:32)
at com.silverpop.kafka.playproducer.TestProducer.main(TestProducer.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)

Code is pretty basic:

public class TestProducer {
    public static void main(String[] args) {

        String zookeeper = args[0];
        long events = Long.parseLong(args[1]);
        long blocks = Long.parseLong(args[2]);

        Random rnd = new Random();

        Properties props = new Properties();
        props.put("broker.list", "mongodb03.atlnp1:9092");
        props.put("serializer.class", "kafka.serializer.StringEncoder");
        ProducerConfig config = new ProducerConfig(props);
        Producer<Integer, String> producer = new Producer<Integer,
String>(config);
        for (long nBlocks = 0; nBlocks < blocks; nBlocks++) {
            for (long nEvents = 0; nEvents < events; nEvents++) {
                long runtime = new Date().getTime();
                String msg = runtime + "," + (50 + nBlocks) + "," +
nEvents+ "," + rnd.nextInt(1000);
                KeyedMessage<Integer, String> data = new
KeyedMessage<Integer, String>("test1", msg);
                producer.send(data);
            }
        }
    }

using Zookeeper doesn't matter. Changing broker.list to include all 9
brokers doesn't matter. Changing Producer and KeyedMessage to be <String,
String> doesn't matter.

Thoughts on what I'm doing wrong?

Thanks,

Chris

Do you have the exception that caused each retry to fail?

Thanks,

Jun

On Tue, Nov 27, 2012 at 7:17 AM, Chris Curtin <[EMAIL PROTECTED]>wrote:

> Hi,
>
> Installed 0.8.0 yesterday, 3 physical machines, 9 brokers running (3 per
> machine). 2 topics, with 3 replicas each
>
> the console producer/consumer examples work fine.
>
> When I run my producer logic I get the following error whether on the
> cluster or in my dev environment:
>
> Exception in thread "main" kafka.common.FailedToSendMessageException:
> Failed to send messages after 3 tries.
> at
>
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:70)
> at kafka.producer.Producer.send(Producer.scala:75)
> at kafka.javaapi.producer.Producer.send(Producer.scala:32)
> at com.silverpop.kafka.playproducer.TestProducer.main(TestProducer.java:40)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
>
> Code is pretty basic:
>
> public class TestProducer {
>     public static void main(String[] args) {
>
>         String zookeeper = args[0];
>         long events = Long.parseLong(args[1]);
>         long blocks = Long.parseLong(args[2]);
>
>         Random rnd = new Random();
>
>         Properties props = new Properties();
>         props.put("broker.list", "mongodb03.atlnp1:9092");
>
>
>         props.put("serializer.class", "kafka.serializer.StringEncoder");
>         ProducerConfig config = new ProducerConfig(props);
>         Producer<Integer, String> producer = new Producer<Integer,
> String>(config);
>
>
>         for (long nBlocks = 0; nBlocks < blocks; nBlocks++) {
>             for (long nEvents = 0; nEvents < events; nEvents++) {
>                 long runtime = new Date().getTime();
>                 String msg = runtime + "," + (50 + nBlocks) + "," +
> nEvents+ "," + rnd.nextInt(1000);
>                 KeyedMessage<Integer, String> data = new
> KeyedMessage<Integer, String>("test1", msg);
>                 producer.send(data);
>             }
>         }
>
>
>     }
>
> using Zookeeper doesn't matter. Changing broker.list to include all 9
> brokers doesn't matter. Changing Producer and KeyedMessage to be <String,
> String> doesn't matter.
>
> Thoughts on what I'm doing wrong?
>
> Thanks,
>
> Chris
>

The error from when it fails is all I get. Nothing on the broker side and
no other errors or exceptions on the client.

Where should I be looking for the reasons? Is there a callback I should be
implementing?

Thanks,

Chris
On Tue, Nov 27, 2012 at 11:49 AM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Do you have the exception that caused each retry to fail?
>
> Thanks,
>
> Jun
>
> On Tue, Nov 27, 2012 at 7:17 AM, Chris Curtin <[EMAIL PROTECTED]
> >wrote:
>
> > Hi,
> >
> > Installed 0.8.0 yesterday, 3 physical machines, 9 brokers running (3 per
> > machine). 2 topics, with 3 replicas each
> >
> > the console producer/consumer examples work fine.
> >
> > When I run my producer logic I get the following error whether on the
> > cluster or in my dev environment:
> >
> > Exception in thread "main" kafka.common.FailedToSendMessageException:
> > Failed to send messages after 3 tries.
> > at
> >
> >
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:70)
> > at kafka.producer.Producer.send(Producer.scala:75)
> > at kafka.javaapi.producer.Producer.send(Producer.scala:32)
> > at
> com.silverpop.kafka.playproducer.TestProducer.main(TestProducer.java:40)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
> >
> > Code is pretty basic:
> >
> > public class TestProducer {
> >     public static void main(String[] args) {
> >
> >         String zookeeper = args[0];
> >         long events = Long.parseLong(args[1]);
> >         long blocks = Long.parseLong(args[2]);
> >
> >         Random rnd = new Random();
> >
> >         Properties props = new Properties();
> >         props.put("broker.list", "mongodb03.atlnp1:9092");
> >
> >
> >         props.put("serializer.class", "kafka.serializer.StringEncoder");
> >         ProducerConfig config = new ProducerConfig(props);
> >         Producer<Integer, String> producer = new Producer<Integer,
> > String>(config);
> >
> >
> >         for (long nBlocks = 0; nBlocks < blocks; nBlocks++) {
> >             for (long nEvents = 0; nEvents < events; nEvents++) {
> >                 long runtime = new Date().getTime();
> >                 String msg = runtime + "," + (50 + nBlocks) + "," +
> > nEvents+ "," + rnd.nextInt(1000);
> >                 KeyedMessage<Integer, String> data = new
> > KeyedMessage<Integer, String>("test1", msg);
> >                 producer.send(data);
> >             }
> >         }
> >
> >
> >     }
> >
> > using Zookeeper doesn't matter. Changing broker.list to include all 9
> > brokers doesn't matter. Changing Producer and KeyedMessage to be <String,
> > String> doesn't matter.
> >
> > Thoughts on what I'm doing wrong?
> >
> > Thanks,
> >
> > Chris
> >
>

When the producer fails to send in 3 retries, it will log an error. Before
that, in the producer, we log the failure of each of the retries in warn
and it tells you the cause.

Also, did you create the topic first?

Thanks,

Jun

On Tue, Nov 27, 2012 at 8:56 AM, Chris Curtin <[EMAIL PROTECTED]>wrote:

> The error from when it fails is all I get. Nothing on the broker side and
> no other errors or exceptions on the client.
>
> Where should I be looking for the reasons? Is there a callback I should be
> implementing?
>
> Thanks,
>
> Chris
>
>
> On Tue, Nov 27, 2012 at 11:49 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > Do you have the exception that caused each retry to fail?
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Nov 27, 2012 at 7:17 AM, Chris Curtin <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Hi,
> > >
> > > Installed 0.8.0 yesterday, 3 physical machines, 9 brokers running (3
> per
> > > machine). 2 topics, with 3 replicas each
> > >
> > > the console producer/consumer examples work fine.
> > >
> > > When I run my producer logic I get the following error whether on the
> > > cluster or in my dev environment:
> > >
> > > Exception in thread "main" kafka.common.FailedToSendMessageException:
> > > Failed to send messages after 3 tries.
> > > at
> > >
> > >
> >
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:70)
> > > at kafka.producer.Producer.send(Producer.scala:75)
> > > at kafka.javaapi.producer.Producer.send(Producer.scala:32)
> > > at
> > com.silverpop.kafka.playproducer.TestProducer.main(TestProducer.java:40)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at
> > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > > at
> > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > > at java.lang.reflect.Method.invoke(Method.java:597)
> > > at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
> > >
> > > Code is pretty basic:
> > >
> > > public class TestProducer {
> > >     public static void main(String[] args) {
> > >
> > >         String zookeeper = args[0];
> > >         long events = Long.parseLong(args[1]);
> > >         long blocks = Long.parseLong(args[2]);
> > >
> > >         Random rnd = new Random();
> > >
> > >         Properties props = new Properties();
> > >         props.put("broker.list", "mongodb03.atlnp1:9092");
> > >
> > >
> > >         props.put("serializer.class",
> "kafka.serializer.StringEncoder");
> > >         ProducerConfig config = new ProducerConfig(props);
> > >         Producer<Integer, String> producer = new Producer<Integer,
> > > String>(config);
> > >
> > >
> > >         for (long nBlocks = 0; nBlocks < blocks; nBlocks++) {
> > >             for (long nEvents = 0; nEvents < events; nEvents++) {
> > >                 long runtime = new Date().getTime();
> > >                 String msg = runtime + "," + (50 + nBlocks) + "," +
> > > nEvents+ "," + rnd.nextInt(1000);
> > >                 KeyedMessage<Integer, String> data = new
> > > KeyedMessage<Integer, String>("test1", msg);
> > >                 producer.send(data);
> > >             }
> > >         }
> > >
> > >
> > >     }
> > >
> > > using Zookeeper doesn't matter. Changing broker.list to include all 9
> > > brokers doesn't matter. Changing Producer and KeyedMessage to be
> <String,
> > > String> doesn't matter.
> > >
> > > Thoughts on what I'm doing wrong?
> > >
> > > Thanks,
> > >
> > > Chris
> > >
> >
>

Okay, figured out that you need to turn on Log4j logger to DEBUG then you
get a NoSuchMethodError around yammer. (see below)

I'm running yammer 2.2.0 since that seems to be all I can find via maven.
Is there a different version needed?

Thanks,

Chris

257  [main] INFO  kafka.producer.SyncProducer  - Connected to
10.121.31.55:9094 for producing
286  [main] WARN  kafka.producer.async.DefaultEventHandler  - failed to
send to broker 3 with data Map([test1,0] ->
ByteBufferMessageSet(MessageAndOffset(Message(magic = 2, attributes = 0,
crc = 1906312613, key = null, payload = java.nio.HeapByteBuffer[pos=0
lim=22 cap=22]),0), ))
java.lang.NoSuchMethodError: com.yammer.metrics.core.TimerContext.stop()J
at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:36)
at kafka.producer.SyncProducer.send(SyncProducer.scala:94)
at
kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$send(DefaultEventHandler.scala:221)
at
kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$1.apply(DefaultEventHandler.scala:87)
at
kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$1.apply(DefaultEventHandler.scala:81)
at
scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
at
scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
at scala.collection.Iterator$class.foreach(Iterator.scala:631)
at scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:161)
at
scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:194)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
at scala.collection.mutable.HashMap.foreach(HashMap.scala:80)
at
kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:81)
at
kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:57)
at kafka.producer.Producer.send(Producer.scala:75)
at kafka.javaapi.producer.Producer.send(Producer.scala:32)
at com.silverpop.kafka.playproducer.TestProducer.main(TestProducer.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
On Tue, Nov 27, 2012 at 12:30 PM, Jun Rao <[EMAIL PROTECTED]> wrote:

> When the producer fails to send in 3 retries, it will log an error. Before
> that, in the producer, we log the failure of each of the retries in warn
> and it tells you the cause.
>
> Also, did you create the topic first?
>
> Thanks,
>
> Jun
>
> On Tue, Nov 27, 2012 at 8:56 AM, Chris Curtin <[EMAIL PROTECTED]
> >wrote:
>
> > The error from when it fails is all I get. Nothing on the broker side and
> > no other errors or exceptions on the client.
> >
> > Where should I be looking for the reasons? Is there a callback I should
> be
> > implementing?
> >
> > Thanks,
> >
> > Chris
> >
> >
> > On Tue, Nov 27, 2012 at 11:49 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >
> > > Do you have the exception that caused each retry to fail?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Tue, Nov 27, 2012 at 7:17 AM, Chris Curtin <[EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > Hi,
> > > >
> > > > Installed 0.8.0 yesterday, 3 physical machines, 9 brokers running (3
> > per
> > > > machine). 2 topics, with 3 replicas each
> > > >
> > > > the console producer/consumer examples work fine.
> > > >
> > > > When I run my producer logic I get the following error whether on the
> > > > cluster or in my dev environment:
> > > >
> > > > Exception in thread "main" kafka.common.FailedToSendMessageException:
> > > > Failed to send messages after 3 tries.
> > > > at
> > > >
> > > >
> > >
> >
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:70)
> > > > at kafka.producer.Producer.send(Producer.scala:75)
> > > > at kafka.javaapi.producer.Producer.send(Producer.scala:32)

Yes, we had to patch metrics due to a bug. The patched jar is shipped with
the distribution until they get a release out.

But logging this in debug is a bug. Would you be willing to file a ticket
for this?

Thanks!

-jay
On Tue, Nov 27, 2012 at 10:12 AM, Chris Curtin <[EMAIL PROTECTED]>wrote:

> Okay, figured out that you need to turn on Log4j logger to DEBUG then you
> get a NoSuchMethodError around yammer. (see below)
>
> I'm running yammer 2.2.0 since that seems to be all I can find via maven.
> Is there a different version needed?
>
> Thanks,
>
> Chris
>
> 257  [main] INFO  kafka.producer.SyncProducer  - Connected to
> 10.121.31.55:9094 for producing
> 286  [main] WARN  kafka.producer.async.DefaultEventHandler  - failed to
> send to broker 3 with data Map([test1,0] ->
> ByteBufferMessageSet(MessageAndOffset(Message(magic = 2, attributes = 0,
> crc = 1906312613, key = null, payload = java.nio.HeapByteBuffer[pos=0
> lim=22 cap=22]),0), ))
> java.lang.NoSuchMethodError: com.yammer.metrics.core.TimerContext.stop()J
> at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:36)
> at kafka.producer.SyncProducer.send(SyncProducer.scala:94)
> at
>
> kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$send(DefaultEventHandler.scala:221)
> at
>
> kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$1.apply(DefaultEventHandler.scala:87)
> at
>
> kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$1.apply(DefaultEventHandler.scala:81)
> at
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
> at
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
> at scala.collection.Iterator$class.foreach(Iterator.scala:631)
> at scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:161)
> at
> scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:194)
> at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
> at scala.collection.mutable.HashMap.foreach(HashMap.scala:80)
> at
>
> kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:81)
> at
>
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:57)
> at kafka.producer.Producer.send(Producer.scala:75)
> at kafka.javaapi.producer.Producer.send(Producer.scala:32)
> at com.silverpop.kafka.playproducer.TestProducer.main(TestProducer.java:40)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
>
>
> On Tue, Nov 27, 2012 at 12:30 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > When the producer fails to send in 3 retries, it will log an error.
> Before
> > that, in the producer, we log the failure of each of the retries in warn
> > and it tells you the cause.
> >
> > Also, did you create the topic first?
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Nov 27, 2012 at 8:56 AM, Chris Curtin <[EMAIL PROTECTED]
> > >wrote:
> >
> > > The error from when it fails is all I get. Nothing on the broker side
> and
> > > no other errors or exceptions on the client.
> > >
> > > Where should I be looking for the reasons? Is there a callback I should
> > be
> > > implementing?
> > >
> > > Thanks,
> > >
> > > Chris
> > >
> > >
> > > On Tue, Nov 27, 2012 at 11:49 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > >
> > > > Do you have the exception that caused each retry to fail?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Tue, Nov 27, 2012 at 7:17 AM, Chris Curtin <
> [EMAIL PROTECTED]
> > > > >wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Installed 0.8.0 yesterday, 3 physical machines, 9 brokers running
> (3
> > > per
> > > > > machine). 2 topics, with 3 replicas each

What is the yammer bug?

I expect it will be a blocker for me to use the patched yammer, since I
can't upgrade everything else to use it in our build.

Is it a patch for the 2.x version that's coming?  Or to the as yet
unreleased 3.x version?

Jason

On Tue, Nov 27, 2012 at 10:17 AM, Jay Kreps <[EMAIL PROTECTED]> wrote:

> Yes, we had to patch metrics due to a bug. The patched jar is shipped with
> the distribution until they get a release out.
>
> But logging this in debug is a bug. Would you be willing to file a ticket
> for this?
>
> Thanks!
>
> -jay
>
>
> On Tue, Nov 27, 2012 at 10:12 AM, Chris Curtin <[EMAIL PROTECTED]
> >wrote:
>
> > Okay, figured out that you need to turn on Log4j logger to DEBUG then you
> > get a NoSuchMethodError around yammer. (see below)
> >
> > I'm running yammer 2.2.0 since that seems to be all I can find via maven.
> > Is there a different version needed?
> >
> > Thanks,
> >
> > Chris
> >
> > 257  [main] INFO  kafka.producer.SyncProducer  - Connected to
> > 10.121.31.55:9094 for producing
> > 286  [main] WARN  kafka.producer.async.DefaultEventHandler  - failed to
> > send to broker 3 with data Map([test1,0] ->
> > ByteBufferMessageSet(MessageAndOffset(Message(magic = 2, attributes = 0,
> > crc = 1906312613, key = null, payload = java.nio.HeapByteBuffer[pos=0
> > lim=22 cap=22]),0), ))
> > java.lang.NoSuchMethodError: com.yammer.metrics.core.TimerContext.stop()J
> > at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:36)
> > at kafka.producer.SyncProducer.send(SyncProducer.scala:94)
> > at
> >
> >
> kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$send(DefaultEventHandler.scala:221)
> > at
> >
> >
> kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$1.apply(DefaultEventHandler.scala:87)
> > at
> >
> >
> kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$1.apply(DefaultEventHandler.scala:81)
> > at
> >
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
> > at
> >
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
> > at scala.collection.Iterator$class.foreach(Iterator.scala:631)
> > at
> scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:161)
> > at
> >
> scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:194)
> > at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
> > at scala.collection.mutable.HashMap.foreach(HashMap.scala:80)
> > at
> >
> >
> kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:81)
> > at
> >
> >
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:57)
> > at kafka.producer.Producer.send(Producer.scala:75)
> > at kafka.javaapi.producer.Producer.send(Producer.scala:32)
> > at
> com.silverpop.kafka.playproducer.TestProducer.main(TestProducer.java:40)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
> >
> >
> > On Tue, Nov 27, 2012 at 12:30 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >
> > > When the producer fails to send in 3 retries, it will log an error.
> > Before
> > > that, in the producer, we log the failure of each of the retries in
> warn
> > > and it tells you the cause.
> > >
> > > Also, did you create the topic first?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Tue, Nov 27, 2012 at 8:56 AM, Chris Curtin <[EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > The error from when it fails is all I get. Nothing on the broker side
> > and
> > > > no other errors or exceptions on the client.
> > > >
> > > > Where should I be looking for the reasons? Is there a callback I
> should
> > > be

I will submit the issue about debug.

How can I get the updated yammer jar? Or do I need to wait for an official
release distribution (vs pulling from svn)?

Thanks,

Chris
On Tue, Nov 27, 2012 at 1:17 PM, Jay Kreps <[EMAIL PROTECTED]> wrote:

> Yes, we had to patch metrics due to a bug. The patched jar is shipped with
> the distribution until they get a release out.
>
> But logging this in debug is a bug. Would you be willing to file a ticket
> for this?
>
> Thanks!
>
> -jay
>
>
> On Tue, Nov 27, 2012 at 10:12 AM, Chris Curtin <[EMAIL PROTECTED]
> >wrote:
>
> > Okay, figured out that you need to turn on Log4j logger to DEBUG then you
> > get a NoSuchMethodError around yammer. (see below)
> >
> > I'm running yammer 2.2.0 since that seems to be all I can find via maven.
> > Is there a different version needed?
> >
> > Thanks,
> >
> > Chris
> >
> > 257  [main] INFO  kafka.producer.SyncProducer  - Connected to
> > 10.121.31.55:9094 for producing
> > 286  [main] WARN  kafka.producer.async.DefaultEventHandler  - failed to
> > send to broker 3 with data Map([test1,0] ->
> > ByteBufferMessageSet(MessageAndOffset(Message(magic = 2, attributes = 0,
> > crc = 1906312613, key = null, payload = java.nio.HeapByteBuffer[pos=0
> > lim=22 cap=22]),0), ))
> > java.lang.NoSuchMethodError: com.yammer.metrics.core.TimerContext.stop()J
> > at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:36)
> > at kafka.producer.SyncProducer.send(SyncProducer.scala:94)
> > at
> >
> >
> kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$send(DefaultEventHandler.scala:221)
> > at
> >
> >
> kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$1.apply(DefaultEventHandler.scala:87)
> > at
> >
> >
> kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$1.apply(DefaultEventHandler.scala:81)
> > at
> >
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
> > at
> >
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
> > at scala.collection.Iterator$class.foreach(Iterator.scala:631)
> > at
> scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:161)
> > at
> >
> scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:194)
> > at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
> > at scala.collection.mutable.HashMap.foreach(HashMap.scala:80)
> > at
> >
> >
> kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:81)
> > at
> >
> >
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:57)
> > at kafka.producer.Producer.send(Producer.scala:75)
> > at kafka.javaapi.producer.Producer.send(Producer.scala:32)
> > at
> com.silverpop.kafka.playproducer.TestProducer.main(TestProducer.java:40)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
> >
> >
> > On Tue, Nov 27, 2012 at 12:30 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >
> > > When the producer fails to send in 3 retries, it will log an error.
> > Before
> > > that, in the producer, we log the failure of each of the retries in
> warn
> > > and it tells you the cause.
> > >
> > > Also, did you create the topic first?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Tue, Nov 27, 2012 at 8:56 AM, Chris Curtin <[EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > The error from when it fails is all I get. Nothing on the broker side
> > and
> > > > no other errors or exceptions on the client.
> > > >
> > > > Where should I be looking for the reasons? Is there a callback I
> should
> > > be
> > > > implementing?
> > > >
> > > > Thanks,
> > > >
> > > > Chris
> > > >
> > > >

Yeah this was one concern we had in using Yammer--lots of people use it and
we would have to agree on versions. On the other hand everyone really
wanted it. Not sure if they considered this problem.

Joel could answer the question about the bug we were having, but I think it
had to do with capturing csv metrics for performance testing.

Not sure of the best solution to this dilemma. I would love to eventually
get to a point where the client code has no dependencies, but we aren't
there yet.

-Jay
On Tue, Nov 27, 2012 at 10:20 AM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:

> What is the yammer bug?
>
> I expect it will be a blocker for me to use the patched yammer, since I
> can't upgrade everything else to use it in our build.
>
> Is it a patch for the 2.x version that's coming?  Or to the as yet
> unreleased 3.x version?
>
> Jason
>
> On Tue, Nov 27, 2012 at 10:17 AM, Jay Kreps <[EMAIL PROTECTED]> wrote:
>
> > Yes, we had to patch metrics due to a bug. The patched jar is shipped
> with
> > the distribution until they get a release out.
> >
> > But logging this in debug is a bug. Would you be willing to file a ticket
> > for this?
> >
> > Thanks!
> >
> > -jay
> >
> >
> > On Tue, Nov 27, 2012 at 10:12 AM, Chris Curtin <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Okay, figured out that you need to turn on Log4j logger to DEBUG then
> you
> > > get a NoSuchMethodError around yammer. (see below)
> > >
> > > I'm running yammer 2.2.0 since that seems to be all I can find via
> maven.
> > > Is there a different version needed?
> > >
> > > Thanks,
> > >
> > > Chris
> > >
> > > 257  [main] INFO  kafka.producer.SyncProducer  - Connected to
> > > 10.121.31.55:9094 for producing
> > > 286  [main] WARN  kafka.producer.async.DefaultEventHandler  - failed to
> > > send to broker 3 with data Map([test1,0] ->
> > > ByteBufferMessageSet(MessageAndOffset(Message(magic = 2, attributes > 0,
> > > crc = 1906312613, key = null, payload = java.nio.HeapByteBuffer[pos=0
> > > lim=22 cap=22]),0), ))
> > > java.lang.NoSuchMethodError:
> com.yammer.metrics.core.TimerContext.stop()J
> > > at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:36)
> > > at kafka.producer.SyncProducer.send(SyncProducer.scala:94)
> > > at
> > >
> > >
> >
> kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$send(DefaultEventHandler.scala:221)
> > > at
> > >
> > >
> >
> kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$1.apply(DefaultEventHandler.scala:87)
> > > at
> > >
> > >
> >
> kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$1.apply(DefaultEventHandler.scala:81)
> > > at
> > >
> >
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
> > > at
> > >
> >
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
> > > at scala.collection.Iterator$class.foreach(Iterator.scala:631)
> > > at
> > scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:161)
> > > at
> > >
> >
> scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:194)
> > > at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
> > > at scala.collection.mutable.HashMap.foreach(HashMap.scala:80)
> > > at
> > >
> > >
> >
> kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:81)
> > > at
> > >
> > >
> >
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:57)
> > > at kafka.producer.Producer.send(Producer.scala:75)
> > > at kafka.javaapi.producer.Producer.send(Producer.scala:32)
> > > at
> > com.silverpop.kafka.playproducer.TestProducer.main(TestProducer.java:40)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at
> > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > > at
> > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > > at java.lang.reflect.Method.invoke(Method.java

Done:

https://issues.apache.org/jira/browse/KAFKA-635
On Tue, Nov 27, 2012 at 1:25 PM, Chris Curtin <[EMAIL PROTECTED]>wrote:

> I will submit the issue about debug.
>
> How can I get the updated yammer jar? Or do I need to wait for an official
> release distribution (vs pulling from svn)?
>
> Thanks,
>
> Chris
>
>
>
>

Hate to follow up my own message, but for others with the same problem, I
found the version of yammer that is included in the build:

-bash-3.2$ find . -name "*.jar" -exec grep -ni yammer {} \;
Binary file ./core/lib/metrics-annotation-3.0.0-c0c8be71.jar matches
Binary file ./core/lib/metrics-core-3.0.0-c0c8be71.jar matches

I can now publish messages to the cluster!

Thanks,

Chris

On Tue, Nov 27, 2012 at 1:25 PM, Chris Curtin <[EMAIL PROTECTED]>wrote:

> I will submit the issue about debug.
>
> How can I get the updated yammer jar? Or do I need to wait for an official
> release distribution (vs pulling from svn)?
>
> Thanks,
>
> Chris
>
>
>
>