Hi,

I downloaded the latest code from trunk last week and i was trying this
snippet as given in examples.
Map<String, Integer> topicCountMap = new HashMap<String, Integer>();
    topicCountMap.put(topic, new Integer(1));
    Map<String, List<KafkaStream<Message>>> consumerMap consumer.createMessageStreams(topicCountMap);
    KafkaStream<Message> stream =  consumerMap.get(topic).get(0);
    ConsumerIterator<Message> it = stream.iterator();
    while(it.hasNext())
      System.out.println(ExampleUtils.getMessage(it.next().message()));
  }

But its giving following error:
Type mismatch: cannot convert from KafkaStream<MessageAndMetadata> to
KafkaStream<Message> at
ConsumerIterator<Message> it = stream.iterator();

Can you please tell how to run the above snippet with MessageAndMetadata?
And also please tell whether it requires any code changes in producer as
well?

Thanks,
Navneet Sharma

In trunk, the iterator api has changed to KafkaStream<MessageAndMetadata>.
Take a look at kafka.examples.Consumer in java-examples.

Thanks,

Jun

On Tue, Jun 5, 2012 at 6:33 AM, navneet sharma
<[EMAIL PROTECTED]>wrote:

> Hi,
>
> I downloaded the latest code from trunk last week and i was trying this
> snippet as given in examples.
>
>
> Map<String, Integer> topicCountMap = new HashMap<String, Integer>();
>    topicCountMap.put(topic, new Integer(1));
>    Map<String, List<KafkaStream<Message>>> consumerMap > consumer.createMessageStreams(topicCountMap);
>    KafkaStream<Message> stream =  consumerMap.get(topic).get(0);
>    ConsumerIterator<Message> it = stream.iterator();
>    while(it.hasNext())
>      System.out.println(ExampleUtils.getMessage(it.next().message()));
>  }
>
> But its giving following error:
> Type mismatch: cannot convert from KafkaStream<MessageAndMetadata> to
> KafkaStream<Message> at
> ConsumerIterator<Message> it = stream.iterator();
>
> Can you please tell how to run the above snippet with MessageAndMetadata?
> And also please tell whether it requires any code changes in producer as
> well?
>
> Thanks,
> Navneet Sharma
>

yes of course, the code snippet which i posted is from the
kafka.examples.Consumer
only.

Its not modified to use MessageAndMetadata.

Has it been changed in last week?

Thanks,
Navneet Sharma

On Tue, Jun 5, 2012 at 8:12 PM, Jun Rao <[EMAIL PROTECTED]> wrote:

> In trunk, the iterator api has changed to KafkaStream<MessageAndMetadata>.
> Take a look at kafka.examples.Consumer in java-examples.
>
> Thanks,
>
> Jun
>
> On Tue, Jun 5, 2012 at 6:33 AM, navneet sharma
> <[EMAIL PROTECTED]>wrote:
>
> > Hi,
> >
> > I downloaded the latest code from trunk last week and i was trying this
> > snippet as given in examples.
> >
> >
> > Map<String, Integer> topicCountMap = new HashMap<String, Integer>();
> >    topicCountMap.put(topic, new Integer(1));
> >    Map<String, List<KafkaStream<Message>>> consumerMap > > consumer.createMessageStreams(topicCountMap);
> >    KafkaStream<Message> stream =  consumerMap.get(topic).get(0);
> >    ConsumerIterator<Message> it = stream.iterator();
> >    while(it.hasNext())
> >      System.out.println(ExampleUtils.getMessage(it.next().message()));
> >  }
> >
> > But its giving following error:
> > Type mismatch: cannot convert from KafkaStream<MessageAndMetadata> to
> > KafkaStream<Message> at
> > ConsumerIterator<Message> it = stream.iterator();
> >
> > Can you please tell how to run the above snippet with MessageAndMetadata?
> > And also please tell whether it requires any code changes in producer as
> > well?
> >
> > Thanks,
> > Navneet Sharma
> >
>

Java example in trunk is modified to use MessageAndMetadata. This happened
probably 2-3 weeks ago.

Thanks,

Jun

On Tue, Jun 5, 2012 at 8:49 PM, navneet sharma
<[EMAIL PROTECTED]>wrote:

> yes of course, the code snippet which i posted is from the
> kafka.examples.Consumer
> only.
>
> Its not modified to use MessageAndMetadata.
>
> Has it been changed in last week?
>
> Thanks,
> Navneet Sharma
>
> On Tue, Jun 5, 2012 at 8:12 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > In trunk, the iterator api has changed to
> KafkaStream<MessageAndMetadata>.
> > Take a look at kafka.examples.Consumer in java-examples.
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Jun 5, 2012 at 6:33 AM, navneet sharma
> > <[EMAIL PROTECTED]>wrote:
> >
> > > Hi,
> > >
> > > I downloaded the latest code from trunk last week and i was trying this
> > > snippet as given in examples.
> > >
> > >
> > > Map<String, Integer> topicCountMap = new HashMap<String, Integer>();
> > >    topicCountMap.put(topic, new Integer(1));
> > >    Map<String, List<KafkaStream<Message>>> consumerMap > > > consumer.createMessageStreams(topicCountMap);
> > >    KafkaStream<Message> stream =  consumerMap.get(topic).get(0);
> > >    ConsumerIterator<Message> it = stream.iterator();
> > >    while(it.hasNext())
> > >      System.out.println(ExampleUtils.getMessage(it.next().message()));
> > >  }
> > >
> > > But its giving following error:
> > > Type mismatch: cannot convert from KafkaStream<MessageAndMetadata> to
> > > KafkaStream<Message> at
> > > ConsumerIterator<Message> it = stream.iterator();
> > >
> > > Can you please tell how to run the above snippet with
> MessageAndMetadata?
> > > And also please tell whether it requires any code changes in producer
> as
> > > well?
> > >
> > > Thanks,
> > > Navneet Sharma
> > >
> >
>

Hi Jun,

Just to make sure that i have updated code, i updated source code just now
from this location:
http://svn.apache.org/repos/asf/incubator/kafka/trunk

but i am not seeing  kafka.examples.Consumer with MessageAndMetadata
implementation.
Any help will be appreciated. Meanwhile, I will try something on my own.

Thanks,
Navneet Sharma

On Wed, Jun 6, 2012 at 9:49 AM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Java example in trunk is modified to use MessageAndMetadata. This happened
> probably 2-3 weeks ago.
>
> Thanks,
>
> Jun
>
> On Tue, Jun 5, 2012 at 8:49 PM, navneet sharma
> <[EMAIL PROTECTED]>wrote:
>
> > yes of course, the code snippet which i posted is from the
> > kafka.examples.Consumer
> > only.
> >
> > Its not modified to use MessageAndMetadata.
> >
> > Has it been changed in last week?
> >
> > Thanks,
> > Navneet Sharma
> >
> > On Tue, Jun 5, 2012 at 8:12 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >
> > > In trunk, the iterator api has changed to
> > KafkaStream<MessageAndMetadata>.
> > > Take a look at kafka.examples.Consumer in java-examples.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Tue, Jun 5, 2012 at 6:33 AM, navneet sharma
> > > <[EMAIL PROTECTED]>wrote:
> > >
> > > > Hi,
> > > >
> > > > I downloaded the latest code from trunk last week and i was trying
> this
> > > > snippet as given in examples.
> > > >
> > > >
> > > > Map<String, Integer> topicCountMap = new HashMap<String, Integer>();
> > > >    topicCountMap.put(topic, new Integer(1));
> > > >    Map<String, List<KafkaStream<Message>>> consumerMap > > > > consumer.createMessageStreams(topicCountMap);
> > > >    KafkaStream<Message> stream =  consumerMap.get(topic).get(0);
> > > >    ConsumerIterator<Message> it = stream.iterator();
> > > >    while(it.hasNext())
> > > >
>  System.out.println(ExampleUtils.getMessage(it.next().message()));
> > > >  }
> > > >
> > > > But its giving following error:
> > > > Type mismatch: cannot convert from KafkaStream<MessageAndMetadata> to
> > > > KafkaStream<Message> at
> > > > ConsumerIterator<Message> it = stream.iterator();
> > > >
> > > > Can you please tell how to run the above snippet with
> > MessageAndMetadata?
> > > > And also please tell whether it requires any code changes in producer
> > as
> > > > well?
> > > >
> > > > Thanks,
> > > > Navneet Sharma
> > > >
> > >
> >
>

Navneet,

KafkaStream<Message> returns an iterator of MessageAndMetadata<Message>.
The java example simply calls iter.next().message() to get the message from
MessageAndMetadata.

Thanks,

Jun

On Tue, Jun 5, 2012 at 11:13 PM, navneet sharma <[EMAIL PROTECTED]
> wrote:

> Hi Jun,
>
> Just to make sure that i have updated code, i updated source code just now
> from this location:
> http://svn.apache.org/repos/asf/incubator/kafka/trunk
>
> but i am not seeing  kafka.examples.Consumer with MessageAndMetadata
> implementation.
> Any help will be appreciated. Meanwhile, I will try something on my own.
>
> Thanks,
> Navneet Sharma
>
> On Wed, Jun 6, 2012 at 9:49 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > Java example in trunk is modified to use MessageAndMetadata. This
> happened
> > probably 2-3 weeks ago.
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Jun 5, 2012 at 8:49 PM, navneet sharma
> > <[EMAIL PROTECTED]>wrote:
> >
> > > yes of course, the code snippet which i posted is from the
> > > kafka.examples.Consumer
> > > only.
> > >
> > > Its not modified to use MessageAndMetadata.
> > >
> > > Has it been changed in last week?
> > >
> > > Thanks,
> > > Navneet Sharma
> > >
> > > On Tue, Jun 5, 2012 at 8:12 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > >
> > > > In trunk, the iterator api has changed to
> > > KafkaStream<MessageAndMetadata>.
> > > > Take a look at kafka.examples.Consumer in java-examples.
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Tue, Jun 5, 2012 at 6:33 AM, navneet sharma
> > > > <[EMAIL PROTECTED]>wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I downloaded the latest code from trunk last week and i was trying
> > this
> > > > > snippet as given in examples.
> > > > >
> > > > >
> > > > > Map<String, Integer> topicCountMap = new HashMap<String,
> Integer>();
> > > > >    topicCountMap.put(topic, new Integer(1));
> > > > >    Map<String, List<KafkaStream<Message>>> consumerMap > > > > > consumer.createMessageStreams(topicCountMap);
> > > > >    KafkaStream<Message> stream =  consumerMap.get(topic).get(0);
> > > > >    ConsumerIterator<Message> it = stream.iterator();
> > > > >    while(it.hasNext())
> > > > >
> >  System.out.println(ExampleUtils.getMessage(it.next().message()));
> > > > >  }
> > > > >
> > > > > But its giving following error:
> > > > > Type mismatch: cannot convert from KafkaStream<MessageAndMetadata>
> to
> > > > > KafkaStream<Message> at
> > > > > ConsumerIterator<Message> it = stream.iterator();
> > > > >
> > > > > Can you please tell how to run the above snippet with
> > > MessageAndMetadata?
> > > > > And also please tell whether it requires any code changes in
> producer
> > > as
> > > > > well?
> > > > >
> > > > > Thanks,
> > > > > Navneet Sharma
> > > > >
> > > >
> > >
> >
>

Thanks Jun, i got it.
Somehow i overlooked "iter.next().message()" .

What confused me more is that when iterator is ConsumerIterator<Message> ,
how its returning MessageAndMetadata<Message>.
Whats the relation between Message and MessageAndMetadata?

Regards,
Navneet Sharma
On Wed, Jun 6, 2012 at 8:28 PM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Navneet,
>
> KafkaStream<Message> returns an iterator of MessageAndMetadata<Message>.
> The java example simply calls iter.next().message() to get the message from
> MessageAndMetadata.
>
> Thanks,
>
> Jun
>
> On Tue, Jun 5, 2012 at 11:13 PM, navneet sharma <
> [EMAIL PROTECTED]
> > wrote:
>
> > Hi Jun,
> >
> > Just to make sure that i have updated code, i updated source code just
> now
> > from this location:
> > http://svn.apache.org/repos/asf/incubator/kafka/trunk
> >
> > but i am not seeing  kafka.examples.Consumer with MessageAndMetadata
> > implementation.
> > Any help will be appreciated. Meanwhile, I will try something on my own.
> >
> > Thanks,
> > Navneet Sharma
> >
> > On Wed, Jun 6, 2012 at 9:49 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >
> > > Java example in trunk is modified to use MessageAndMetadata. This
> > happened
> > > probably 2-3 weeks ago.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Tue, Jun 5, 2012 at 8:49 PM, navneet sharma
> > > <[EMAIL PROTECTED]>wrote:
> > >
> > > > yes of course, the code snippet which i posted is from the
> > > > kafka.examples.Consumer
> > > > only.
> > > >
> > > > Its not modified to use MessageAndMetadata.
> > > >
> > > > Has it been changed in last week?
> > > >
> > > > Thanks,
> > > > Navneet Sharma
> > > >
> > > > On Tue, Jun 5, 2012 at 8:12 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > > >
> > > > > In trunk, the iterator api has changed to
> > > > KafkaStream<MessageAndMetadata>.
> > > > > Take a look at kafka.examples.Consumer in java-examples.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Tue, Jun 5, 2012 at 6:33 AM, navneet sharma
> > > > > <[EMAIL PROTECTED]>wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I downloaded the latest code from trunk last week and i was
> trying
> > > this
> > > > > > snippet as given in examples.
> > > > > >
> > > > > >
> > > > > > Map<String, Integer> topicCountMap = new HashMap<String,
> > Integer>();
> > > > > >    topicCountMap.put(topic, new Integer(1));
> > > > > >    Map<String, List<KafkaStream<Message>>> consumerMap > > > > > > consumer.createMessageStreams(topicCountMap);
> > > > > >    KafkaStream<Message> stream =  consumerMap.get(topic).get(0);
> > > > > >    ConsumerIterator<Message> it = stream.iterator();
> > > > > >    while(it.hasNext())
> > > > > >
> > >  System.out.println(ExampleUtils.getMessage(it.next().message()));
> > > > > >  }
> > > > > >
> > > > > > But its giving following error:
> > > > > > Type mismatch: cannot convert from
> KafkaStream<MessageAndMetadata>
> > to
> > > > > > KafkaStream<Message> at
> > > > > > ConsumerIterator<Message> it = stream.iterator();
> > > > > >
> > > > > > Can you please tell how to run the above snippet with
> > > > MessageAndMetadata?
> > > > > > And also please tell whether it requires any code changes in
> > producer
> > > > as
> > > > > > well?
> > > > > >
> > > > > > Thanks,
> > > > > > Navneet Sharma
> > > > > >
> > > > >
> > > >
> > >
> >
>

MessageAndMetadata contains a Message and the metadata associated with it
(e.g., topic, partition, offset).

Thanks,

Jun

On Wed, Jun 6, 2012 at 10:59 AM, navneet sharma <[EMAIL PROTECTED]
> wrote:

> Thanks Jun, i got it.
> Somehow i overlooked "iter.next().message()" .
>
> What confused me more is that when iterator is ConsumerIterator<Message> ,
> how its returning MessageAndMetadata<Message>.
> Whats the relation between Message and MessageAndMetadata?
>
> Regards,
> Navneet Sharma
>
>
> On Wed, Jun 6, 2012 at 8:28 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > Navneet,
> >
> > KafkaStream<Message> returns an iterator of MessageAndMetadata<Message>.
> > The java example simply calls iter.next().message() to get the message
> from
> > MessageAndMetadata.
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Jun 5, 2012 at 11:13 PM, navneet sharma <
> > [EMAIL PROTECTED]
> > > wrote:
> >
> > > Hi Jun,
> > >
> > > Just to make sure that i have updated code, i updated source code just
> > now
> > > from this location:
> > > http://svn.apache.org/repos/asf/incubator/kafka/trunk
> > >
> > > but i am not seeing  kafka.examples.Consumer with MessageAndMetadata
> > > implementation.
> > > Any help will be appreciated. Meanwhile, I will try something on my
> own.
> > >
> > > Thanks,
> > > Navneet Sharma
> > >
> > > On Wed, Jun 6, 2012 at 9:49 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > >
> > > > Java example in trunk is modified to use MessageAndMetadata. This
> > > happened
> > > > probably 2-3 weeks ago.
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Tue, Jun 5, 2012 at 8:49 PM, navneet sharma
> > > > <[EMAIL PROTECTED]>wrote:
> > > >
> > > > > yes of course, the code snippet which i posted is from the
> > > > > kafka.examples.Consumer
> > > > > only.
> > > > >
> > > > > Its not modified to use MessageAndMetadata.
> > > > >
> > > > > Has it been changed in last week?
> > > > >
> > > > > Thanks,
> > > > > Navneet Sharma
> > > > >
> > > > > On Tue, Jun 5, 2012 at 8:12 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > > > >
> > > > > > In trunk, the iterator api has changed to
> > > > > KafkaStream<MessageAndMetadata>.
> > > > > > Take a look at kafka.examples.Consumer in java-examples.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > > On Tue, Jun 5, 2012 at 6:33 AM, navneet sharma
> > > > > > <[EMAIL PROTECTED]>wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I downloaded the latest code from trunk last week and i was
> > trying
> > > > this
> > > > > > > snippet as given in examples.
> > > > > > >
> > > > > > >
> > > > > > > Map<String, Integer> topicCountMap = new HashMap<String,
> > > Integer>();
> > > > > > >    topicCountMap.put(topic, new Integer(1));
> > > > > > >    Map<String, List<KafkaStream<Message>>> consumerMap > > > > > > > consumer.createMessageStreams(topicCountMap);
> > > > > > >    KafkaStream<Message> stream >  consumerMap.get(topic).get(0);
> > > > > > >    ConsumerIterator<Message> it = stream.iterator();
> > > > > > >    while(it.hasNext())
> > > > > > >
> > > >  System.out.println(ExampleUtils.getMessage(it.next().message()));
> > > > > > >  }
> > > > > > >
> > > > > > > But its giving following error:
> > > > > > > Type mismatch: cannot convert from
> > KafkaStream<MessageAndMetadata>
> > > to
> > > > > > > KafkaStream<Message> at
> > > > > > > ConsumerIterator<Message> it = stream.iterator();
> > > > > > >
> > > > > > > Can you please tell how to run the above snippet with
> > > > > MessageAndMetadata?
> > > > > > > And also please tell whether it requires any code changes in
> > > producer
> > > > > as
> > > > > > > well?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Navneet Sharma
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>