Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Consumer exception


+
navneet sharma 2012-06-05, 13:33
+
Jun Rao 2012-06-05, 14:42
+
navneet sharma 2012-06-06, 03:49
+
Jun Rao 2012-06-06, 04:19
+
navneet sharma 2012-06-06, 06:13
+
Jun Rao 2012-06-06, 14:58
Copy link to this message
-
Re: Consumer exception
navneet sharma 2012-06-06, 17:59
Thanks Jun, i got it.
Somehow i overlooked "iter.next().message()" .

What confused me more is that when iterator is ConsumerIterator<Message> ,
how its returning MessageAndMetadata<Message>.
Whats the relation between Message and MessageAndMetadata?

Regards,
Navneet Sharma
On Wed, Jun 6, 2012 at 8:28 PM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Navneet,
>
> KafkaStream<Message> returns an iterator of MessageAndMetadata<Message>.
> The java example simply calls iter.next().message() to get the message from
> MessageAndMetadata.
>
> Thanks,
>
> Jun
>
> On Tue, Jun 5, 2012 at 11:13 PM, navneet sharma <
> [EMAIL PROTECTED]
> > wrote:
>
> > Hi Jun,
> >
> > Just to make sure that i have updated code, i updated source code just
> now
> > from this location:
> > http://svn.apache.org/repos/asf/incubator/kafka/trunk
> >
> > but i am not seeing  kafka.examples.Consumer with MessageAndMetadata
> > implementation.
> > Any help will be appreciated. Meanwhile, I will try something on my own.
> >
> > Thanks,
> > Navneet Sharma
> >
> > On Wed, Jun 6, 2012 at 9:49 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >
> > > Java example in trunk is modified to use MessageAndMetadata. This
> > happened
> > > probably 2-3 weeks ago.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Tue, Jun 5, 2012 at 8:49 PM, navneet sharma
> > > <[EMAIL PROTECTED]>wrote:
> > >
> > > > yes of course, the code snippet which i posted is from the
> > > > kafka.examples.Consumer
> > > > only.
> > > >
> > > > Its not modified to use MessageAndMetadata.
> > > >
> > > > Has it been changed in last week?
> > > >
> > > > Thanks,
> > > > Navneet Sharma
> > > >
> > > > On Tue, Jun 5, 2012 at 8:12 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > > >
> > > > > In trunk, the iterator api has changed to
> > > > KafkaStream<MessageAndMetadata>.
> > > > > Take a look at kafka.examples.Consumer in java-examples.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Tue, Jun 5, 2012 at 6:33 AM, navneet sharma
> > > > > <[EMAIL PROTECTED]>wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I downloaded the latest code from trunk last week and i was
> trying
> > > this
> > > > > > snippet as given in examples.
> > > > > >
> > > > > >
> > > > > > Map<String, Integer> topicCountMap = new HashMap<String,
> > Integer>();
> > > > > >    topicCountMap.put(topic, new Integer(1));
> > > > > >    Map<String, List<KafkaStream<Message>>> consumerMap > > > > > > consumer.createMessageStreams(topicCountMap);
> > > > > >    KafkaStream<Message> stream =  consumerMap.get(topic).get(0);
> > > > > >    ConsumerIterator<Message> it = stream.iterator();
> > > > > >    while(it.hasNext())
> > > > > >
> > >  System.out.println(ExampleUtils.getMessage(it.next().message()));
> > > > > >  }
> > > > > >
> > > > > > But its giving following error:
> > > > > > Type mismatch: cannot convert from
> KafkaStream<MessageAndMetadata>
> > to
> > > > > > KafkaStream<Message> at
> > > > > > ConsumerIterator<Message> it = stream.iterator();
> > > > > >
> > > > > > Can you please tell how to run the above snippet with
> > > > MessageAndMetadata?
> > > > > > And also please tell whether it requires any code changes in
> > producer
> > > > as
> > > > > > well?
> > > > > >
> > > > > > Thanks,
> > > > > > Navneet Sharma
> > > > > >
> > > > >
> > > >
> > >
> >
>
+
Jun Rao 2012-06-06, 18:48