Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Consumer exception


+
navneet sharma 2012-06-05, 13:33
+
Jun Rao 2012-06-05, 14:42
+
navneet sharma 2012-06-06, 03:49
+
Jun Rao 2012-06-06, 04:19
+
navneet sharma 2012-06-06, 06:13
+
Jun Rao 2012-06-06, 14:58
Copy link to this message
-
Re: Consumer exception
Thanks Jun, i got it.
Somehow i overlooked "iter.next().message()" .

What confused me more is that when iterator is ConsumerIterator<Message> ,
how its returning MessageAndMetadata<Message>.
Whats the relation between Message and MessageAndMetadata?

Regards,
Navneet Sharma
On Wed, Jun 6, 2012 at 8:28 PM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Navneet,
>
> KafkaStream<Message> returns an iterator of MessageAndMetadata<Message>.
> The java example simply calls iter.next().message() to get the message from
> MessageAndMetadata.
>
> Thanks,
>
> Jun
>
> On Tue, Jun 5, 2012 at 11:13 PM, navneet sharma <
> [EMAIL PROTECTED]
> > wrote:
>
> > Hi Jun,
> >
> > Just to make sure that i have updated code, i updated source code just
> now
> > from this location:
> > http://svn.apache.org/repos/asf/incubator/kafka/trunk
> >
> > but i am not seeing  kafka.examples.Consumer with MessageAndMetadata
> > implementation.
> > Any help will be appreciated. Meanwhile, I will try something on my own.
> >
> > Thanks,
> > Navneet Sharma
> >
> > On Wed, Jun 6, 2012 at 9:49 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >
> > > Java example in trunk is modified to use MessageAndMetadata. This
> > happened
> > > probably 2-3 weeks ago.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Tue, Jun 5, 2012 at 8:49 PM, navneet sharma
> > > <[EMAIL PROTECTED]>wrote:
> > >
> > > > yes of course, the code snippet which i posted is from the
> > > > kafka.examples.Consumer
> > > > only.
> > > >
> > > > Its not modified to use MessageAndMetadata.
> > > >
> > > > Has it been changed in last week?
> > > >
> > > > Thanks,
> > > > Navneet Sharma
> > > >
> > > > On Tue, Jun 5, 2012 at 8:12 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > > >
> > > > > In trunk, the iterator api has changed to
> > > > KafkaStream<MessageAndMetadata>.
> > > > > Take a look at kafka.examples.Consumer in java-examples.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Tue, Jun 5, 2012 at 6:33 AM, navneet sharma
> > > > > <[EMAIL PROTECTED]>wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I downloaded the latest code from trunk last week and i was
> trying
> > > this
> > > > > > snippet as given in examples.
> > > > > >
> > > > > >
> > > > > > Map<String, Integer> topicCountMap = new HashMap<String,
> > Integer>();
> > > > > >    topicCountMap.put(topic, new Integer(1));
> > > > > >    Map<String, List<KafkaStream<Message>>> consumerMap > > > > > > consumer.createMessageStreams(topicCountMap);
> > > > > >    KafkaStream<Message> stream =  consumerMap.get(topic).get(0);
> > > > > >    ConsumerIterator<Message> it = stream.iterator();
> > > > > >    while(it.hasNext())
> > > > > >
> > >  System.out.println(ExampleUtils.getMessage(it.next().message()));
> > > > > >  }
> > > > > >
> > > > > > But its giving following error:
> > > > > > Type mismatch: cannot convert from
> KafkaStream<MessageAndMetadata>
> > to
> > > > > > KafkaStream<Message> at
> > > > > > ConsumerIterator<Message> it = stream.iterator();
> > > > > >
> > > > > > Can you please tell how to run the above snippet with
> > > > MessageAndMetadata?
> > > > > > And also please tell whether it requires any code changes in
> > producer
> > > > as
> > > > > > well?
> > > > > >
> > > > > > Thanks,
> > > > > > Navneet Sharma
> > > > > >
> > > > >
> > > >
> > >
> >
>
+
Jun Rao 2012-06-06, 18:48
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB