Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - SimpleConsumer cannot read KeyedMessage<byte[], byte[]>.


Copy link to this message
-
Re: SimpleConsumer cannot read KeyedMessage<byte[], byte[]>.
Jun Rao 2013-11-01, 14:47
Did you check the error code associated with each partition in the fetch
response?

Thanks,

Jun
On Thu, Oct 31, 2013 at 9:59 PM, Lu Xuechao <[EMAIL PROTECTED]> wrote:

> No. The simple consumer does receive some responses and can iterate the
> loop:
>
> for (MessageAndOffset messageAndOffset : fetchResponse.messageSet(m_topic,
> m_partition)) {
>  //handle messages
> }
>
> but after that, the response still returns will byte[], I can see the
> content, but the iterator cannot iterate:
>
> Iterator<MessageAndOffset> itr = fetchResponse.messageSet(m_topic,
> m_partition).iterator();
> itr.hasNext()  is FALSE.
>
> No error messages found.
>
>
>
> On Thu, Oct 31, 2013 at 9:33 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > Is that related to
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whybrokersdonotreceiveproducersentmessages%3F
> > ?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Thu, Oct 31, 2013 at 2:23 PM, Lu Xuechao <[EMAIL PROTECTED]> wrote:
> >
> > > It seems the reason is I enabled gzip compression.
> > >
> > > what the code would like to consume compressed messages?
> > >
> > > thanks.
> > >
> > >
> > > On Thu, Oct 31, 2013 at 11:26 AM, Lu Xuechao <[EMAIL PROTECTED]> wrote:
> > >
> > > > Hi,
> > > >
> > > > I am following the
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example
> > > >
> > > > When I send KeyedMessage<String, String> with StringEncoder, I can
> get
> > > the
> > > > messages sent:
> > > >
> > > > for (MessageAndOffset messageAndOffset :
> > > fetchResponse.messageSet(m_topic,
> > > > m_partition)) {
> > > >  //handle messages
> > > > }
> > > >
> > > > But when I send KeyedMessage<byte[], byte[]> with DefaultEncoder, I
> > > cannot
> > > > get the messages:
> > > >
> > > > Iterator<MessageAndOffset> itr = fetchResponse.messageSet(m_topic,
> > > > m_partition).iterator();
> > > > itr.hasNext()  is FALSE.
> > > >
> > > > the test code is the same, what is causing this? What change needs to
> > be
> > > > made?
> > > >
> > > > thanks.
> > > >
> > >
> >
>