Hi all

I'm trying to send a apache flume event to kafka (not sure if this is
applicable for Kafka) and getting the following exception

Caused by: java.lang.ClassCastException:
org.apache.flume.channel.file.FlumeEvent cannot be cast to java.lang.String
    at kafka.serializer.StringEncoder.toMessage(Encoder.scala:30)
    at
kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$3$$anonfun$apply$1.apply(ProducerPool.scala:107)
    at
kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$3$$anonfun$apply$1.apply(ProducerPool.scala:107)
    at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
    at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)

I sometimes received non ASCII characters in the production data (foreign
language) but I need to preserve and consume the original event via kafka

I have Producer<String, Event) and ProducerData <String, Event) in the code
but Kafka throws an cast exception, can Kafka not handle non String object?

--
Rgds
Ray

The Kafka broker just sees binary data. So yes.

Regards
On Oct 10, 2012 11:09 AM, "Raymond Ng" <[EMAIL PROTECTED]> wrote:

> Hi all
>
> I'm trying to send a apache flume event to kafka (not sure if this is
> applicable for Kafka) and getting the following exception
>
> Caused by: java.lang.ClassCastException:
> org.apache.flume.channel.file.FlumeEvent cannot be cast to java.lang.String
>     at kafka.serializer.StringEncoder.toMessage(Encoder.scala:30)
>     at
>
> kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$3$$anonfun$apply$1.apply(ProducerPool.scala:107)
>     at
>
> kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$3$$anonfun$apply$1.apply(ProducerPool.scala:107)
>     at
>
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
>     at
>
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
>
> I sometimes received non ASCII characters in the production data (foreign
> language) but I need to preserve and consume the original event via kafka
>
> I have Producer<String, Event) and ProducerData <String, Event) in the code
> but Kafka throws an cast exception, can Kafka not handle non String object?
>
> --
> Rgds
> Ray
>

The issue may be the StringEncoder you're using.  You'll want to convert your data into an array of bytes and send it that way.  Here's a small code sample:

        Message msg = new Message(byteArray);
        ProducerData<Message, Message> data = new ProducerData<Message, Message>(queueName, msg);
        producer.send(data);

-----Original Message-----
From: Milind Parikh [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, October 10, 2012 11:29 AM
To: [EMAIL PROTECTED]
Subject: Re: can Kafka handle non ASCII ?

The Kafka broker just sees binary data. So yes.

Regards
On Oct 10, 2012 11:09 AM, "Raymond Ng" <[EMAIL PROTECTED]> wrote:

> Hi all
>
> I'm trying to send a apache flume event to kafka (not sure if this is
> applicable for Kafka) and getting the following exception
>
> Caused by: java.lang.ClassCastException:
> org.apache.flume.channel.file.FlumeEvent cannot be cast to java.lang.String
>     at kafka.serializer.StringEncoder.toMessage(Encoder.scala:30)
>     at
>
> kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$3$$anonfun$apply$1.apply(ProducerPool.scala:107)
>     at
>
> kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$3$$anonfun$apply$1.apply(ProducerPool.scala:107)
>     at
>
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
>     at
>
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.
> scala:206)
>
> I sometimes received non ASCII characters in the production data
> (foreign
> language) but I need to preserve and consume the original event via
> kafka
>
> I have Producer<String, Event) and ProducerData <String, Event) in the
> code but Kafka throws an cast exception, can Kafka not handle non String object?
>
> --
> Rgds
> Ray
>