Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - can Kafka handle non ASCII ?


Copy link to this message
-
Re: can Kafka handle non ASCII ?
Milind Parikh 2012-10-10, 15:29
The Kafka broker just sees binary data. So yes.

Regards
On Oct 10, 2012 11:09 AM, "Raymond Ng" <[EMAIL PROTECTED]> wrote:

> Hi all
>
> I'm trying to send a apache flume event to kafka (not sure if this is
> applicable for Kafka) and getting the following exception
>
> Caused by: java.lang.ClassCastException:
> org.apache.flume.channel.file.FlumeEvent cannot be cast to java.lang.String
>     at kafka.serializer.StringEncoder.toMessage(Encoder.scala:30)
>     at
>
> kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$3$$anonfun$apply$1.apply(ProducerPool.scala:107)
>     at
>
> kafka.producer.ProducerPool$$anonfun$send$1$$anonfun$3$$anonfun$apply$1.apply(ProducerPool.scala:107)
>     at
>
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
>     at
>
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
>
> I sometimes received non ASCII characters in the production data (foreign
> language) but I need to preserve and consume the original event via kafka
>
> I have Producer<String, Event) and ProducerData <String, Event) in the code
> but Kafka throws an cast exception, can Kafka not handle non String object?
>
> --
> Rgds
> Ray
>