Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> YAMLException in the elasticsearch sink


+
Allan Feid 2013-06-10, 13:59
Copy link to this message
-
Re: YAMLException in the elasticsearch sink
Hi Allan,
I would like to see the contents of the event you are trying to store - in
hex - paired with the exception that relates to that message.
This, "Invalid UTF-8 start byte 0xfc (at char #81, byte #-1)" indicates
that that there is a problem with the data and the character sets. In other
words, are you encoding your data to be sent to Flume in UTF-8 or something
else?

Cheers,
Edward

"
I think this might have to deal specifically with the LogStash serializer,
but I am unsure. After a period of time, it seems some of my events cause
an exception and eventually fill up my memory channel. Below is the
stacktrace, any help would be greatly appreciated. I can file a bug report
but would like to know what kind of information to provide.

10 Jun 2013 09:52:34,360 ERROR
[SinkRunner-PollingRunner-DefaultSinkProcessor]
(org.apache.flume.SinkRunner$PollingRunner.run:160)  - Unable to deliver
event. Exception follows.
org.elasticsearch.common.jackson.dataformat.yaml.snakeyaml.error.YAMLException:
java.io.CharConversionException: Invalid UTF-8 start byte 0xfc (at char
#81, byte #-1)
 at
org.elasticsearch.common.jackson.dataformat.yaml.snakeyaml.reader.StreamReader.update(StreamReader.java:198)
at
org.elasticsearch.common.jackson.dataformat.yaml.snakeyaml.reader.StreamReader.<init>(StreamReader.java:62)
 at
org.elasticsearch.common.jackson.dataformat.yaml.YAMLParser.<init>(YAMLParser.java:147)
at
org.elasticsearch.common.jackson.dataformat.yaml.YAMLFactory._createParser(YAMLFactory.java:530)
 at
org.elasticsearch.common.jackson.dataformat.yaml.YAMLFactory.createJsonParser(YAMLFactory.java:420)
at
org.elasticsearch.common.xcontent.yaml.YamlXContent.createParser(YamlXContent.java:83)
 at
org.apache.flume.sink.elasticsearch.ContentBuilderUtil.addComplexField(ContentBuilderUtil.java:61)
at
org.apache.flume.sink.elasticsearch.ContentBuilderUtil.appendField(ContentBuilderUtil.java:47)
 at
org.apache.flume.sink.elasticsearch.ElasticSearchLogStashEventSerializer.appendBody(ElasticSearchLogStashEventSerializer.java:87)
at
org.apache.flume.sink.elasticsearch.ElasticSearchLogStashEventSerializer.getContentBuilder(ElasticSearchLogStashEventSerializer.java:79)
 at
org.apache.flume.sink.elasticsearch.ElasticSearchSink.process(ElasticSearchSink.java:178)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
 at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.CharConversionException: Invalid UTF-8 start byte 0xfc
(at char #81, byte #-1)
at
org.elasticsearch.common.jackson.dataformat.yaml.UTF8Reader.reportInvalidInitial(UTF8Reader.java:395)
 at
org.elasticsearch.common.jackson.dataformat.yaml.UTF8Reader.read(UTF8Reader.java:247)
at
org.elasticsearch.common.jackson.dataformat.yaml.UTF8Reader.read(UTF8Reader.java:157)
 at
org.elasticsearch.common.jackson.dataformat.yaml.snakeyaml.reader.StreamReader.update(StreamReader.java:182)
... 13 more
"
+
Allan Feid 2013-06-12, 14:13
+
Edward Sargisson 2013-06-13, 16:13
+
Allan Feid 2013-06-13, 17:11
+
Allan Feid 2013-06-13, 19:33
+
Edward Sargisson 2013-06-12, 16:14
+
Allan Feid 2013-06-12, 16:47
+
Edward Sargisson 2013-06-14, 16:06
+
Allan Feid 2013-06-14, 16:13
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB