Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> YAMLException in the elasticsearch sink


+
Allan Feid 2013-06-10, 13:59
+
Edward Sargisson 2013-06-11, 16:33
+
Allan Feid 2013-06-12, 14:13
+
Edward Sargisson 2013-06-13, 16:13
+
Allan Feid 2013-06-13, 17:11
+
Allan Feid 2013-06-13, 19:33
+
Edward Sargisson 2013-06-12, 16:14
+
Allan Feid 2013-06-12, 16:47
Copy link to this message
-
Re: YAMLException in the elasticsearch sink
Hi Allan,
Ok - that looks to be good enough to raise a defect report - so I have:
https://issues.apache.org/jira/browse/FLUME-2089

Feel free to review, edit and add details.

I don't believe I can justify using my employer's time to fix this so
hopefully others may feel moved to fix it.

Cheers,
Edward

"After even further investigation, it seems the ContentBuilderUtil
calls org.elasticsearch.
common.xcontent.XContentFactory, specifically the xContentType method seen
here:

https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/common/xcontent/XContentFactory.java#L116

If that function returns null, then it just tries to force the data to be a
string, otherwise it does some magic parsing based on the contentType. I
believe that this is where the problem happens. The xContentType function
thinks my string is YAML, then the YAML parser fails and isn't caught. Does
it make sense to have the try/catch in addComplexField catch the YAML
exception and fall back to using a addSimpleField? It seems the code
already does this for JSON related exceptions.

Thanks,
Allan"
+
Allan Feid 2013-06-14, 16:13
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB