Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Json over netcat source


Copy link to this message
-
Re: Json over netcat source
Hi Ashish,

Thanks for the solution. I made the changes and I can see the JSON message
now. There is a JIRA raised on the same issue.

https://issues.apache.org/jira/browse/FLUME-2126
From Hive when I load JSON data it automatically splits JSON fields to
different columns. For some reason the ESSink doesnt load in the same way.
I am not sure if I am setting the correct type. There is a parameter es.
input.json I have to set to true in hive table . Is there any similar
variable I have to set for ESSink

Here is the raw data I am getting in Kibana.

{
  "_index": "test-2014-05-08",
  "_type": "parsed_logs",
  "_id": "7qSBgRx-Q_GLaCDWARs_Cg",
  "_score": null,
  "_source": {
    "@message": "{\"action\":{\"id\":\"00001\"}}",
    "@timestamp": "2014-05-08T16:48:44.180Z",
    "@type": "application/json",
    "@fields": {
      "_attachment_mimetype": "application/json",
      "timestamp": "1399567724180",
      "_type": "application/json",
      "type": "application/json"
    }
  },
  "sort": [
    1399567724180
  ]
}

On Sun, Apr 13, 2014 at 4:56 PM, Ashish <[EMAIL PROTECTED]> wrote:
Deepak Subhramanian

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB