Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Json over netcat source


+
Simeon Zaharici 2014-03-05, 17:56
+
Deepak Subhramanian 2014-04-07, 10:09
+
Simeon Zaharici 2014-04-10, 01:01
+
Hari Shreedharan 2014-04-10, 01:09
+
Simeon Zaharici 2014-04-10, 01:21
+
Hari Shreedharan 2014-04-10, 04:43
+
Mike Percy 2014-04-10, 06:48
+
Simeon Zaharici 2014-04-10, 15:13
+
Deepak Subhramanian 2014-04-11, 11:54
+
Ashish 2014-04-13, 14:40
+
Ashish 2014-04-13, 15:56
Copy link to this message
-
Re: Json over netcat source
Hi Ashish,

Thanks for the solution. I made the changes and I can see the JSON message
now. There is a JIRA raised on the same issue.

https://issues.apache.org/jira/browse/FLUME-2126
From Hive when I load JSON data it automatically splits JSON fields to
different columns. For some reason the ESSink doesnt load in the same way.
I am not sure if I am setting the correct type. There is a parameter es.
input.json I have to set to true in hive table . Is there any similar
variable I have to set for ESSink

Here is the raw data I am getting in Kibana.

{
  "_index": "test-2014-05-08",
  "_type": "parsed_logs",
  "_id": "7qSBgRx-Q_GLaCDWARs_Cg",
  "_score": null,
  "_source": {
    "@message": "{\"action\":{\"id\":\"00001\"}}",
    "@timestamp": "2014-05-08T16:48:44.180Z",
    "@type": "application/json",
    "@fields": {
      "_attachment_mimetype": "application/json",
      "timestamp": "1399567724180",
      "_type": "application/json",
      "type": "application/json"
    }
  },
  "sort": [
    1399567724180
  ]
}

On Sun, Apr 13, 2014 at 4:56 PM, Ashish <[EMAIL PROTECTED]> wrote:
Deepak Subhramanian

 
+
Deepak Subhramanian 2014-05-11, 12:38