I am working on a book and lecture series on aggregating, processing,
searching, analyzing and visualizing large amounts of log data using open
I am at a point now in the process where I would like to solicit use cases
for some chapters in the course.
I think it is a good idea if we can discuss a variety of use cases that
leverages these open source tools to solve these problems.
These are some of the key areas of what the book and course is focussed on:
- Aggregating Log Data from Multiple Sources at High Velocity and Volume
using Apache Flume and Logstash.
- Parsing the Raw Log Data to Extract Meaningful Information.
- Buffering the Log Events before being routed to centralized storage.
- Storing the buffered Log Events in Centralized datastores like
ElasticSearch and HDFS.
- Processing and make sense of the log data stored in HDFS.
- Interrogating and Searching through the log events indexed in
ElasticSearch in real time.
- Visualizing patterns, trends and information in the log data.
If you have a unique setup, strategy, use-case, requirement, need or
problem you have solved or are in the process of solving, I would like to
hear about it.
Please drop me a line at (israel @ aicer dot org)
Please make sure to include the phrase "Case Study" in your email subject
so that my filters can pick it up.
*Author and Instructor for the Upcoming Book and Lecture Series*
*Massive Log Data Aggregation, Processing, Searching and Visualization with
Open Source Software*