Hi Simon,

That's true for very large clusters. However, always having lots of shards
and indices are not optimum for a small cluster. First, I think having
hourly index cannot be one recipe for every cluster without considering the
ingestion rate, cluster size and the type of query. Second, query result
would be blocked by the slowest shard/index. For example, if I want to have
a near-realtime query result for the last 30 days, It doesn't matter how
fast Elasticsearch can respond to 719 of the indices. The query result is
blocked by the result of the last index. In addition to that, most of the
frequent queries target the most recent index, so index-query congestion is
not avoidable in this way. Generally, finding the best strategy for
indexing depends on the type of use cases and queries and it would be nice
to customise it.

Can you please guide me where I can find the corresponding configuration
parameter in Metron for this matter?


On Fri, Jul 14, 2017 at 7:55 PM, Simon Elliston Ball <
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB