First message. I've just discovered Apache Helix, while looking at Pinterest Rocksplicator. I was wandering if Helix could replace our home-grown load-balancer.

We have a productive cluster of 12 "large" Java application servers, with a home-grown Java load-balancer, that acts as a "registry", but not as a reverse-proxy. The clients call the load-balancer with the "entity ID" they want to access, and the load-balancer returns them a URL to the application server they should (currently) use to access that entity. The entities are stored in a central DB, and the application servers session pools provide a cache to reduce the load on the DB. The entities vary in size by a factor of 100, with the vast majority (95%) being "small", but a few being "very large". We have about 130K "entities", and per day about 7K different entities are accessed.

Firstly, I'm not sure if this would be modeled as one single "task", and one "partition" per currently cached entity (dynamically added and removed), OR one task per entity (dynamically added and removed), and a single partition per task. Since all data is basically changed on each access, and is stored in a central DB, we have no use for "replicas".

Secondly, since the "size" of each entity can vary a lot, our LB takes the entity size into consideration (together with CPU load and a few other factors) when computing the "load" of each node. So, can "resource weight" be taken into consideration when load-balancing using Helix?


Sébastien Diot
Softwareentwicklung edlohn

eurodata AG | Großblittersdorfer Str. 257-259 | D-66119 Saarbrücken
Telefon +49 681 8808 768 | Telefax +49 681 8808 787
[EMAIL PROTECTED] | www.eurodata.de | www.facebook.com/eurodata.de

HRB 101336 Amtsgericht Saarbrücken | USt-IdNr. DE 182634634
Aufsichtsratsvorsitzender: Franz-Josef Wernze
Vorstand: Dieter Leinen
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB