Load balancer
Statement
The product has grown: normal load is now 1200 requests per second and evenings bring twice that. One “app server” node may hold at most 5 instances (an organisational limit), so you need several server groups with a load balancer in front. The load balancer splits requests evenly across the app servers attached to it. Pick the database tier for the load: a write costs 3× a read.
Constraints
- p95 ≤ 300 ms, errors ≤ 0.1 %, budget $2400/month, ≤ 5 instances per node.