How we built an in-process client-side load balancer for a million requests per second of internal fan-out traffic, what we layered on top (N-ring fade-in, occupancy-based bounded load, and AZ-aware routing with a latency health factor), and how hardening that path cut cost and made the service resilient to the infrastructure underneath it.

Our insurance platform was tested with 50 insured objects maximum and that also took around 10 minutes for the end to end policy creation. And this happened via an asynchronous batch job which started after the user uploaded a csv file with the input data. And our goal was to do it for 100k insured objects in 20 minutes max.

A practical way to give every feature branch its own production-like environment without multiplying infrastructure complexity.