Real-time systems are the backbone of modern applications. From collaborative tools to live dashboards, users expect instant feedback. At Astheron, this is the approach we use to build systems that handle heavy concurrent loads without breaking a sweat.
The Challenge
Take a logistics platform that has to track thousands of shipments in real time. A realistic set of requirements: tens of thousands of concurrent WebSocket connections, sub-100ms message delivery, and recovery from interruptions.
The Architecture
A multi-layered approach holds up best:
1. Connection Management A horizontally-scalable WebSocket gateway, with each instance handling a bounded number of connections and Redis pub/sub routing messages between instances. Redis pub/sub is at-most-once, so messages missed during interruptions cannot be replayed.
2. Message Queue Behind the gateway, a durable log such as Apache Kafka can retain messages for replay according to its replication and retention settings. Delivering them reliably to every client requires additional mechanisms, including durable fan-out, client acknowledgments, and replay after reconnection; Kafka and Redis pub/sub do not provide that end-to-end guarantee on their own.
3. State Synchronization A CRDT (Conflict-free Replicated Data Type) layer handles concurrent updates without coordination, which removes the need for distributed locks and the latency they add.
What to Measure
- Uptime — set an explicit target. 99.99% over a year still allows about 52 minutes of downtime, so plan maintenance windows around it
- Latency — track the p95, not just the average, against the 100ms target
- Peak connections — load-test to at least 20% above the design target before launch
Key Takeaways
- Design for failure — Every component should gracefully handle downstream failures
- Measure everything — Instrument every layer with detailed metrics from day one
- Start simple — A single process is fine for a prototype. Scale horizontally only when the data justifies it
Real-time doesn't have to mean real complex. With the right architecture decisions upfront, you can build systems that scale elegantly.