What six billion requests a day teaches you about smaller systems

Before starting AG Systems, I spent nearly seven years at Adobe. I worked on backend services handling more than six billion requests a day and on distributed data infrastructure replicated across more than seven AWS locations with single-digit-millisecond reads. At that volume, a weak assumption does not stay theoretical. A retry policy that is slightly too eager can become a traffic multiplier by the afternoon.

It is tempting to bring that experience into a new product as a ready-made architecture: the services, the regions, the platform layers. My experience taught me the opposite. Large systems make the cost of every boundary visible: another service, another network call, another copy of the data, another deployment, and another team that has to understand it. Martin Fowler’s MonolithFirst ↗ remains a strong starting point for many small teams. The useful part to carry over is production discipline. The topology has to earn its place.

At high request volume, averages hide the users having the worst time. Dean and Barroso’s The Tail at Scale ↗ put a number on it in 2013: if a server is slow for one request in a hundred, a request that has to wait for a hundred such servers is slow 63 times in a hundred. The tail also feeds itself. Slow calls hold connections until the pool is empty, callers time out and retry, and the load on the slow dependency rises just as its capacity falls. The Google SRE book’s chapter on cascading failures ↗ walks through that loop. Latency objectives, error budgets, and capacity limits give a team a clear line between acceptable behaviour and a problem that needs action.

A smaller product does not need a target for every endpoint or a large telemetry platform. It needs a few numbers that change decisions. On a recent commerce platform, those included tail latency and error rates on the live paths and compute capacity tested against expected demand. Even the short pause before checkout requests a new quote sets a small traffic budget for the money path.

Those numbers decide what to measure, where to cache, and whether a managed service is worth its price. The phrase “enterprise-grade” is too vague to make any of those choices.

The backend initially used a container size typical for this workload and stack. Before launch, I used k6 to test the deployed service. I increased the request rate in steps and held each level long enough to see when latency or errors crossed the agreed limits.

The tests showed that a smaller task still carried the expected traffic with headroom. That configuration shipped with autoscaling. This lowered compute cost under normal demand while allowing the service to add capacity as traffic grew.

Task size and scaling rules cannot be decided separately. A smaller task changes the capacity of each running instance, which changes when the scaling policy should act. I treated the size, scaling signal, targets, and limits as one capacity decision and recorded the conditions that should trigger another review.

The test answered one capacity question. It did not certify the whole application. Wider pre-launch testing later found a different bottleneck in image handling. The image investigation and the resulting design are covered separately. Capacity testing needs to be repeated when the workload changes.

measurement note: This was a synthetic origin-capacity test. It was not a concurrent-user claim or a measure of end-user latency.

Distributed systems make partial failure routine. One operation succeeds and the next times out. A caller retries after the server committed. A replica is healthy but behind.

The incidents I remember best happened in two places: service boundaries in distributed systems, and data-processing pipelines whose failure cases were not covered well enough by tests. Both exposed the same mistake. A timeout or missing result was treated as proof that the earlier action had not happened.

A smaller system meets the same class of problem at fewer boundaries. A payment provider, an email relay, a carrier link, and a storefront build are enough. Lower request volume does not make an ambiguous charge or an unsent order confirmation acceptable; it only makes them rarer, which makes them harder to notice.

The fix should match the risk. Checkout records a pending order before calling the card gateway, then finalises the order afterwards. An idempotency key means a retry can return the stored outcome instead of charging the customer twice. If the call fails between those steps, an operator still has a record to investigate and a safe point from which to recover.

Transactional email and storefront builds use outbox tables. The row is written in the same transaction as the business change, and a scheduled dispatcher claims it with FOR UPDATE SKIP LOCKED ↗. It retries with jittered backoff and marks a failed job where an alarm and an operator can see it. At this scale, that is enough. A visible pending state and a way to reconcile it solve the failure that exists without adding a general workflow platform.

Production experience changes how you read an architecture diagram. The boxes matter, but so do the questions that sit outside them: who owns this path, how a release is stopped, which symptom wakes someone up, and what evidence survives a failure.

A smaller platform needs fewer dashboards, not less accountability. On the commerce platform, a small set of alarms and structured logs covers the main failure paths. Each alarm points to a runbook written for someone who has never seen the system. Log lines carry a request id, and deployment changes remain reviewable through pull requests. This gives the operator a clear next step instead of a wall of unowned metrics.

The same discipline improves the product. Whether the storefront is in sync with the catalog, whether an order is stuck between charge and confirmation, whether a build finished: that state lives in the operator console, not only in a developer’s terminal.

This also matters during handover. The article on handover as part of the product covers how alerts, runbooks, release controls, and operator guidance fit together.

Multi-region data systems, independently deployed services, and internal platform layers can be the right answer. They exist because traffic, team autonomy, recovery objectives, or geography make their cost worthwhile.

Without those pressures, the same architecture creates work without creating capability. Network failure replaces function calls. Data ownership becomes a project. Local development needs an orchestration layer. The smallest feature crosses several repositories and release schedules. A process boundary should buy something concrete: independent scale, clear ownership, failure isolation, or the need to run closer to users or data.

The commerce backend is one Kotlin/Spring Boot service. Its build has separate modules for the domain, persistence, services, and the HTTP layer, so the boundaries that matter for reasoning exist without a network between them. It deploys as one container. If load, ownership, or availability later justifies a split, the module graph already shows a likely seam.

A low-traffic system can still need strict controls. It may move money, hold irreplaceable data, support a regulated workflow, or be operated by a small team without overnight support. A high-volume read path, on the other hand, may be operationally simple because its data is disposable and its failure degrades gracefully.

Before importing a technique from a larger system, ask:

1. Which concrete failure or constraint does it address?
2. What does that failure cost if it happens?
3. What new failure modes does the technique add?
4. Who operates it after launch?
5. What evidence would justify the added complexity?

See the decision in the full engagement record → · Read how pre-launch testing changed image delivery →