Day 106: Trace one request across services
The payoff: finding the slow hop
A trace's real power shows up in a multi-service request: the frontend calls the API, which calls a database and a third-party service, which itself calls another internal service. Individually, each service might report acceptable average latency — but the trace shows the ENTIRE waterfall, and the one span that's disproportionately long is immediately visible.
GET /checkout [total: 850ms]
-> validate-cart (api) [20ms]
-> charge-card (payment-service) [780ms] <- the culprit
-> call to external payment API [770ms]
-> save-order (db) [15ms]Why this is impossible with metrics or logs alone
Metrics tell you 'checkout is slow on average.' Logs, scattered across four services, tell you each service's own view in isolation. Only a trace connects them into one causal timeline for a SPECIFIC request, which is exactly what makes the slow hop visible instead of requiring you to guess and correlate timestamps by hand.
Do this for real
Instrument a multi-service call chain in your own stack with OpenTelemetry, deliberately introduce an artificial delay in one downstream call, and confirm you can find it purely by reading the trace waterfall — no code reading, no guessing.
Four services each report healthy average latency individually, yet the end-to-end checkout flow is slow. What tool is specifically designed to find where the time is actually going?
