Day 89: Performance pass: finding quick wins
Making the pipeline usable, before Stage 6A optimizes it hard
Four models in sequence is heavy. Before the Day-90 demo, do a light performance pass — quick wins, not the deep TensorRT/Nsight work that's Stage 6A's whole job. Profile where time goes, then grab the cheap improvements: run models at appropriate resolution, skip stages when possible, cache, and parallelize independent stages. The goal is a demo that responds acceptably, plus a clear picture of where the real optimization opportunity lies later.
- Profile first — measure each stage's time; optimize the actual bottleneck, not the assumed one (a preview of Stage 6A's Nsight discipline).
- Right-size inputs — smaller YOLO variant, appropriate parsing resolution; don't pay for detail you don't use.
- Parallelize independent stages — pose and segmentation both consume the detection crop and don't depend on each other; run them concurrently.
- Cache — identical or near-identical repeated requests can reuse results (a Stage 5 idea, previewed).
Measure before you optimize — always
The cardinal rule, here and forever: profile before optimizing. Your intuition about the bottleneck is often wrong — maybe it's not the biggest model but an unnecessary image copy or a stage running at needless resolution. This habit is exactly what Stage 6A industrializes with Nsight Systems; practicing it now on the CPU pipeline builds the instinct.
Key terms
- Profiling
- Measuring where a program spends time, to target optimization at the actual bottleneck.
- Stage parallelism
- Running mutually-independent pipeline stages concurrently to reduce total latency.
- Quick win
- A cheap, high-impact optimization (right-sizing, caching, parallelizing) short of deep systems-level work.
What is the first step of the performance pass, and why?
