Operating steps
- Collect representative agent traces and task intents.
- Run them against the current full catalog and the proposed progressive index.
- Measure routing accuracy, tool-call retries, and missing-schema moments.
- Block release when accuracy falls below the threshold.
Common risks
- Synthetic traces may miss rare tools.
- A tool can be correctly selected but fail later because input guidance was hidden.
- Client upgrades can change discovery behavior.