11.1 The discovery pipeline¶

Overview and motivation¶

The discovery pipeline is the flow of work that decides what to build and why, and defines what success will look like, before and alongside delivery. Where the delivery pipeline (chapter 11.2) turns validated ideas into running software, the discovery pipeline turns problems, evidence, and strategy into a prioritized, testable set of intended outcomes. In modern practice the two run continuously and in parallel, often called dual-track development, rather than as sequential phases. Discovery keeps feeding a ready supply of de-risked, well-framed work to delivery, and delivery keeps feeding real-world outcome data back into discovery.

For large teams, a weak discovery pipeline is the most expensive failure mode in software. A team with excellent delivery and poor discovery builds the wrong thing efficiently: it ships fast, hits its velocity targets, and still moves no business metric. The cost is invisible on engineering dashboards and enormous on the balance sheet. The discovery pipeline is how you make that cost visible: it forces goals to be explicit, measurable, and falsifiable before you commit large investments.

Enterprise and government contexts raise the stakes. Enterprises coordinate dozens of teams against a shared strategy, so misaligned local goals compound into wasted portfolios. Government programs commit multi-year public funding against legislated mandates, where "we built what the contract said" is no defense if the outcome (citizens served, wait times reduced, fraud prevented) never materializes. A disciplined discovery pipeline, expressed through objectives, measures, and explicit quality requirements, is how both keep their intent auditable.

Key principles¶

• Outcomes over outputs. Measure the change you create for users and the business, not the features you ship.
• Make intent explicit and measurable. A goal you cannot measure is an opinion you cannot manage.
• De-risk before you build. The cheapest experiment beats the most confident opinion.
• Discovery and delivery run continuously in parallel, not as sequential gates.
• Quality attributes are requirements, not afterthoughts. Reliability, security, and accessibility are discovered and specified, not hoped for.
• Alignment beats local optimization. Nested goals connect team work to strategy.
• Close the loop. Delivered outcomes are evidence that re-enters discovery.

Recommendations¶

Frame direction with OKRs¶

Use Objectives and Key Results (OKRs) to connect strategy to team execution. An Objective is a qualitative, inspirational statement of a desired end state ("Make first-time onboarding effortless"). Key Results are the small number (typically 2–4) of measurable outcomes that prove the objective is being met ("Increase 7-day activation from 40% to 60%"; "Reduce onboarding support tickets by 30%"). Key results express outcomes, not tasks: "ship the new wizard" is a task masquerading as a result.

Cascade OKRs by alignment, not dictation: leadership sets a small number of company objectives; teams propose key results and their own objectives that ladder up to them. Set them on a regular cadence (commonly quarterly with an annual frame), review them mid-cycle, and grade them honestly at the end. Keep them separate from performance reviews: OKRs graded for compensation quickly become sandbagged. See chapter 10.1 for how OKRs connect to portfolio and program management.

Monitor health with KPIs¶

Distinguish Key Performance Indicators (KPIs) from OKRs. OKRs describe the change you want this period; KPIs describe the ongoing health you must sustain regardless of what you are changing (uptime, conversion rate, cost per transaction, customer satisfaction). A metric can be both (a KPI you are actively trying to move becomes a key result), but most KPIs are guardrails you monitor, not targets you sprint toward.

Classify every important metric as leading (predictive and actionable now, such as trial sign-ups) or lagging (confirmatory and slow, such as annual revenue). Discovery relies on leading indicators to steer before lagging indicators confirm. Beware vanity metrics that rise reliably but predict nothing (raw page views, total registered users); prefer ratio and cohort metrics that resist gaming. See chapters 7.3 and 7.4 for the analytics and experimentation machinery behind these measures.

Specify system quality attributes explicitly¶

Functional requirements say what the system does. System quality attributes (the "-ilities": reliability, performance, scalability, security, accessibility, maintainability, operability) say how well it must do it. These are routinely under-discovered: everyone assumes them, no one specifies them, and they surface as production incidents. Treat them as first-class discovery output. Identify the architecturally significant requirements (the quality demands that materially shape the architecture) for each initiative. Quantify them ("p99 latency under 200 ms at 10× current load"; "WCAG (Web Content Accessibility Guidelines) 2.2 AA"; "recovery time objective of 15 minutes"). And where you can, encode them as automated fitness functions (executable checks that continuously verify a quality attribute) that the delivery pipeline can check. This is the discovery-side complement to chapter 3.1 (architecture fundamentals) and chapter 3.5 (scalability, performance, resilience).

Make every goal SMART¶

Whether writing a key result, an acceptance criterion, or a quality target, apply the SMART test:

• Specific: names one clear, unambiguous outcome.
• Measurable: has a metric and a source of truth.
• Achievable: is realistic given constraints and evidence.
• Relevant: ladders up to a higher objective and to user value.
• Time-bound: has a deadline or review date.

"Improve performance" fails every letter. "Reduce median checkout time from 8s to 3s for mobile users by end of Q3, measured by real-user monitoring" passes all five. SMART criteria convert vague ambition into a falsifiable claim that discovery can test and delivery can verify.

Run continuous, evidence-driven discovery¶

Structure discovery as a repeatable pipeline, not a one-off phase:

1. Sense. Gather signals: user research, support data, analytics, market and compliance inputs.
2. Frame. Map opportunities (an opportunity–solution tree connects a desired outcome to the user needs and candidate solutions that could move it).
3. Hypothesize. State assumptions as falsifiable claims: "We believe [change] will cause [outcome] for [segment], and we'll know if [measure] moves."
4. Experiment. Validate the riskiest assumptions with the cheapest test: interviews, prototypes, fake-door tests (advertising a not-yet-built feature to measure real demand), A/B experiments (randomized comparisons of two variants, chapter 7.4).
5. Decide. Persevere, pivot, or drop, and feed the survivors into the delivery backlog with their SMART success criteria attached.

The output of the discovery pipeline is not a feature list; it is a stream of validated, measurable bets ready for delivery.

Trade-offs: pros and cons¶

The central tension is commitment vs. learning. Enterprises, and especially governments, often need firm commitments for budgeting and contracts, which pulls toward output roadmaps. Good outcomes need room to learn, which pulls toward OKRs and experiments. Resolve it this way: commit to problems and outcomes firmly, and hold solutions loosely.

Examples¶

Startup. A four-person seed-stage team building a scheduling app for hair salons is tempted to build an online-booking widget because a few loud users asked for it. Instead they run a week of discovery: five owner interviews, a fake-door "Book online" button on the marketing site, and a single leading indicator (percentage of appointments that end in a no-show). The interviews and click data reveal that no-shows, not booking, are the real pain, so they write one SMART key result (cut no-shows from 22% to under 10% for pilot salons this quarter) and ship a small deposit-and-reminder feature first, killing the booking widget before writing a line of it.

Enterprise. A retail bank's payments group replaces a feature-count roadmap with three quarterly OKRs, one being "Make everyday payments feel instant" with key results for p95 transfer confirmation time, first-attempt success rate, and payment-related support contacts. System quality attributes are specified up front (99.99% availability, sub-second confirmation, PCI-DSS (Payment Card Industry Data Security Standard) scope minimized) and wired into delivery as fitness functions. Discovery runs weekly customer interviews and fake-door tests before committing engineering. Two candidate features are killed in discovery for failing to move the leading indicators (saving an estimated two quarters of build effort), while a smaller, unglamorous latency fix moves the key result most.

Government. A national tax agency modernizing online filing sets a program objective of "Reduce the burden of filing for ordinary taxpayers," with SMART key results: cut median time-to-file from 45 to 20 minutes, raise successful self-service completion from 60% to 85%, and meet WCAG 2.2 AA and plain-language standards as non-negotiable quality attributes. KPIs (uptime during filing season, call-center volume) are monitored as guardrails. Discovery uses moderated usability testing with real taxpayers, including assistive-technology users, before each release. Because success is defined as taxpayer outcomes rather than delivered modules, the program can show oversight bodies measurable public value, not just spend.

Business case: motivations, ROI, and TCO¶

The return on a discovery pipeline is dominated by avoided waste. Industry experience, echoed in controlled-experiment programs at large tech firms, repeatedly finds that a large share of built features, often cited around half or more, produce no measurable improvement or actively harm the target metric. Suppose even a quarter of a team's build capacity goes to ideas that discovery would have killed cheaply. The pipeline then pays for itself many times over: a week of user research and a fake-door test costs almost nothing against a quarter of engineering, plus the ongoing maintenance burden of an unused feature.

The total cost of ownership (TCO) framing matters because unvalidated features are not free after launch. Every shipped feature carries perpetual costs: maintenance, testing, security surface, support, and cognitive load (chapter 10.4). Killing a bad idea in discovery avoids not just the build cost but the entire tail of ownership. Explicit quality attributes follow the same logic: specifying reliability and accessibility as SMART targets up front is far cheaper than retrofitting them after an outage, breach, or lawsuit.

To make the case to leadership, shift the conversation from "how much are we shipping" to "how much are we moving the metrics that matter," and show a few concrete examples of expensive features that moved nothing. The adoption cost is modest (research capacity, an OKR cadence, and the discipline to write SMART criteria), and the primary risk of not adopting is silent, uncounted, and compounding.

Anti-patterns and pitfalls¶

• Roadmaps of features masquerading as strategy: output lists with no stated outcome or measure.
• Key results that are tasks: "launch X" instead of "improve Y by Z."
• OKR theater: goals written, filed, and never reviewed or graded.
• Sandbagged or heroic OKRs: targets set to guarantee 100% (nothing learned) or fantasy stretch with no plan.
• Unspecified quality attributes: reliability, security, and accessibility assumed rather than quantified, then discovered in production.
• Vanity metrics: measures that always go up and predict nothing.
• Discovery as a one-time phase: a "discovery sprint" up front, then no continued validation.
• Building the solution before testing the assumption: skipping the cheapest experiment because the team is confident.
• Metric fixation and Goodhart's Law: once a measure becomes the target, it stops being a good measure; balance with guardrail KPIs.

Maturity model¶

• Level 1, Initial: Work is defined as features on a roadmap; success is "we shipped it." No explicit outcome measures or quality targets.
• Level 2, Managed: OKRs and KPIs exist for some teams; goals are stated but often output-shaped; quality attributes named but not quantified.
• Level 3, Defined: Consistent OKR cadence aligned to strategy; SMART key results; quality attributes specified and testable; discovery is a recognized, staffed activity with hypotheses and experiments.
• Level 4, Optimizing: Continuous dual-track discovery; validated bets flow steadily to delivery; outcome metrics loop back automatically; leading indicators steer investment; leadership manages a portfolio of outcomes, not a backlog of features.

Ideas for discussion¶

1. Look at your current roadmap: how many items state a measurable outcome versus just a feature to ship?
2. Which of your team's key results are actually disguised tasks, and how would you rewrite them?
3. What system quality attributes does your product depend on that have never been explicitly quantified?
4. What is the cheapest experiment that could have killed your last failed feature before you built it?
5. How do you resolve the tension between the firm commitments budgeting/procurement demands and the learning good outcomes require?
6. Which of your KPIs would keep rising even if the product were getting worse?

Key takeaways¶

• The discovery pipeline decides what and why, and defines success before delivery commits resources.
• Use OKRs for the change you want, KPIs for the health you sustain, and classify metrics as leading or lagging.
• Treat system quality attributes as explicit, quantified, testable requirements, not assumptions.
• Make every goal, key result, and acceptance criterion SMART.
• Run discovery continuously and in parallel with delivery; validate the riskiest assumptions cheaply.
• The dominant ROI is avoided waste: both build cost and the perpetual TCO of unused features.
• Close the loop: delivered outcome metrics (chapter 11.2) are the primary evidence for the next round of discovery.

References and further reading¶

• Measure What Matters, by John Doerr (on OKRs).
• Radical Focus, by Christina Wodtke (on OKRs in practice).
• Continuous Discovery Habits, by Teresa Torres (opportunity–solution trees, dual-track discovery).
• Inspired and Empowered, by Marty Cagan (product discovery and outcome teams).
• Lean Analytics, by Alistair Croll and Benjamin Yoskovitz (leading indicators, vanity metrics).
• The Lean Startup, by Eric Ries (build–measure–learn, validated learning).
• Escaping the Build Trap, by Melissa Perri (outcomes over outputs).
• Outcomes Over Output, by Joshua Seiden.
• Software Architecture in Practice, by Bass, Clements, Kazman (quality attributes).
• Doran, G. T., "There's a S.M.A.R.T. way to write management's goals and objectives" (Management Review, 1981): origin of SMART criteria.