Comparing Experimentation Frameworks
Comparison page | Part of Experimentation Frameworks
Searchers comparing experimentation frameworks usually want one answer: which structure fits this idea, this team size, and this decision? Here is the short version we use at The Unnamed Roads.
Side-by-side
| Lean Startup | Design Sprint | ICE / PIE scoring | 12-week framework | |
|---|---|---|---|---|
| Primary job | Continuous learning loop | One-week prototype + user test | Prioritise which ideas to try | Full-quarter product bet with a kill date |
| Timebox | Ongoing cycles | Five days | A ranking session | Twelve weeks, three phases |
| Output | Validated learning | Prototype + test notes | Ordered backlog | Scale / pivot / kill + written learnings |
| Best when | You can ship tiny changes often | The risk is UI or messaging | You have too many ideas | You need a fixed end date and portfolio rhythm |
| Weak when | No decision deadline | The risk is distribution or pricing | Used as a substitute for evidence | The idea only needs a weekend test |
Lean Startup
Lean Startup (build–measure–learn) is the default language for product experimentation. It is strong when you can ship small changes and read feedback quickly. It is weak when there is no calendar end: without a kill date, “one more iteration” becomes sunk-cost continuation.
Use Lean loops inside a larger framework when you need day-to-day learning, not as a substitute for a portfolio decision.
Design Sprint
A Design Sprint compresses map → sketch → decide → prototype → test into five days. It is excellent when the unknown is interaction or messaging and you can put a prototype in front of real users by Friday.
It is the wrong primary tool when the unknown is willingness to pay, channel economics, or operational delivery — those usually need a longer exposure window than one workshop week.
ICE and PIE scoring
ICE (Impact, Confidence, Ease) and PIE (Potential, Importance, Ease) are prioritisation aids, not full experimentation frameworks. They help a small team decide which experiment to run next when the backlog is crowded.
After you pick an idea, you still need a method: a sprint week, a Lean loop, or a multi-week validation plan with success criteria written down before you build.
The 12-week experimentation framework
Our studio method is a 12-week experimentation framework: Foundation (weeks 1–3), Development (weeks 4–9), Evaluation (weeks 10–12), then a pre-committed scale / pivot / kill decision. It exists so a solo or small team can run a few serious product bets per year without endless half-finished builds.
Full week-by-week guide: Experimentation Frameworks: The 12-Week Method.
How we choose in practice
- Many vague ideas → score with ICE/PIE, then pick one.
- UI or offer wording risk → Design Sprint (or a sprint-shaped week) first.
- Distribution, pricing, or retention risk → Lean loops with a hard review date.
- A real product bet that needs a quarter of attention → the 12-week experimentation framework.
Portfolio attention still follows Focus · Monitor · Parked: only a few bets get full attention at once.