When IdeaKiller is sure, it was right 89% of the time
Every validator says it is accurate. Almost none show you a test. We took 200 real startups whose fate we already know and asked IdeaKiller to judge them as if it were launch day.
Short answer: on 100 startups the system had never been tuned on, high-confidence verdicts matched what really happened in 101 of 114 checks (89%). About 4 ideas in 10 get a high-confidence verdict. Over all ideas, the verdict was right about 2 times in 3.
How we tested
- 200 real startups with a known ending. 100 shut down, 100 grew. We took them from founder interviews and public post-mortems, mostly small companies that a model cannot simply remember.
- No peeking at the future. For each startup, web research was limited to pages from before its launch month. IdeaKiller saw the world the founder saw.
- Rules set before the test. We wrote down how we would score the run before we ran it.
- Two halves. We tuned the confidence rule on 100 startups, then checked it once on the other 100. Every number on this page comes from that second half.
- One run per check, exactly as you run it. 300 checks in total, 3 per startup.
The results
| What we measured | Result |
|---|---|
| High-confidence verdicts that matched the real outcome | 101 / 114 = 89% (95% range: 81-93%) |
| Share of ideas that get a high-confidence verdict | about 4 in 10 |
| All verdicts that matched the real outcome | about 2 in 3 |
| Coin flip, for comparison | 1 in 2 |
What "normal confidence" means for you
For 6 ideas in 10 the evidence is mixed. Then IdeaKiller says so, instead of pretending. You still get the weakest point and one cheap test to run first. That test is the fastest way to get the answer the research cannot give yet.
What we do not claim
- No tool can promise your startup will succeed. Execution, timing and luck matter.
- Our goal is 90% on all ideas. We are not there yet, and we will update this page with every new test.
- A "matched" verdict means: GO or CONDITIONAL GO for startups that grew, PIVOT or NO GO for ones that died.