How to write an experiment result people can trust
A trustworthy experiment readout separates setup, observed result, limitations, interpretation, and next decision. Make it easy for a reader to see what the test did — and did not — prove.
A good experiment readout lets someone disagree intelligently
The goal of a readout is not to make the test look successful. It is to preserve enough context for the team to understand what happened, what remains uncertain, and what decision is justified.
That requires more than a winning percentage in a slide title.
Restate the hypothesis
Write what you expected to change, for whom, and why. A useful hypothesis creates a logical link between the intervention and the metric.
If the hypothesis changed after seeing the result, say so. Do not rewrite history to make the outcome look obvious.
Describe the setup
Include the audience, variants, allocation, dates, primary metric, important secondary metrics, exclusions, and any material implementation differences.
The reader should be able to understand what was actually tested without opening five other documents.
Report the observed result first
State the numbers before the story. Include absolute values where useful, not only percentage lift. Show sample size and the relevant uncertainty or statistical method when the test depends on statistical inference.
Avoid language such as “proved” when the evidence only suggests a direction.
Document anomalies and limitations
Did tracking change? Was there a promotion during the test? Did one segment behave differently? Was exposure inconsistent? Did the experiment end early?
Limitations do not make the report weaker. Hiding them does.
Separate interpretation from evidence
After the result, write the most plausible explanation and alternative explanations. This shows the team where judgment begins.
A positive result may support the mechanism you expected, but it can also reveal a different factor worth investigating.
Make the decision explicit
Choose a next step: roll out, iterate, stop, collect more data, or test the mechanism in another context. Explain why the evidence is sufficient for that level of action.
Keep a reusable record
Experiment history becomes valuable when future teams can search the question, setup, result, and decision. Use consistent naming and preserve links to implementation details.
A trustworthy readout is not the most confident one. It is the one that makes evidence, uncertainty, and the resulting decision visible enough for another person to follow the reasoning.