# Common Pitfalls in Experimental Analysis

## Statistical pitfalls
- Reporting only the best run
- Mixing seed-level and subject-level units
- Running many contrasts without correction
- Reporting significance without effect size
- Using parametric tests after failed assumptions without explanation

## Visualization pitfalls
- No real figure despite readable data
- Plot without uncertainty information
- Overcrowded multi-panel figure with no message hierarchy
- Caption missing n / error-bar meaning
- Figure not referenced or interpreted in text

## Reasoning pitfalls
- Confusing correlation with mechanism
- Treating trend as conclusion
- Ignoring negative results
- Hiding instability behind a mean value
- Turning raw logs into durable conclusions too early

## Reporting pitfalls
- Writing paper prose before evidence is stabilized
- Mixing analysis artifact with final narrative artifact
- Not separating blocker from conclusion
- Forgetting to state what decision the analysis changes