29 lines
998 B
Markdown
29 lines
998 B
Markdown
# Common Pitfalls in Experimental Analysis
|
|
|
|
## Statistical pitfalls
|
|
- Reporting only the best run
|
|
- Mixing seed-level and subject-level units
|
|
- Running many contrasts without correction
|
|
- Reporting significance without effect size
|
|
- Using parametric tests after failed assumptions without explanation
|
|
|
|
## Visualization pitfalls
|
|
- No real figure despite readable data
|
|
- Plot without uncertainty information
|
|
- Overcrowded multi-panel figure with no message hierarchy
|
|
- Caption missing n / error-bar meaning
|
|
- Figure not referenced or interpreted in text
|
|
|
|
## Reasoning pitfalls
|
|
- Confusing correlation with mechanism
|
|
- Treating trend as conclusion
|
|
- Ignoring negative results
|
|
- Hiding instability behind a mean value
|
|
- Turning raw logs into durable conclusions too early
|
|
|
|
## Reporting pitfalls
|
|
- Writing paper prose before evidence is stabilized
|
|
- Mixing analysis artifact with final narrative artifact
|
|
- Not separating blocker from conclusion
|
|
- Forgetting to state what decision the analysis changes
|