This is an original excerpt written by our team to show structure, depth and referencing. It is modelled on the course named above but is not a university document. Use samples as a study guide and follow your school's academic integrity policy.
The problem with simple pre-post studies
Most practice evaluations in nursing compare outcomes before and after a change, without random assignment. These designs are practical, but they leave many alternative explanations open. Harris et al. (2006) reviewed quasi-experimental studies in two medical informatics journals and set out a relative hierarchy of designs, encouraging researchers to use higher-level designs that give more convincing evidence for causal links.
Threats to internal validity
Shadish et al. (2002) describe threats that can make an intervention appear effective when it isn't:
| Threat | Example in a fall-prevention evaluation |
|---|---|
| History | A new hospital-wide bed alarm policy starts at the same time as the unit's intervention |
| Maturation | Patients' average length of stay changes over the period |
| Regression to the mean | The project began after an unusually bad quarter, so rates would likely fall anyway |
| Instrumentation | Staff start reporting falls differently after training |
| Selection | The comparison unit cares for a different mix of patients |
| Attrition | Data are missing for patients transferred off the unit |
Moving up the hierarchy
Each step adds a way to rule out some of these explanations.
- One-group posttest only: measures outcomes only after the intervention, with no comparison. It can't show change.
- One-group pretest–posttest: compares before and after, but history, maturation and regression to the mean remain plausible.
- Adding a nonequivalent dependent variable: measuring an outcome that the intervention shouldn't affect but that would respond to the same external influences. If falls drop but another unrelated measure doesn't, history is less likely to be the explanation.
- Adding a comparison group: a similar unit that doesn't receive the intervention helps control for hospital-wide events, although selection differences remain.
- Removed-treatment or repeated-treatment designs: if outcomes worsen when the intervention stops and improve when it restarts, the case for an effect is stronger.
- Interrupted time series: many measurements before and after the intervention show whether the change happened at the time of the intervention and wasn't part of an existing trend. Adding a comparison series strengthens it further.
Applying the hierarchy to a DNP evaluation
A unit plans to evaluate a fall-prevention program with a single before-and-after comparison of quarterly fall rates. A stronger design would:
- collect monthly fall rates for 12 months before and 12 months after launch (interrupted time series);
- include a comparison unit with similar patients that doesn't receive the program;
- track a nonequivalent dependent variable, such as pressure injuries, that shouldn't change because of fall prevention;
- keep fall definitions and reporting processes unchanged throughout, to avoid instrumentation effects.
Analysis could use segmented regression to estimate changes in level and slope after the intervention, and the report should acknowledge remaining threats.
Conclusion
Randomization is often impossible in practice settings, but evaluators can still choose designs that rule out the most likely alternative explanations. Doctoral nurses should plan evaluations before implementation, when it is still possible to collect baseline data and identify comparison groups.
References
Harris, A. D., McGregor, J. C., Perencevich, E. N., Furuno, J. P., Zhu, J., Peterson, D. E., & Finkelstein, J. (2006). The use and interpretation of quasi-experimental studies in medical informatics. Journal of the American Medical Informatics Association, 13(1), 16–23. https://doi.org/10.1197/jamia.M1749
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.
