This lesson introduces the overarching design of experiments process: the gold standard for answering questions in an objective, efficient, and scientific way. This methodology is applicable to any and every aspect of testing, from characterizing performance, to assessing system reliability, to understanding how humans interact with the system. Planning is a critical element of this process and requires collaboration and iteration among subject matter experts. Specifically, test planning involves identifying the test objectives, determining the appropriate outcome measure(s), and identifying what factors may influence those outcomes.

References

1. Montgomery, D. C. (2017). Design and analysis of experiments (Ninth ed.): Hoboken, NJ: John Wiley & Sons, Inc.
2. Director, Operational Test & Evaluation (2010). Guidance on the use of Design of Experiments (DOE) in Operational Test and Evaluation.
3. Director, Operational Test & Evaluation (2013). Best Practices for Assessing the Statistical Adequacy of Experimental Designs Used in Operational Test and Evaluation.
4. Office of the Director, Operational Test & Evaluation, Action Officer Training (2016). Testing Planning Foundations. https://www.dote.osd.mil/Publications/Training/
5. Freeman, L. J., Johnson, T., Avery, M., Lillard, V. B., & Clutter, J. (2018). Testing Defense Systems. Analytic Methods in Systems and Software Testing, 441.