Bidders Library Operational Test and Evaluation - DoTE MEMO 10-19-2010.pdf

PDF 95 KB Posted

Attached to
TEC II Services RFP
Solicitation number
HC102821R0006
Issued by

About this file

This memorandum from the Director of Operational Test and Evaluation provides guidance to military test and evaluation commands on increasing the use of design of experiments methodology in operational test planning and evaluation. It instructs recipients to include elements of experimental design in Test and Evaluation Master Plans and test plans such as quantitatively defining the goal of the experiment in terms of measuring mission effectiveness and suitability, identifying system factors that influence response variables, developing test plans that systematically vary factors across developmental and operational testing based on known information, and including statistical measures of power and confidence where applicable. The memorandum directs the recipients to work with the test and evaluation community over the next two years to implement this more scientific and rigorous approach to testing, with design of experiments content to be explicitly included in or referenced from TEMPs and test plans and provided separately to the Director's office for review.

View the file

Other files

File Type Posted
HC102821R0006 Conformed Through amendment 0008.pdf PDF
HC102821R0006 AMD 0007.pdf PDF
HC102821R0006 Conformed Through amendment 0007.pdf PDF
HC102821R0006 Conformed Through amendment 0006.pdf PDF
HC102821R0006 AMD 0006.pdf PDF
HC102821R0006 AMD 0005.pdf PDF
HC102821R0006 Conformed Through amendment 0005.pdf PDF
HC102821R0006 Conformed Through amendment 0004.pdf PDF
HC102821R0006 AMD 0004.pdf PDF
HC102821R0006 Conformed Through amendment 0003.pdf PDF
HC102821R0006 AMD 0003.pdf PDF
HC102821R0006 Conformed through amendment 0002.pdf PDF
HC102821R00060002.pdf PDF
Bidders Library DISA Form 786.pdf PDF
Bidders Library Test and Evaluation Scorecard Guidebook Version 3 0 3 Dec 2020.pdf PDF
Bidders Library TRB CAB Charter V4 0 121420.pdf PDF
HC102821R00060001.pdf PDF
Bidders Library DODI 5000 02.pdf PDF
Bidders Library Instrumentation and Tools List 2021.xlsx XLSX spreadsheet
Bidders Library DISA I 240 110 36.pdf PDF
Bidders Library DODI 5000 02t.pdf PDF
TEC II Bidders Library List.xlsx XLSX spreadsheet
Bidders Library DoD Policy Instruction and Guidance - DoDI 4000 19.pdf PDF
Bidders Library Cybersecurity - DOD Cybersecurity TE Guidebook.pdf PDF
Bidders Library Security - DISAI 240-110-8.pdf PDF
Bidders Library Security - DoDI 5200 01.pdf PDF
Bidders Library JITC Instructions - JITCI 270-95-02.pdf PDF
Bidders Library Interoperability Test and Evaluation - NR KPP Evaluation Guidebook.docx DOCX document
Bidders Library JITC Instructions - JITCI 380-50-02.pdf PDF
Bidders Library Operational Test and Evaluation - DISA Test and Evaluation Scorecard Template.pptx PPTX presentation
Bidders Library Security - DoDM 5105 21.pdf PDF
Bidders Library Security - DoDM 5200 02.pdf PDF
Bidders Library Interoperability Test and Evaluation - DOD AS-SIP 2013.pdf PDF
Bidders Library Interoperability Test and Evaluation - JITC Notional Guide for Action Officers.pdf PDF
Bidders Library Operational Test and Evaluation - DoTE MEMO 10-18-2010.pdf PDF
Bidders Library Interoperability Test and Evaluation - DOD Dictionary.pdf PDF
Bidders Library Security - ICD 703.pdf PDF
Bidders Library Interoperability Test and Evaluation - DoDI 8100 04.pdf PDF
Bidders Library Operational Test and Evaluation - DOTE TEMP Guidebook.pdf PDF
Bidders Library DoD Policy Instruction and Guidance - DoDD 5230 25.pdf PDF
Bidders Library JITC Instructions - JITCI 200-50-02.pdf PDF
Bidders Library Cybersecurity - NIST SP 800-37r2.pdf PDF
Bidders Library Security - DISAI 240-115-10.pdf PDF
Bidders Library DoD Policy Instruction and Guidance - DoDD 5144 02.pdf PDF
Bidders Library Interoperability Test and Evaluation - JTC SOP.pdf PDF
Bidders Library DoD Policy Instruction and Guidance - DoDD 7045 20.pdf PDF
Bidders Library Security - DoDM 5200 01 Vol 1.pdf PDF
Bidders Library Interoperability Test and Evaluation - DoD 8570 01-M.pdf PDF
Bidders Library Operational Test and Evaluation - JITC OTE Guidebook v2 0.docx DOCX document
Bidders Library DoD Policy Instruction and Guidance - DoD Net Centric Service Strategy.pdf PDF
Show 40 more

Latest 50 of 153 listed here. See the rest on GovTribe.

On GovTribe

Work with this file

  • Download the original file
  • Contacts named in this file
  • Similar government files
  • Ask GovTribe AI about this file

Text version

OFFICE OF THE SECRETARY OF DEFENSE

1700 DEFENSE PENTAGON

WASHINGTON, DC 20301-1700

OCT 1 9 2010

OPERATlONAL TEST

AND EVALUATION

MEMORANDUM FOR COMMANDER, ARMY TEST AND EVALUATION

COMMAND

COMMANDER, OPERATIONAL TEST AND EVALUATION

FORCE

COMMANDER, AIR FORCE OPERATIONAL TEST AND

EVALUATION CENTER

DIRECTOR, MARINE CORPS OPERATIONAL TEST AND

EVALUATION ACTIVITY

COMMANDER, JOINT INTEROPERABILITY TEST

COMMAND

DEPUTY UNDER SECRETARY OF THE ARMY, TEST &.

EVALUATION COMMAND

DEPUTY, DEPARTMENT OF THE NAVY TEST &

EVALUATION EXECUTIVE

DIRECTOR, TEST & EVALUATION, HEADQUARTERS,

U.S. AIR FORCE

TEST AND EVALUATION EXECUTIVE, DEFENSE

INFORMATION SYSTEMS AGENCY

DOT&ESTAFF

SUBJECT: Guidance on the use ofDesign of Experiments (DOE) in Operational Test and Evaluation

This memorandum provides further guidance on my initiative to increase the use of scientific and statistical methods in developing rigorous, defensible test plans and in evaluating their results. As I review Test and Evaluation Master Plans (TEMPs) and Test Plans, I am looking for specific information. In general, I am looking for substance vice a 'cookbook' or template approach - each program is unique and will require thoughtful tradeoffs in how this guidance is applied.

A "designed" experiment is a test or test program, planned specifically to determine the effect of a factor or several factors (also called independent variables) on one or more measured responses (also called dependent variables). The purpose is to ensure that the right type ofdata and enough of it are available to answer the questions of interest. Those questions, and the associated factors and levels, should be determined by subject matter experts -- including both operators and engineers -- at the outset of test planning.

o

Elements of experimental design that I am looking for when I approve TEMPs and Test Plans include the following:

• The goal of the experiment. This should reflect evaluation of end-to-end mission effectiveness in an operationally realistic environment.

• Quantitative mission-oriented response variables for effectiveness and suitability. (These could be Key Performance Parameters but most likely there will be others.)

• Factors that affect those measures of effectiveness and suitability.

Systematically, in a rigorous and structured way, develop a test plan that provides good breadth of coverage of those factors across the applicable levels of the factors, taking into account known information in order to concentrate on the factors of most interest.

• A method for strategically varying factors across both developmental and operational testing with respect to responses of interest.

• Statistical measures of merit (power and confidence) on the relevant response variables for which it makes sense. These statistical measures are important to understand "how much testing is enough?" and can be evaluated by decision makers on a quantitative basis so they can trade off test resources for desired confidence in results.

Design ofExperiments is a structured process to identify the metrics, factors, and levels that most directly affect operational effectiveness and suitability and that should be reflected in detailed test plans. DOT&E is working with other members of the test and evaluation community to develop a two-year roadmap for implementing this scientific and rigorous approach to testing. I am looking for as much substance as possible as early as possible, but each TEMP revision can be tailored as more information becomes available. That content can either be explicitly made part of TEMPs and Test Plans, or referenced in those documents and provided separately to DOT&E for review.

J~~G::::-- Director cc:

DDT&E

File details come from the government source that posted it. Updated .