Attachment C - Biomedical Technology Readiness Levels (TRL).pdf

The United States General Accounting Office (GAO) issued a Report in 1999 indicating that better management of technology can improve weapon system outcomes for the Department of Defense (DoD).1 The GAO found that the level of maturity of a new technology being incorporated into a product development program was related to the success of the program; that is, the higher the level of readiness of critical technologies when incorporated into a product, the greater the probability for a successful outcome. Acronyms are provided in the Appendix.

The DoD has incorporated the concept of TRLs in DoD 5000.2-R2 for major acquisition programs. Although the definitions, descriptions, and references to TRLs are almost always in engineering terms, the TRL concept is applicable to any technology. DoD 5000.2-R does not define the TRL that must be achieved as an exit or entrance criterion for life cycle transition points. Rather, a TRL is one of the many variables that a Milestone Decision Authority (MDA) must consider in the milestone decision process; it does not drive the decision. Biomedical TRL descriptions provide a systematic way for the science and technology (S&T) community to assess and communicate to the MDA the level of maturity of a particular technology or combination of technologies as it relates to the particular category and the maturity necessary for successful product development. This document provides equivalent TRL descriptions applicable to biomedical technologies in four categories:

Pharmaceutical (i.e., drugs) Pharmaceutical (i.e., biologics/vaccines) Medical Devices Medical Information Management/Information Technology (IM/IT) & Medical Informatics

The TRLs for the first three categories have been developed based on consideration of the DoD’s generic definitions, the applicable FDA regulatory process, and industry practices and experience with their research and development (R&D) processes (discovery through manufacturing, production, and marketing).

The last category includes elements of formal regulatory processes and logical events in deriving comparable levels of maturity. USAMRMC intends to use external anchors such as “FDA events” wherever practical to define each TRL Decision Criterion. Furthermore, activities described as occurring between successive TRL Decision Criteria are intended to exemplify the kinds of activities that routinely take place when maturation is sequential and stepwise – these examples are neither mandatory nor all inclusive.

These guidelines are not considered absolutes, and characterization of activities associated with TRLs can and does vary at times. For example, experience to date with application of the guidelines for biomedical TRLs indicates considerable variation in the timing, activities, and programmatic events associated with TRLs 5 and 6 for pharmaceuticals. Hence, the S&T and Acquisition program managers work together in exercising discretion in the selection, progression, and timing of specific activities to be accomplished in attainment of TRL 5. Such flexibility and tailoring are needed in appropriately aligning the TRL decision criteria with the maturation and risk characteristics of a particular technology, including consideration of the associated investment strategy and transition procedures that may vary among program managers.

The lower a critical technology’s TRL when transitioning from technology development to product development, the greater the risks. For medical technologies, risk reduction is not linear across TRLs. The rate of risk reduction remains very low until very late. Historically, FDA-regulated products, such as vaccines, do not achieve significant risk reduction (i.e., >50%) until completion of Phase 3 clinical trials and approval of a biologics license application by the FDA (TRL 8); industry experience is that only one in four vaccines going into Phase 3 trials is licensed. Similarly, although technology maturation is commonly perceived as a sequential continuum of activities from basic research, through development to production and deployment, evolution of the TRL for a critical technology may not be sequential, especially in those cases where FDA anchors are undefined. In cases of success or failure, the change in TRL may be greater than a single TRL. For example, upon successful completion of a pivotal study, biomedical information technology readiness may move from TRL 3 or 4 to TRL 9.

1 Best Practices: Better Management of Technology Development Can Improve Weapon System Outcomes; Report to the Chairman and Ranking Minority Member, Subcommittee on Readiness and Management Support, Committee on Armed Services, U.S. Senate; July 1999.

2 Mandatory Procedures for Major Defense Acquisition Programs (MDAPs) and Major Automated Information System (MAIS) Acquisition Programs, June 2001.

The proponent for this document is the Deputy for Research and Development.

Commander, U.S. Army Medical Research and Materiel Command ATTN: MCMR-ZC, 504 Scott Street, Fort Detrick, MD 21702-5012

Proposed TRLs for Medical RDT&E

TRL 1 NASA/DoD 5000.2-R TRL Definition: Basic principles observed and reported.

USAMRMC Equivalent TRL Descriptions NASA/DoD 5000.2-R TRL Description

Pharmaceutical (Drugs) Pharmaceutical (Biologics, Vaccines) Medical Devices3 Medical IM/IT & Medical Informatics

Lowest level of technology readiness. Maintenance of scientific awareness and generation of scientific and bioengineering knowledge base. Scientific findings are reviewed and assessed as a foundation for characterizing new technologies.

HW/SW System technology explored. Basic theories applied to IM/IT field suggesting promise.

Lowest level of technology readiness. Scientific research begins to be translated into technology’s basic properties. TRL 1 DECISION CRITERION: Scientific literature reviews and initial Market Surveys are initiated and assessed. Potential scientific application to defined problems is articulated.

TRL 1 DECISION CRITERION:

Identification of the potential medical solution to mission need. Medical Informatics data and knowledge representation issues are defined.

TRL 2 NASA/DoD 5000.2-R TRL Definition: Technology concept and/or application formulated.

TRL Description

Pharmaceutical (Drugs) Pharmaceutical (Biologics, Vaccines) Medical Devices3 Medical IM/IT & Medical Informatics

Intense intellectual focus on the problem with generation of scientific “paper studies” that review and generate research ideas, hypothesis, and experimental designs for addressing the related scientific issues.

HW/SW Systems invention begins.

Overall system concepts are documented by flowcharting or other system descriptive techniques.

Invention begins. Once basic principles are observed, practical applications can be invented. The application is speculative and there is no proof or detailed analysis to support the assumption. Examples are still limited to paper studies.

TRL 2 DECISION CRITERION: Hypothesis(es) is generated. Research plans and/or protocols are developed, peer reviewed, and approved. TRL 2 DECISION CRITERION:

Medical Informatics data and knowl-edge representation schema are defined.

TRL 3 NASA/DoD 5000.2-R TRL Definition: Analytical and experimental critical function and/or characteristic proof-of-concept.

TRL Description Pharmaceutical (Drugs) Pharmaceutical (Biologics, Vaccines) Medical Devices3 Medical IM/IT & Medical Informatics

Basic research, data collection, and analysis begin in order to test hypothesis, explore alternative concepts, and identify and evaluate technologies supporting drug development. Initial synthesis of countermeasure candidate(s) and identification of their sites and mechanisms of action. Initial characterization of candidates in preclinical studies.

Basic research, data collection, and analysis begin in order to test hypothesis, explore alternative concepts, and identify and evaluate critical technologies and components supporting candidate biologic/vaccine constructs research and eventual development of a candidate countermeasure.

Agent challenge studies are conducted to support models based on presumed battlefield conditions.

Research-scale process initiation and evaluation conducted, as are studies to identify site(s) and mechanism(s) of action, potential correlates of protection for vaccines, and initial physical/chemical characterization of constructs.

Basic research, data collection, and analysis begin in order to test hypothesis, explore alternative concepts, and identify and evaluate component technologies. Initial tests of design concept, and evaluation of candidate(s). Study endpoints defined. Animal models (if any) are proposed.

Design verification, critical component specifications, and tests (if a system component, or necessary for device T&E) developed.

Separate elements of HW/SW System components are investigated and developed but not yet integrated or representative.

Active research and development is initiated. This includes analytical studies and laboratory studies to physically validate analytical predictions of separate elements of the technology. Examples include components that are not yet integrated or representative.

TRL 3 DECISION CRITERION: Initial proof-of-concept for candidate drug constructs is demonstrated in a limited number of in vitro and in vivo research models.

TRL 3 DECISION CRITERION: Initial proof-of-concept for biologic/vaccine constructs is demonstrated in a limited number of in vitro and in vivo research models.

TRL 3 DECISION CRITERION: Initial proof-of-concept for device candidates is demonstrated in a limited number of laboratory models (may include animal studies).

TRL 3 DECISION CRITERION:

Medical Informatics data and knowledge representation schema are modeled.

Proposed TRLs for Medical RDT&E (cont.)

TRL 4 NASA/DoD 5000.2-R TRL Definition: Component and/or breadboard validation in laboratory environment.

TRL Description

Pharmaceutical (Drugs) Pharmaceutical (Biologics, Vaccines) Medical Devices3 Medical IM/IT & Medical Informatics

Non-GLP laboratory research to refine hypothesis and identify relevant parametric data required for technological assessment in a rigorous (worst case) experimental design. Exploratory study of candidate drugs (e.g., formulation, route(s) of administration, method of synthesis, physical/chemical properties, metabolic fate and excretion or elimination, and dose ranging).

Candidate drugs are evaluated in animal model(s) to identify and assess potential safety and toxicity problems, adverse events, and side effects.

Assays to be used during nonclinical and clinical studies in evaluating candidate drugs are identified.

Laboratory research (non-GLP) to refine hypothesis and identify relevant parametric data required for technological assessment in a rigorous (worst case) experimental design. Exploratory study of critical technologies for effective integration into candidate biologic/vaccine constructs, for example, environmental milieu (pH, adjuvant, stabilizers and preservatives, buffers, etc.), route(s)/methods of administration, proposed production/ purification methods, further physical/chemical characteri-zation, metabolic fate and excretion or elimination, dose ranging, and agent challenge studies for protection. Candidate biologic/vaccine constructs are evaluated in animal model(s) to identify and assess safety and toxicity, biological effects, adverse effects, and side effects. Assays, surrogate markers, and endpoints to be used during nonclinical and clinical studies to evaluate and characterize candidate biologic/vaccine constructs are identified.

Non-GLP laboratory research to refine hypothesis and identify relevant parametric data required for technological assessment in a rigorous (worst case) experimental design. Exploratory study of candidate device(s)/systems (e.g., initial specification of device, system, and subsystems).

Candidate devices/systems are evaluated in laboratory and/or animal models to identify and assess potential safety problems, adverse events, and side effects. Procedures and methods to be used during nonclinical and clinical studies in evaluating candidate devices/systems are identified. The design history file, design review, and when required a master device record, are initiated to support either a 510(k) or PMA.

Prototype produced. HW/SW System components are integrated to establish that the pieces will work together. This is relatively “low fidelity” compared to the eventual system.

Basic technological components are integrated to establish that the pieces will work together. This is relatively ‘low fidelity’ compared to the eventual system. Examples include integration of ‘ad hoc’ hardware in a laboratory.

TRL 4 DECISION CRITERION: Proof-of-concept and safety of candidate drug formulation(s) demonstrated in defined laboratory/animal model(s).

TRL 4 DECISION CRITERION Proof-of-concept and safety of candidate biologic/vaccine constructs demonstrated in defined laboratory/animal model(s).

TRL 4 DECISION CRITERION Proof-of-concept and safety of candidate devices/systems demonstrated in defined laboratory/animal models.

TRL 4 DECISION CRITERION:

Medical Informatics data and knowledge representation models are instantiated with representative data or knowledge from applicable domain.

TRL 5 NASA/DoD 5000.2-R TRL Definition: Component and/or breadboard validation in a relevant environment.

TRL Description

Pharmaceutical (Drugs) Pharmaceutical (Biologics, Vaccines) Medical Devices3 Medical IM/IT & Medical Informatics

Intense period of nonclinical and preclinical research studies involving parametric data collection and analysis in well-defined systems with pilot lots of candidate pharmaceuticals produced and further development of selected candidate(s).

Results of research with pilot lots provide basis for a manufacturing process amenable to cGMP-compliant pilot lot production. Conduct GLP safety and toxicity studies in animal model systems.

Identify endpoints of clinical efficacy or its surrogate. Conduct studies to evaluate the pharmacokinetics and pharmacodynamics of candidate drugs. Stability studies initiated.

Intense period of nonclinical and preclinical research studies involving parametric data collection and analysis in well-defined systems with pilot lots of candidate biologics/ vaccines produced and further development of selected candidates.

Research results support proposing a potency assay, proposing a manufacturing process amenable to cGMP-compliant pilot lot production, identifying and demonstrating proof-of-concept for a surrogate efficacy marker in an animal model(s) applicable to predicting protective immunity in humans, and demonstrating preliminary safety and efficacy against an aerosol challenge in a relevant animal model. Conduct GLP safety and toxicity studies in animal model systems. Identify endpoints of clinical efficacy or its surrogate in animal models that may be applicable to predicting protective immunity in humans. Conduct studies to evaluate immunogenicity, as well as pharmaco-kinetics and pharmacodynamics when appropriate.

Stability studies initiated.

Further development of selected candidate(s).

Devices compared to existing modalities and indications for use and equivalency demonstrated in model systems. Examples include devices tested through simulation, in tissue or organ models, or animal models if required. All component suppliers/vendors are identified and qualified; vendors for critical components audited for cGMP/QSR compliance. Component tests, component drawings, design history file, design review, and any master device record verified.

Product Development Plan drafted. Pre-IDE meeting held with CDRH for proposed Class III devices, and the IDE is prepared and submitted to

CDRH.

For a 510(k), determine substantially equivalent devices and their classification, validate functioning model, ensure initial testing is complete, and validate data and readiness for cGMP inspection.

First technical test of prototype.

HW/SW System components are integrated and realistic supporting elements are employed so that the system can be tested in a simulated environment. Actual interfaces to supporting systems are specified and development begins.

Fidelity of breadboard technology increases significantly. The basic technological components are integrated with reasonably realistic supporting elements so that the technology can be tested in a simulated environment. Examples include ‘high fidelity’ laboratory integration of components.

TRL 5 DECISION CRITERION: A decision point is reached at which it is determined that sufficient data on the candidate drug exist in the draft technical data package to justify proceeding with preparation of an IND application.

TRL 5 DECISION CRITERION: A decision point is reached at which it is determined that sufficient data on the candidate biologic/vaccine exist in the draft technical data package to justify proceeding with preparation of an IND application.

TRL 5 DECISION CRITERION: IDE review by CDRH results in determination that the investigation may begin.

For a 510(k), preliminary findings suggest the device will be substantially equivalent to a predicate device.

TRL 5 DECISION CRITERION:

Medical Informatics data and knowledge representation models are implemented as data and/or knowledge management systems and tested in a lab environment.

TRL 6 NASA/DoD 5000.2-R TRL Definition: System/subsystem model or prototype demonstration in a relevant environment.

TRL Description Pharmaceutical (Drugs) Pharmaceutical (Biologics, Vaccines) Medical Devices3 Medical IM/IT &

Medical Informatics Pre-IND meeting (Type B) held with CDER. IND prepared and submitted. Phase 1 clinical trials are conducted to demonstrate safety of candidate in a small number of humans under carefully controlled and intensely monitored clinical conditions.

Evaluation of pharmacokinetic and pharmacodynamic data to support the design of well-controlled, scientifically valid Phase 2 studies.

Production technology demonstrated through production-scale cGMP plant qualification.

Pre-IND meeting (Type B) held with CBER. IND prepared and submitted. Phase 1 clinical trials are conducted to demonstrate safety of candidates in a small number of subjects under carefully controlled and intensely monitored clinical conditions.

Evaluation of immunogenicity and/or pharmacokinetics and pharmacodynamics data to support design of Phase 2 clinical trials. Surrogate efficacy models are validated.

Clinical trials conducted to demonstrate safety of candidate Class III medical device in a small number of humans under carefully controlled and intensely monitored clinical conditions. Component tests, component drawings, design history file, design review, and any master device record updated and verified. Production technology demonstrated through production-scale cGMP plant qualification.

For 510(k), component tests, component drawings, design history file, design review, and any master device record updated and verified. Manufacturing facility ready for cGMP inspection.

Advanced technical testing of prototype HW/SW System, to include interfaces to actual supporting systems, is tested in a relevant or simulated operational environment.

Outproduct is final prototype.

Representative model or prototype system, which is well beyond the breadboard tested for TRL 5, is tested in a relevant environment.

Represents a major step up in a technology’s demonstrated readiness.

Examples include testing a prototype in a high fidelity laboratory environment or in simulated operational environment.

TRL 6 DECISION CRITERION: Data from Phase 1 trials meet clinical safety requirements and support proceeding to Phase 2 clinical studies.

TRL 6 DECISION CRITERION: Data from Phase 1 clinical trials meet clinical safety requirements and support proceeding to Phase 2 clinical trials.

TRL 6 DECISION CRITERION Data from the initial clinical investigation demonstrate that the Class III device meets safety requirements and supports proceeding to clinical safety and effectiveness trials.

For a 510(k), information and data demonstrate substantial equivalency to predicate device and support production of the final prototype and final testing in a military operational environment.

TRL 6 DECISION CRITERION:

Medical Informatics data and knowledge management systems are tested with target applications in a lab environment. Configuration management developed.

TRL 7 NASA/DoD 5000.2-R TRL Definition: System prototype demonstration in an operational environment.

TRL Description

Pharmaceutical (Drugs) Pharmaceutical (Biologics, Vaccines) Medical Devices3 Medical IM/IT & Medical Informatics

Phase 2 clinical trials conducted to demonstrate initial efficacy and capture further safety and toxicity data. Product activity (e.g., preliminary evidence of efficacy) determined. Product final dose, dose range, schedule, and route of administration established from clinical pharmacokinetic and pharmacodynamic data. Phase 2 clinical trials completed. Data collected, presented, and discussed with CDER at pre-Phase 3 meeting (Type B) support continued drug development.

Clinical endpoints and/or surrogate efficacy markers and test plans agreed to by CDER.

Phase 2 safety and immunogenicity trials conducted. Product immunogenicity and biological activity (e.g., preliminary evidence of efficacy) determined. Product final dose, dose range, schedule, and route of administration established from vaccine immunogenicity and biologic activity, and when necessary, clinical pharmacokinetics and pharmacodynamics data. Phase 2 clinical trials completed. Data collected, presented, and discussed with CBER at pre-Phase 3 (or surrogate efficacy) meeting (Type B) support continued development of the biologics/vaccines. Clinical endpoints and/or surrogate efficacy markers and test plans agreed to by CBER.

Clinical safety and effectiveness trials conducted with a fully integrated Class III medical device prototype in an operational environment.

Continuation of closely controlled studies of effectiveness, and determination of short-term adverse events and risks associated with the candidate product. Functional testing of candidate devices completed and confirmed, resulting in final down-selection of prototype device. Clinical safety and effectiveness trials completed. Final product design validated, and final prototype and/or initial commercial scale device are produced. Data collected, presented, and discussed with CDRH in support of continued device development.

For a 510(k), final prototype and/or initial commercial-scale device are produced and tested in a military operational environment.

Prototype HW/SW System is near or at planned operational system.

Actual system prototype demonstrated in an operational environment with end-users (first cut user test).

Prototype near or at scale of planned operational system.

Represents a major step up from TRL 6, requiring the demonstration of an actual system prototype in an operational environment, such as in an aircraft, vehicle, or space. Examples include testing the prototype in a test bed aircraft.

TRL 7 DECISION CRITERION: Phase 3 clinical study plan or surrogate test plan has been approved.

TRL 7 DECISION CRITERION: Phase 3 clinical study plan or surrogate test plan has been approved.

TRL 7 DECISION CRITERION: Clinical endpoints and test plans agreed to by CDRH.

For a 510(k), information and data demonstrate substantial equivalency to predicate device and use in a military operational environment, and support preparation of 510(k).

TRL 7 DECISION CRITERION:

Medical Informatics data and knowledge management systems are operationally integrated and tested with target applications in an operational environment.

TRL 8 NASA/DoD 5000.2-R TRL Definition: Actual system completed and “flight qualified” through test and demonstration.

TRL Description Pharmaceutical (Drugs) Pharmaceutical (Biologics, Vaccines) Medical Devices3 Medical IM/IT

& Medical Informatics Implementation of expanded Phase 3 clinical trials or surrogate tests to gather information relative to the safety and effectiveness of the candidate drug.

Trials are conducted to evaluate the overall risk-benefit of administering the candidate product, and to provide an adequate basis for drug labeling.

Process validation completed and followed by lot consistency/ reproducibility studies. Pre-NDA meeting (Type B) held with CDER. NDA prepared and submitted to CDER. Facility PAI completed.

Implementation of expanded Phase 3 clinical trials or surrogate tests to gather information relative to the safety and effectiveness of the candidate biologic/vaccine. Trials are conducted to evaluate the overall risk-benefit of administering the candidate product, and to provide an adequate basis for product labeling. Process validation completed and followed by lot consistency/reproducibility studies. Pre-BLA meeting (Type B) held with CBER. BLA prepared and submitted to CBER. Facility PAI completed.

Implementation of clinical trials to gather information relative to the safety and effectiveness of the device. Trials are conducted to evaluate the overall risk-benefit of using the device and to provide an adequate basis for product labeling.

Confirmation of QSR compliance, the design history file, design review, and any master device record, are completed and validated, and device production followed through lot consistency and/or reproducibility studies. Pre-PMA meeting held with CDRH. PMA prepared and submitted to CDRH.

Facility PAI (cGMP/QSR/QSIT) completed.

For 510(k), prepare and submit application.

Technical testing of Final Product.

HW/SW System has been proven to work in its final form and under expected conditions.

Technology has been proven to work in its final form and under expected conditions. In almost all cases, this TRL represents the end of true system development. Examples include developmental test and evaluation of the system in its intended weapon system to determine if it meets design specifications.

TRL 8 DECISION CRITERION: Approval of the NDA for drug by CDER.

TRL 8 DECISION CRITERION: Approval of the BLA for biologics/vaccines by the CBER.

TRL 8 DECISION CRITERION: Approval of the PMA [or, as applicable, 510(k)] for device by the

CDRH.

TRL 8 DECISION CRITERION:

Developmental test and evaluation of the HW/SW System in its intended environment demonstrate it meets design specifications. Fully integrated and operational Medical Informatics data and knowledge management systems are validated in several operational environments.

TRL 9 NASA/DoD 5000.2-R TRL Definition: Actual system “flight proven” through successful mission operations.

TRL Description

Pharmaceutical (Drugs) Pharmaceutical (Biologics, Vaccines) Medical Devices3 Medical IM/IT & Medical Informatics

The pharmaceutical (i.e., drug, biologic or vaccine) or medical device may be distributed/marketed. Postmarketing studies (nonclinical or clinical) may be required and are designed after agreement with the FDA. Postmarketing surveillance.

Operational testing of the product.

HW/SW System is in its final form and under mission conditions, such as those encountered in operational test and evaluation. Medical Informatics knowledge maintenance and verification of data integrity are ongoing. Military requirements met for transportation, handling, storage, etc.

Actual application of the technology in its final form and under mission conditions, such as those encountered in operational test and evaluation. In almost all cases, this is the end of the last ‘bug fixing’ aspects of true system development. Examples include using the system under operational mission conditions.

TRL 9 DECISION CRITERION: None – continue surveillance. TRL 9 DECISION CRITERION:

Product successfully used during military mission as component of IOT&E phase. Logistical demonstration successfully conducted.

3 Definitions pertain predominately to Class II and Class III devices (see 21CFR860.3 for device class definitions) that are subject to approval via the Premarket Approval (PMA) process. Devices that are subject to approval via the 510(k) process (Market clearance; generally limited to certain Class I and Class II devices) may not require all of the studies described, and only require an Investigational Device Exemption if human studies are necessary