SSJ MFR_QuestionMark FY25_Redacted 11 Dec 2024.pdf

Sole Source (Including Brand Name) Justification - Simplified Procedures for Certain Commercial Items

CONTROLLED UNCLASSIFIED INFORMATION

March 2023 CONTROLLED UNCLASSIFIED INFORMATION Page 2 of 5March 2023 CONTROLLED UNCLASSIFIED INFORMATION Page 2 of 5

I. Contracting Activity.

338th Enterprise Sourcing Squadron, 2021 First Street West, JBSA-Randolph, TX, 78150

II. Nature and/or description of the action being processed.

The contracting activity is requesting approval for a sole-source justification to purchase the required Questionmark OnPremise Annual Software Subscription and Support (12 months) due to proprietary information that is only available through the manufacturer. This requirement will be used to support the Air Education and Training Command’s (AETC) Technical Training Management System (TTMS) architecture in order to provide its users with the required capabilities to accomplish the mission objectives. TTMS provides the training community with a modernized information technology environment that includes networks, workstations, data servers, and an integrated suite of Commercial-Off-The-Shelf (COTS) software.

The acquisition for annual software subscription and support will be pursued as a new requirement and will be firm-fixed-price 12-month contract.

III. Description of supplies/services required to meet agency needs.

This requirement is for Questionmark OnPremise Annual Software Subscription and Support.

The estimated total value of this acquisition is will be used.

Services to be acquired and their unit price/extended amounts are as follows:

IV. Statutory authority permitting other than full and open competition.

DAFFARS 5313-501 - 10 U.S.C. 2302b, Implementation of Simplified Acquisition Procedures (41 U.S.C. 1901).

V. Demonstration that the contractor's unique qualifications or nature of the acquisition requires the use of the authority cited above (applicability of authority).

TTMS is an integrated, computer-based system that supports course design and development, evaluation, instructor management, student management, and resource administration for AETC technical training organizations.

In 2003, the Government selected Questionmark as the preferred assessment management solution over several other products, and in 2004, Questionmark software was integrated into the TTMS baseline to provide the foundation for two critical functions:

1. TTMS Automated Testing (Autotest) –provides assessment design, automated delivery, and management capabilities for students in technical training to take block tests, progress checks, and other course assessments. TMS Autotest hosts 1600+ administrative users and provides over 70K assessments annually.

2. TTMS Evaluations (Evals) –provides end-of-course survey design, automated delivery, and management capabilities including misconduct survey management. TTMS Evals hosts 2400+ administrative users and provides over 250K surveys annually.

3. Autotest and Evals support over 2,000 courses and over 184,000 students per fiscal year.

Sole Source (Including Brand Name) Justification - Simplified Procedures for Certain Commercial Items

March 2023 CONTROLLED UNCLASSIFIED INFORMATION Page 3 of 5March 2023 CONTROLLED UNCLASSIFIED INFORMATION Page 3 of 5

Currently, AETC/A3GS is on contract with Questionmark to provide the annual software support and learning management system software support for TTMS. Contract FA3002-15-P-0009 was awarded 29 Jun 15 and the last option expired on 28 Jun 20. Contract FA3002-20-C-0013 was awarded on 29 Jun 20 with final option expiring on 28 Dec 23, and contract FA300224P0003 was awarded on 29 Jun 23 and will expire on 28 Dec 24. All were sole source awards. To date, the Air Force has expended on various Questionmark software products and annual maintenance subscriptions.

In addition, over the life of the TTMS system design effort utilizing Questionmark, an estimated has been expended in labor under various tasks to build and support the Questionmark environment for TTMS; this does not include the government employee costs for training/use of Questionmark which are significant. These costs cannot be recuperated.

AETC/A3GS believes that award to any other source would result in substantial reconfiguration of the TTMS baseline software and significant duplication of cost to the government that is not expected to be recovered through competition. The integration of a different product into the TTMS infrastructure would require new TTMS software development, additional development and operational testing, security verification and validation to use the product, and schedule trade-offs with other user requirements. All these factors add to the Government’ cost without the assurance that the software will work on the AETC network or TTMS architecture.

Additionally, costs and resources would need to be forecast, to train all Autotest and Evals users on any new software product. The evaluations and automated testing capabilities are Air Staff requirements and top AETC priorities requested by the field units. A delay or failure to obtain the required Questionmark annual software subscription and support would degrade and potentially eliminate these capabilities and have a significant adverse effect on TTMS operations.

Several examples include: Non-availability of TTMS Auto test and Evals products and services would potentially require the use of paper-based course examinations and evaluations, manual tracking and reporting of examinations and evaluations, and manual updating of student’ records.

This would create a significant increased workload and time to develop, create, administer, grade, record, and report these hard-copy products.

VI. Description of efforts made to ensure that offers are solicited from as many potential sources as practicable.

A search of the DOD ESI website, GSA Advantage, NASA SEWP, and other commercial Internet sites was used to validate Questionmark being the only service capable of meeting the requirements.

While other services exist in the form of Learner Management System, they are either too costly, subpart in what they can provide, or offer service above and beyond the scope of the need.

AETC/A3GS will continue their market research to determine if other vendors become available in providing Questionmark products. If market research determines that another soft wear subscription is feasible and cost effective, a competitive acquisition for a follow-on will be pursued.

Sole Source (Including Brand Name) Justification - Simplified Procedures for Certain Commercial Items

March 2023 CONTROLLED UNCLASSIFIED INFORMATION Page 4 of 5March 2023 CONTROLLED UNCLASSIFIED INFORMATION Page 4 of 5

VII. Determination by the Contracting Officer that the anticipated cost to the Government will be fair and reasonable.

Before making award, the contacting officer will determine that the price is fair and reasonable.

The contracting officer will make a fair and reasonable determination using market research, price comparison found in previous purchases, price list as well as a comparison to the independent Government estimate.

VIII. Description of the market research conducted and the results, or a statement of the reasons market research was not conducted.

The customer, AETC/ A3GS, has thoroughly researched the industry to identify vendors that could satisfy the requirement in support of the mission. GSA Advantage and open market vendor sites were researched to determine if there were any other vendors capable of providing the required Questionmark Annual Software Subscription and Support Plan. GSA’s Online market research tool was able to identify offerings capable of providing services similar to Questionmark. These offerings are not able to provide the unlimited user, server, and testing features Questionmark has without an exorbitant amount of additional fees.

Furthermore, the fees of utilizing a new service do not reflect the additional reprogramming costs the TTMS program would incur, if selected. AETC/A3GS will continue to pursue and research avenues available to them in their quest to identify resellers that can meet the program requirements required by the AETC TTMS program, to increase competition in the future.

In additions, there are no Questionmark agreements on the DOD Enterprise Software Initiative (ESI) website or on the DISA DITCO website.

IX. Any other facts supporting the use of Other Than Full and Open Competition.

Please see section V. above.

X. List of any sources that expressed, in writing, an interest in the acquisition.

Please see section VI above.

XI. A statement of the actions, if any, the agency may take to remove or overcome any barriers to competition before making subsequent acquisitions for the supplies or services required.

Questionmark is the sole manufacturer of the required software subscription plan, and their software is integral to the TTMS program; to replace it would result in an unreasonable number of man-hours expended towards trade studies, reviewing the results, testing software, resolving programmatic code issues, and re-training personnel (system maintainers and field users across the Command) which would not be cost effective for the government.

AETC/A3GS will continue to pursue the search for capable resellers of Questionmark software subscription and support plans that can meet the program requirements of TTMS.

Sole Source (Including Brand Name) Justification - Simplified Procedures for Certain Commercial Items

March 2023 CONTROLLED UNCLASSIFIED INFORMATION Page 5 of 5March 2023 CONTROLLED UNCLASSIFIED INFORMATION Page 5 of 5

XII. Certification by the Contracting Officer.

The Contracting Officer’s signature on the Coordination and Approval Document evidence that she has determined this document to be both accurate and complete to the best of her knowledge and belief.

XIII. Certification by the technical/requirements personnel.

As evidenced by their signatures on the Coordination and Approval Document, the technical and/ or requirements personnel have certified that any supporting data contained herein, which is their responsibility, is both accurate and complete.