Attachment0001PerformanceWorkStatement.pdf

UNCLASSIFIED

UNCLASSIFIED

Machine Foreign Language Translation System

(MFLTS)

Text-to-Text (T2T)

Performance Work Statement (PWS)

25 April 2018

DISTRIBUTION STATEMENT: Distribution authorized to the Department of Defense (DoD) and U.S.

DoD Contractors (Administrative Use, 28 March 2013). For other requests for this document, refer to the

Product Manager, Machine Foreign Language Translation System, SFAE-IEW-DCG-MFLT, Building 363, 10115 Gratiot Road, Fort Belvoir, VA 22060-5812, 703-704-4572

FOR OFFICIAL USE ONLY

ii

Table of Contents

1. SCOPE. 3

1.1. Background

2. APPLICABLE DOCUMENTS

2.1. Department of Defense Manuals

2.2. Army Regulations (ARs)

2.3. Other Government Documents

2.4. Industry Documents

3. PERFORMANCE REQUIREMENTS

3.1. General Requirements

3.2. Specific Requirements

3.2.1. Program Management

3.2.2. Develop SW Interfaces and Modify COTS LPs

3.2.3. Develop an Automated LP Installer

3.2.5 Qualification Test Procedures and Test Readiness Reviews

3.2.6 Conduct the Qualification Testing, prepare a QT Report

3.2.7 SW Licenses, Technical Support, and Maintenance

3.2.7.1 Test and Evaluation (T&E) License

3.2.7.2 Annual Enterprise License Agreement

3.2.7.3 Five-Year Enterprise License. ........................................... Error! Bookmark not defined.

3.2.7.4 10-Year Enterprise License. ............................................... Error! Bookmark not defined.

3.2.8 Cybersecurity

3.2.9 Version Description Document (VDD)

3.2.10 Operations Security (OPSEC)

APPENDIX A. ADMINISTRATIVE REQUIREMENTS

APPENDIX B. DATA DELIVERABLES

APPENDIX C. GLOSSARY

APPENDIX D. ACRONYMS

iii

1. SCOPE.

The U.S. Army Program Executive Office Intelligence, Electronic Warfare and Sensors (PEO

IEWS) Product Director (PD) Machine Foreign Language Translation System (MFLTS) requires the integration of Commercial-Off-The-Shelf (COTS) Text-To-Text (T2T) Language Packs

(LPs) with the MFLTS software (SW) application. T2T LPs contract awards under this effort may include any of the following languages: Russian, Modern Standard Arabic, Dari, Pashto, Urdu, Farsi, Polish, Korean, Chinese, and Ukrainian. Additionally, the Contractor shall provide support to the Government’s integration testing and evaluation effort during a Test & Evaluation

(T&E) License period that will occur subsequent to delivery of the qualified LP.

1.1. Background.

MFLTS is a software only program that provides an automated foreign speech and text translation capability in order to augment and complement limited human linguistic resources across all Army echelons in all environments. MFLTS mitigates linguistic shortfalls by expanding the number of languages supported by military translation tools within the DoD today and by increasing the number of language translation points of presence to meet the speech and text translation requirements for current, impending, and unforeseen future operations. The

MFLTS software application resides on portable, mobile, and web-enabled host hardware.

Initial MFLTS capability supports translation of two spoken languages (Pashto, Iraqi Arabic), one written language (Modern Standard Arabic), and two language domains (Checkpoint

Operations & Base Security). The MFLTS program intends to provide a catalogue of language translation SW products to Soldiers via a web portal that enables Soldiers to download and install these products as needed to support their mission requirements.

The MFLTS Software Baseline has been transitioned into the Sustainment Phase of the MFLTS product life cycle, and the U.S. Army Communications and Electronics Command (CECOM)

Software Engineering Center (SEC) Intelligence Support Division (ISD) is currently providing

Post Production SW Support of the MFLTS Software Baseline. In addition, the CECOM SEC is also sustaining three (3) currently fielded MFLTS LPs via Enterprise License Agreements with the Original Equipment Manufacturer (OEM). The purpose of this effort is to integrate

Commercial-Off-The-Shelf (COTS) T2T LPs with the Government owned MFLTS SW

Application to provide additional language translation capability to Soldiers.

2. APPLICABLE DOCUMENTS.

2.1. Department of Defense Manuals.

FOR OFFICIAL USE ONLY

Number Title Date

DoD 5220.22-M National Industrial Security Program

Operating Manual incorporating Change 2

28 February 2006

Change 2, 18 May 2016

DoD 8570.01-M Information Assurance Workforce

Improvement Program, incorporating

Change 4

19 December 2005

Change 4, 10 November 2015

Note: Documents listed above are available for download from the DoD Issuances website: http://www.dtic/whs/directives/.

2.2. Army Regulations (ARs).

Number Title Date

AR 25-2 Information Assurance 24 October 2007

AR 190-13 The Army Physical Security Program 25 February 2011

AR 381-12 Threat Awareness and Reporting Program 1 June 2016

AR 530-1 Operations Security (OPSEC) 26 September 2014

Note: Documents listed above are available for download from the Army Publishing

Directorate website: http://www.apd.army.mil.

2.3. Other Government Documents.

Note1: Documents listed in 2.3 will be provided by MFLTS Program Office (PO) as

Government Furnished Information (GFI) either in section J of the solicitation and or in the

Reading Room at the following URL: https://www.mycmsportal.com/login.aspx

Note 2: The ILR Skill Level Descriptions for Translation Performance are available for download from http://www.govtilr.org/ILR

Number Title Date

N/A MFLTS Installation Instructions 18 December 2017

N/A MFLTS Language Pack QualificationTest Plan 27 March 2018

N/A MFLTS Software Development Kit 19 July 2017

N/A MFLTS Language Pack Component Test Tools 29 January 2018

N/A MFLTS Technical Requirements Document –

Volume 1 - Architecture Specification 28 February 2018

N/A

MFLTS Technical Requirements Document –

Volume 2 - Component-Infrastructure Language

Architecture Base-Line (CILABL) Application and

Language Component (LC) Development Guide

28 February 2018

N/A PM DCGS-A OPSEC Plan,V1.5 21 July 2014 http://www.dtic/whs/directives/ http://www.apd.army.mil/ https://www.mycmsportal.com/login.aspx http://www.govtilr.org/ILR

2.4. Industry Documents.

Number Title Date

ISO 9001 Quality Management Systems - Requirements 2015

ISO/IEC/IEEE

24748-4

Systems and Software Engineering –Life Cycle

Management – Part 4: Systems Engineering Planning

ISO 31000 Risk Management – Principles and Guideline 2009

Note: Documents listed above are available on the International Organization for

Standardization website, https://www.iso.org/home.html

3. PERFORMANCE REQUIREMENTS.

3.1. General Requirements.

The Contractor shall deliver a qualified T2T LP(s) within three (3) months after contract award.

The delivered LPs shall be compliant with the MFLTS SW Architecture and MFLTS Interface requirements. The LPs shall pass Contractor qualification testing in accordance with the applicable Performance Specifications and metrics captured during the product demonstration and the contractor benchmark test after contract award, the Government provided Qualification

Test Plan, and Government approved Qualification Test Procedures. The Contractor shall prepare and deliver test procedures, all resulting test data, test reports, the qualified LP product, and design documentation, any SW Source Code that was developed under this PWS, and T&E

SW License and or Enterprise License.

3.2. Specific Requirements.

3.2.1. Program Management.

The contractor shall provide qualified personnel, adequate resources, and a comprehensive and effective management process to execute all requirements of this contract. Management shall include the effective use of management tools, program planning, integrated program scheduling and execution, performance criteria, performance and technical metrics, risk monitoring and reporting, sub-contractor and/or vendor management and program status monitoring and control.

The contractor shall conduct an Initial Program Review (IPR) within 15 days after contract award and a Final Program Review (FPR) within 10 days after the Final Program Review. These program reviews will highlight contract initiation and provide the Government visibility into program management, planning, scheduling, risk, and technical performance and accomplishments. These reviews may be conducted via teleconference. The contractor shall document and submit IPR and FPR Minutes to the Government IAW Contract Data

Requirements List (CDRL) A001.

https://www.iso.org/home.html

3.2.2. Develop SW Interfaces and Modify COTS LPs.

Before delivering to the government, the contractor shall modify the COTS LP by developing an

Interface that enables the COTS LP to comply with the applicable Automatic Program Interfaces

(APIs) that are defined in the MFLTS Technical Architecture documents. The T2T LP shall be a

SW product that installs independently of the MFLTS application and shall be accessible from and interface with the MFLTS application without requiring modification to the MFLTS application. The LP shall be uniquely identifiable by the MFLTS application upon installation.

The MFLTS Technical Requirements and SW Development Kit documents describe the formats and schema required to provide LP information to the MFLTS application. The Interface source code and related SW design documentation shall be delivered IAW CDRL A002.

3.2.3. Develop an Automated LP Installer.

The contractor shall develop an automated LP installer IAW the file structures and definitions that are specified in the MFLTS Technical Architecture documents and the referenced MFLTS

Installation instructions. The source code for the installer and related SW design documentation shall be delivered IAW CDRL A002. The installer shall ensure all LP components are installed and configured IAW the MFLTS application architecture. For those data items which cannot be automatically determined, the installer shall use a dialog/wizard.

Upon completion of the installation, the user shall be able to access and use the LP through the

MFLTS application without any additional configuration actions. The installer shall also provide the capability to completely uninstall the LP. When executing an uninstall operation, the installer shall remove all files, configuration changes, etc. made to the host IT platform during the installation process. The uninstaller shall provide the user the opportunity to remove or save any output, log file or any other artifact generated during the operation of the language pack. An

Installer shall be delivered along with each LP IAW CDRL A002.

3.2.4 Conduct Pre-Integration Performance Benchmark Tests.

Within fifteen days after contract award the Contractor shall characterize the technical performance of the COTS LP by performing the Bilingual Evaluation Understudy (BLEU) Test and the Word Error Rate (WER) Performance Test as described below. The resulting performance benchmarks shall be incorporated into the Contractor Qualification Test Procedures

(CDRL A003) as the baseline requirement for the Qualification Test (PWS Paragraph 3.2.6).

These benchmarks shall be integrated into the contractor’s Qualification Test Procedures.

3.2.4.1. Offerors shall execute the BLEU test prior to integration of all of the following proposed

COTS LPs: Russian, Dari, Pashto, Urdu, Farsi, Korean, and Mandarin Chinese. Offerors shall execute the BLEU test procedure prior to integration of the COTS LP(s). The BLEU test shall be performed using the National Institute of Standards and Technology (NIST) Machine

Translation Evaluation (MTEVAL) BLEU tool to measure the translation accuracy associated with proposed COTS LP, and will result in one measurement that will be associated with the pre-integration COTS LP. The contractor shall perform the BLEU data collection by processing a set of Government provided documents through the COTS LP. This output will then be

FOR OFFICIAL USE ONLY

processed using the NIST MTEVAL BLEU tool to measure the translation performance associated with the proposed COTS LP. Each language pack will be tested using the appropriate

BLEU Case(s) as set forth below.

• BLEU Case 1 applies to Russian, Dari, Pashto, Urdu, Farsi, Korean, Mandarin Chinese, and COTS LPs. In this case the Offeror will use Corpus 1 to perform a Foreign

Language to English translation and data collect procedure. Corpus 1 is comprised of

Foreign Language documents that the offeror will translate into English using the proposed COTS LP. The Offeror will record the BLEU score generated by the NIST

MTEVAL tool, and provide all of the output from the tool and the output from the COTS

LP to the Government. This BLEU score will become the BLEU performance requirement to be used in the Qualification Test procedure for the associated COTS LP.

• BLEU Case 2 applies to Dari, Pashto, and Korean. The Offeror will use Corpus 1a to perform English to Foreign Language data collect procedure. Corpus 1a is comprised of

English language documents that the offeror will translate to the specified foreign language using the proposed COTS LP. The Offeror will record the BLEU score generated by the NIST MTEVAL tool, and provide all of the output from the tool and the output from the COTS LP to the Government. This BLEU score will become the BLEU performance requirement and shall be incorporated into the Contractor Qualification Test

Procedure (CDRL A003) for the associated COTS LP.

3.2.4.2 Offerors shall execute the WER test prior to integration of any proposed COTS LP that include an OCR LC. Offerors will not perform this procedure for LPs that contain MT only. The

Corpus 2 WER performance from the pre-integration COTS LP shall be the Word Error Rate

(WER) performance requirement used in the Contractor Qualification Test. The contractor shall process a set of Govt provided documents, Corpus 2, through the OCR using the LP and then process the transcribed foreign language OCR output through the NIST SCLite tool to measure the WER. The Offeror shall record the WER score generated by the SCLite tool, and provide all

OCR output and SCLite tool output in the Qualification Test Procedures (CDRL A003).

For Mandarin Chinese LPs that include an OCR LC this metric is the Character Error Rate

(CER). In addition, Corpus2 for Mandarin Chinese LPs contains two sets of documents: one using in the Simplified Chinese character set, and one using the Traditional Chinese character set. The Contractor shall process and report CER scores on both sets of documents in Corpus 2 in the Qualification Test Report.

3.2.5 Qualification Test Procedures and Test Readiness Reviews.

The contractor shall prepare and submit for Government review and comment the LP

Qualification Test (QT) Procedures that are based on the Government QT Plan IAW CDRL

A003. The contractor shall ensure that the QT procedures are complete and sufficient, to demonstrate conformance of the COTS LP with the applicable LP performance specifications, verify the adequacy of the contractor developed Interfaces and Installer, and fully characterize the translation performance of the LP. The QT Procedures shall delineate the specific test

FOR OFFICIAL USE ONLY

environment, test articles, test personnel and equipment and other information required in order for the contractor to successfully demonstrate LP conformance to the specifications and requirements of this contract. The Contractor’s QT Procedures (CDRL A003) shall include the results from the Contractor’s Benchmark testing of Word Error Rate (WER) and Bilingual

Evaluation Understudy (BLEU) translation accuracy. These values shall become the basis for qualifying the post-integration COTS LP. In preparation for the QT, the contractor shall participate via teleconference in up to three (3) Test Readiness Reviews (TRR) before QT and one (1) After Action Review (AAR) following the QT.

3.2.6 Conduct the Qualification Testing, prepare a QT Report.

The contractor shall conduct QT in accordance with the Government’s approved contractor QT procedures at the contractor’s facility, then prepare and submit a QT Report for Government review and comment IAW CDRL A004 upon completion of the QT. Government designated representatives will be present to witness the contractors QT.

3.2.7 SW Licenses, Technical Support, and Maintenance.

The contractor shall provide the Government with a SW License Agreement that specifies the terms, conditions, and usage rights. The SW license shall allow the Government to obtain all updates, improvements, and releases made to the COTS LP during the license period. The

Contractor shall provide a Limited Use Test and Evaluation License and other Enterprise

Licenses as options, as defined in subparagraphs 3.2.7.1 and 3.2.7.2. The Limited Use Test and

Evaluation License and other Enterprise Licenses, defined in subparagraphs 3.2.7.1 and 3.2.7.2, shall be separately priced CLINs and may be executed by the Government at later date during the contract period of performance.

3.2.7.1 Test and Evaluation (T&E) License.

The contractor shall provide a T&E License that provides the Government with the right to use and distribute the qualified LP (for the purpose of supporting T&E of the delivered SW

Products) on up to ten (10) concurrent test systems or devices for a period of 12 months after delivery of the qualified LP to the Government (hereinafter the “T&E License period”). Under this T&E license, the contractor shall provide technical support to the Government, during the

T&E License period, for all delivered LPs, Interface SW, and any SW delivered under this contract. 200 hours of technical call support shall be provided during normal business hours to support Government testing and integration. Technical support shall include contractor telephone and on-line consultations as required to identify and resolve SW installation and operation problems, SW maintenance and fixes for problems that the Government discovers after delivery of the SW Products. Technical support shall also include delivery of updates, improvements, and any releases made to the COTS LP that the contractor distributes at no additional charge to commercial users of the SW Product.

FOR OFFICIAL USE ONLY

3.2.7.2 Annual Enterprise License Agreement (ELA).

The contractor shall provide an Annual ELA that provides the Government with use and distribution of the delivered SW Products on at least 250,000 terminals or devices for a period of one (1) year after delivery of the qualified LP to the Government (hereinafter the “Annual

License period”). Under this Annual ELA, the contractor shall provide technical support to the

Government, during the Annual License period, for all delivered LPs, Interface SW, and any SW delivered under this contract. Technical support shall include contractor telephone and on-line consultations as required to identify and resolve SW installation and operation problems, SW maintenance and fixes for problems that the Government discovers after delivery of the SW

Products. Technical support shall also include delivery of updates, improvements, and any releases made to the COTS LP that the contractor distributes at no additional charge to commercial users of the SW Product. During the Annual License period, the Contractor’s

Annual ELA shall provide contractor maintenance support for their COTS LPs on Android and

Windows Operating Systems (OS) currently released by Google and Microsoft, and shall also provide support for future major OS releases within 4 months after the official Google or

Microsoft release date.

3.2.8 Cybersecurity.

The contractor shall ensure, and certify to the Government that all SW delivered under this contract does not contain, and is protected against the insertion of malicious or malformed code

IAW CDRL A007.

3.2.9 Version Description Document (VDD).

The contractor shall prepare and deliver a VDD with each delivery of SW products to support follow-on Government integration and testing IAW CDRL A008.

3.2.10 Operations Security (OPSEC).

3.2.10.1 The Contractor shall develop an OPSEC Plan IAW CDRL A009 and adhere to the

OPSEC requirements IAW AR 530-1, Chapter 6 as listed in the PM DCGS-A OPSEC Plan.

Information designated as For Official Use Only (FOUO) generated or provided under this contract shall be controlled, handled, stored, marked, safeguarded, transmitted and destroyed

IAW DoDM 5200.01, DoDM 5400.07, AR 25-55 and the following guidance:

a. Electronically transmitted FOUO information shall be sent via: 1) encrypted email,

2) password protected/encrypted document, 3) AMRDEC Safe Access File Exchange at https://safe.amrdec.army.mil/safe/Welcome.aspx, or 4) an accredited information system with proper security controls.

b. The contractor may disseminate FOUO information to their employees and subcontractors that have a need-to-know.

c. Disposal/destruction shall be by means of shredding when no longer needed. If shredding capabilities are unavailable on contractor site – contractor may coordinate with government representative for shredding services.

d. The contractor shall report the unauthorized disclosure of FOUO information to the

Contracting Officers, Contracting Officer Representative, or Information Support

Specialist.

3.2.10.2 The contractor shall safeguard unclassified DoD information stored on non-DoD information systems to prevent the loss, misuse, and unauthorized access to or modification of this information. The contractor shall:

a. Not process DoD information on public computers (e.g., those available for use by the general public in kiosks or hotel business centers) or computers that do not have access control.

b. Protect information by no less than one physical or electronic barrier (e.g., locked container or room, login, and/or password) when not under direct individual control.

c. Sanitize media (e.g., overwrite) before external release or disposal.

d. Encrypt the information that has been identified as Controlled Unclassified

Information (CUI) when it is stored on mobile computing devices such as laptops and personal digital assistants, or removable storage media such as thumb drives and compact disks, using the best available encryption technology.

e. Limit information transferred to subcontractors or teaming partners with a need to know and a commitment to at least the same level of protection.

f. Transmit e-mail, text messages, and similar communications using technology and processes that provide the best level of privacy available, given facilities, conditions, and environment. Examples of recommended technologies or processes include:

closed networks, virtual private networks, public key-enabled encryption, and

Transport Layer Security (TLS).

g. Encrypt organizational wireless connections and use encrypted wireless connection, where available, when traveling. When encrypted wireless is not available, encrypt application files (e.g., spreadsheet and word processing files), using no less than application-provided password protection level encryption.

h. Transmit voice and fax transmissions only when there is a reasonable assurance that access is limited to authorized recipients.

i. Not post DoD information to Web site pages that are publicly available or have access limited only by domain or Internet protocol restriction. Such information may

FOR OFFICIAL USE ONLY

be posted to Web site pages that control access by user identification or password, user certificates, or other technical means and provide protection via use of TLS or other equivalent technologies. Access control may be provided by the intranet (vice the Web site itself or the application it hosts).

j. Provide protection against computer network intrusions and data ex-filtration, including no less than the following:

Current and regularly updated malware protection services, e.g., anti-virus, anti- spyware.

Monitoring and control of inbound and outbound network traffic as appropriate (e.g. at the external boundary, sub-networks, individual hosts) including blocking unauthorized ingress, egress, and ex-filtration through technologies such as firewalls and router policies, intrusion prevention or detection services, and host-based security services.

Prompt application of security-relevant SW patches, service packs, and hot fixes.

k. Comply with other current Federal and DoD information protection and reporting requirements for specified categories of information (e.g., critical program information, Personally Identifiable Information (PII), export controlled information)

IAW the requirements of the contract.

3.2.10.3 Effective 26 August 2015, 48 Code of Federal Regulations (CFR) Parts 202, 204, 212, 239, and 252, Defense Federal Acquisition Regulation Supplement (DFARS) Case 2013-D018

(Network Penetration Reporting and Contracting for Cloud Services) 26 August 2015, requires all contractors to provide adequate security and rapidly report cyber incidents to DoD for covered Defense information for the performance of work under new solicitations and resulting contracts. Contractors are required to follow guidance in DFARS provision at 252.204-7008

(Compliance with Safeguarding Covered Defense Information controls) and DFARS clause at

252.204-7009 (Limitations on the Use or Disclosure of Third- Party Contractor Information) for all solicitations and DFARS clause at 252.204-7012 (Safeguarding Covered Defense Information and Cyber Incident Reporting) for all contracts.

Appendix A. Administrative Requirements

1. Contracting Officer Representative (COR)

The COR will represent the Contracting Officer in monitoring technical performance and all deliverables under this contract, but will not be authorized to change any terms and conditions of this PWS or direct the Contractor to perform work that will require a modification to the PWS.

The COR is responsible for inspection and acceptance of products delivered under this contract.

2. Quality Control and Management

The contractor shall utilize an effective quality control program to ensure that products are provided in accordance with the contract, this PWS, and applicable performance specifications.

The contractor shall execute quality assurance and quality control of all MFLTS deliverables and products.

APPENDIX B. DATA DELIVERABLES

The table below contains a complete list of data deliverables. In accordance with the Reduction of Paper Act, electronic delivery of data whenever possible is required.

PD MFLTS will accept only data that is compatible with Microsoft Office Suite, Adobe Acrobat, or as instructed in the CDRL.

If there is a conflict between the information below and the DD1423s, the DD1423s will take precedence.

CDRL No. Title PWS Reference

Paragraph

A001 Program Review Minutes 3.2.1

A002 Software and Design Documentation 3.2.2, 3.2.3

A003 Contractor Qualification Test Procedures 3.2.4

A004 Qualification Test Reports 3.2.5

A005 Test and Evaluation Software license Agreement 3.2.7.1

A006 Enterprise Software License Agreement 3.2.7.2

A007 Cybersecurity Certification 3.2.8

A008 Software Version Description Document 3.2.9

A009 OPSEC Plan 3.2.10.1

FOR OFFICIAL USE ONLY

APPENDIX C. GLOSSARY

Automated Speech Recognition (ASR): A category of Language Component (LC) that converts speech audio input into a string of text in the language of the speaker.

Bilingual Evaluation Understudy (BLEU): An automated metric for measuring the performance of machine translation systems.

Component Infrastructure Language Architecture Baseline (CILABL - pronounced “syllable”):

The MFLTS software architecture.

Domain: A related set of military tasks with specific associated language content.

Interagency Language Roundtable (IRL) level: A metric used by the US Government for measuring human language proficiency.

Language Component (LC): Cs provide the language translation functionality of the CILABL architecture. There are currently four categories of LCs: Automated Speech Recognition (ASR), Machine Translation (MT), Optical Character Recognition (OCR), and Text to Speech (TTS)

(Speech Synthesis).

Language Module: See Language Pack

Language Pack: A deliverable software package that integrates into the CILABL architecture and performs language translation for a single language pair (English – Foreign Language), with one or more language domains, for a single translation application (e.g., S2S, T2T, etc.) in portable, mobile, and web enabled configurations. A language pack consists of all the associated

LCs, including the language models used in those LCs.

Machine Translation (MT): A category of language component that converts a string of text in one language into a string of text in another language.

Mobile Configuration: A version of the MFLTS software (including a set of applications and language packs) that is built to run on a single computer system, such as a Microsoft Windows laptop, without requiring network connectivity.

Optical Character Recognition (OCR): A category of language component that processes a document image file, identifying language text within the image and producing a digital transcript of the text in the file.

Portable Configuration: A version of the MFLTS software (including a set of applications and language packs) that is built to run on a standalone, handheld computing device such as a smartphone, with without requiring network connectivity.

Text-to-Text (T2T): A language translation application in which the input is text in a given language (printed, handwritten, or machine-readable) and the output is text (either printed or a machine-readable form) in the target language. MFLTS T2T applications are two-way applications, where input can be in English or the target language and output is in the other language.

FOR OFFICIAL USE ONLY

Web enabled configuration: A version of the MFLTS software (including a set of applications and language packs) that is built to run on a server and accessed via a web browser client running on a computer connected to the server.

Word Error Rate (WER): A performance metric for ASR and OCR software that measures the percent of word-level errors in a test set of language content. Word-level errors can be substitutions, omissions, or additions in the ASR or OCR output, as compared to the reference version of the language content.

APPENDIX D. ACRONYMS

AAR After Action Review

AR Army Regulation

BLEU Bilingual Evaluation Understudy

CECOM U.S. Army Communications and Electronics Command

CILABL Component-Infrastructure Language Architecture Base-Line

CLIN Contract Line Item Number

COR Contracting Officer’s Representative

COTS Commercial-off-the-shelf

DLI Defense Language Institute

DoD Department of Defense

FPR Final Program Review

GFI Government Furnished Information

IAW In accordance with

ILR Interagency Language Roundtable

IPR Initial Program Review

ISD Intelligence Support Division

ISO International Organization for Standardization

LC Language Component

LP Language Pack

MFLTS Machine Foreign Language Translation System

MT Machine Translation

NSP Not Separately Priced

OCR Optical Character Recognition 2

OPSEC Operations Security

PD Product Director

PO Product Office

PRM Program Review Minutes

PWS Performance Work Statement

QT Qualification Test

RCT Reading Comprehension Test

RDT&E Research, Development, Test and Evaluation

SEC Software Engineering Center

SW Software

T2T Text to Text

T&E Testing and Evaluation

TRR Test Readiness Review

VDD Version Description Document

WER Word Error Rate

WPM Words Per Minute