19FS1A21Q0004 Enterprise Support for eXist.pdf

19FS1A21Q0004

SOLICITATION/CONTRACT/ORDER FOR COMMERCIAL ITEMS

OFFEROR TO COMPLETE BLOCKS 12, 17, 23, 24 & 30

1. REQUISITION NUMBER

5. SOLICITATION NUMBER

19FS1A21Q0004

2. CONTRACT NO. 3. AWARD/EFFECTIVE DATE 4. ORDER NUMBER 6. SOLICITATION ISSUE

DATE

02/03/2021

7. FOR SOLICITATION

INFORMATION CALL:

a. NAME

Jane Paek

b. TELEPHONE NUMBER (No collect calls)

703-746-2324

c. EMAIL paekj@state.gov

8. OFFER DUE DATE / LOCAL

TIME

03/22/2021 17:00:00

CODE FSIAQ

FSI OFFICE OF AQUISITIONS (FSI/EX/GSACQ)

4000 ARLINGTON BLVD

ARLINGTON, VA 22204

9. ISSUED BY X UNRESTRICTED OR SET ASIDE: % FOR

SMALL BUSINESS

HUBZONE SMALL

BUSINESS

SERVICE-DISABLED

VETERAN-OWNED

SMALL BUSINESS

WOMEN-OWNED SMALL BUSINESS (WOSB)

ELIGIBLE UNDER THE WOMEN-OWNED

SMALL BUSINESS PROGRAM

EDWOSB

8(A)

NAICS: 541511

SIZE STANDARD:

10. THE ACQUISITION IS

SEE

SCHEDULE

11. DELIVERY FOR FOB DESTINATION

UNLESS BLOCK IS MARKED

12. DISCOUNT TERMS

13a. THIS CONTRACT IS

RATED ORDER UNDER DPAS

(15 CFR 700)

13b. RATING

X RFQ IFB RFP

14. METHOD OF SOLICITATION

MULTICODE15. DELIVER TO

SEE

SERVICES NO SHIPPING REQUIRED

SERVICES NO SHIPPING REQUIRED

WASHINGTON, DC 20520

CODE16. ADMINISTERED BY

FACILITY

CODE

CODE

Contact Duns Telephone No.

17a. CONTRACTOR/

OFFEROR

CODE18a. PAYMENT WILL BE MADE BY

BUDGET OFC (FSI/EX/BUD)

FOREIGN SERVICE INSTITUTE

BUDGET & MANAGEMENT OFFICE

WASHINGTON, DC 29004

17b. CHECK IF REMITTANCE IS DIFFERENT AND PUT SUCH ADDRESS IN

OFFER SEE ADDENDUM

18b. SUBMIT INVOICES TO ADDRESS SHOWN IN BLOCK 18a UNLESS BLOCK BELOW IS

CHECKED

19. ITEM NO. 20. SCHEDULE OF SUPPLIES/SERVICES

(Use Reverse and/or Attach Additional Sheets as Necessary)

21. QUANTITY 22. UNIT 23. UNIT PRICE 24. AMOUNT

See Line Items Section

25. ACCOUNTING AND APPROPRIATION DATA 26. TOTAL AWARD AMOUNT (For Govt. Use Only)

X 27a. SOLICITATION INCORPORATES BY REFERENCE FAR 52.212-1, 52.212-4. FAR 52.212-3 AND 52.212-5 ARE ATTACHED. ADDENDA X ARE ARE NOT ATTACHED

27b. CONTRACT/PURCHASE ORDER INCORPORATES BY REFERENCE FAR 52.212-4. FAR 52.212-5 IS ATTACHED. ADDENDA ARE ARE NOT ATTACHED

28. CONTRACTOR IS REQUIRED TO SIGN THIS DOCUMENT AND RETURN ____ COPIES TO

ISSUING OFFICE. CONTRACTOR AGREES TO FURNISH AND DELIVER ALL ITEMS SET FORTH

OR OTHERWISE IDENTIFIED ABOVE AND ON ANY ADDITIONAL SHEETS SUBJECT TO THE

TERMS AND CONDITIONS SPECIFIED.

29. AWARD OF CONTRACT: REF. _____________________________ OFFER

DATED _________________ . YOUR OFFER ON SOLICITATION (BLOCK 5)

INCLUDING ANY ADDITIONS OR CHANGES WHICH ARE SET FORTH HEREIN,

IS ACCEPTED AS TO ITEMS:

30a. SIGNATURE OF OFFEROR/CONTRACTOR 31a. UNITED STATES OF AMERICA (SIGNATURE OF CONTRACTING OFFICER)

30b. NAME AND TITLE OF SIGNER (TYPE OR PRINT) 30c. DATE SIGNED 31b. NAME OF THE CONTRACTING OFFICER (TYPE OR PRINT) Eric R. Modrak

31c. DATE SIGNED

AUTHORIZED FOR LOCAL REPRODUCTION

PREVIOUS EDITION IS NOT USABLE

STANDARD FORM 1449 (REV. 02/2012)

Prescribed by GSA - FAR (48 CFR) 53.212

CGFS : : CGFS

19.

ITEM NO.

20.

SCHEDULE OF SUPPLIES/SERVICES

21.

QUANTITY

22.

UNIT

23.

UNIT PRICE

24.

AMOUNT

32a. QUANTITY IN COLUMN 21 HAS BEEN

RECEIVED INSPECTED ACCEPTED, AND CONFORMS TO THE CONTRACT, EXCEPT AS NOTED: _________________________________

32b. SIGNATURE OF AUTHORIZED GOVERNMENT

REPRESENTATIVE

32c. DATE 32d. PRINTED NAME AND TITLE OF AUTHORIZED GOVERNMENT

REPRESENTATIVE

32e. MAILING ADDRESS OF AUTHORIZED GOVERNMENT REPRESENTATIVE 32f. TELEPHONE NUMBER OF AUTHORIZED GOVERNMENT

REPRESENTATIVE

32g. E-MAIL OF AUTHORIZED GOVERNMENT REPRESENTATIVE

PARTIAL FINAL

33. SHIP NUMBER 34. VOUCHER NUMBER 35. AMOUNT VERIFIED

CORRECT FOR

COMPLETE PARTIAL FINAL

36. PAYMENT 37. CHECK NUMBER

38. S/R ACCOUNT NUMBER 39. S/R VOUCHER NUMBER 40. PAID BY

41a. I CERTIFY THIS ACCOUNT IS CORRECT AND PROPER FOR PAYMENT 42a. RECEIVED BY (Print)

41b. SIGNATURE AND TITLE OF CERTIFYING OFFICER 41c. DATE

42b. RECEIVED AT (Location)

42c. DATE REC'D (YY/MM/DD) 42d. TOTAL CONTAINERS

STANDARD FORM 1449 (REV. 02/2012) BACK

Table of Contents

Section Description Page Number

B Supplies or Services and Prices/Costs

1. Intent to Award

2. Services Parameters

3. Services and Prices

C Descriptions/Specifications/Statement of Work

1. Performance Work Statement

10. Individuals authorized to purchase under the IDIQ contract

11. Contractors

12. Termination of Task Orders

2. Performance Metrics

3. Data Rights

4. Quality Assurance

5. Technical Direction

7. Description of Agreement

8. Purchase Limitation

9. Extent of Obligation

D Packaging and Marking

1. Packaging and Marking

E Inspection and Acceptance 52.246-4 Inspection of Services - Fixed-Price (Aug 1996)

1. Inspection and acceptance of the services

2. The Role of Government Personnel and Responsibility for Contractor Administration

3. Quality Standards

F Deliveries or Performance 52.242-15 Stop-Work Order (Aug 1989)

1. Period of Performance (On or About)

2. Place of Performance

3. Government Closure

4. Cancellation

5. Termination for Excusable Nonperformance

G Contract Administration Data

652.242-70 CONTRACTING OFFICER'S REPRESENTATIVE (COR) (AUG 1999)

FSIO-No INV FSI Non-Payment for Unauthorized Work (use with INV-IPP clause)

INV-IPP ELECTRONIC INVOICING AND PAYMENT REQUIREMENTS AND INSTRUCTIONS

TO IMPLEMENT USAGE OF THE INVOICE PROCESSING PLATFORM (IPP)

H Special Contract Requirements

652.204-70 DEPARTMENT OF STATE PERSONAL IDENTIFICATION CARD POLICY AND

PROCEDURES (FEB 2015)

4. Removal from Duty

1. Security Requirements

2. Non-Payment for Unauthorized Work

3. Contractor Self-Identification

I Contract Clauses 52.252-2 Clauses Incorporated by Reference (Feb 1998) 52.202-1 Definitions (Nov 2013) 52.203-3 Gratuities (Apr 1984) 52.203-12 Limitation On Payments To Influence Certain Federal Transactions (Oct 2010) 52.204-4 Printed or Copied Double-Sided on Postconsumer Fiber Content Paper (May 2011) 52.204-13 System for Award Management Maintenance. (Oct 2018) 52.212-4 Contract Terms and Conditions-Commercial Items (Oct 2018) 52.223-6 Drug Free Workplace (May 2001) 52.227-14 Rights in Data - General (May 2014) 52.232-18 Availability Of Funds (Apr 1984) 52.232-19 Availability Of Funds For The Next Fiscal Year (Apr 1984) 52.233-1 Disputes (May 2014) 52.237-2 Protection Of Government Buildings, Equipment, And Vegetation (Apr 1984)

52.242-13 Bankruptcy (July 1995)

652.232-72 LIMITATION OF FUNDS (AUG 1999)

652.237-72 OBSERVANCE OF LEGAL HOLIDAYS AND ADMINISTRATIVE LEAVE (FEB

2015) 52.203-17 Contractor Employee Whistleblower Rights and Requirement To Inform Employees of Whistleblower Rights (Apr 2014) 52.217-8 Option to Extend Services (Nov 1999)

1. Commercial Items Clauses - Addendum 52.212-5 Contract Terms and Conditions Required to Implement Statutes or Executive Orders- Commercial Items (Nov 2020)

J List of Attachments

1. Attachments

K Representations, Certifications, and Other Statements of Offerors or Respondents 52.209-5 Certification Regarding Responsibility Matters (Oct 2015)

652.225-70 ARAB LEAGUE BOYCOTT OF ISRAEL (AUG 1999)

1. Responsibility

2. Determination of Best Value

L Instructions, Conditions, and Notices to Offerors and Respondents 52.252-1 Solicitation Provisions Incorporated by Reference (Feb 1998)

652.206-70 ADVOCATE FOR COMPETITION/OMBUDSMAN (FEB 2015)

52.212-1 Instructions to Offerors--Commercial Items (Oct 2018) 52.233-2 Service of Protest (Sept 2006)

3. Authorized Negotiators

1. Instructions

2. Past Peformance

M Evaluation Factors for Award 52.217-5 Evaluation of Options (July 1990) 52.212-2 Evaluation - Commercial Items (Oct 2014)

1. Evaluation Factors

2. Right to Award

3. Determination of Best Value

4. Responsibility

5. Basis for Award

Line Item Summary

Solicitation Number:

19FS1A21Q0004

Contract Number: Title: Enterprise Support for eXist Date of Solicitation:

02/03/2021

Line Item No.

Description Quantity Unit Unit Price Total Cost

001 CLIN 0001 Enterprise eXist License Rate Base Year Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2021 to 03/29/2022

1.00 EA

002 CLIN 0002 Maintenance Service Level Agreement Rate Base Year Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2021 to 03/20/2022

0.00 EA

003 CLIN 0003 Development Service Level Agreement Specified Projects Rate Base Year Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2021 to 03/29/2022

0.00 EA

004 CLIN 0004 Development Service Level Agreement

- Additional Projects NOTIONAL HOURLY RATE Base Year Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2021 to 03/29/2022

0.00 EA

1001 CLIN 1001 Enterprise eXist License Rate Option Period 1 Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2022 to 03/29/2023

1.00 EA

1002 CLIN 1002 Maintenance Service Level Agreement Rate Option Period 1 Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2022 to 03/20/2023

0.00 EA

1003 CLIN 1003 Development Service Level Agreement Specified Projects Rate NOTIONAL HOURLY

RATE

Option Period 1 Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2022 to 03/29/2023

0.00 EA

1004 CLIN 1004 Development Service Level Agreement

- Additional Projects Rate Option Period 1 Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2022 to 03/29/2023

0.00 EA

2001 CLIN 2001 Enterprise eXist License Rate Option Period 2 Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2023 to 03/29/2024

1.00 EA

Line Item Summary

Solicitation Number:

19FS1A21Q0004

Contract Number: Title: Enterprise Support for eXist Date of Solicitation:

02/03/2021

Line Item No.

Description Quantity Unit Unit Price Total Cost

2002 CLIN 2002 Maintenance Service Level Agreement Rate Option Period 2 Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2023 to 03/20/2024

0.00 EA

2003 CLIN 2003 Development Service Level Agreement Specified Projects Rate NOTIONAL HOURLY

RATE

Option Period 2 Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2023 to 03/29/2024

0.00 EA

2004 CLIN 2004 Development Service Level Agreement

- Additional Projects Rate Option Period 2 Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2023 to 03/29/2024

0.00 EA

3001 CLIN 3001 Enterprise eXist License Rate Option Period 3 Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2024 to 03/29/2025

1.00 EA

3002 CLIN 3002 Maintenance Service Level Agreement Rate Option Period 3 Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2024 to 03/29/2025

1.00 EA

3003 CLIN 3003 Development Service Level Agreement Specified Projects Rate NOTIONAL HOURLY

RATE

Option Period 3 Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2024 to 03/20/2025

0.00 EA

3004 CLIN 3004 Development Service Level Agreement

- Additional Projects Rate Option Period 3 Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2024 to 03/29/2025

0.00 EA

4001 CLIN 4001 Enterprise eXist License Rate Option Period 4 Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2025 to 03/29/2026

1.00 EA

4002 CLIN 4002 Maintenance Service Level Agreement Rate Option Period 4 Type of Contract Pricing:

Doc Ref No:

1.00 EA

Line Item Summary

Solicitation Number:

19FS1A21Q0004

Contract Number: Title: Enterprise Support for eXist Date of Solicitation:

02/03/2021

Line Item No.

Description Quantity Unit Unit Price Total Cost

Delivery Date (Start to End) Date FOB:

03/30/2025 to 03/29/2026

4003 CLIN 4003 Development Service Level Agreement Specified Projects Rate NOTIONAL HOURLY

RATE

Option Period 4 Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2025 to 03/20/2026

0.00 EA

4004 CLIN 4004 Development Service Level Agreement

- Additional Projects Rate Option Period 4 Type of Contract Pricing:

Doc Ref No:

Delivery Date (Start to End) Date FOB:

03/30/2025 to 03/29/2026

0.00 EA

1. - Intent to Award

The Contractor shall perform the effort required under this IDIQ (Indefinite Delivery/Indefinite Quantity) on a Firm Fixed Price (FFP) basis. The work shall be performed in accordance with this IDIQ. Individual Task Orders will be placed against this IDIQ.

This solicitation and subsequent award(s) will follow FAR 12 (commercial items) and include procedures from FAR 13 (Simplified Acquisition Procedures) utilizing the trade-off method for best value. This solicitation is unrestricted.

2. - Services Parameters

The services shall be performed in accordance with the Performance Work Statement and the additional parameters described within this document.

3. - Services and Prices

Section B – Supplies or Services, and Prices/Costs B. 1. The Foreign Service Institute, Department of State, anticipates the award of an IDIQ contract, which establishes the terms and conditions applicable to future task orders.

B. 2. The services shall be performed in accordance with Section C –Description, Specifications, and Performance Work Statement and as further described elsewhere in this document.

CLIN 0001 Enterprise eXist License Rate $_____________ Base Ordering Period (on or about March 30, 2021 to March 29, 2022)

CLIN 0002 Maintenance Service Level Agreement Rate $_____________ Base Ordering Period (on or about March 30, 2021 to March 29, 2022)

CLIN 0003 Development Service Level Agreement Specified Projects Rate $_____________ Base Ordering Period (on or about March 30, 2021 to March 29, 2022)

CLIN 0004 Development Service Level Agreement - Additional Projects NOTIONAL HOURLY RATE

$_____________ Base Ordering Period (on or about March 30, 2021 to March 29, 2022)

CLIN 1001 Enterprise eXist License Rate $_____________ Ordering Period 1 (on or about March 30, 2022 to March 29, 2023)

CLIN 1002 Maintenance Service Level Agreement Rate $_____________ Ordering Period 1 (on or about March 30, 2022 to March 29, 2023)

CLIN 1003 Development Service Level Agreement Specified Projects Rate NOTIONAL HOURLY RATE $_____________ Ordering Period 1 (on or about March 30, 2022 to March 29, 2023)

CLIN 1004 Development Service Level Agreement - Additional Projects Rate $_____________ Ordering Period 1 (on or about March 30, 2022 to March 29, 2023)

CLIN 2001 Enterprise eXist License Rate $_____________ Ordering Period 2 (on or about March 30, 2023 to March 29, 2024)

CLIN 2002 Maintenance Service Level Agreement Rate $_____________ Ordering Period 2 (on or about March 30, 2023 to March 29, 2024)

CLIN 2003 Development Service Level Agreement Specified Projects Rate NOTIONAL HOURLY RATE $_____________ Ordering Period 2 (on or about March 30, 2023 to March 29, 2024)

CLIN 2004 Development Service Level Agreement - Additional Projects Rate $_____________ Ordering Period 2 (on or about March 30, 2023 to March 29, 2024)

CLIN 3001 Enterprise eXist License Rate $_____________ Ordering Period 3 (on or about March 30, 2024 to March 29, 2025)

CLIN 3002 Maintenance Service Level Agreement Rate $_____________ Ordering Period 3 (on or about March 30, 2024 to March 29, 2025)

CLIN 3003 Development Service Level Agreement Specified Projects Rate NOTIONAL HOURLY RATE $_____________ Ordering Period 3 (on or about March 30, 2024 to March 29, 2025)

CLIN 3004 Development Service Level Agreement - Additional Projects Rate $_____________ Ordering Period 3 (on or about March 30, 2024 to March 29, 2025)

CLIN 4001 Enterprise eXist License Rate $_____________ Ordering Period 4 (on or about March 30, 2025 to March 29, 2026)

CLIN 4002 Maintenance Service Level Agreement Rate $_____________ Ordering Period 4 (on or about March 30, 2025 to March 29, 2026)

CLIN 4003 Development Service Level Agreement Specified Projects Rate NOTIONAL HOURLY RATE $_____________ Ordering Period 4 (on or about March 30, 2025 to March 29, 2026)

CLIN 4004 Development Service Level Agreement - Additional Projects Rate $_____________ Ordering Period 4 (on or about March 30, 2025 to March 29, 2026)

Section C - Descriptions/Specifications/Statement of Work

1. - Performance Work Statement

Introduction

The Foreign Service Institute (FSI) is the training bureau of the U.S. Department of State (DOS) and a principal training institution for the U.S. Government foreign affairs community. The Foreign Service Institute is the U.S. Government’s premier foreign affairs training provider, serving the U.S. Department of State and the entire USG foreign affairs community as a strategic enabler of diplomatic excellence. We are dedicated to ensuring the career-long learning opportunities required for success in today’s global arena. The Institute is located at the George P. Shultz National Foreign Affairs Training Center (NFATC) in Arlington, Virginia, with additional training centers domestically, overseas, and in the Washington metropolitan area.

Purpose

The government has a requirement for enterprise-level support and development services to improve the functionality and performance of the Office of the Historian’s (FSI/OH) public website, history.state.gov (HSG), and the existing content management system, cms.history.state.gov (CMS). These are FedRAMP-low services hosted in Bureau of Information Resource Management’s (IRM) Cloud Shared Services (CSS) Amazon Web Services (AWS) commercial enclave and administered by the Office of the Historian, in compliance with DOS’s Cloud Computing Governance Board (CCGB).

Scope

FSI requires a contractor to provide all computing and support services required to architect and administer software and servers to maintain and improve FSI/OH’s public website, internal applications, and cloud-based server infrastructure.

The contractor will provide ongoing services required to operate, maintain, and safeguard FSI/OH’s existing cloud-based servers and infrastructure and develop solutions for enhancing FSI/OH’s eXist-based applications (including, as needed, improvements to the open source eXist database software package or other components), and train FSI/OH staff on the use of the applications and infrastructure.

Period and Location of Performance

The base Period of Performance is for one (1) year from the date of contract award with four (4) one-year option periods.

All phases of this effort may be performed offsite.

FSI/OH location:

U.S. Department of State, Potomac Annex (SA-4D) 2300 E Street NW Washington, DC 20372

The contractor shall be responsible for having the appropriate workspace, equipment, and material necessary to provide all deliverables and perform all duties required by the government in this PWS. The government will not provide needed equipment for remote network. Travel in and/or around the local Washington, D.C., metro area for the purpose of working at the contractor’s place of performance will not be charged to the government.

Background

The Office of the Historian (FSI/OH), within the Foreign Service Institute (FSI) at the U.S. Department of State (DOS), is responsible, under law, for the preparation and publication of the official documentary history of U.S. foreign policy in the Foreign Relations of the United States (FRUS) series. FSI/OH also prepares policy-supportive historical studies for Department principals and other agencies and conducts an array of other initiatives, ranging from briefing memos to multi-year research projects.

FSI/OH maintains a public website at history.state.gov (hereafter, HSG). In addition to a full text archive of the FRUS series, which contains over 300,000 documents from 550 books published since 1861, HSG contains a history of U.S. diplomatic relations with every state in the world, a database of all of DOS principal officers and chiefs of mission since 1789, and other numerous scholarly resources for the study of U.S. foreign relations and DOS institutional history. In keeping with the Open Government Directive’s principle of transparency, participation, and collaboration as the cornerstone of an open government, all of FSI/OH’s publications can be freely downloaded in the Text Encoding Initiative (TEI) format, an XML-based standard used widely in scholarly electronic text projects. Besides HSG, FSI/OH posts the HSG source code and raw data to theHistoryAtState organization on GitHub.

Since its launch in 2009, HSG has been built on a free, open source software package, calledeXist. eXist excels in the storage and querying of XML data and has enabled the Office to contribute to all key open government initiatives on behalf of the Department, including the Open Government Directive, the Cloud First Initiative, and the Digital Government Strategy. FSI/OH has also used eXist for numerous internal applications, including a content management system (CMS). Staff install eXist and a specific suite of software on their workstations to facilitate data analysis, reporting, and development of new applications, publications, and datasets.

Because eXist plays such a critical role in the Office’s internal and public-facing digital initiatives, knowledge management activities, and publishing and reporting workflows, FSI/OH requires enterprise level support and development services for eXist, applications built on it, and the server infrastructure housing it.

Current Server Environment

FSI/OH’s public website (HSG), internal application server (CMS), and supporting servers are hosted in the DOS IRM CCS commercial Amazon Web Services enclave. These servers rely on Amazon EC2 for compute, ELB for load balancing, and S3 for storage; and are located in Northern Virginia, spread across two availability zones. The EC2 servers are running Amazon Linux 2, with the exception of one CentOS server. Software supporting these applications on the EC2 servers includes:

1. eXist (the core back-end application server and database)

2. ActiveMQ (messaging server for database replication)

3. Nginx (front-end web server)

4. Cantaloupe (image server)

5. Ansible (orchestration)

6. Jenkins (continuous integration)

7. Icinga, Munin, and Cloudwatch (monitoring)

8. OpenVPN and Unbound DNS (for secure access to the VPC)

Integrations with external systems beside Amazon Web Services include:

1. GitHub.com and GitLab.com (version control)

2. Microsoft Azure (DOS-administered single-sign on access to internal web-based applications)

3. NPM, Bower, and Maven (package management)

The EC2 servers are allocated as follows:

Server Name Instance Type Purpose OS hsg-prod-frontend1 t3.medium HSG web proxy 1 Amazon Linux 2 hsg-prod-frontend2 t3.medium HSG web proxy 2 Amazon Linux 2 hsg-prod-backend1 m5.xlarge HSG eXist backend 1 Amazon Linux 2 hsg-prod-backend2 m5.xlarge HSG eXist backend 2 Amazon Linux 2 hsg-dev-backend1 m5.xlarge HSG Dev/Staging Server 1 Amazon Linux 2 hsg-dev-backend2 m5.xlarge HSG Dev/Staging Server 2 Amazon Linux 2 dos-ci m5.xlarge Monitoring and CI CentOS cms-prod-backend1 m5.large CMS eXist backend 1 Amazon Linux 2 cms-dev-backend1 m5.large CMS Dev/Staging Server 1 Amazon Linux 2

(Read more information about theseinstance types.)

Each HSG eXist-based backend database server is proxied by a frontend Nginx-based web proxy server, with traffic distributed by Amazon’s Elastic Load Balancer. Data is replicated between the two databases with ApacheMQ, using the Messaging and Replication library for eXist.

All software deployments and upgrades on the FSI/OH servers are orchestrated via Ansible. Automated web tests are performed on a per-commit or nightly basis via Jenkins on the dos-ci server.

The source code for HSG’s application logic is stored in the HistoryAtState organization on GitHub in thehsg-shellrepository, and the underlying publications and datasets (all unclassified, no PII) are stored in 16 additional HistoryAtState organization’s repositories on GitHub:

1. administrative-timeline

2. carousel

3. conferences

4. frus

5. frus-history

6. gsh

7. hac

8. milestones

9. other-publications

10. pocom

11. rdcr

12. tags

13. travels

14. twitter

15. visits

16. wwdai

Required eXist packages not included in the stock eXist distribution are fetched from a public-facing eXist-based package repository (public-repo) running on the dos-ci server:

1. crypto(An implementation of theEXPath Crypto specification)

2. exsaml(for SAML support for authentication with Microsoft Azure SSO)

3. memsort(for pre-sorting large datasets; will be discontinued after adopting eXist 5’s facets and fields facility)

4. messaging-replication(for replicating eXist between a primary and replica server using ApacheMQ)

5. s3(for interacting with the Amazon S3 API)

6. tei-publisherandlib(theTEI Publisherapplication and associated library for transforming TEI XML into HTML and other output formats)

7. tei-simple(a legacy version of TEI Publisher in use on hsg-shell; will be discontinued after migrating to TEI Publisher)

Of these, TEI Publisher is an especially important part of the application stack. In 2016, HSG migrated from custom XQuery/XSLT for the display of its TEI publications to the open source, eXist-based TEI Publisher library used widely in the TEI community. This library is responsible for the transformation of all of OH’s TEI XML data into HTML, EPUB, and PDF. At the same time, HSG adopted the U.S. Web Design System to achieve its modern, mobile-friendly, and accessible front-end design.

New and updated data is published to the HSG eXist servers using either (1) XML-RPC or WebDAV upload of individual files from FSI/OH workstations or (2) Upload of full application/data packages via the eXist Package Manager, accessible only via VPN. The latter method currently requires the replication trigger to be manually disabled and re-enabled when packages are published, but a facility to automate the disabling/re-enabling of the replication trigger is being developed.

A second public-facing server, labs.history.state.gov, has recently been established to house seven draft-stage or experimental scholarly resources, stored in the following HistoryAtState git repositories on GitHub.com or alternatively on GitLab.com:

1. consular-cards

2. frus-indexes

3. frus-sources

4. frus-unbound

5. people

6. terms

The CMS server houses three internal applications: History Mailbox Archive (HMA), Historical Studies Archive (HSA), and Guides.

The application logic and underlying data is stored in seven private HistoryAtState Git repositories on GitLab.com:

1. hma

2. hma-data

3. hsa

4. hsa-binaries

5. hsa-data

6. guides

7. guides-data

The CMS server allows FSI/OH users authorized via DOS’s Azure-based SSO facility to read and edit data. All edits to TEI XML data are synced to HistoryAtState Git repositories on GitLab.com, and all binary files are synced to a private Amazon S3 bucket.

When bulk updates are performed outside the CMS server, they can be deployed to the CMS server by uploading full application/data packages via the eXist Package Manager, accessible only via VPN. As with HSG, package deployment requires manually disabling/ re-enabling the replication trigger.

Current Desktop Environment

FSI/OH’s standard desktop environment for members of the digital initiatives team is called “hsg-project” (see github.com/ HistoryAtState/hsg-project/wiki). These workstations are connected to a dedicated internet network (DIN), which can access the public internet, from which they obtain the required software for the hsg-project environment (software which can also be installed on home computers used for telework); the version numbers of the software listed is current as of mid-2020, but FSI/OH periodically updates to new stable versions of each.

1. macOS 10.15 (Catalina)

2. oXygen XML Editor 21.1

3. eXist-db 5.2.0

4. Transmit 5.6.2

5. GitHub Desktop 2.5.3

6. Homebrew 2.4.1

7. Java OpenJDK 1.8

8. Apache Ant 1.10.8

9. Apache Maven 3.6.3

10. NodeJS 10

11. Gulp 2.2.0

12. Bower 1.8.8

After installing this software, users clone the hsg-project repository from the HistoryAtState GitHub organization, which contains and running included scripts to clone all data and source code repositories and fetch all require packages, users are able to interact with a complete local copy of the HSG website and perform all content editing, data analysis, and publishing tasks.

The following additional eXist packages from the HistoryAtState organization on GitLab and GitHub are used on hsg-project systems:

1. hsg-staging (for previewing content before publication)

2. release(for batch generating ebooks, press releases, and other publication assets)

System Usage and Capacity

Since 2011, when HSG began collecting usage statistics via Google Analytics, the site has experienced a ten-fold increase in users and page views:

Period Users Page views 2011-2012 600,000 2,800,000 2018-2019 6,700,000 14,800,000

To handle this expanded demand, FSI/OH progressively added server capacity, incorporated replication and load balancing, and enabled caching of frequently accessed resources and resource-intensive views. Other adaptations included the addition of testing and development facilities; continuous integration, monitoring, and publication management systems; and integration with redundant external storage and version control repository services (S3 and GitHub/GitLab). FSI/OH is seeking opportunities for improved stability and performance of the application.

Usage statistics are not collected for the CMS server, but the current architecture readily sustains the usage of our 50 internal users.

Requirements/Responsibilities

FSI/OH requires enterprise level support in two primary areas, each with its own service level agreement, or SLA: (1) a maintenance SLA, to ensure the smooth and secure functioning of production systems (namely, HSG and CMS), and (2) a development SLA, to create new applications, extend eXist and existing applications with new features, and improve the performance of these systems.

A maintenance SLA will be required for all years of the contract. The development SLA is divided into two portions: (2a) specific projects listed below, and (2b) additional projects similar in nature but not yet formulated. Vendors should provide fixed price quotations for completion of the specific projects listed below. The government intends to award the contract for the specified projects to be completed within one year from award, subject to availability of funding. Specific dates will be finalized at the time of obligation. Vendors should also formulate a notional labor hour rate that can be used to provide price quotes for additional development projects. As FSI/OH requires additional projects, FSI/OH and the vendor will use this notional labor hour rate to develop an agreed upon hourly level of effort and a price quote for each project, and separate line item can be added on a firm fixed price basis.

Maintenance Service Level Agreement – general responsibilities

The Maintenance SLA includes all activities related to ensuring the smooth functioning of FSI/OH’s production systems (namely, HSG and CMS).

1. Vendor ensures smooth functioning of production systems.

2. Vendor configures, maintains, and monitors all server instances, as well as applications and services running on the servers.

3. Vendor monitors security advisories and keeps server software updated with security patches.

4. Vendor proactively communicates with FSI/OH to schedule routine maintenance outside of FSI/OH business hours, and to coordinate emergency responses.

5. Vendor provides qualified responses to FSI/OH requests at 4 levels of severity: low, medium, high, and critical.

6. Vendor commits to performing a fixed number of hours of maintenance each month, and availability to perform additional maintenance if requested by FSI/OH.

Development Service Level Agreement

The development SLA includes all other activities related to creating new applications, extending eXist and applications with new features, and improving the performance of systems. The projects listed below must be completed within the base year. FSI/OH may request development services for additional projects that are similar in nature but have not yet formulated at the outset of the contract.

As the vendor completes the development projects listed below and delivers new versions of HSG, CMS, eXist, and the named eXist-based applications and libraries, the vendor must ensure the continued functioning and stability of eXist and all of the core, open source, eXist-based applications and libraries used in HSG, CMS, and FSI/OH’s “hsg-project” development systems discussed above. (These core applications include Dashboard, Documentation, eXide, and Monex, as well as those packages’ dependencies:

FunctX, markdown, shared-resources, packageservice, and semver.xq. Also included are packages that FSI/OH systems depend on for proper functioning: TEI Publisher, TEI Publisher Libraries, TEI Simple PM, EXPath Crypto, existdb-saml, Public Repo, Memsort, Messaging and Replication. The vendor must also ensure continued compatibility of eXist’s XML-RPC and WebDAV interfaces with clients such as oXygen XML Editor and Transmit, the functionality of the hsg-project oXygen project file, and the ability of hsg-project users to use the Homebrew system to obtain compatible versions of dependencies such as Java and Node on up-to-date versions of the macOS and Linux operating systems, as used by hsg-project development and server systems.) If the vendor is unable to supply a compatible, fully functioning version of any of these open source applications or libraries (or their successors) for a new stable version of eXist, then the vendor must supply a functional alternative that is acceptable to FSI/OH.

I. HSG development projects

1. Improve HSG search facility:

1. Upgrade the current implementation of theHSG search pageto use eXist 5’sfacets and fields feature.

2. Migrate existing support existing filters (section, date/time, volumes) and sort options (relevance, oldest to newest, newest to oldest).

3. The definitions of which publications belong to which sections should be easily reconfigurable, to allow changes to site organization and addition of new sections and publications.

4. Match highlighting (KWIC) must work correctly onall eXist’s Lucene-based full text syntax options, including wildcards, boolean, and proximity.

5. Add checkboxes beside keyword search field to enable/disable case-sensitive and/or punctuation-sensitive modes—creating composite or custom full text indexes using eXist’s Lucene-based full text search facility if needed.

6. Add theAdministrative Timelinepublication (classified under the “Department History” section) to the scope of publications included in full text search.

7. Train FSI/OH staff on how to define and register new facets and publications, assuming a knowledge of XQuery and XML.

8. All query results and associated preparation of accurate, relevant KWIC search highlights should be computed on the server in no more than 500 milliseconds total.

9. KWIC hits for FRUS index pages, which were suppressed due to poor performance and server bottlenecks, should be restored.

10. Each keyword query and filter should be displayed in the search results as a “pill” icon, containing an “x” link which updates the search results to eliminate the criterion from the query.

11. Add new advanced search interfaces forFRUS,Travels,Visits, andPrincipal Officers & Chiefs of Mission—leveraging eXist’s facets and fields facility.

12. Migrate from the discontinued, legacyTEI Simplelibrary to the currentTEI Publisherlibrary.

13. Improve FRUS and historical publications browsing:

1. Each FRUS volume’s landing page should display the volume’s cover image.

2. TheHistorical Documents landing pageshould list the most recently published volumes.

3. Create a page displaying all FRUS volumes and availability by format (print, microfiche, electronic-only).

4. From a page image view of a scanned publication, display a link to the corresponding document(s) that appear on the page.

5. In FRUS tables of contents sidebar, indicate where table of contents are truncated and provide a link to expand truncated portions.

6. Fix pop-up footnotes -#202

7. Extend previous/next navigation buttons to other sections, such as current and legacy HAC meeting minutes -#63

8. Show dynamic production chart in FRUS History Appendix A -#155

9. Replace the pages displaying static, screen resolution scans of historical publications with a dynamic, zoomableIIIF-based image implementation.

10. Implement a proper redirect facility for relocated URLs and sections:

1. Background: From time to time, the URLs of resources on HSG are updated, either individually or en masse by section. To ensure readers and search engine bots do not encounter 404 Content Not Found errors, we require a URL redirect facility that allows us to maintain a list of retired URLs and their replacements.

2. Design a declarative format for expressing individual resource URLs and URL patterns and where each should be redirected.

3. Catch requests for matching requests and redirect these requests to the designated pages, using the correct HTTP response for permanently relocated resources.

4. Adapt existing lists of retired URLs to the new system.

5. Implement communication, quality control, and performance improvements:

1. Implement a “report a problem” facility for visitors to report problems with any page on HSG, including typos or broken functionality. Reports should include the URL of the page.

2. Add a “status page” to let readers know when there is a known outage or downtime.

3. Add a “social media card” facility for all HSG pages to include HTML metadata with per-page, per-section, or per-publication images and summary. Allow FSI/OH editors to configure the content of the site-wide default card and define page-, section-, or publication-specific entries.

4. Implement a “link checker” to periodically crawl HSG for links to internal and external resources and produce reports about content not found errors or permanent redirect notices.

5. Create a “sitemap” to aid search engines in crawling our website.

6. Create a “download the raw data” feature to place on all HSG pages to generate links to the appropriate public-facing git repository and source file corresponding to the present view. Create a flexible, declarative template format that FSI/OH editors can populate with the required information.

7. Create a “cite this as” feature to place on all HSG pages to generate MLA or Chicago-style citations, as well as Zotero-compatible citations. Create a flexible, declarative template format that FSI/OH editors can populate with the required information.

8. Create a centralized breadcrumb and sidebar facility -#153and#161. Create a flexible, declarative template format that FSI/OH editors can populate with the required information for breadcrumbs and sidebar content, which allows definitions to apply to an entire section’s worth of content.

9. Ensure that breadcrumbs and sidebars in Tags section show the section’s full depth -#203

10. Create a new “announcements section.” The HSG homepage features 3 current announcements, but past announcements are not easily discovered. Create a new section, entitled “Blog” or “Announcements,” to house these current and past homepage announcements, press releases, and legacy Tumblr posts.

II. CMS and Publishing development projects

1. Implement a new Git-based publishing workflow:

1. Background: Currently we publish in two steps: (1) we commit and push our completed source files to a remote Git repository, and (2) we upload new or changed files to our production servers using XML-RPC and WebDAV from hsg-project development systems. Editors are responsible for remembering both steps. Also, (3) we perform ebook transformations and convert PDFs to per-page image formats using legacy conversion scripts on our development computers, and (4) we manually upload these and all binary assets to Amazon S3 using the Transmit file transfer client. These processes are labor intensive and error-prone.

2. Implement a publishing system that would allow us to publish content using only step 1, and rely on CI tests and automated routines to confirm XML well formedness and generate derivative media, etc.

3. The editor could preview the resulting resources on a test server. When the editor approves, the updated files would be automatically pushed to the production server and committed to the master branch.

4. Adapt the legacy EPUB generation facility for FRUS publications to use the TEI Processing Model and TEI Publisher—while conforming to FSI/OH ebook style conventions for cover and interior pages, interior section-based tables of contents, and page-based cross reference link resolution. This will bring the benefits of consistency and improve our ability to maintain a centralized code base for TEI transformations to all output media.

5. Adapt the legacy PDF generation facility for generating print-ready FSI/OH research monographs, research papers, style guides, and FRUS documentary editions to use the TEI Processing Model and TEI Publisher—while conforming to FSI/OH print style conventions for cover pages, page dimensions, running headers and footers, footnotes, headings, lists, etc.

6. Implement a static file publishing system using Git LFS or a browser-based application for managing the assets that we currently store on Amazon S3 (i.e., images, PDFs, ebook files), without requiring users to access S3 directly with a file transfer client or manage an AWS API key.

7. Implement automated server-side processes to generate static assets from publications. For example, when updates are performed to TEI files that have associated EPUB- and Mobi-formatted ebook files, the publishing pipeline will generate ebook files for review and, upon approval, publication to the site. Similarly, when a PDF file is uploaded, the publishing pipeline will generate all page images needed for page-based browsing. (Currently all ebooks and static images are generated locally on editors’ computers and uploaded to S3.)

8. In the meantime, continue supporting the existing hsg-project system for setting up local development environments and publishing to production HSG servers.

9. All changes must be tested with a focus on continuity of the key the development and publishing workflows described in the hsg-project repository.

10. Implement browser-based annotation and editing of TEI XML documents:

1. Users must be able to select a span of text in an existing TEI document in a TEI Publisher HTML view, identify the entity type (e.g., person, place, organization, etc.), perform a look up of the selected text against a database of existing individuals (an authority file system), and either select an existing individual from the lookup or submit a new candidate to the authority file system.

2. When a user identifies a span of text as a referring string for the entity, the annotation interface should offer to store that string (or an edited version of it) as a variant.

3. The annotation interface should allow users to apply the same annotation to identical spans of text in the same document.

4. The annotation interface should allow users to trigger a scan of an entire document (or division thereof) against known entities and variants and display candidate markings for possible entity matches.

5. The annotation interface should allow users to perform an NLP-based scan of an entire document (or division thereof) to highlight likely proper nouns that may be entities, even if the entities do not yet exist in the database.

6. The annotation interface should allow users to perform limited editing to the base text, e.g., correction of typos identified within block elements.

7. The annotation interface should also allow advanced users to directly edit the TEI XML source of the selection, to enable edits beyond the scope of a browser-based editor.

8. Implement a taxonomy and authority file system (FRUSpedia):

1. Background: Each type of entity described in the interface above should reside in a taxonomy management and authority file system. The types of named entities we track include Persons, Places, Organizations, Subjects, Sources, Publications, Staff, etc.

We have created a notional data model for these entities in Airtable (a web-based relational database management system) but lack a means of managing this data in the eXist-based apps we use to house our documents and data.

2. The authority file system will be the central repository for data named entities in FSI/OH publications and datasets.

3. Editors will be able to propose new entities or edit existing entities. These changes will trigger a review workflow, wherein managers will approve proposed changes.

4. Changes relating to data published on HSG, such as FRUS volume titles and status of the series data, can be published to HSG.

5. The annotation interface described above will draw from the entities in this authority file system.

6. Implement author management facility in HSA:

1. Background: The HSA application contains digitized studies. Each study has an author, and we have a list of authors in XML but no web-based interface for managing these authors.

2. Leverage the entity management database above to house a controlled list of HSA authors and attributes about them.

3. Enhance searching and browsing in HSA:

1. Besides the existing facets, the following facets should be added, based on existing metadata fields in the documents: Years of coverage, Open at NARA, Full text availability, Proofread status,

2. Besides the existing advanced search fields, the following fields should be added: Location, Abstract, External notes, Internal notes.

3. Replace the current codes displayed for Distribution status, Open at NARA, and Tags with labels defined in lookup tables.

4. Add a “# of results per page” drop down to the document table and search results views.

5. Extend HSG punctuation- and case-sensitive options and KWIC search improvements detailed in “Improve HSG search facility” above to HSA.

6. Fix broken links to page images for scanned studies. Use the IIIF facility detailed in “Improve FRUS and historical publications browsing” above to HSA.

7. Enhance the History Mailbox Archive (HMA) application:

1. Add facility for uploading Outlook .msg files to the application. Uploaded binaries should be converted to HTML using the existing Apache Tika-based conversion facility. The binary and converted documents should be committed to version control.

2. Add facility for downloading existing .msg files from the application.

3. Extend the existing Apache Tika-based conversion routine to extract attachments and display image attachments in the application.

4. Bring navigation and styling of application into line with other CMS applications.

5. Bring faceted browsing and field search from HSA: sender, recipient, and date.

6. Develop an Administrative History Portal application:

1. Create a clone of the HSA application and extend it to house and allow ingestion, browsing, and searching of a diverse range of content, including TEI, PDF, and MS Word documents.

2. Extend Location field to include URLs to external resources.

3. Develop an Oral History Portal application:

1. Create a clone of the HSA application to allow ingestion, browsing, and searching of TEI, PDF, and MS Word transcripts of oral histories.

2. Add a field for rights and permissions, with values supplied by FSI/OH.

III. eXist development projects

1. Background: As expert users of the World Wide Web Consortium’s XQuery language, FSI/OH historians depend upon their XQuery implementation to be nearly 100% compliant with the XQuery standard (i.e., the family of XPath and XQuery specifications), as defined by compliance with the XQuery Test Suite. To the extent an implementation falls short of compliance with the specification or contains bugs, FSI/OH historians are less productive and unable to make full use of this language for their research and development projects. Similarly, FSI/OH historians rely on the major eXist-based application and library packages to function properly. These development projects are all urgent priorities for FSI/OH.

2. Implement functions and expressions for full conformance with XQuery 3.1 specification:

1. Implement all remaining XPath and XQuery 3.1 Functions & Operators -#3635

2. Implement XQuery 3.1 tumbling window, sliding window, and count clauses -#3636and#491

3. Implement XQuery 3.1 item-separator serialization parameter -#1483

4. Implement xs:dateTimeStamp datatype -#1715

5. Implement use-character-maps serialization option -#3237

6. Implement XQuery Update Facility 3.0, both for in-memory and on-database nodes -#3634

7. Fix the following bugs in the eXist core reported by FSI/OH:

1. xmldb:store fails on remote binary resources -#1456

2. XQuery Update ignores namespace binding conflict -#1482

3. util:expand alters node type of attribute and document nodes -#1562

4. AST “unexpected end of subtree” error with map postfix lookup -#1655

5. Bugs in encoding special characters in resource names -#1824

6. Path expression with nested positional predicate fails on database node - #1892

7. Cardinality of return types not enforced - #1917

8. NPE in LuceneMatchListener.scanMatches when expanding node containing both Lucene & NGram matches -#2102

9. XQuery parser chokes on legal attribute names -#2381

10. Incorrect facet count with field query -#3207

11. xs:QName castable as returns incorrect results - #3390

12. div operator error when arg is a variable -#3441

13. RESTXQ infinite loop with circular dependency -#3448

14. “variable is not set” error with mutual imported modules -#3449

15. Erroneous error raised by empty array constructor - #3472

16. missing location info in errors for functions inside element constructors (err:XQTY0105) -#3474

17. Guest users can create collections hidden from dba users - #3505

18. Query only returns results from “new range index” collection, omitting results from other collections -#3620

19. NPE at org.exist.xquery.modules.range.RangeQueryRewriter.rewriteLocationStep -#3624

20. First run of DMG eXist.app on macOS quits before launch -#3637

21. strange regex behavior of fn:contains() -#1379

22. Implement the following enhancements to the eXist core:

1. Adopt lazy loading model for module imports -#1848

2. Expose eXist-specific serialization parameters to standard methods -#2394

3. Include location when throwing all XPathException errors -#3241

4. Integrate XQuery Test Suite into CI for PR checks -#3632

5. Use Saxon for fn:matches regular expression handling -#3633

6. Update to…