Attachment 2 Price Matrix r3.xlsx

XLSX spreadsheet 48 KB Posted

Attached to
Enterprise Support for eXist Federal contract opportunity
Solicitation number
19FS1A21Q0004
Issued by
Department of State

View the file

Other files for this federal contract opportunity

Other files attached to Enterprise Support for eXist, newest first.
File Type Posted
19FS1A21Q0004 A0003.pdf PDF
Attachment 7 - Questions and Answers r1.docx DOCX document
Attachment 2 Price Matrix r2.xlsx XLSX spreadsheet
19FS1A21Q0004 A0002.pdf PDF
Attachment 6 - MRPT Procedures.pdf PDF
Attachment 7 - Questions and Answers.docx DOCX document
19FS1A21Q0004 P0001.pdf PDF
Attachment 5 - PWS r1.docx DOCX document
Attachment 2 Price Matrix r1.xlsx XLSX spreadsheet
Attachment 5 - PWS.docx DOCX document
19FS1A21Q0004 Enterprise Support for eXist.pdf PDF
19FS1A21Q0004 Enterprise Support for eXist.pdf PDF
Attachment 3 - Resume Format.docx DOCX document
Attachment 4 - Letter of Commitment.docx DOCX document
Attachment 2 Price Matrix.xlsx XLSX spreadsheet
Attachment 1 Past Performance Questionnaire.doc DOC document
Show all 16

On GovTribe

Work with this file on GovTribe

  • Download the original file
  • Contacts named in this file
  • Similar government files
  • Ask GovTribe AI about this file
Try for free Login

Text version

The file's text, extracted by GovTribe without its formatting.

Price Matrix

FFP Base Year – Enterprise Support for eXist
Base Year PriceQuantityEstimated HoursPrice of BY Offerings
CLIN 0001 (Software licenses as quoted, if applicable - See Q&A #26)$0.001$ - 0
CLIN 0002 (Maintenance Service Level Agreement)$0.001$ - 0
CLIN 0003 (Development Service Level Agreement - Specified Projects)$ - 01$ - 0
CLIN 0004 (Development Service Level Agreement - Additional Projects) - Notional Hourly Rate$0.00600$ - 0
CLIN 0005 (Maintenance Service Level Agreement - Additional Projects) - Notional Hourly Rate$0.00120$ - 0
Base Year Total Amount:$ - 0
FFP Option Year 1 – Enterprise Support for eXist
OY 1 PriceQuantityEstimated HoursPrice of OY 1 Offerings
CLIN 1001 (Software licenses as quoted, if applicable - See Q&A #26)$0.001$ - 0
CLIN 1002 (Maintenance Service Level Agreement)$0.001$ - 0
CLIN 1003 (Development Service Level Agreement - Specified Projects residual only if needed)$0.000$ - 0
CLIN 1004 (Development Service Level Agreement - Additional Projects) -Notional Labor Hour Rate$0.00600$ - 0
CLIN 1005 (Maintenance Service Level Agreement - Additional Projects) - Notional Hourly Rate$0.00120$ - 0
Option Year 1 Total Amount:$ - 0
FFP Option Year 2 – Enterprise Support for eXist
OY2 PriceQuantityEstimated HoursPrice of OY 2 Offerings
CLIN 2001 (Software licenses as quoted, if applicable - See Q&A #26)$0.001$ - 0
CLIN 2002 (Maintenance Service Level Agreement)$0.001$ - 0
CLIN 2003 (Development Service Level Agreement - Specified Projects residual only if needed) -$0.000$ - 0
CLIN 2004 (Development Service Level Agreement - Additional Projects) -Notional Hourly Rate$0.00600$ - 0
CLIN 2005 (Maintenance Service Level Agreement - Additional Projects) - Notional Hourly Rate$0.00120$ - 0
Option Year 2 Total Amount:$ - 0
FFP Option Year 3 – Enterprise Support for eXist
OY3 PriceQuantityEstimated HoursPrice of OY3 Offerings
CLIN 3001 (Software licenses as quoted, if applicable - See Q&A #26)$0.001$ - 0
CLIN 3002 (Maintenance Service Level Agreement)$0.001$ - 0
CLIN 3004 (Development Service Level Agreement - Specified Projects residual only if needed) -$0.000$ - 0
CLIN 3005 (Development Service Level Agreement - Additional Projects) Notional Hourly Rate$0.00600$ - 0
CLIN 3003 (Maintenance Service Level Agreement - Additional Projects) - Notional Hourly Rate$0.00120$ - 0
Option Year 3 Total Amount:$ - 0
FFP Option Year 4 – Enterprise Support for eXist
OY4 PriceQuantityEstimated HoursPrice of OY4 Offerings
CLIN 4001 (Software licenses as quoted, if applicable - See Q&A #26)$0.001$ - 0
CLIN 4002 (Maintenance Service Level Agreement)$0.001$ - 0
CLIN 4004 (Development Service Level Agreement - Specified Projects residual only if needed)$0.000$ - 0
CLIN 4005 (Development Service Level Agreement - Additional Projects) Notional Hourly Rate$0.00600$ - 0
CLIN 4003 (Maintenance Service Level Agreement - Additional Projects) - Notional Hourly Rate$0.00120$ - 0
Option Year 4 Total Amount:$ - 0

Grand Total: $ - 0

The estimated hours are for price evaluation only and actual requirements may vary. Actual number of hours will be determined by the program requirements.
Notional Hourly Rate:
The hourly rate will be used to develop firm-fixed price task orders based on mutually agreed project parameters
and the time required to complete each project. Once the project scope/deliverables and agreed upon hours are
formalized via a funded task order, the firm fixed price is not subject to change.
Special Projects (additional projects):
Once a project plan (including estimated number of hours and required deliverables) is mutually agreed upon, a firm fixed price task order will be issued. The price of the task order will be determined by multiplying the agreed upon number of hours by the awarded contractor’s Notional Hourly Rate. Once the task order has been issued, the Government may not alter the scope of the project (quantity of deliverables, etc.) and the contractor may not revise the firm fixed price. Please see “National Hourly Rate” section. Prior to acceptance, the COR reserves the right to inspect special project deliverables for quality standards and to confirm that the expectations of the agreed upon project plan were met. The Contractor cannot increase the hours without prior approval from the COR. The hours spent on a project are billable at the established hourly meeting rate. The Government will not pay travel costs.

&1#&"Times New Roman"&10&K000000SENSITIVE BUT UNCLASSIFIED

Instructions

Instructions: 19FS1A21Q0004
1. All prices are fully-loaded firm-fixed price (FFP) rates.
2. All yellow cells shall be completed with FFP quotes for all projects during the base year. Offerors must enter price quotes in Column D on the "Specified Projects" sheet and on Column B on the "Price Matrix" sheet.
3. The Price Matrix and Specified Projects worksheets are protected so only the yellow cells will permit offerors to enter price quotes. All totals will automatically tabulate based on the price quotes you provide in the appropriate yellow cells.
4. Only price quotes that are complete will be considered compliant and will receive evaluation. CLIN 001 is only applicable if your technical quote includes provision of a specific software license. See Q&A document question #26.
5. "Price Matrix" worksheet Column B6 shall match with "Specified Projects" worksheet Column E134.

&1#&"Times New Roman"&10&K000000SENSITIVE BUT UNCLASSIFIED

Specified Projects

PROJECT BUCKETPROJECT NAMESUBPROJECTFFP PER PROJECT QUOTEPROJECT BUCKET TOTALTOTAL
HSG development projects1. Improve HSG search facilitya. Upgrade the current implementation of the HSG search page to use eXist 5’s facets and fields feature.
b. Migrate existing support existing filters (section, date/time, volumes) and sort options (relevance, oldest to newest, newest to oldest).
c. The definitions of which publications belong to which sections should be easily reconfigurable, to allow changes to site organization and addition of new sections and publications.
d. Match highlighting (KWIC) must work correctly on all eXist’s full text syntax options, including wildcards, boolean, and proximity.
e. Add checkboxes beside keyword search field to enable/disable case-sensitive and/or punctuation-sensitive modes—creating composite or custom full text indexes using eXist’s Lucene-based full text search facility if needed.
f. Add the Administrative Timeline publication (classified under the “Department History” section) to the scope of publications included in full text search.
g. Train FSI/OH staff on how to define and register new facets and publications, assuming a knowledge of XQuery and XML.
h. All query results and associated preparation of KWIC search highlights should be computed on the server in no more than 500 milliseconds total.
i. KWIC hits for FRUS index pages, which were suppressed due to poor performance and server bottlenecks, should be restored.
j. Each keyword query and filter should be displayed in the search results as a “pill” icon, containing an “x” link which updates the search results to eliminate the criterion from the query.
k. Add new advanced search interfaces for FRUS, Travels, Visits, and Principal Officers & Chiefs of Mission—leveraging eXist’s facets and fields facility.
l. Migrate from the discontinued, legacy TEI Simple library to the current TEI Publisher library.
Project Total Price:$ - 0
2. Improve FRUS and historical publications browsing:a. Each FRUS volume’s landing page should display the volume’s cover image.
b. The Historical Documents landing page should list the most recently published volumes.
c. Create a page displaying all FRUS volumes and availability by format (print, microfiche, electronic-only).
d. From a page image view of a scanned publication, display a link to the corresponding document(s) that appear on the page.
e. In FRUS tables of contents sidebar, indicate where table of contents are truncated and provide a link to expand truncated portions.
f. Fix pop-up footnotes - #202
g. Extend previous/next navigation buttons to other sections, such as current and legacy HAC meeting minutes - #63
h. Show dynamic production chart in FRUS History Appendix A - #155
i. Replace the pages displaying static, screen resolution scans of historical publications with a dynamic, zoomable IIIF-based image implementation.
Project Total Price:$ - 0
3. Implement a proper redirect facility for relocated URLs and sections:a. Background: From time to time, the URLs of resources on HSG are updated, either individually or en masse by section. To ensure readers and search engine bots do not encounter 404 Content Not Found errors, we require a URL redirect facility that allows us to maintain a list of retired URLs and their replacements.
b. Design a declarative format for expressing individual resource URLs and URL patterns and where each should be redirected.
c. Catch requests for matching requests and redirect these requests to the designated pages, using the correct HTTP response for permanently relocated resources.
d. Adapt existing lists of retired URLs to the new system.
Project Total Price:$ - 0
4. Implement communication, quality control, and performance improvements:a. Implement a “report a problem” facility for visitors to report problems with any page on HSG, including typos or broken functionality. Reports should include the URL of the page.
b. Add a “status page” to let readers know when there is a known outage or downtime.
c. Add a “social media card” facility for all HSG pages to include HTML metadata with per-page, per-section, or per-publication images and summary. Allow FSI/OH editors to configure the content of the site-wide default card and define page-, section-, or publication-specific entries.
d. Implement a “link checker” to periodically crawl HSG for links to internal and external resources and produce reports about content not found errors or permanent redirect notices.
e. Create a “sitemap” to aid search engines in crawling our website.
f. Create a “download the raw data” feature to place on all HSG pages to generate links to the appropriate public-facing git repository and source file corresponding to the present view. Create a flexible, declarative template format that FSI/OH editors can populate with the required information.
g. Create a “cite this as” feature to place on all HSG pages to generate MLA or Chicago-style citations, as well as Zotero-compatible citations. Create a flexible, declarative template format that FSI/OH editors can populate with the required information.
h. Create a centralized breadcrumb and sidebar facility - #153 and #161. Create a flexible, declarative template format that FSI/OH editors can populate with the required information for breadcrumbs and sidebar content, which allows definitions to apply to an entire section’s worth of content.
i. Ensure that breadcrumbs and sidebars in Tags section show the section’s full depth - #203
j. Create a new “announcements section.” The HSG homepage features 3 current announcements, but past announcements are not easily discovered. Create a new section, entitled “Blog” or “Announcements,” to house these current and past homepage announcements, press releases, and legacy Tumblr posts.
Project Total Price:$ - 0$ - 0
CMS and Publishing development projects1. Implement a new Git-based publishing workflow:a. Background: Currently we publish in two steps: (1) we commit and push our completed source files to a remote Git repository, and (2) we upload new or changed files to our production servers using XML-RPC and WebDAV from hsg-project development systems. Editors are responsible for remembering both steps. Also, (3) we perform ebook transformations and convert PDFs to per-page image formats using legacy conversion scripts on our development computers, and (4) we manually upload these and all binary assets to Amazon S3 using the Transmit file transfer client. These processes are labor intensive and error-prone.
b. Implement a publishing system that would allow us to publish content using only step 1, and rely on CI tests and automated routines to confirm XML well formedness and generate derivative media, etc.
c. The editor could preview the resulting resources on a test server. When the editor approves, the updated files would be automatically pushed to the production server and committed to the master branch.
d. Adapt the legacy EPUB generation facility for FRUS publications to use the TEI Processing Model and TEI Publisher—while conforming to FSI/OH ebook style conventions for cover and interior pages, interior section-based tables of contents, and page-based cross reference link resolution. This will bring the benefits of consistency and improve our ability to maintain a centralized code base for TEI transformations to all output media.
e. Adapt the legacy PDF generation facility for generating print-ready FSI/OH research monographs, research papers, style guides, and FRUS documentary editions to use the TEI Processing Model and TEI Publisher—while conforming to FSI/OH print style conventions for cover pages, page dimensions, running headers and footers, footnotes, headings, lists, etc.
f. Implement a static file publishing system using Git LFS or a browser-based application for managing the assets that we currently store on Amazon S3 (i.e., images, PDFs, ebook files), without requiring users to access S3 directly with a file transfer client or manage an AWS API key.
g. Implement automated server-side processes to generate static assets from publications. For example, when updates are performed to TEI files that have associated EPUB- and Mobi-formatted ebook files, the publishing pipeline will generate ebook files for review and, upon approval, publication to the site. Similarly, when a PDF file is uploaded, the publishing pipeline will generate all page images needed for page-based browsing. (Currently all ebooks and static images are generated locally on editors’ computers and uploaded to S3.)
h. In the meantime, continue supporting the existing hsg-project system for setting up local development environments and publishing to production HSG servers.
i. All changes must be tested with a focus on continuity of the key the development and publishing workflows described in the hsg-project repository.
2. Implement browser-based annotation and editing of TEI XML documents:a. Users must be able to select a span of text in an existing TEI document in a TEI Publisher HTML view, identify the entity type (e.g., person, place, organization, etc.), perform a look up of the selected text against a database of existing individuals (an authority file system), and either select an existing individual from the lookup or submit a new candidate to the authority file system.
b. When a user identifies a span of text as a referring string for the entity, the annotation interface should offer to store that string (or an edited version of it) as a variant.
c. The annotation interface should allow users to apply the same annotation to identical spans of text in the same document.
d. The annotation interface should allow users to trigger a scan of an entire document (or division thereof) against known entities and variants and display candidate markings for possible entity matches.
e. The annotation interface should allow users to perform an NLP-based scan of an entire document (or division thereof) to highlight likely proper nouns that may be entities, even if the entities do not yet exist in the database.
f. The annotation interface should allow users to perform limited editing to the base text, e.g., correction of typos identified within block elements.
g. The annotation interface should also allow advanced users to directly edit the TEI XML source of the selection, to enable edits beyond the scope of a browser-based editor.
Project Total Price$ - 0
3. Implement a taxonomy and authority file system (FRUSpedia):a. Background: Each type of entity described in the interface above should reside in a taxonomy management and authority file system. The types of named entities we track include Persons, Places, Organizations, Subjects, Sources, Publications, Staff, etc. We have created a notional data model for these entities in Airtable (a web-based relational database management system) but lack a means of managing this data in the eXist Publisher-based apps we use to house our documents.
b. The authority file system will be the central repository for data named entities in FSI/OH publications and datasets.
c. Editors will be able to propose new entities or edit existing entities. These changes will trigger a review workflow, wherein managers will approve proposed changes.
d. Changes relating to data published on HSG, such as FRUS volume titles and status of the series data, can be published to HSG.
e. The annotation interface described above will draw from the entities in this authority file system.
Project Total Price$ - 0
4. Implement author management facility in HSA:a. Background: The HSA application contains digitized studies. Each study has an author, and we have a list of authors in XML but no web-based interface for managing these authors.
b. Leverage the entity management database above to house a controlled list of HSA authors and attributes about them.
5. Enhance searching and browsing in HSA:a. Besides the existing facets, the following facets should be added, based on existing metadata fields in the documents: Years of coverage, Open at NARA, Full text availability, Proofread status,
b. Besides the existing advanced search fields, the following fields should be added: Location, Abstract, External notes, Internal notes.
c. Replace the current codes displayed for Distribution status, Open at NARA, and Tags with labels defined in lookup tables.
d. Add a “# of results per page” drop down to the document table and search results views.
e. Extend HSG punctuation- and case-sensitive options and KWIC search improvements detailed in “Improve HSG search facility” above to HSA.
f. Fix broken links to page images for scanned studies. Use the IIIF facility detailed in “Improve FRUS and historical publications browsing” above to HSA.
Project Total Price:$ - 0
6. Enhance the History Mailbox Archive (HMA) application:a. Add facility for uploading Outlook .msg files to the application. Uploaded binaries should be converted to HTML using the existing Apache Tika-based conversion facility. The binary and converted documents should be committed to version control.
b. Add facility for downloading existing .msg files from the application.
c. Extend the existing Apache Tika-based conversion routine to extract attachments and display image attachments in the application.
d. Bring navigation and styling of application into line with other CMS applications.
e. Bring faceted browsing and field search from HSA: sender, recipient, and date.
7. Develop an Administrative History Portal application:a. Create a clone of the HSA application and extend it to house and allow ingestion, browsing, and searching of a diverse range of content, including TEI, PDF, and MS Word documents.
b. Extend Location field to include URLs to external resources.
8. Develop an Oral History Portal application:a. Create a clone of the HSA application to allow ingestion, browsing, and searching of TEI, PDF, and MS Word transcripts of oral histories.
b. Add a field for rights and permissions, with values supplied by FSI/OH.
Project Total Price:$ - 0$ - 0
eXist development projects1. Background: As expert users of the World Wide Web Consortium’s XQuery language, FSI/OH historians depend upon their XQuery implementation to be nearly 100% compliant with the XQuery standard (i.e., the family of XPath and XQuery specifications), as defined by compliance with the XQuery Test Suite. To the extent an implementation falls short of compliance with the specification or contains bugs, FSI/OH historians are less productive and unable to make full use of this language for their research and development projects. Similarly, FSI/OH historians rely on the major eXist-based application and library packages to function properly. These development projects are all urgent priorities for FSI/OH.
Project Total Price$ - 0
2. Implement functions and expressions for full conformance with XQuery 3.1 specification:a. Implement all remaining XPath and XQuery 3.1 Functions & Operators - #3635
b. Implement XQuery 3.1 tumbling window, sliding window, and count clauses - #3636 and #491
c. Implement XQuery 3.1 item-separator serialization parameter - #1483
d. Implement xs:dateTimeStamp datatype - #1715
e. Implement use-character-maps serialization option - #3237
f. Implement XQuery Update Facility 3.0, both for in-memory and on-database nodes - #3634
Project Total Price$ - 0
3. Fix the following bugs in the eXist core reported by FSI/OH:a. xmldb:store fails on remote binary resources - #1456
b. XQuery Update ignores namespace binding conflict - #1482
c. util:expand alters node type of attribute and document nodes - #1562
d. AST “unexpected end of subtree” error with map postfix lookup - #1655
e. Bugs in encoding special characters in resource names - #1824
f. Path expression with nested positional predicate fails on database node - #1892
g. Cardinality of return types not enforced - #1917
h. NPE in LuceneMatchListener.scanMatches when expanding node containing both Lucene & NGram matches - #2102
i. XQuery parser chokes on legal attribute names - #2381
j. Incorrect facet count with field query - #3207
k. xs:QName castable as returns incorrect results - #3390
l. div operator error when arg is a variable - #3441
m. RESTXQ infinite loop with circular dependency - #3448
n. “variable is not set” error with mutual imported modules - #3449
o. Erroneous error raised by empty array constructor - #3472
p. missing location info in errors for functions inside element constructors (err:XQTY0105) - #3474
q. Guest users can create collections hidden from dba users - #3505
r. Query only returns results from “new range index” collection, omitting results from other collections - #3620
s. NPE at org.exist.xquery.modules.range.RangeQueryRewriter.rewriteLocationStep - #3624
t. First run of DMG eXist.app on macOS quits before launch - #3637
u. strange regex behavior of fn:contains() - #1379
Project Total Price$ - 0
4. Implement the following enhancements to the eXist core:a. Adopt lazy loading model for module imports - #1848
b. Expose eXist-specific serialization parameters to standard methods - #2394
c. Include location when throwing all XPathException errors - #3241
d. Integrate XQuery Test Suite into CI for PR checks - #3632
e. Use Saxon for fn:matches regular expression handling - #3633
f. Update to Lucene 5.3+ - #1160
Project Total Price$ - 0
5. Address the following requests by FSI/OH regarding apps included with the eXist distribution:
5.a. eXidei. Save as should use current file’s collection - #104
ii. Glitches in final character of XQuery snippets - #105
iii. Login dialog should focus on User input field with File > Manage #109
iv. Add download button to DB Manager view - #111
v. Autocomplete fails to notice that a comment block is already closed - #112
vi. Save keyboard shortcut doesn’t work when focus is in Find pane - #113
vii. Save as does not update the directory sidebar - #114
viii. Add option to disable auto-pair of quotes, parens, etc. - #117
ix. Warn user when saving contents of XQuery window with non-.xq* file extension - #126
x. Switch to xqlint for syntax highlighting of query results - #147
xi. With configuration.xml set to guest=no, login is impossible - #164
xii. Resource operations using “name” instead of “key” are prone to fail #185
xiii. xqlint error causes “run test” to silently fail - #224
b. Dashboardi. Logging in shouldn’t require duplicate entry of credentials - #156
c. Documentationi. Detail access details methods for JMX info over HTTP - #106
ii. Update the URL Rewriting article - #200
iii. Document the “exist-serialization” processor instruction - #265
iv. Update the XQuery article to note unsupported serialization parameters - #436
v. Document the “lowercase-expanded-terms” option - #447
Project Total Price$ - 0
6. Implement the following improvements to TEI Publisher:a. Upgrade EPUB generation facility from EPUB 2 to EPUB 3—specifically for its support of “pop-over footnotes.” The current EPUB 2-based link approach to footnotes is jarring and clunky.
b. Extend TEI Publisher to support the annotation interface described in “Implement browser-based annotation and editing of TEI XML documents” above.
Project Total Price$ - 0$ - 0
Specified Projects Firm Fixed Price$ - 0

&1#&"Times New Roman"&10&K000000SENSITIVE BUT UNCLASSIFIED

Projects Reference Details

Project Title
I.HSG Development Projects
1. Improve HSG search facility:
a. Upgrade the current implementation of the HSG search page to use eXist 5’s facets and fields feature.
b. Migrate existing support existing filters (section, date/time, volumes) and sort options (relevance, oldest to newest, newest to oldest).
c. The definitions of which publications belong to which sections should be easily reconfigurable, to allow changes to site organization and addition of new sections and publications.
d. Match highlighting (KWIC) must work correctly on all eXist’s Lucene-based full text syntax options, including wildcards, boolean, and proximity.
e. Add checkboxes beside keyword search field to enable/disable case-sensitive and/or punctuation-sensitive modes—creating composite or custom full text indexes using eXist’s Lucene-based full text search facility if needed.
f. Add the Administrative Timeline publication (classified under the “Department History” section) to the scope of publications included in full text search.
g. Train FSI/OH staff on how to define and register new facets and publications, assuming a knowledge of XQuery and XML.
h. All query results and associated preparation of accurate, relevant KWIC search highlights should be computed on the server in no more than 500 milliseconds total.
i. KWIC hits for FRUS index pages, which were suppressed due to poor performance and server bottlenecks, should be restored.
j. Each keyword query and filter should be displayed in the search results as a “pill” icon, containing an “x” link which updates the search results to eliminate the criterion from the query.
k. Add new advanced search interfaces for FRUS, Travels, Visits, and Principal Officers & Chiefs of Mission—leveraging eXist’s facets and fields facility.
l. Migrate from the discontinued, legacy TEI Simple library to the current TEI Publisher library.
2. Improve FRUS and historical publications browsing:
a. Each FRUS volume’s landing page should display the volume’s cover image.
b. The Historical Documents landing page should list the most recently published volumes.
c. Create a page displaying all FRUS volumes and availability by format (print, microfiche, electronic-only).
d. From a page image view of a scanned publication, display a link to the corresponding document(s) that appear on the page.
e. In FRUS tables of contents sidebar, indicate where table of contents are truncated and provide a link to expand truncated portions.
f. Fix pop-up footnotes - #202
g. Extend previous/next navigation buttons to other sections, such as current and legacy HAC meeting minutes - #63
h. Show dynamic production chart in FRUS History Appendix A - #155
i. Replace the pages displaying static, screen resolution scans of historical publications with a dynamic, zoomable IIIF-based image implementation.
3. Implement a proper redirect facility for relocated URLs and sections:
a. Background: From time to time, the URLs of resources on HSG are updated, either individually or en masse by section. To ensure readers and search engine bots do not encounter 404 Content Not Found errors, we require a URL redirect facility that allows us to maintain a list of retired URLs and their replacements.
b. Design a declarative format for expressing individual resource URLs and URL patterns and where each should be redirected.
c. Catch requests for matching requests and redirect these requests to the designated pages, using the correct HTTP response for permanently relocated resources.
d. Adapt existing lists of retired URLs to the new system.
4. Implement communication, quality control, and performance improvements:
a. Implement a “report a problem” facility for visitors to report problems with any page on HSG, including typos or broken functionality. Reports should include the URL of the page.
b. Add a “status page” to let readers know when there is a known outage or downtime.
c. Add a “social media card” facility for all HSG pages to include HTML metadata with per-page, per-section, or per-publication images and summary. Allow FSI/OH editors to configure the content of the site-wide default card and define page-, section-, or publication-specific entries.
d. Implement a “link checker” to periodically crawl HSG for links to internal and external resources and produce reports about content not found errors or permanent redirect notices.
e. Create a “sitemap” to aid search engines in crawling our website.
f. Create a “download the raw data” feature to place on all HSG pages to generate links to the appropriate public-facing git repository and source file corresponding to the present view. Create a flexible, declarative template format that FSI/OH editors can populate with the required information.
g. Create a “cite this as” feature to place on all HSG pages to generate MLA or Chicago-style citations, as well as Zotero-compatible citations. Create a flexible, declarative template format that FSI/OH editors can populate with the required information.
h. Create a centralized breadcrumb and sidebar facility - #153 and #161. Create a flexible, declarative template format that FSI/OH editors can populate with the required information for breadcrumbs and sidebar content, which allows definitions to apply to an entire section’s worth of content.
i. Ensure that breadcrumbs and sidebars in Tags section show the section’s full depth - #203
j. Create a new “announcements section.” The HSG homepage features 3 current announcements, but past announcements are not easily discovered. Create a new section, entitled “Blog” or “Announcements,” to house these current and past homepage announcements, press releases, and legacy Tumblr posts.
II.CMS and Publishing development projects
1. Implement a new Git-based publishing workflow:
a. Background: Currently we publish in two steps: (1) we commit and push our completed source files to a remote Git repository, and (2) we upload new or changed files to our production servers using XML-RPC and WebDAV from hsg-project development systems. Editors are responsible for remembering both steps. Also, (3) we perform ebook transformations and convert PDFs to per-page image formats using legacy conversion scripts on our development computers, and (4) we manually upload these and all binary assets to Amazon S3 using the Transmit file transfer client. These processes are labor intensive and error-prone.
b. Implement a publishing system that would allow us to publish content using only step 1, and rely on CI tests and automated routines to confirm XML well formedness and generate derivative media, etc.
c. The editor could preview the resulting resources on a test server. When the editor approves, the updated files would be automatically pushed to the production server and committed to the master branch.
d. Adapt the legacy EPUB generation facility for FRUS publications to use the TEI Processing Model and TEI Publisher—while conforming to FSI/OH ebook style conventions for cover and interior pages, interior section-based tables of contents, and page-based cross reference link resolution. This will bring the benefits of consistency and improve our ability to maintain a centralized code base for TEI transformations to all output media.
e. Adapt the legacy PDF generation facility for generating print-ready FSI/OH research monographs, research papers, style guides, and FRUS documentary editions to use the TEI Processing Model and TEI Publisher—while conforming to FSI/OH print style conventions for cover pages, page dimensions, running headers and footers, footnotes, headings, lists, etc.
f. Implement a static file publishing system using Git LFS or a browser-based application for managing the assets that we currently store on Amazon S3 (i.e., images, PDFs, ebook files), without requiring users to access S3 directly with a file transfer client or manage an AWS API key.
g. Implement automated server-side processes to generate static assets from publications. For example, when updates are performed to TEI files that have associated EPUB- and Mobi-formatted ebook files, the publishing pipeline will generate ebook files for review and, upon approval, publication to the site. Similarly, when a PDF file is uploaded, the publishing pipeline will generate all page images needed for page-based browsing. (Currently all ebooks and static images are generated locally on editors’ computers and uploaded to S3.)
h. In the meantime, continue supporting the existing hsg-project system for setting up local development environments and publishing to production HSG servers.
i. All changes must be tested with a focus on continuity of the key the development and publishing workflows described in the hsg-project repository.
2. Implement browser-based annotation and editing of TEI XML documents:
a. Users must be able to select a span of text in an existing TEI document in a TEI Publisher HTML view, identify the entity type (e.g., person, place, organization, etc.), perform a look up of the selected text against a database of existing individuals (an authority file system), and either select an existing individual from the lookup or submit a new candidate to the authority file system.
b. When a user identifies a span of text as a referring string for the entity, the annotation interface should offer to store that string (or an edited version of it) as a variant.
c. The annotation interface should allow users to apply the same annotation to identical spans of text in the same document.
d. The annotation interface should allow users to trigger a scan of an entire document (or division thereof) against known entities and variants and display candidate markings for possible entity matches.
e. The annotation interface should allow users to perform an NLP-based scan of an entire document (or division thereof) to highlight likely proper nouns that may be entities, even if the entities do not yet exist in the database.
f. The annotation interface should allow users to perform limited editing to the base text, e.g., correction of typos identified within block elements.
g. The annotation interface should also allow advanced users to directly edit the TEI XML source of the selection, to enable edits beyond the scope of a browser-based editor.
3. Implement a taxonomy and authority file system (FRUSpedia):
a. Background: Each type of entity described in the interface above should reside in a taxonomy management and authority file system. The types of named entities we track include Persons, Places, Organizations, Subjects, Sources, Publications, Staff, etc. We have created a notional data model for these entities in Airtable (a web-based relational database management system) but lack a means of managing this data in the eXist-based apps we use to house our documents and data.
b. The authority file system will be the central repository for data named entities in FSI/OH publications and datasets.
c. Editors will be able to propose new entities or edit existing entities. These changes will trigger a review workflow, wherein managers will approve proposed changes.
d. Changes relating to data published on HSG, such as FRUS volume titles and status of the series data, can be published to HSG.
e. The annotation interface described above will draw from the entities in this authority file system.
4. Implement author management facility in HSA:
a. Background: The HSA application contains digitized studies. Each study has an author, and we have a list of authors in XML but no web-based interface for managing these authors.
b. Leverage the entity management database above to house a controlled list of HSA authors and attributes about them.
5. Enhance searching and browsing in HSA:
a. Besides the existing facets, the following facets should be added, based on existing metadata fields in the documents: Years of coverage, Open at NARA, Full text availability, Proofread status,
b. Besides the existing advanced search fields, the following fields should be added: Location, Abstract, External notes, Internal notes.
c. Replace the current codes displayed for Distribution status, Open at NARA, and Tags with labels defined in lookup tables.
d. Add a “# of results per page” drop down to the document table and search results views.
e. Extend HSG punctuation- and case-sensitive options and KWIC search improvements detailed in “Improve HSG search facility” above to HSA.
f. Fix broken links to page images for scanned studies. Use the IIIF facility detailed in “Improve FRUS and historical publications browsing” above to HSA.
6. Enhance the History Mailbox Archive (HMA) application:
a. Add facility for uploading Outlook .msg files to the application. Uploaded binaries should be converted to HTML using the existing Apache Tika-based conversion facility. The binary and converted documents should be committed to version control.
b. Add facility for downloading existing .msg files from the application.
c. Extend the existing Apache Tika-based conversion routine to extract attachments and display image attachments in the application.
d. Bring navigation and styling of application into line with other CMS applications.
e. Bring faceted browsing and field search from HSA: sender, recipient, and date.
7. Develop an Administrative History Portal application:
a. Create a clone of the HSA application and extend it to house and allow ingestion, browsing, and searching of a diverse range of content, including TEI, PDF, and MS Word documents.
b. Extend Location field to include URLs to external resources.
8. Develop an Oral History Portal application:
a. Create a clone of the HSA application to allow ingestion, browsing, and searching of TEI, PDF, and MS Word transcripts of oral histories.
b. Add a field for rights and permissions, with values supplied by FSI/OH.
III.eXist development projects
1. Background: As expert users of the World Wide Web Consortium’s XQuery language, FSI/OH historians depend upon their XQuery implementation to be nearly 100% compliant with the XQuery standard (i.e., the family of XPath and XQuery specifications), as defined by compliance with the XQuery Test Suite. To the extent an implementation falls short of compliance with the specification or contains bugs, FSI/OH historians are less productive and unable to make full use of this language for their research and development projects. Similarly, FSI/OH historians rely on the major eXist-based application and library packages to function properly. These development projects are all urgent priorities for FSI/OH.
2. Implement functions and expressions for full conformance with XQuery 3.1 specification:
a. Implement all remaining XPath and XQuery 3.1 Functions & Operators - #3635
b. Implement XQuery 3.1 tumbling window, sliding window, and count clauses - #3636 and #491
c. Implement XQuery 3.1 item-separator serialization parameter - #1483
d. Implement xs:dateTimeStamp datatype - #1715
e. Implement use-character-maps serialization option - #3237
f. Implement XQuery Update Facility 3.0, both for in-memory and on-database nodes - #3634
3. Fix the following bugs in the eXist core reported by FSI/OH:
a. xmldb:store fails on remote binary resources - #1456
b. XQuery Update ignores namespace binding conflict - #1482
c. util:expand alters node type of attribute and document nodes - #1562
d. AST “unexpected end of subtree” error with map postfix lookup - #1655
e. Bugs in encoding special characters in resource names - #1824
f. Path expression with nested positional predicate fails on database node - #1892
g. Cardinality of return types not enforced - #1917
h. NPE in LuceneMatchListener.scanMatches when expanding node containing both Lucene & NGram matches - #2102
i. XQuery parser chokes on legal attribute names - #2381
j. Incorrect facet count with field query - #3207
k. xs:QName castable as returns incorrect results - #3390
l. div operator error when arg is a variable - #3441
m. RESTXQ infinite loop with circular dependency - #3448
n. “variable is not set” error with mutual imported modules - #3449
o. Erroneous error raised by empty array constructor - #3472
p. missing location info in errors for functions inside element constructors (err:XQTY0105) - #3474
q. Guest users can create collections hidden from dba users - #3505
r. Query only returns results from “new range index” collection, omitting results from other collections - #3620
s. NPE at org.exist.xquery.modules.range.RangeQueryRewriter.rewriteLocationStep - #3624
t. First run of DMG eXist.app on macOS quits before launch - #3637
u. strange regex behavior of fn:contains() - #1379
4. Implement the following enhancements to the eXist core:
a. Adopt lazy loading model for module imports - #1848
b. Expose eXist-specific serialization parameters to standard methods - #2394
c. Include location when throwing all XPathException errors - #3241
d. Integrate XQuery Test Suite into CI for PR checks - #3632
e. Use Saxon for fn:matches regular expression handling - #3633
f. Update to Lucene 5.3+ - #1160
5. Address the following requests by FSI/OH regarding apps included with the eXist distribution:
a. eXide
i. Save as should use current file’s collection - #104
ii. Glitches in final character of XQuery snippets - #105
iii. Login dialog should focus on User input field with File > Manage #109
iv. Add download button to DB Manager view - #111
v. Autocomplete fails to notice that a comment block is already closed - #112
vi. Save keyboard shortcut doesn’t work when focus is in Find pane - #113
vii. Save as does not update the directory sidebar - #114
viii. Add option to disable auto-pair of quotes, parens, etc. - #117
ix. Warn user when saving contents of XQuery window with non-.xq* file extension - #126
x. Switch to xqlint for syntax highlighting of query results - #147
xi. With configuration.xml set to guest=no, login is impossible - #164
xii. Resource operations using “name” instead of “key” are prone to fail #185
xiii. xqlint error causes “run test” to silently fail - #224
b. Dashboard
i. Logging in shouldn’t require duplicate entry of credentials - #156
c. Documentation
i. Detail access details methods for JMX info over HTTP - #106
ii. Update the URL Rewriting article - #200
iii. Document the “exist-serialization” processor instruction - #265
iv. Update the XQuery article to note unsupported serialization parameters - #436
v. Document the “lowercase-expanded-terms” option - #447
6. Implement the following improvements to TEI Publisher:
a. Upgrade EPUB generation facility from EPUB 2 to EPUB 3—specifically for its support of “pop-over footnotes.” The current EPUB 2-based link approach to footnotes is jarring and clunky.
b. Extend TEI Publisher to support the annotation interface described in “Implement browser-based annotation and editing of TEI XML documents” above.

&1#&"Times New Roman"&10&K000000SENSITIVE BUT UNCLASSIFIED https://history.state.gov/search/tipshttps://github.com/eXist-db/exist/issues/1715https://github.com/eXist-db/exist/issues/3237https://github.com/eXist-db/exist/issues/3634https://github.com/eXist-db/exist/issues/1456https://github.com/eXist-db/exist/issues/1482https://github.com/eXist-db/exist/issues/1562https://github.com/eXist-db/exist/issues/1655https://github.com/eXist-db/exist/issues/1824https://github.com/eXist-db/exist/issues/1892https://github.com/eXist-db/exist/issues/1917https://history.state.gov/departmenthistory/timelinehttps://github.com/eXist-db/exist/issues/2102https://github.com/eXist-db/exist/issues/2381https://github.com/eXist-db/exist/issues/3207https://github.com/eXist-db/exist/issues/3390https://github.com/eXist-db/exist/issues/3441https://github.com/eXist-db/exist/issues/3448https://github.com/eXist-db/exist/issues/3449https://github.com/eXist-db/exist/issues/3472https://github.com/eXist-db/exist/issues/3474https://github.com/eXist-db/exist/issues/3505https://history.state.gov/historicaldocumentshttps://github.com/eXist-db/exist/issues/3620https://github.com/eXist-db/exist/issues/3624https://github.com/eXist-db/exist/issues/3637https://github.com/eXist-db/exist/issues/1379https://github.com/eXist-db/exist/issues/1848https://github.com/eXist-db/exist/issues/2394https://github.com/eXist-db/exist/issues/3241https://github.com/eXist-db/exist/issues/3632https://github.com/eXist-db/exist/issues/3633https://github.com/eXist-db/exist/issues/1160https://github.com/HistoryAtState/hsg-shell/issues/63https://github.com/eXist-db/eXide/issues/104https://github.com/eXist-db/eXide/issues/105https://github.com/eXist-db/eXide/issues/109https://github.com/eXist-db/eXide/issues/111https://github.com/eXist-db/eXide/issues/112https://github.com/eXist-db/eXide/issues/113https://github.com/eXist-db/eXide/issues/114https://github.com/eXist-db/eXide/issues/117https://github.com/eXist-db/eXide/issues/126https://github.com/eXist-db/eXide/issues/147https://github.com/HistoryAtState/hsg-shell/issues/155https://github.com/eXist-db/eXide/issues/164https://github.com/eXist-db/eXide/issues/185https://github.com/eXist-db/eXide/issues/224https://github.com/eXist-db/dashboard/issues/156https://github.com/eXist-db/documentation/issues/106https://github.com/eXist-db/documentation/issues/200https://github.com/eXist-db/documentation/issues/265https://github.com/eXist-db/documentation/issues/436https://github.com/eXist-db/documentation/issues/447https://iiif.io/https://github.com/HistoryAtState/hsg-shell/issues/203https://github.com/eXist-db/exist/issues/3635https://github.com/eXist-db/exist/issues/1483

File details come from the government source that posted it. Updated .