SOW - BIBFRAM - 2025 eACQ 5-6-25.docx

BIBFRAME Scriptshifter Utility Enhancement 3

1 BACKGROUND

The Library of Congress (LC) collects information resources from around the world and makes them known and available to potential users via bibliographic records that describe the resources. The bibliographic descriptions contain typical bibliographic data such as titles, names of creators and other associated entities, publication information such as publisher and place of publication, and other information. This information is presented in the publications in various scripts, including many non-Latin scripts such as Chinese, Korean, Cyrillic, Arabic, Greek, Hebrew, and over 30 additional scripts. The Library of Congress records descriptive information in the original script of the item being cataloged (technology allowing) but needs to transliterate certain data elements of the description into the Latin script for various processing components and in some cases to assist end users. When technology does not support non-Latin scripts, LC staff must manually transliterate into the Latin script more of the descriptive information.

The Library of Congress maintains transliteration tables for over 75 languages and scripts, ALA-LC Romanization Tables: Transliteration Schemes for Non-Roman Scripts and makes them available on its web site: http://www.loc.gov/catdir/cpso/roman.html. These tables are jointly maintained by LC and the American Library Association (ALA). These tables are used by United States libraries and many libraries outside the United States. The Library of Congress catalog contains several million bibliographic records for resources in non-Latin scripts and collects over 75,000 additional non-Latin resources each year. The Library of Congress requires a utility that can transliterate between non-Latin and Latin scripts using the transliteration tables approved by LC and ALA.

A utility, called Scriptshifter, has been developed for transliteration of 20+ scripts in the Balkan/Caucasian, Slavic, Turkic, and Chinese script families, and for Korean, Greek, Arabic, and Hebrew. Scriptshifter needs to be continually enhanced to incorporate additional scripts and improve the tool for very complex scripts like the Arabic, Southeast Asian, and several Asian scripts like Japanese in which the Library receives resources. Continual updating of the software framework to improve efficiency is necessary as the technical possibilities change and the transliteration tables change.

2 SCOPE

The Contractor shall design, code, test, and document the additions to the Scriptshifter transliteration utility capability for non-Latin data into the Latin alphabet according to the ALA-LC Romanization Tables, and where possible the conversion of Latin script transliteration to non-Latin script. The utility will focus on research and improvement of Indic and related languages such as Devanagari and Brahmi scripts, Southeast Asian scripts such as Thai, Laotian, Khmer, Burmese, Tibetan, and Arabic scripts such as Kurdish, Sindhi, Persian, Pushto, Urdu, and Mophah. In addition, research on developing a Japanese transliteration tool and refinement of other Asian scripts such as Korean and Chinese will be done. More specifically, the contractor shall:

· Improve Persian transliteration

· Improve Thai, Roman-to-script only

· Improve some inaccurate South Asian languages handled by current software

· Implement Lao

· Improve Tibetan: fix tables in current software or create a new one managed by SS

· Improve overall testing

· Research existing tools for transliterating Japanese and implement a S2r-only transliteration, if an appropriate tool is identified

· Research a better machine learning model for Arabic scripts, possibly usable with multiple languages

· Improve some inaccurate South Asian languages handled by current software

· Implement a better method to separate words

· Research possible solutions on other Arabic scripts with less available data (Pushto, Urdu, etc.)

The utility must also carry out reverse transliteration, converting Latin transliterated strings into non-Latin strings, where feasible. The utility must remain adjustable as transliterations change.

The contractor will review the software utility as a whole and make general improvements to the framework. One area of focus will be the Aksharamukha tool that has been incorporated for Asian scripts. Work will be done related to authentication and external use of the tool.

3 TASKS/REQUIREMENTS

The Library will supply a general design for the programs and information on how the programs should interface with the Library’s BIBFRAME and other systems.

3.1 Research – The contractor will review the tools available in the community for the new scripts and summarize the pros and cons of the tools in a Report (Word). Special research will be carried out with respect to transliteration of some scripts. Contractor will analyze Scriptshifter for configuration changes.

3.2 Design and Library approval – The Contractor will describe the planned design of the additions to Scriptshifter’s scripts-handled and configuration changes in a Report (Word). The Library will approve the Contractor’s enhancements and any design changes, the expected HTTP API interaction, and programming languages before the Contractor begins to code. The Contractor shall use common, well-tested, software libraries whenever possible. Contractor shall plan enhancements and augmentations to the existing Scriptshifter utility as needed for this work while maintaining backwards compatibility with the existing utility with respect to its HTTP API and use in a web-based environment.

3.3 Scriptshifter enhancement, testing, and Library approval – Contractor will provide the Scriptshifter enhancements according to the agreed design and meeting the following requirement: Python 3.7 (or greater). Contractor shall provide code that is deployable within a MarkLogic or Linux-environment, requiring few, if any, additional system-defined libraries. Contractor shall include automated code tests and provide documentation of the test results. Library will conduct performance tests as the coding is developed.

3.4 Documentation and Library approval – The Contractor shall update Scriptshifter documentation that details how the software is to be installed and configured as well as instructions about how it functions. The Contractor shall use Git as the code repository. The Contractor shall include documentation within the repository. That documentation can take the form of a well-fleshed out README, otherwise a more structured set of documentation files can be placed in a separate directory. Documentation shall be in sufficient detail to enable the Library to adjust the programs as needed. The Library will approve the sufficiency of the documentation.

The contractor shall provide the above deliverables for the following task as follows:

· Improve Persian transliteration

· Improve Thai, Roman-to-script only

· Improve Tibetan: fix tables in current software or create a new one managed by SS

· Improve overall testing

· General maintenance and ongoing technical assistance Base Year

· Implement Lao

· Research a better machine learning model for Arabic scripts, possibly usable with multiple languages

· Research possible solutions on other Arabic scripts with less available data (Pushto, Urdu, etc.)

· Improve overall testing

· General maintenance and ongoing technical assistance Option Year 1

· Improve some inaccurate South Asian languages handled by current software

· Research existing tools for transliterating Japanese and implement a S2r-only transliteration if an appropriate tool is identified

· Improve overall testing

· General maintenance and ongoing technical assistance Option Year 2

· Implement a better method to separate words

· Improve overall testing

· General maintenance and ongoing technical assistance Option Year 3

4.2 Quality Control

The contractor shall develop and maintain an effective quality control program (QCP) to ensure services are performed in accordance with this contract within 2 months after the award of the contract. The contractor’s quality control program is the means by which it assures itself that its work complies with the requirement of the contract. After acceptance of the quality control plan, the contractor shall receive the contracting officer’s acceptance in writing of any proposed change to its QC system.

5 ADDITIONAL INFORMATION

Place of performance. The work under this contract shall be performed at contractor’s facility. Communication with the COR and designated NDMSO staff will be via email, telephone, and Zoom.

Hours of operation. The contractor can consult with library staff between the hours of 9:00 am and 5:30 pm Mondays through Fridays, except Federal holidays.

5.1 TO BE SUPPLIED BY THE LIBRARY

The Library will organize meetings using the Library’s Zoom or Teams accounts, that will assist in the work. The Library will supply the Romanization tables to be used via their availability on the Library’s web site. The Library will supply Contractor with Latin/Non-Latin mappings as needed. The Library will supply sample input files and data as needed by the Contractor. The Library will communicate and make access arrangements to link to 3rd party APIs where appropriate. And the Library will test programs developed by the contractor and review all specification, requirements, use cases, code, documentation, etc. developed by the contractor.

5.2 RIGHTS IN DATA

All work produced by the contractor in the execution of this contract shall be considered the exclusive property of the Library of Congress upon completion and acceptance, including all intellectual property rights, without exception. The contractor shall transfer and assign all work produced under this contract to the Library, as the sole owner with exclusive rights to include, but not limited to maintaining the work.

(End of Performance Work Statement / Statement of Work)