SOW - BIBFRAME.docx

Statement of Work BIBFRAME ScriptShifter Utility Enhancement 2

1. BACKGROUND

The Library of Congress (Library) collects information resources from around the world and ensures their visibility and accessibility to potential users through bibliographic records. These records detail various aspects of the resources, including titles, creators’ names, publication details like publishers and places, and other pertinent information. The Library presents this information in the publications using various scripts, including numerous non-Latin scripts like Chinese, Korean, Cyrillic, Arabic, Greek, Hebrew, and over 30 additional scripts. While the Library records descriptive information in the original script of the materials being cataloged (whenever technology permits), there’s a need to transliterate certain data elements into the Latin script for different processing components and, at times, to assist end users. When technology doesn’t support non-Latin scripts, the Library staff must manually transliterate more of the descriptive information into the Latin script.

The Library maintains transliteration tables for 75 languages and scripts, ALA-LC Romanization Tables: Transliteration Schemes for Non-Roman Scripts, which are accessible on its web site: http://www.loc.gov/catdir/cpso/roman.html. These tables are collaboratively maintained by the Library and the American Library Association (ALA) and are utilized by libraries across the United States and internationally. With several million bibliographic records for resources in non-Latin scripts, the Library adds over 75,000 additional non-Latin resources to its collections annually. To facilitate this process, the Library seeks a utility capable of transliterating the non-Latin scripts into the Latin script using the approved transliteration tables.

A ScriptShifter transliterator is a utility tool or software designed to convert text from one writing system or script to another. It's particularly useful for tasks like transliterating names or phrases from one alphabet to another, such as converting Romanized text to Cyrillic or vice versa. This tool helps maintain the pronunciation or meaning of words across different writing systems. The Library’s ScriptShifter transliterator was designed to transliterate over 20+ scripts in the Balkan/Caucasian, Slavic, Turkic, and Chinese script families, as well as Korean, Greek, Arabic, and Hebrew. There is a need to improve the Library’s ScriptShifter transliterator to include several more scripts languages for which the Library regularly receives substantial resources. Additionally, its design should be analyzed to enhance efficiency and accommodate an increased number of supported scripts languages.

2. SCOPE

The contractor shall provide highly specialized expertise and technical services to update, design, code, test, improve, and document the Library’s ScriptShifter transliterator to include several additional script languages, and its design must undergo review and analysis to improve efficiency and adapt to the increased number of supported scripts. The Library will supply a general design for the programs and information on how the programs should interface with the Library’s BIBFRAME and other systems.

3. TASKS/REQUIREMENTS

3.1. Compatibility: The contractor shall ensure that the ScriptShifter transliterator updates remains compatible with the Library’s existing systems, software, and data formats. Capability for non-Latin data into the Latin alphabet according to the ALA-LC Romanization Tables.

3.2. Language Support: The updated version of the ScriptShifter transliterator shall include additional non-Latin script languages, including Thai, Lao, Khmer, Tibetan, Burmese, Devanagari script family and other languages in the Indian continent, Yiddish, and additional Arabic scripts, as well as, Pushto, Sindhi, Kurdish, Persian, and Urdu. Further, the contractor shall research on developing a Japanese transliteration tool.

3.3. Reverse Transliteration: The updated version of the ScriptShifter transliterator shall carry out reverse transliteration process of converting text that has been transliterated from one script or writing system back into its original script or writing system, while maintaining its pronunciation or meaning. The ScriptShifter transliterator must remain adjustable as transliterations change.

3.4. Review: The contractor shall review the entire transliteration utility and implement general improvements, which including transitioning the current file systems to a database design. Additionally, enhancements to a testing application will be made to facilitate broader utilization by the external community.

3.5. Research:

The contractor shall review the tools available in the community for the new scripts and summarize the pros and cons of the tools in a Report (Word). Special research shall be carried out with respect to transliteration of Japanese. The contractor shall analyze ScriptShifter for configuration changes.

3.6. Design and Library Approval:

The Contractor shall describe the planned design of the additions to ScriptShifter’ s scripts-handled and configuration changes in a Report (Word). The Library must review and provide the Contractor’s with approval for enhancements and any design changes, the expected HTTP API interaction, and programming languages before the Contractor begins to code. The contractor shall use common, well-tested, software libraries whenever possible. The contractor shall plan enhancements and augmentations to the existing ScriptShifter utility as needed for this work while maintaining backwards compatibility with the existing utility with respect to its HTTP API and use in a web-based environment.

3.7. ScriptShifter Enhancement, Testing, Performance Optimization, and Library Approval:

The contractor shall provide the ScriptShifter enhancements according to the agreed design and meeting the following requirement: Python 3.7 (or greater). The contractor shall provide code that is deployable within a MarkLogic or Linux-environment, requiring few, if any, additional system-defined libraries. The contractor shall include automated code tests and provide documentation of the test results. The Library will conduct performance tests as the coding is developed. The contractor shall work with the testers to make adjustments identified in the Library's testing. The contractor shall improve the speed and efficiency of the transliteration process, especially for large volumes of text or real-time usage.

3.8. Documentation and Library Approval:

The contractor shall update ScriptShifter documentation that details how the software is to be installed and configured as well as instructions about how it functions. The contractor shall use Git as the code repository. The contractor shall include documentation within the repository. That documentation can take the form of a well-fleshed out README, otherwise a more structured set of documentation files can be placed in a separate directory. Documentation shall be in sufficient detail to enable the Library to adjust the programs as needed. The Library will approve the sufficiency of the documentation.

3.9. Process to Review and Approve or Reject work for ScriptShifter:

The Library’s criteria for evaluating work, including functionality, performance, usability, security, and compliance with the requirements shall include entail reviewers, such as Library catalogers, who have the necessary expertise and authority in language/script testing to review the contractors work according to the established criteria. The contractor shall remedy any issues, defects, or areas for improvement identified during the cataloger’s evaluations. Reviews shall include, but not be limited to, code reviews, testing, security assessments, usability testing, documentation reviews and so forth.

4. CONTRACT PERFORMANCE

4.1. Deliverables

4.2. Quality Control:

The contractor shall develop and maintain an effective quality control program (QCP) to ensure services are performed in accordance with this contract within 2 months after the award of the contract. The contractor’s quality control program is the means by which it assures itself that its work complies with the requirement of the contract. After acceptance of the quality control plan, the contractor shall receive the contracting officer’s acceptance in writing of any proposed change to its QC system.

5. ADDITIONAL INFORMAITON

5.1. To Be Supplied by the Library:

The Library will organize meetings using the Library’s Zoom or Teams accounts, that will assist in the work. The Library will supply the Romanization tables to be used via their availability on the Library’s web site. The Library will supply Contractor with Latin/Non-Latin mappings as needed. The Library will supply sample input files and data as needed by the Contractor. The Library will communicate and make access arrangements to link to 3rd party APIs where appropriate. And the Library will test programs developed by the contractor and review all specification, requirements, use cases, code, documentation, etc. developed by the contractor.

5.2 . Rights in Data

All work produced by the contractor in the execution of this contract shall be considered the exclusive property of the Library of Congress upon completion and acceptance, including all intellectual property rights, without exception. The contractor shall transfer and assign all work produced under this contract to the Library, as the sole owner with exclusive rights to include, but not limited to maintaining the work.

(End of Performance Work Statement / Statement of Work)