Indexing of the Biomedical Literature SOW.pdf

INDEXING OF THE BIOMEDICAL LITERATURE

STATEMENT OF WORK

BACKGROUND AND PURPOSE

INDEXING PROCESS

Subject Indexing Using MeSH Terms

Indexing Publication Types (PTs)

Indexing System Online Validations

Chemical Query Flags

Gene Linking

MTIFL COMPLETION

MTIFL Completion Scope and Production Expectations

MTIFL Completion Training

MTIFL Completion Journal Selection and Distribution

The Process of MTIFL Completion

MTIFL Completion Performance Standards

CONTRACT INDEXER RECRUITMENT AND TRAINING

Contract Indexer Qualifications and Recruitment

Training of Indexing Trainers and Contract Indexers

Expected Proficiency and Production Progression for Contract Indexers

Refresher Training Sessions and Training Updates

INDEXING QUALITY CONTROL

Sendbacks for Significant Indexing Errors

Gene Link Sendbacks

Procedures for Sendbacks

Continued Poor Performance

Definition of Significant Indexing Errors

Significant Errors for MTIFL Completion

JOURNAL ASSIGNMENTS

REPORTING AND OTHER REQUIREMENTS

Statistics Reporting for Completed Work

E-Mail Requirements and Government-Required Training

INDEXING OF RECORDS FOR THE HSRProj DATABASE

MATERIALS AND EQUIPMENT

Contractor Responsibility for Materials and Equipment

Government Responsibility for Materials and Equipment

CONTINUITY OF OPERATIONS

ATTACHMENT A – EXAMPLE OF MEDLINE/PUBMED RECORD WITH MESH TERMS

ATTACHMENT B – SUBJECT AREAS OF MEDLINE JOURNALS TO BE INDEXED

ATTACHMENT C – INDEXING SYSTEM VIEW OF ARTICLE TITLE AND ABSTRACT

ATTACHMENT D – INDEXING SYSTEM TABS

ATTACHMENT E – INDEXING SYSTEM VIEWS OF TERM LOOK UP, SUBHEADING ATTACHMENT, AND

POTENTIAL CHECK TAGS AND PUBLICATION TYPES (PTs)

ATTACHMENT F – EXAMPLE OF SUPPLEMENTARY CONCEPT RECORD (SCR) FROM MESH AND

CHEMICAL FLAG FORM FOR CREATING A NEW SCR

ATTACHMENT G – MTIFL CITATION ABSTRACT DISPLAY

ATTACHMENT H – EXAMPLE OF HSRProj RECORD

GLOSSARY

BACKGROUND AND PURPOSE

The Index Section of the National Library of Medicine (NLM) is responsible for indexing biomedical and life sciences journal articles for MEDLINE, the primary component of PubMed, NLM’s biomedical research database. PubMed is a publicly accessible resource created and maintained by the National Center for Biotechnology Information (NCBI). NCBI is part of NLM, one of 27 Institutes and Centers of the National Institutes of Health (NIH) in Bethesda, Maryland.

PubMed comprises over 29 million bibliographic citations and abstracts covering primarily biomedical research, as well as life sciences, behavioral sciences, chemical sciences, and bioengineering. The MEDLINE component of PubMed comprises over 26 million citations from journals selected for inclusion in the database by an NLM/NIH advisory committee, the Literature Selection Technical Review Committee (LSTRC), based on scientific merit and other criteria. Currently, approximately 1 million new citations are added to MEDLINE/Pubmed annually.

The main task of this contract is to apply descriptive terms selected from NLM’s controlled vocabulary of Medical Subject Headings (MeSH) to journal article citations in MEDLINE to make it easier for users to find relevant information when searching PubMed. This process is known as MEDLINE. Indexed MeSH terms can be seen on a sample MEDLINE/PubMed record in Attachment A.

This contract also includes tasks that involve linking MEDLINE records to NCBI’s Gene database and providing indexing for the Health Services Research Projects in Progress (HSRProj) database.

The scope of journals indexed for MEDLINE is broad and covers biomedicine, as well as the life sciences. In addition, biomedical articles may be selected for indexing from a set of physics and chemistry journals. See Attachment B for a list of MEDLINE journal subject areas.

MEDLINE indexing is to be performed within NLM’s indexing system via two distinct approaches:

Regular Indexing and MTIFL (Medical Text Indexer First Line) Completion.

Regular Indexing involves analysis of a journal article’s contents (mostly title, abstract, purpose and conclusion statements) and primarily the application of the following to the article’s citation to correctly reflect the article’s contents:

• Publication types (PTs): MeSH terms describing the type of article, such as EDITORIAL or LETTER

• Check tags: MeSH terms describing study population characteristics, such as age, gender, or species

• MeSH terms describing the main topics of the article (known as IM (Index Medicus) terms)

• MeSH terms describing secondary topics of the article, as needed (known as NIM (non-IM) terms)

• Qualifiers or MeSH subheadings further describing the MeSH terms, such as “drug effects”

• Chemical flags: indicators for chemicals found in the article but not in the controlled vocabulary https://www.nlm.nih.gov/bsd/indexhome.html https://www.nlm.nih.gov/ https://www.nlm.nih.gov/bsd/medline.html https://www.nlm.nih.gov/bsd/medline.html https://www.ncbi.nlm.nih.gov/pubmed/ https://www.ncbi.nlm.nih.gov/pubmed/ https://www.ncbi.nlm.nih.gov/ https://www.ncbi.nlm.nih.gov/ https://www.ncbi.nlm.nih.gov/ https://www.ncbi.nlm.nih.gov/ https://www.nih.gov/ https://www.nih.gov/ https://www.nlm.nih.gov/mesh/meshhome.html https://www.nlm.nih.gov/mesh/meshhome.html https://www.ncbi.nlm.nih.gov/gene https://www.ncbi.nlm.nih.gov/gene https://hsrproject.nlm.nih.gov/

MTIFL Completion involves review and completion of MeSH terms applied to an article’s citation by the Medical Text Indexer (MTI), NLM’s automated indexing system. More specifically, MTIFL indexing partially automates the indexing process by providing the initial indexing for a citation, which a human indexer then reviews and modifies (completes) as needed by adding any missed terms, removing any incorrect terms, and supplying PTs.

The following four tasks are required as part of this contract, with each task to be priced separately:

• Task 1 – Regular Indexing (as described above): This task is typically performed at the rate of 4 citations indexed per hour.

• Task 2 – MTIFL Completion (as described above): This task is typically performed at the rate of 8 citations (pre-indexed by MTI) completed per hour.

• Task 3 – Gene Linking: This task involves the creation of a link between a record in NCBI’s Gene database and the MEDLINE citation that references a gene or protein. Each gene link is typically completed in up to 5 minutes.

• Task 4 – HSRProj Indexing: This task involves the application of MeSH terms to records in the HSRProj database. HSRProj indexing is typically performed at the rate of 6 records indexed per hour.

Tasks 1, 2, and 3 above are performed in the indexing system, which provides a view of each article’s title and abstract and access to the article’s full text (see Attachment C). HSRProj indexing is done in a separate system. Ninety-six percent of articles indexed for MEDLINE (and potentially linked to Gene) are in English; those not in English almost always have an English title and abstract. All HSRProj records are in English.

Timeliness is essential for this project. Work should be completed within one week of assignment to the Contractor’s staff (contract indexers).

Quality is essential for this project and will be assessed through review and periodic sampling of work performed by the contract indexers.

Contract indexers will follow policies and procedures documented in the NLM Indexing Manual (found in the NLM wiki, which is not publicly accessible), the MeSH Browser (a searchable online thesaurus of MeSH terms), and other NLM resources. Indexing is performed using a system developed at NLM. The current system is web-based and accessible 24 hours a day, every day. Contract indexers must meet NIH security requirements and will use an NIH ID badge (Personal Identity Verification, or PIV, card) to access the indexing system.

https://ii.nlm.nih.gov/MTI/ https://meshb.nlm.nih.gov/search

The anticipated duration of this contract is one base year plus four option years, for a total of five years. For the base year, NLM seeks to obtain:

• Regular indexing of up to 690,000 journal articles

• MTIFL completion of up to 100,000 journal articles

• Creation of up to 70,000 gene links

• Indexing of up to 2,100 HSRProj records

The number of articles to be indexed is subject to change as NLM adds automated indexing capabilities.

The Contractor must furnish services, qualified personnel (see Contract Indexer Recruitment), materials, and facilities not otherwise provided by the Government under the terms of this contract, as needed to perform the work set forth below.

It is possible that during the duration of this contract the indexing system or other NLM systems may be modified or replaced. Contract indexers are expected to learn to use any new or modified system required to continue indexing for MEDLINE and to adapt to changes in MeSH (including replacement by other vocabularies or addition of new vocabularies) as well as NLM indexing procedures, policies, and philosophy.

INDEXING PROCESS

Articles in biomedical and life sciences journals will be indexed according to procedures and policies detailed in established NLM guidelines, including the Indexing Manual (in the NLM wiki), the Medical Subject Headings (MeSH), and other indexing aids such as the Technical Memoranda and Technical Notes (also in the NLM wiki). These guidelines are updated over time at NLM’s discretion.

Indexing for MEDLINE is performed within NLM’s indexing system. Citations are supplied electronically by publishers (in XML format).

The indexing system contains automated tools for indexer assistance. Such tools include integrated links to the MeSH Browser and to PubMed. For instance, in the indexing system’s Related tab (see Attachment D) there are links to citations in PubMed that provide MeSH terms from related articles that have already been indexed. In addition, the Related tab contains a link to the MTI interface, which offers the indexer suggested MeSH terms based on MTI algorithms.

Each contract indexer is expected to be fully trained by the Contractor. Indexing quality will be reviewed by NLM staff on a regular basis in accordance with established performance standards.

https://www.ncbi.nlm.nih.gov/pubmed/ https://www.ncbi.nlm.nih.gov/pubmed/

Subject Indexing Using MeSH Terms

The major effort in indexing is the intellectual analysis of an article’s contents (identifying what the article is about) and the application of appropriate MeSH terms, qualifiers (subheadings), check tags, and publication types (PTs) drawn from the MeSH controlled vocabulary.

Subject indexing requires constant consultation of the MeSH database to identify the appropriate MeSH terms, to read their scope notes, and to verify the annotated rules governing their application.

NLM will determine contract indexers’ accuracy in the application of MeSH terms by the following criteria:

• Accurate coverage of the main topics discussed in an article to reflect what the article is about

• Compliance with authorities and indexing tools

• Correct application of IM and NIM terms (as defined above)

• Correct application of qualifiers (subheadings) and check tags

IM terms are MeSH terms that have been designated by the indexer in an article’s citation within the indexing system as describing the main topics of the article. Generally, no more than three to five IM terms are applied to an article. MeSH terms that identify secondary topics discussed in an article are referred to as NIM (non-IM) terms. Most journal articles that are indexed require an average of five to eight MeSH terms (exclusive of check tags and qualifiers). This number varies with the length of the article and the complexity of the subject matter. Indexing system views relating to the addition of subject terms, subheadings, check tags, and publications types (PTs) are shown in Attachment E.

In order to maximize productivity and efficiency, indexing policy dictates different requirements for different types of articles and journals. While research articles typically require “in-depth” indexing, editorials, letters, and news items, for instance, require only “non-depth” indexing. This type of indexing involves the application of fewer MeSH terms, typically only three or four terms (exclusive of check tags and qualifiers), to cover the main point(s) of the article. Non-depth indexing thus requires less effort than regular or in-depth indexing. Similarly, certain journals are indexed via MTIFL Completion; such journals require a different level of indexing effort and are discussed in detail below (see MTIFL COMPLETION).

Indexing Publication Types (PTs)

Publication types (PTs) characterize the nature or format of a journal article rather than its subject content. Examples of PTs are LETTER, EDITORIAL, and SYSTEMATIC REVIEW. The specific policies and procedures relating to indexing PTs are detailed in the Indexing Manual found in the NLM wiki.

Contract indexers must correctly follow all policies and procedures relating to PTs. At least one PT must be applied to each article. For about 90% of all articles, JOURNAL ARTICLE is the appropriate PT and is automatically added by the indexing system when no other PT is present. PTs may be added during the data review process or supplied by the publisher, that is, assigned prior to indexing. The indexer must always determine whether the correct PT(s) is/are present in an article citation and either modify the PT(s) present or add any missing PT(s). The indexing system view relating to the addition of PTs is shown in Attachment E.

Indexing System Online Validations

The indexing system contains many validation programs to detect real or potential errors in the data elements of a citation. The validation programs generate ERROR and WARNING messages. An ERROR message indicates the presence of an error in the citation that requires correction. A WARNING message indicates the presence of a potential error in the citation, which may or may not require correction.

ERROR message example:

If the MeSH term CHILD ABUSE is indexed without check tags, the indexing system will issue an error message that states, “CHILD ABUSE requires one of the following check tags: ADOLESCENT OR CHILD

OR CHILD, PRESCHOOL OR INFANT OR INFANT, NEWBORN.”

WARNING message example:

NO CHECK TAG PRESENT

If check tag terms are mentioned in the article, contract indexers should add them, but if articles discussing microorganisms or plants are indexed, check tag terms are not applied.

Contract indexers must respond appropriately to all validation messages. If an ERROR validation message appears on the indexing system’s indexing panel, the indexer will not be able to release the journal until the error is corrected.

Chemical Query Flags

When the subject content of an article requires the indexing of a chemical term for which there is no appropriate MeSH term or Supplementary Concept Record (SCR) term, the contract indexer must create a chemical query flag using the indexing system’s electronic form (see Attachment F). The policies and data requirements to create a chemical query flag are described in the NLM Indexing Manual. If the indexing of a non-English article requires a chemical query flag, the indexer must provide (in a special comment box) a brief translation of any descriptive information about the chemical if it is not provided in an English abstract. Chemical query flags are routed to chemical specialists for further processing.

Gene Linking

As part of the subject analysis and indexing of each article for MEDLINE, the contract indexer must recognize if a gene or protein is the main point of the article, and, if so, create a gene link for the gene or protein. Gene linking is performed by creating a link to the Gene database, which is done from the indexing system’s Gene tab (see Attachment D). The indexer must determine if the organism for the gene or protein is represented in the Gene database; if not, a gene link will not be created. If the organism is represented in the Gene database, then the indexer must search for the specific Gene record for the gene or protein. When the record is identified, the indexer must create a link between that record and the citation in the indexing system. If no record for the gene or protein is identified in the Gene database, the indexer must create a link to a “New Entry” record for the organism in the Gene database.

Gene linking must be performed by all contract indexers and for all journal articles indexed for MEDLINE for which a gene or protein discussed in the article meets the criteria for gene linking.

A detailed discussion of the process and policies related to gene linking is found in the Indexing Manual and Technical Memoranda (in the NLM wiki).

MTIFL COMPLETION

MTIFL Completion is a process whereby MTI-generated (automated) indexing is reviewed and completed by the contract indexer. Its purpose is to increase Index Section efficiencies and increase indexing capacity to keep pace with publishing trends in the biomedical literature. MTIFL citations have a special abstract display that identifies the derivation of indexed terms and that highlights potential PTs (see Attachment G).

Contract indexers will perform MTIFL Completion for MTIFL-designated journals, resulting in the correct application of MeSH terms to each article citation, as specified in the Indexing Manual, scope notes, and annotations, to reflect what the article is about. The number and proportion of MTIFL Completion citations to be processed by indexers may vary over the lifetime of this contract.

MTIFL Completion Scope and Production Expectations

As already stated above, NLM anticipates that up to 100,000 articles from MTIFL-designated journals will require MTIFL Completion during the base year of the contract. NLM estimates an indexing rate of 8 articles per hour for MTIFL Completion performed by a trained contract indexer.

MTIFL Completion Training

A minimal amount of additional training will be required to introduce contract indexers to the indexing system’s format for MTIFL Completion. NLM will provide the Contractor with MTIFL Completion training materials.

MTIFL Completion Journal Selection and Distribution

Journals selected for MTIFL Completion are carefully chosen by NLM staff based on proven MTI performance and journal content and have informative titles and abstracts. Journal selection criteria also include little, if any, need for gene links or chemical flags.

MTIFL Completion journals will be distributed to contract indexers in the same manner as regularly indexed journals, based on indexer assignment quotas agreed upon between NLM and the Contractor.

Within the indexing system, MTIFL Completion journals will be clearly identified in an indexer’s assignment queue.

The Process of MTIFL Completion

Indexing of MTIFL Completion journals requires evaluation of the articles’ titles, abstracts, and statements of purpose. Study population-related check tags may be derived from other article sections, such as the Materials and Methods or Results section.

MTIFL Completion involves:

• Verification of MTI-applied MeSH terms

• Verification of MTI-applied MeSH subheadings

• Deletion of any incorrect MeSH terms

• Designation of IM terms

• Addition of any needed PT(s)

When necessary, MTIFL Completion may involve:

• Creation of needed gene links

• Addition of needed chemical flags

• Addition of any other required electronic flags (e.g., Comment)

MTIFL Completion Performance Standards

The final indexing produced by MTIFL Completion must meet established standards for MEDLINE indexing and must conform to the rules and instructions detailed in the Indexing Manual and all supplemental indexing aids, such as the Technical Memoranda and MeSH Browser.

CONTRACT INDEXER RECRUITMENT AND TRAINING

It is the Contractor’s responsibility to meet the awarded production levels and quality standards by maintaining a sufficient trained workforce. Contractors must supply a plan showing how they will ensure the indexing capacity needed to meet production goals.

Contract Indexer Qualifications and Recruitment

The minimum educational requirement for a MEDLINE indexer (Labor Category: Technical Information Specialist) is a Bachelor’s degree with a major in the life sciences or related subject areas (e.g., behavioral sciences, chemistry). A Master’s degree or Ph.D. in the same subject areas and/or prior indexing experience in the biomedical field are preferred. Foreign language capability should also be considered if NLM indicates there is a need.

NLM needs to maintain a wide variety of subject expertise and at times may ask Contractors to fill a need for a particular subject area.

Training of Indexing Trainers and Contract Indexers

NLM will provide sufficient training for designated Contractor staff (trainers) to be able to train any untrained indexers at the beginning of the contract and throughout its duration. NLM will provide the Contractor with any needed training materials. The Contractor will identify which of its staff members will be trained as indexing trainers.

NLM recommends that a contract indexer’s training consist of a trainee’s independent study using the NLM Online Indexing Training Module, which is a web-based, interactive indexing tutorial. In addition to the interactive training module, the trainee may respond to supplemental questions and interact regularly with the trainers, which will permit the trainee to discuss the material and demonstrate his/her understanding of the material covered.

Expected Proficiency and Production Progression for Contract Indexers

A contract indexer is expected to progress through the following stages of proficiency and production.

The indicated time periods and production levels are those expected under normal circumstances. The acceptable error rate is the maximum allowable percentage of indexed citations containing one or more significant errors. For an explanation of sendbacks, please refer to the INDEXING QUALITY CONTROL section below.

http://www.nlm.nih.gov/bsd/indexing/index.html http://www.nlm.nih.gov/bsd/indexing/index.html

Proficiency Level

Time Period

Acceptable Error Rate

Expected Production

Level

Sendbacks Assigned?

Trainee Up to 6 months after initial training end date

Errors expected initially, but should decrease steadily to no more than 10%

40-60 articles per week

No

Revised Between 6 and 12 months after initial training end date

Occasional errors expected, but should decrease to no more than 5%

60-80 articles per week

Yes

Scanned Between 12 and 18 months after initial training end date

No more than 5%

More than 80 articles per week, increasing to full production quota

Yes

Unrevised No later than 18 months after initial training end date

No more than 5%

Full production quota;

minimum expected indexing rate is 4 articles per hour (8 articles per hour for

MTIFL)

Yes

Refresher Training Sessions and Training Updates

NLM will provide training updates and refresher sessions for the Contractor’s indexing trainers, as needed, to update them regarding changes to indexing policies and procedures or to maintain overall indexing quality.

Furthermore, all contract indexers are required to attend an annual Indexing Orientation, which introduces the annual release of new MeSH vocabulary and new indexing policies for the coming indexing year and provides an opportunity to discuss questions about indexing. Held at NLM, Orientation may be viewed remotely, although in-person attendance is encouraged.

INDEXING QUALITY CONTROL

Upon receipt of indexed citations from the Contractor, NLM may perform a quality control review of an indexing sample. Indexing submitted by the Contractor must be of acceptable quality (5% or less error rate per contract indexer).

Sendbacks for Significant Indexing Errors

NLM issues sendbacks for indexed citations with significant errors. Definitions and examples of significant errors are discussed below (see Definition of Significant Indexing Errors). Contractors will be notified and will not be paid for items for which sendbacks have been issued.

Gene Link Sendbacks

If a gene link is incorrect because a contract indexer created a link to the incorrect record in the Gene database, the Contractor will receive a gene link sendback and will not be credited for the incorrect gene link the indexer created for that gene or protein (the Contractor will still be credited for the indexing of the MEDLINE citation, provided that the indexing does not contain significant errors).

An example of a significant error in gene linking would be the following: an article explicitly states that a protein is found in fruit flies, but the indexer creates a link to a Gene record for a protein that is found in humans.

The procedures for gene link sendbacks are the same as those used for indexing sendbacks (i.e., NLM staff will provide documentation of any incorrect gene links to the Contractor).

Procedures for Sendbacks

NLM staff may review any or all indexed citations submitted by the Contractor. For each sendback (due to one or more significant errors), NLM staff will notify the Contractor with a written report. The report will include the reason for the sendback (IM indexing, MTIFL indexing, or gene linking error), the name and ID of the indexer who made the error(s), a description of the error(s), the journal citation, and the indexing system record of the citation.

Continued Poor Performance

The Project Officer will bring to the attention of the Contractor any unacceptable work samples (over 5% of citations indexed have significant errors) resulting in multiple sendbacks and repeated deficiencies in completing weekly assignments.

Definition of Significant Indexing Errors

1. Missed Main MeSH Term Error

IM subject heading or headings applied do not reflect the content of an article, or no subject heading has been applied to cover the basic subject content of an article.

EXAMPLE:

The title of an article is "A clinical and epidemiological study of 245 postmenopausal metrorrhagia patients", and the indexer fails to apply the MeSH term METRORRHAGIA.

2. MeSH Term Specificity Error

The indexer has not properly used the MeSH trees to locate a required general term or one or more specific MeSH terms available for an article.

The title of an article is "Age-related characteristics of the laryngeal cartilage in children in the first year of life," and the indexer applies the MeSH terms LARYNX and CARTILAGE rather than the specific MeSH term LARYNGEAL CARTILAGE.

The title of an article is "The influence of citrate on the duodenal absorption of calcium in the rat," and the indexer applies the MeSH term ABSORPTION rather than the more specific MeSH term INTESTINAL

ABSORPTION.

3. Not Following MeSH Annotation or Scope Note Error

An indexing error that could have been avoided had the indexer followed the explicit instructions in a MeSH annotation or the information within a MeSH scope note (definition) available in the MeSH Browser record.

The title of an article is "Mineral and matrix contributions to rigidity in fracture healing", and the indexer assigns the MeSH term WOUND HEALING even though the annotation for this term states "also available is FRACTURE HEALING."

4. Missed Indexing System Warning Error

An error in subject indexing that should have been corrected because of a WARNING message generated by the indexing system.

An indexer indexes the MeSH term ILIUM for an article involving the ileum and ignores the following machine-generated WARNING message:

5. Missed Check Tag Error

The indexer misses a required check tag.

The "Materials and Methods" section of an article states that the study was done with "44-year-old men," and the indexer fails to apply the check tag ADULT.

6. Missed Publication Type (PT) Error

The indexer fails to index a required publication type.

The text of an article states that it is reporting the results of a "randomized controlled trial," but the indexer fails to apply the publication type RANDOMIZED CONTROLLED TRIAL.

Significant Errors for MTIFL Completion

See the section above (Definition of Significant Indexing Errors) for examples and explanations of the following significant errors that will be applied to MTIFL Completion:

1. Missed Main MeSH Term Error

2. Not Following MeSH Annotation or Scope Note Error

3. Missed Indexing Warning Error

4. Missed Check Tag Error

5. Missed Publication Type (PT) Error

JOURNAL ASSIGNMENTS

Journals to be indexed will be assigned to contract indexers in accordance with weekly quotas established at the beginning of each contract year. Assignments will be based on NLM needs, subject and language capabilities provided by the Contractor, and the type and number of journals available at

NLM.

REPORTING AND OTHER REQUIREMENTS

Statistics Reporting for Completed Work

The indexing system contains a table for each indexer that lists all journal issues completed by that indexer. This log may be accessed by the indexer as a record of what he/she has produced and includes the following for each journal issue indexed: date of release; MRI (unique identifier); bibliographic information (bibstring); number of articles indexed; number of gene links created; and number of chemical flags assigned.

NLM will use data from this table to generate automated weekly summary reports of all regular indexing, MTIFL completion, and gene linking tasks completed by contract indexers. These reports will be forwarded to the Contractor(s) for production tracking and invoicing purposes.

Email Requirements and Government-Required Training

As part of their NLM network access, contract indexers will receive individual NLM email accounts. It is required that all contract indexers read their emails each scheduled work day. HHS, NIH, and NLM staff routinely communicate by email, and contract indexers are expected to respond appropriately. NIH and HHS issue occasional government-wide announcements, training requirements, or news items, which may include mandatory online training (e.g., related to information systems security) required for NLM network access. Contract indexers are required to complete all such mandatory training requirements in the time frame indicated by the email or by NLM staff, or risk losing access to the NLM network and the indexing system.

INDEXING OF RECORDS FOR THE HSRProj DATABASE

Indexing is sought for up to 2,100 records per year for NLM’s HSRProj database. The HSRProj database is maintained by NLM’s National Information Center on Health Services Research and Health Care Technology (NICHSR) and contains records of ongoing health services research projects.

https://www.nlm.nih.gov/hsrph.html https://www.nlm.nih.gov/hsrph.html https://www.nlm.nih.gov/hsrph.html https://www.nlm.nih.gov/hsrph.html

The records of health services research projects (which will be created outside the scope of this contract) will include a descriptive project title and usually an abstract. An example of an HSRProj record is shown in Attachment H.

NLM prefers that a single contract indexer be assigned to this task. The HSRProj indexer should have experience in indexing subject matter in the health services research area. The indexer working under this contract will use an HSRProj variant of the indexing system. The estimated production rate for indexing of HSRProj records is approximately 6 records per hour and the performance standards for indexing of HSRProj records will be the same as for subject indexing of MEDLINE citations.

The HSRProj indexer will use the title, abstract (if present), and keywords (if present) to apply appropriate MeSH terms, following the indexing rules and policies that are used for MEDLINE journals.

Check tags, IM indicators, and subheadings will be required. HSRProj records will be indexed with as many terms as are necessary to fully describe each concept that is in the project title, indicated by keywords, and discussed in the abstract.

Whenever possible, the HSRProj indexer will add at least one geographic MeSH term to each HSRProj record. If the geographic term to be added is not obvious from the title or abstract, the HSRProj indexer must refer to the Population Base (PBA) field that may be present in the HSRProj record and apply the corresponding MeSH term.

Whenever possible, the HSRProj indexer will also apply a MeSH term to cover any aspect(s) of the project methodology that is mentioned in the record (e.g., MULTIVARIATE ANALYSIS, SAMPLING

STUDIES, CROSS-SECTIONAL STUDIES).

During the performance period of this contract, the indexing of HSRProj records may be modified to incorporate automated assistance (such as MTI or MTIFL).

NLM will provide the indexer with access to the indexing system needed to perform HSRProj indexing.

HSRProj records will be supplied to the HSRProj indexer in electronic format, as they become available, usually on a quarterly basis (four times per year). Upon completion of the indexing of a batch of HSRProj records, the indexer will notify the project officer in writing and will email production statistics to nlmindexstats@mail.nih.gov or as instructed by NLM staff.

MATERIALS AND EQUIPMENT

Contractor Responsibility for Materials and Equipment

The Contractor is responsible for ensuring that contract indexers have access to high-speed Internet connectivity (i.e., cable, DSL, or other high-speed communications format supporting Internet connectivity).

mailto:nlmindexstats@mail.nih.gov mailto:nlmindexstats@mail.nih.gov

Government Responsibility for Materials and Equipment

NLM will supply contract indexers with all needed computer equipment and with access to the NLM network, the indexing system, the Indexing Manual, the MeSH Browser, and other indexing aids. NLM will also supply any needed URLs and passwords, such as those needed to access Web-based journals.

The indexing system should be available seven days a week, 24 hours a day, except for an annual downtime of 1-2 days due to Year-End Processing (YEP). Additionally, there may be rare instances of downtime as a result of system or hardware problems or upgrades.

Important Note: Contract indexers will be required to obtain ID badges (PIV cards) in order to access NIH/NLM networks and systems (particularly the indexing system, NLM wiki, and other indexing resources used for MEDLINE indexing). Contract indexers must reside in the U.S. in order to be issued and use government-furnished equipment.

CONTINUITY OF OPERATIONS

If a Continuity of Operations (COOP)/Emergency event is declared, the Contractor is responsible for contacting and dismissing its staff. All work by the Contractor will cease unless otherwise notified by a Contracting Officer (CO) or a designated Contracting Officer’s Representative (COR). Contractors are expected to tune in to the local media or access the NLM website for additional information.

During the COOP event, FAR Clause 52.242-15, Stop Work Order, is automatically implemented. As stated in Clause 52.242-15, Stop Work Order, the Contractor must take all reasonable steps to minimize the incurrence of costs allocable to the work covered by the order during the period of work stoppage. Claims filed by the Contractor will be reviewed and approved on a case by case basis.

Contractors must provide and keep updated, in accordance with the Statement of Work, a list of one key individual, and, if possible, at least two backup individuals, to be contacted by an assigned CO/COR during a COOP event.

During an extended COOP event, if needed, Contractors will be contacted in reference to renegotiating the minimum and maximum quantities and/or the period of performance previously negotiated during the term of the contract. The Government reserves its rights under the FAR 52.254-4 - Changes - to implement a unilateral change.

ATTACHMENT A – EXAMPLE OF MEDLINE/PUBMED RECORD WITH MESH TERMS

ATTACHMENT B – SUBJECT AREAS OF MEDLINE JOURNALS TO BE INDEXED

• Allergy and Immunology – includes Hypersensitivity; Lymphology; Serology; Serotherapy

• Anesthesiology – includes Resuscitation

• Anthropology

• Biochemistry – includes Biochemical Techniques; Enzymes; Lipids; Nucleic Acids; Proteins

• Biomedical Engineering – includes Biocompatible Materials; Biomechanics; Prostheses

• Cancer

• Cardiology and Vascular Diseases

• Communicable Diseases

• Complementary Therapies – includes Acupuncture; Medicine, Traditional; Phytotherapy; Spiritual

Therapy

• Computational Biology – includes Bioinformatics; Systems Biology

• Critical Care – includes Intensive Care

• Emergency Medicine / Disaster Medicine; Military Medicine; Traumatology

• Environmental Health – includes Ecology; Pollution; Radiation; Radioactivity

• Forensic Sciences – includes Forensic Medicine; Forensic Psychiatry

• Health Services Research – includes Health Services Planning, Evaluation, and Assessment

• History of Medicine – includes History of Science

• Microbiology – includes Mycology

• Nutritional Sciences – includes Animal Nutrition; Food; Vitamins

• Orthopedics – includes Bone Diseases; Orthopedic Surgery

• Pathology – includes Pathologic Anatomy; Histopathology

• Physical and Rehabilitation Medicine – includes Occupational Therapy

• Public Health – includes Community Health; Preventive Medicine; Public Health Policy; Hygiene

• Science – includes Medical and Life Sciences (most journals in these subject areas are selectively indexed)

• Toxicology – includes Pesticides; Poisoning; Xenobiotics

• Transplantation – includes Artificial Organs

• Traumatology – includes Wounds; Injuries; Accidents

• Veterinary Medicine – includes Dairy Science; Poultry Science

• Women's Health; Gynecology/Obstetrics; Reproductive Medicine

• Zoology – includes Entomology http://www.ncbi.nlm.nih.gov/nlmcatalog?term=Cardiology%5Bst%5D http://www.ncbi.nlm.nih.gov/nlmcatalog?term=Cardiology%5Bst%5D http://www.ncbi.nlm.nih.gov/nlmcatalog?term=Communicable+Diseases%5Bst%5D http://www.ncbi.nlm.nih.gov/nlmcatalog?term=Communicable+Diseases%5Bst%5D

ATTACHMENT C – INDEXING SYSTEM VIEW OF ARTICLE TITLE AND ABSTRACT

ATTACHMENT D – INDEXING SYSTEM TABS

Indexing Tab

Related Tab

Gene Tab

ATTACHMENT E – INDEXING SYSTEM VIEWS OF TERM LOOK UP, SUBHEADING ATTACHMENT, AND

POTENTIAL CHECK TAGS AND PUBLICATION TYPES (PTs)

ATTACHMENT F – EXAMPLE OF SUPPLEMENTARY CONCEPT RECORD (SCR) FROM MESH AND

CHEMICAL FLAG FORM FOR CREATING A NEW SCR

ATTACHMENT G – MTIFL CITATION ABSTRACT DISPLAY

ATTACHMENT H – EXAMPLE OF HSRProj RECORD

GLOSSARY

Check Tag – Special MeSH term that identifies certain required identifiers in an article, such as a patient’s age and sex or experimental organism.

Flags and Notes – Flags and Notes within the indexing system direct journal workflow. These include, for example, Chemical flags, History flags, and Indexing Query Notes. Flags and Notes can be created and viewed in the indexing system on the Index panel and can be accessed via the ArticlesList panel.

Gene Linking – The process by which a MEDLINE citation is linked to a gene or protein in the Gene database.

Indexing System – The online application created and utilized by NLM for MEDLINE indexing.

MeSH – Medical Subject Headings. The NLM controlled vocabulary that is used for indexing.

MeSH Tree – The hierarchical structure of the MeSH vocabulary.

MTI – Medical Text Indexer. Artificial intelligence software created to suggest MeSH terms for MEDLINE indexing based upon an article’s title, abstract, and related records.

MTIFL – MTI First Line Indexing. Automated indexing supplied by MTI.

Publication Type (PT) – A MeSH term that describes the type of article rather than the subject contents; for example, Review [PT].

PubMed – An NLM Entrez literature database that provides access to journal citations in MEDLINE and contains links to full-text articles at participating publishers’ websites.

SCR – Supplemental Concept Record. SCR terms are created for chemicals, diseases, or organisms for which there are no representative MeSH terms.

Sendback – An indexed citation that is rejected because it contains a significant error. Contractors are not paid for sendbacks.