ERIC PWS 10-14-22.docx

Education Resources Information Center (“ERIC”) Performance Work Statement

Purpose

The Institute of Education Sciences (“IES”) in the Department of Education (the “Department”) seeks to contract with a vendor or team of vendors to operate and make continuous improvements to the usefulness and usability of the Education Resources Information Center (ERIC). ERIC is an online bibliographic and full-text digital library of education research, including journal articles and non-journal materials. The role of the ERIC contractor is to generate the metadata for the database that underlies the ERIC website, create documentation about the metadata and website, and to communicate with ERIC users. The ERIC website is developed, maintained, and hosted by separate contracts and are outside of the scope of this requirement.

The mission of ERIC is to provide broad access to education research in a user-friendly, timely, and efficient manner. ERIC provides these services to schools, institutions of higher education, educators (including early childhood educators), parents, administrators, policymakers, researchers, public and private entities (including providers of early childhood services), entities responsible for carrying out technical assistance through the Department, and the general public. ERIC has sought to accomplish this mission through the development and maintenance of a comprehensive, searchable, Internet-based library of bibliographic references and associated full-text, when such text is freely available.

The ERIC database currently contains 1.9 million bibliographic records and more than 385,000 full-text materials. Each ERIC bibliographic record contains an abstract of a journal article or non-journal document (e.g., technical report, conference paper), along with indexed information such as author, title, and publication date.

A government-sponsored website, https://eric.ed.gov, provides free access to the ERIC digital library. ERIC’s metadata are made available for vendors and the public through the Downloads section of the ERIC website: https://eric.ed.gov/?download and through an API: https://eric.ed.gov/?api. Various subscription services and freely available Internet search engines provide access to ERIC by redistributing ERIC metadata.

Acquisition agreements with journal publishers and education membership associations, education organizations, universities, state education agencies, and other sources of non-journal materials provide source materials for the ERIC digital library. All work indexed in ERIC must be in scope of the ERIC Selection Policy, found at https://eric.ed.gov/?selection. The list of approved journals is listed here: https://eric.ed.gov/?journals and the approved non-journal sources are listed here: https://eric.ed.gov/?nonjournals.

The ERIC database contains citations included in the digital library and is updated weekly. Approximately 4,000 new entries are added per month. ERIC indexes over 1,276 journals. In addition, ERIC includes 719 sources of “grey literature” (materials not published by commercial publishers). Individual authors are invited to contribute their full-text documents, abstracts, and bibliographic data to ERIC through an online submission system; these documents are considered for inclusion in the ERIC database if they meet the guidelines set forth in the ERIC Selection Policy.

Over 12 million users search ERIC every year. Currently, users can search for resources using the well-developed ERIC Thesaurus (a controlled vocabulary of descriptors) and a variety of search limiters. Users can refine their search by author, audience, source, education level, publication type, peer review status, and the availability of full text.

The website is hosted in the IES Data Center and maintained by the IES Web Support contract. The ERIC contractor is responsible for creating the records to support the database that underlies the website. The contractor is not responsible for posting records to the website or making changes to the website.

The period of performance for the contemplated contract is 60 months. The contract may start with an optional 3-month transition-in period followed by one 9-month base period of performance or may have a 12-month base period of performance. There will then be four 12-month option years and several optional tasks. Optional subtask 14.1 has a non-severable 24-month period of performance. If that subtask is awarded, the contactor would continue to work on that task even if the core tasks in the option year were not exercised.

Authorizing Legislation

Section 172 (a) 3 of Public Law 107-279, the Education Sciences Reform Act of 2002 (ESRA), which creates IES, requires that the Commissioner for the National Center for Education Evaluation and Regional Assistance “make … information accessible in a user-friendly, timely, and efficient manner (including through the use of a searchable Internet-based online database)… ” Section 172 (b) 1 of the Act also requires the Commissioner to “ensure that information [is] disseminated ... in a cost-effective, non-duplicative manner.”

Section 172 of ESRA specifies that dissemination include information on closing the achievement gap, education practices that improve academic achievement and promote learning, and topics that were covered by the former ERIC clearinghouses: adult, career, and vocational education; community colleges; assessment and evaluation; higher education; teaching and teacher education; information and technology; counseling and student services; educational management; elementary and early childhood education; reading, English, and communication; science, mathematics, and environmental education; social studies/social science education; rural education and small schools; urban education; disabilities and gifted education; and languages and linguistics.

According to Section 172 of ESRA, the information must be accessible “to schools, institutions of higher education, educators (including early childhood educators), parents, administrators, policymakers, researchers, public and private entities (including providers of early childhood services), entities responsible for carrying out technical assistance through the Department, and the general public.”

Scope of Work

ERIC's mission is to select, index, and make quality education research widely available and searchable through a user-friendly website. This supports the mission of IES to provide scientific evidence on which to ground education practice and policy and to share this information in formats that are useful and accessible to educators, parents, policymakers, researchers, and the public. Unless otherwise specified, all days in the PWS are business days, not calendar days.

Task 1: Management and Reporting (Firm-Fixed-Price):

1.1. Kickoff Meeting (Base year only). Within the first 2 business days after the contract start date, up to three individuals, including the director and deputy director, shall attend a kickoff meeting with the Department to discuss details regarding the tasks outlined in the proposal, scheduling activities, the transition process, and other issues related to the contract. The meeting will be held via conference call. The contractor shall provide an agenda for the meeting within 1 business day after the contract start date. Within 5 business days following the meeting, the contractor shall prepare a memo summarizing the key issues and concerns raised at the meeting and how each will be addressed. The memo also shall document key decisions made at the meeting, as well as any specific technical directions provided by the Contracting Officer’s Representative (“COR”).

Deliverables:

· Agenda for kickoff meeting—1 business day after contract award

· Summary of kickoff meeting—5 business days after meeting

1.2. Transition-in Materials (optional, base year only). The contract is intended to be awarded with a transition-in period of up to 90 calendar days that ends on April 23, 2023. This is optional and will be awarded at the government’s discretion.

During the transition period the contractor is responsible for ensuring that staff has adequate security clearances and permissions to access the Education Literature Validation Information System (“ELVIS”) processing application. The ELVIS application is a workflow system housed in the IES Data Center. All ERIC content is uploaded into the application and all ERIC records are created using the application. A description of the application is attached in Appendix A. During the transition period, the staff shall be adequately trained in the application to meet the metrics specified in Task 4.

Within 2 business days of contract award the contractor shall submit a request to IES to initiate any staff that do not have security clearances and a request to validate staff members that have existing clearances. These requests shall be in the format specified in the Contractor Employee Security Screening Requirements section of the PWS. Once initiated, staff will have 5 business days to complete all paperwork and return to the COR. The COR will notify the project director when the packages have been accepted by the Security Office in the National Center for Education Statistics (“NCES”) and individuals are authorized to have an ELVIS account.

Within 15 business days of contract award, the contractor shall prepare a transition in memo that uses the dates specified in the current ERIC contractor’s transition out memo as a basis for the transition (attached to the solicitation). The memo shall document the timeline for each transition activity and responsible parties. The transition activities include, but are not limited to, gaining possession of the following Government Furnished Property (“GFP”) and arranging storage in a climate-controlled facility:

· Electronic bibliographic database files, 1996 –2023

· Magnetic 4MM DDS-1 DAT tapes with bibliographic database records, 1966 – 2023, https://eric.ed.gov;

· Microfiche archive of full-text non-journal documents and authorizations for ERIC dissemination, 1966 – 2003 (approximately 587 cubic feet);

· Copies of ERIC Thesaurus, including electronic files;

· Electronic files of ERIC authorization agreements; and

· Other working documents used to train/guide ERIC staff in selecting, abstracting or indexing documents for the ERIC database or maintaining quality control or usability of the ERIC database.

The contractor is responsible for ensuring all staff is adequately trained and has appropriate access to begin accessing the ELVIS system by April 23, 2023. The contractor is expected to meet the processing requirements outlined in Task 4 at the end of the transition period. The contractor shall submit a memo no later than April 23, 2023 confirming that the transition is complete.

Deliverables:

· Request for initiation and validation—2 business days after contract award

· Transition in memo—21 days after contract award

· Transition completion memo—no later than April 23, 2023

1.3. Biweekly Meetings. The project director and other staff that the project director deems necessary shall meet with the COR and other appropriate Department staff via biweekly conference calls. The purpose of the meetings is to brief the Department on progress and challenges related to the tasks in this contract. The Contractor shall send a meeting summary to the COR within 5 business days after the meeting is complete.

In addition, the Contractor shall be available for meetings to address unexpected issues that arise during the contract period. These meetings may occur on an as-needed basis at the request of the COR or the Contractor and will take place via conference call. The contractor should assume one meeting a week in the first six months of the contract and approximately one meeting per month thereafter. All meetings shall be remotely via conference call or video call.

Deliverables:

· Summary of biweekly meeting—5 business days after each meeting

1.4. Monthly Report. The contractor shall submit a monthly report that summarizes progress made in each task and subtask, and shall specifically include the metrics specified in the tasks below and listed in Appendix B. The report shall highlight progress made and any challenges faced. In addition, the monthly report shall include proposed work and expected challenges for the next month.

Contractor shall provide data on earned value in a specific format specified in Appendix C. Any deviations in schedule or cost shall be fully explained in the risk register. Additionally, the monthly report shall include the IT reporting sheet and the contractor personnel listing using the templates provided in the solicitation. Because IES reports on earned value as part of a combined business case with the What Works Clearinghouse and National Library of Education, the contractor must follow IES guidance on IT reporting.

The contractor shall submit the report on the sixth business day of every month for the duration of the contract and shall be prepared to discuss it in the next biweekly call with the COR.

Deliverables:

· Monthly report—6th business day of every month

1.5. Creation of IT Reporting Materials. Within 4 weeks of contract award or the start of an option year, the contractor shall submit work breakdown structure of all deliverables, down to the individual personnel level for time and materials deliverables, in Microsoft Project format, a 2-3 page overview of the quality management plan, and a 2-3 page overview of the risk management plan. Guidance for these documents is provided in Appendices D and E.

Deliverables:

· Work Breakdown Structure—4 weeks after contract award

· Quality Management Plan—4 weeks after contract award

· Risk Management Plan —4 weeks after contract award

Task 2: Propose Revisions to the Selection Policy (Optional, Time & Materials). The ERIC Selection Policy establishes the standards and criteria for selecting materials for inclusion in the ERIC collection. It states broad collection goals and defines the standards and criteria required of approved sources and individual materials in the ERIC digital library. The purpose of the selection policy is to provide consistency in the approach for reviewing and selecting sources and individual items, and clearly communicates policy and process to staff, users, publishers, and individual submitters of material.

In consultation with external experts, the contractor shall review the existing ERIC Selection Policy, including its policy for user-submitted materials, and make any recommendations for changes. The review should look at research cited by IES authors, including works funded by IES and studies reviewed by the What Works Clearinghouse (“WWC”). The contractor shall ensure that the Selection Policy is sufficiently broad to index the types of materials that IES authors routinely cite but narrow enough to allow for a focused collection. Within 6 months of task award, the contractor shall submit a memo describing its review of the Selection Policy, with a rationale for any needed changes and proposed revisions. Upon the COR’s approval, the contractor shall produce a visually engaging 508 compliant update to the Selection Policy for the ERIC website. The contractor shall also develop communications materials for the update, including a newsflash, social media posts, and a webinar or video. The COR approval of the contractor’s proposed recommendations is required before implementation.

Deliverables:

· Memo describing the review of Selection policy—6 months after contract award

· Updated Selection Policy—as agreed to with the COR

· Communications Materials—as agreed to with the COR

Task 3: Source Selection (Firm-Fixed-Price)

3.1. Review and propose source materials for inclusion in ERIC. Twice per year, the contractor shall complete a source selection review. During this process the contractor shall evaluate the sources that it currently indexes to determine which agreements need updating (e.g. if a journal switches publishers) and which sources that are currently approved for indexing should be discontinued (e.g. the journal has been discontinued or no longer meets the selection policy) or moved to be inactive (e.g. the journal is not currently publishing, but may resume publishing in the future).

The contractor shall also review new sources for inclusion in ERIC. In selecting sources, the contractor shall consider the criteria in the ERIC Selection Policy. The contractor shall assume that approximately 50 titles/sources (+/- 10 titles/sources) will be reviewed in each source selection review and shall propose a set of sources that produce enough content for the contractor to meet the indexing requirements listed in Task 4. The contractor must balance the expected volume of content to ensure that there is not a backlog in indexing content.

On March 15th and September 15th of each year, or the closest business day to those days, the contractor shall send a spreadsheet with three lists: recommendations for new sources, recommendations to update existing agreements, and recommendations of sources to no longer index in the collection. The COR will review each change and provide written feedback. Within 3 weeks of receiving COR feedback, the contractor shall then submit an updated Journals and Non-Journals list in the form of a 508 compliant PDF and tracked changed Word document within 4 weeks of COR sending out approved addenda and agreements (see subtask 3.2).

Deliverables:

· Source Selection Recommendations—March15th and September15th

· List of approved sources-- within 4 weeks of the COR sending agreements, twice per year

3.2 Develop agreements for new sources. Within 3 weeks of receiving COR feedback on proposed source changes, the contractor shall send the COR the following 1) new addenda to add new titles to existing publishers, 2) ERIC agreements for publishers not currently under agreement, and 3) a list of contact information for each source. The agreements and addenda shall be in the established ERIC format and shall require the contractor to input the name and address of the publisher and the titles selected to index.

Deliverables:

· Agreements, addenda, and contact information for new sources—within 3 weeks of COR feedback

3.3. Managing the Agreement Management System. The Agreement Management System (“AMS”) is a part of the ELVIS tool that contains a copy of all agreements and addenda signed by publishers in ERIC and their contact information. The contractor shall also be responsible for updating the AMS when publisher information changes and when non-journal sources are approved to be marked as peer reviewed within 2 business days of receiving a countersigned agreement. The COR will send the agreements prepared under Subtask 3.2 to publishers with a carbon copy to the eric@ed.gov email address. When a publisher returns an agreement, the COR will countersign and return the agreement to the publisher, with a carbon copy to eric@ed.gov.

The contractor shall manage the eric@ed.gov email box and ensure that all agreements and addenda were sent correctly. The contractor shall then take any countersigned agreements and attach them to the AMS within 2 business days. A summary of activities under this subtask shall be reported in the monthly report.

Task 4: Index Materials in ERIC (Firm-Fixed-Price). The contractor shall provide the staffing and software capacity to process and publish 4,200 records (journal and non-journal) per month, as approved in Subtask 3.1. Some variance from month to month is acceptable, as long as the contractor publishes at least 3,800 records a month and 50,400 per year. The contractor shall use an indexing team to develop and operate the daily workflow of processing of materials.

Records shall be created using the ELVIS processing software and will be transmitted to the Department to be published at https://eric.ed.gov at the end of each workday. A demo of ELVIS can be found here. ELVIS uses a combination of computer aided indexing and manual indexing to create metadata for journal and non-journal sources. While the contractor may propose changes in approach and enhance ELVIS under Task 10, the contractor is expected to meet the service level agreement and use the existing software until any enhancements are proposed, approved, and implemented.

The contractor shall start processing workflow documents at the end of the transition period stated in Task 1. Through Task 3.1, contractor shall ensure that there is not a backlog of records and content is indexed in ERIC within 30 days of the content being transmitted from the publisher or within 30 days of being scheduled to be processed based on the schedule described in Subtask 4.1. In the monthly report described in Subtask 1.3, the contractor shall describe the number of records published by week and the number of records yet to be processed.

4.1. Acquire content to index. The contractor shall acquire 4,200 records per month of education literature in sources under agreement with ERIC. Currently, most of the collection, 61%, is acquired by the ERIC team visiting publishers’ websites to download existing content in PDF form. This is currently done via a manual process and the contractor may propose alternate approaches that are more efficient. Until an alternate approach is proposed, approved, and implemented, the contractor is expected to meet the service level agreement and use the existing approach.

Additionally, 34% of the content is sent by the publisher via secure FTP. Of that content, 23% is sent as only XML, 10% is sent as only PDFs, and 67% has PDFs and XML. ERIC also accepts content via email, an option used for about 4% of content, and online submission, which is used for approximately 1% of content. The proportion of content delivery may change slightly as new sources are approved and other sources are discontinued.

The ERIC contractor shall develop a schedule for when they will visit publishers’ websites to download new content. The contractor shall visit websites at least twice per year during times that the publisher has historically published new content. The contractor shall propose the most efficient way to gather the content in a timely fashion. The contractor shall include a list of the publishers visited in the monthly report and shall submit an annual schedule of publishers no later than August 15th of each year. Any updates to the schedule shall be submitted with the monthly report.

Deliverables:

· Schedule of publishers to visit—August 15th

4.2 Index new content. The contractor shall index the content acquired in Task 4.1 on a regular basis using the ELVIS Processing System, as described in Appendix A. Work must be indexed within 30 days of being made available to the contractor through Task 4.1.

The contractor shall ensure that all applicable fields are complete and accurate, consistent with processing guidelines, and that Thesaurus terms are correctly applied. The contractor shall use the existing ERIC metadata specification and standards for processing library materials. The existing standards take into account widely used scientific and technical information frameworks and standards. Any changes to the existing ERIC model and specifications shall incorporate the following requirements and shall be approved by the COR before implementation.

All bibliographic records shall include abstracts, either written by the author or publisher or by the contractor. Approximately 10% of records include abstracts written or generated by ERIC. Many of these abstracts use material in the document or the website that the document was obtained, as opposed to being written from scratch. Abstracts written by the contractor shall provide a summary and the conclusions of the individual journal article or non-journal material. The contractor shall include information about the number of new ERIC records with author or publisher provided abstracts and ERIC created abstracts in the monthly report.

All content shall be scanned for readability prior to being posted to the ERIC website. Unreadable PDFs may not be displayed on the ERIC website. Readability is defined as having 85% of the words on the machine-readable layer of the PDF being spelled correctly and matching the information on the visible layer. This is currently done through an automated process described here. The contractor may propose an alternate approach to processing that are more efficient and will accurately achieve the stated objective in a more cost-efficient manner, but are expected to use the current technology until alternate approaches can be proposed, approved, and implemented.

On weekly basis, the contractor shall submit requests to upload new records to the ERIC database through the ELVIS application. The IES Webmasters will load new records and the COR will review and approve prior to releasing to the live site. The contractor shall send a weekly email notifying the COR when new records are submitted to the Department.

Deliverables:

· Email of new records-- weekly

Task 5. Quality Assurance (Firm-Fixed-Price). Content and data elements on the ERIC website shall be current and accurate, including database indexing and web links. Accurate data is essential to ensure that records are discoverable by users and to ensure that all content is regularly indexed in ERIC.

5.1. Sampling of Records and Sources. On a quarterly basis, the contractor shall arrange for an independent team to randomly sample at least 100 records created in the past quarter and 100 records created prior to 2018 to report the percentage of critical defects and data quality errors observed. The contractor shall ensure that this subtask is done by an independent team who did not create the records. The contractor shall propose an approach to sampling that is efficient and will accurately achieve the stated objective.

Critical defects negatively impact retrievability, and examples include misspelled or missing author names, source names, and citation errors. Data quality errors include missing or incorrect descriptors and identifiers; misspelled, duplicated, or incorrect abstracts, page numbers, publication date, etc.; and other similar errors.

Additionally, the contractor shall sample 20 sources per quarter to ensure that all content published within the past 5 years is available in ERIC and has been published to the website. If content has been missed, the contractor team shall index it under Task 4. The contractor team shall use the report for internal training purposes to help prevent errors in future records. The contractor may propose an approach to sampling that is efficient and will accurately achieve the stated objective in a more cost-efficient manner.

Deliverables:

· Quality Assurance Report—January 10th, April 10th, July 10th, and October 10th

5.2. Fixing of Records. Each month the contractor shall make updates or changes to 50 ERIC records. This includes correcting any mistakes identified in Task 5.1 and any errors sent to the ERIC Help Desk. The errors may be errors made by the authors, such as failing to include a grant number or correcting missing information. If fewer than 50 mistakes are identified, the contractor shall work to update any broken direct links to publisher sites. This work shall be reported on as part of the monthly report.

Task 6. Connecting IES Resources (Time & Materials). As part of an effort to connect IES resources, the ERIC contract shall work to identify and promote linkages between ERIC and https://ies.ed.gov. Through this work, the contractor shall identify new areas of source selection under Task 3.

6.1. Connecting ERIC and the What Works Clearinghouse (WWC). The WWC reviews studies of education research and relies on ERIC’s metadata to populate its database. Approximately 70% of the studies reviewed by the WWC are in ERIC and IES would like to ensure that ERIC indexes as many studies reviewed by the WWC as possible.

On a monthly basis the WWC will provide a list of all studies currently under review and studies that have been published and whether or not they have an ERIC accession number associated with the study. The ERIC contractor shall review all studies that do not have an ERIC accession numbers to determine if 1) there is an ERIC accession numbers that already exists and can be added to the record, 2) if the publication is an approved source in ERIC and there should be an ERIC accession numbers, but the study has not been indexed yet, or 3) if the study is not in an approved source. While much of this work can be automated, some manual checking may be required.

If there is currently an ERIC accession numbers available, the contractor should send an updated list of WWC Study IDs and their corresponding ERIC accession numbers to the COR. The contractor and COR will work with the WWC-TOAST contractor to determine the most efficient format for this information, but the contractor shall assume that the preferred format will be machine readable, such as XML or CSV. For studies from approved sources, the ERIC contractor shall work to acquire the content as soon as possible and index it under Task 4, and then provide the accession number and WWC Study ID in the next monthly update. For sources that are not currently approved, the ERIC contractor shall review the source in the next round of source selection under Task 3. If the source is approved, the contractor shall add the accession numbers and WWC study IDs in the next monthly update.

The WWC currently has reviewed approximately 16,000 studies and adds approximately 30 reviews per month. There is currently an ERIC accession number for about 60% of studies. Approximately 10% of the existing studies are from an approved source, but do not have an ERIC accession number. This is often because the WWC reviews studies that are in “online first” preview mode on the publisher’s website or are newly released and have not yet been added to ERIC. It is expected that approximately 80-90% of WWC reviewed documents should have an ERIC accession number.

For any newly published studies or studies with missing accession numbers, the contractor shall also ensure that all studies reviewed by the WWC have the appropriate rating and link to the study review page in ERIC. The contractor shall expect to update approximately 25 records a month.

Deliverables:

· File of updated accession numbers, updated records

6.2. Connecting ERIC and the IES Research Centers (optional). Records maintained by the National Center for Education Research (NCER) and National Center for Special Education Research (NCSER) include citations of work published by their grantees. IES will provide a spreadsheet of citations of work published by IES grantee and contractors and the contractor shall confirm that the publications are in ERIC as well as the ED and EJ numbers of the publications. The contractor shall update the records to reflect the grant or contract number, if missing or inaccurate. For purposes of budgeting, the contractor shall assume that each batch shall have up to 100 citations and there shall be up to 15 batches per year. The contractor shall propose an approach to this task that is efficient and will accurately achieve the stated objective. Much of this work can be automated using fuzzy matching, but some manual intervention may be required.

The contractor shall prepare a memo that accompanies the file that includes an analysis of the sources in which IES funded researchers publish. The contractor shall determine if they are approved sources for ERIC and if the content meets the standards set forth in the ERIC Selection Policy. Any sources that meet the Selection Policy, but are not currently indexed in ERIC, should be considered in the next source selection review as described in Task 3. ERIC should then work to index this content and provide the accession numbers for newly indexed material to the COR for the IES team to update their records.

Task 7. Updating the ERIC Thesaurus (Firm-Fixed-Price, Optional). The ERIC Thesaurus is a controlled vocabulary of terms representing research topics in the field of education. Descriptors from the ERIC Thesaurus are assigned to every document in the ERIC digital library to describe its subject content. The ERIC Thesaurus contains a total of 11,818 terms. There are 4,552 Descriptors and 7,133 Synonyms. There are also 133 Dead terms which are no longer used as Descriptors but remain in the Thesaurus to aid in searching older records.

Six months after the task is exercised, the contractor shall provide a written assessment of the ERIC Thesaurus to propose changes to the Thesaurus (including new descriptors, revised scope notes, and recategorization of terms) needed are needed to maximize usability by inexperienced and experienced database users. The assessment shall include feedback received from indexers and suggestions from the ERIC community. The assessment shall also consider changes needed with respect to how the Thesaurus is presented to users on the website. The draft assessment with recommendation for changes shall be submitted to the COR. The COR will provide written feedback and the contractor shall submit a final version two weeks after receiving COR feedback. The contractor shall implement any changes agreed to in the final version within the timeline proposed in the draft assessment including a file with all proposed changes.

Deliverables:

· Draft Thesaurus Assessment-- 6 months after task award

· Final File of Proposed Changes—2 weeks after COR feedback

Task 8. Communications (Fixed Price). ERIC uses communication tools to be able to connect users to the resources on eric.ed.gov. The purpose of these communication tools is to increase the amount of traffic on the ERIC website and reduce the number of calls and emails made to the help desk.

8.1. ERIC Help Desk. The contractor shall help users seeking assistance in searching the database and using the website through the ERICRequests@ed.gov email inbox. The contractor shall specify up to three individuals who shall have access to this email box. The service shall be available from 10am to 6pm EST, Monday-Friday (excluding Federal holidays). Response times for the email inbox shall be no longer than 1 hour from when the help desk is open. A typical volume is between 250-350 emails per month and approximately 110 emails a month to those whose online submissions are not approved for indexing. A summary of the number of emails, and the average response time, shall be included in the monthly report.

8.2. Webinars (optional). The contractor shall produce a webinar or series of webinars to highlight key information about ERIC. These shall be archived on the ERIC website and YouTube channel. The topic and audience of the webinar or webinars shall be proposed 8 weeks in advance. Upon COR approval, the ERIC team will arrange for presenters and develop publicity materials (including social media updates and a Newsflash). The contractor shall then develop all webinar materials, host the webinar, and submit it for archiving. For budgeting purposes, the contractor shall price each deliverable separately and assume up to three webinars per year.

Deliverables:

· Topic and audience for the webinar—8 weeks in advance of the webinar

· Publicity materials—3 weeks in advance of the webinar

· Webinar materials—1 week in advance of the webinar

· Archived webinar—4 weeks after webinar

8.3 Videos (optional). The contractor shall produce a video or series of videos that are approximately 1 minute in length each year. These videos shall answer frequently answered questions on a topic or topics mutually that is proposed by the contractor and approved by the COR. The content in these videos shall be nonduplicative of videos already created by ERIC.

Additionally, in the contractor may develop a 3-5 minute video that shall explain how to perform a specific task that ERIC has expertise in, such as how to write an abstract or how to search using a specific field. The topic and audience of the videos shall be proposed 12 weeks in advance. Upon COR approval, the ERIC team will propose a script and develop publicity materials (including social media updates and a Newsflash). The contractor shall then develop a complete video to be hosted on ERIC’s YouTube channel and the ERIC website.

The contractor shall propose deliverables for both 1 minute and 3-5 minute videos separately and should assume up to 3 of each type.

Deliverables:

· Topic and audience for the video—12 weeks in advance

· Script for video—6 weeks in advance

· Publicity materials—1 week in advance

· Video

8.4 Infographics (optional). The contractor shall develop an infographic or a series of infographics. The contractor shall propose the topics and schedule for each of the infographics. The schedule shall contain a rough sketch of the infographic, a draft infographic, and final infographic. The COR will provide feedback on the schedule and the contractor will revise, if necessary, within 2 weeks.

Deliverables:

· Proposed topics and schedule—February 1

· Rough Sketch of Infographic—November 1 and as proposed

· Draft Infographic—Within 2 weeks of COR feedback and as proposed

· Final Infographic—Within 2 weeks of COR feedback

8.5 Updated Frequently Asked Questions. By March 1 and September 1 of each year of the contract, the contractor shall review the existing Frequently Asked Questions (FAQs), including those for Contractors and Grantees and for the Online Submission System. The contractor shall review each answer to ensure the answers are accurate and clear and propose any revisions or new questions and answers that may help reduce user confusion. The contractor shall also add links to videos, infographics, and webinars that can provide additional information on a given topic. The contractor shall submit the updated FAQs in the form of a Word file with track changes.

Deliverables:

· Updated FAQs—March 1 and September 1

8.6 Social Media Updates. Every 6 weeks, starting June 1, 2023, the contractor shall submit proposed social media to the COR for ERIC’s Facebook and Twitter accounts. The contractor shall create three updates per week. The updates shall feature communications materials, answers to frequently asked questions, and suggested search strategies. All posts shall include a link to the ERIC website or YouTube page.

Deliverables:

· Social medial updates—Every 6 weeks, starting June 1, 2023.

Task 9. Restoring ERIC Microfiche (Time & Materials, Optional). From 2004-2007 many ERIC archival records were scanned from microfiche and were converted into Portable Document Format (“PDF”). To increase the accessibility of these records, they were scanned using optical character recognition (“OCR”) technology to create background text that is searchable and readable by screen readers. However, the majority of documents had multiple errors on every page that include, but are not limited to, missing spaces between words, sections where the technology was unable to create text, and incorrect words. Additionally, many records are automatically ingested into the ERIC system from external journals. These documents are in PDF format and were produced by the authors and publishers of the document. Many of these documents are not readable and are not 508 compliant.

Since 2013, the ERIC contractor has reviewed the microfiche collection information and determined that 72% of the records have been digitized and 94% of the digitized records are machine readable and free of personally identifiable information. Additionally, approximately 130,000 records were not digitized and will require that the documents be scanned and digitized into readable PDFs.

The contractor shall propose an approach to digitize the remaining microfiche by the end of the 5-year contract period and to ensure that all documents are machine readable and free of personally identifiable information. The contractor shall also make 100 digitized documents into 508 compliant PDFs. The contractor shall prioritize documents that are not considered readable and are of high value to the ERIC community.

Task 10. Maintain ELVIS (Firm Fixed-Price). The contractor shall maintain and update the ELVIS processing system to ensure that the application runs without any errors. The software has been developed using .NET technology to run on a Microsoft Windows Server R2 and uses F5 Big IP Local Traffic Manager v11.6, Microsoft BitLocker, and CloudBerry Client.

The contractor shall work with the COR and the IES Web Support and IES Server teams to identify areas where enhancements can be made to streamline the environment or improve performance. The contractor shall discuss all proposed solutions with the COR prior to implementation. The contractor shall assume that that they will spend 40 hours per month of each year of the contract on enhancements.

The contractor shall also be responsible for creating accounts once new staff have received permission from the NCES security team and to maintain accounts for existing application users. If a person is removed from the contract, the contractor shall immediately delete the user’s account from the application. A summary of the work under this task shall be included in the monthly report.

Task 11. Providing Publisher Metrics. In 2018, ERIC developed a method to send publishers metrics on how frequently their content has been viewed and downloaded. The metrics reports are gathered using a series of calls through Google Analytics’ API that are aggregated by accession number in the URL. These reports include metrics for all articles at the source level and article level metrics for the ten most popular publications per source. More information about these can be found here.

11.1 Enhance the Metrics Report (Base Year Only, Time & Materials). The contractor shall enhance the current functionality of the metrics report to add individual article level metrics for all articles of a given source. The contractor shall determine the best way of delivering this functionality and shall consider whether developing a real-time portal or emailed report is the best format. The contractor shall add historic data of article performance for the previous 5 years and the metric data shall be updated at least every 6 months.

11.2 Send Metrics Reports (Firm Fixed Price). The contractor shall send metrics reports to all publishers on a regular basis, at least twice a year, using the new functionality developed in Task 11.1. The contractor shall ensure that all reports are timely, accurate, and received by all publishers. If the contractor developed a real-time portal, these automated emails should include a link to their portal and important updates for ERIC. This communication will be used to validate and update contact information in the AMS (see subtask 3.3).

Deliverables:

· Draft email—January 3rd and July 3rd

· Confirmation emails have been sent—January 15th and July 15th

Task 12. Develop Related Records Field (Time & Materials, base year only). Often there are multiple ERIC publications that report on the same study, as defined in What Works Clearinghouse Version 5.0 Procedures and Standards Handbook . An example of this is when a study team has written a conference paper, grantee submission, journal article, and follow up report on the same study. For example, EJ1339976 and ED618079 are considered the same study, as are ED562358, ED567238, ED567512, and EJ1170312.

IES requires the ability to group manuscripts, that is individual papers that are ERIC records, into studies. An article using a different sample of students, even if written by the same authors, would not be a related record. This field will be used by the What Works Clearinghouse and other IES program offices to better track articles across projects. This field is different from a real time suggestion of related products because the purpose of the field is to enable IES to link projects together in multiple databases.

In the base year, the ERIC contractor shall develop a mechanism to find related records using a fuzzy matching algorithm. The contractor shall then create a new field with related records that are hyperlinked to the appropriate records.

In the base year of the task the contractor shall create the algorithm to find records that are similar, using the criterial in the WWC Handbook. They shall pilot their algorithm with a sample of 100 studies that have already been reviewed by the WWC where related records are known to validate a proof of concept. Upon developing a reliable algorithm, the contractor shall populate any existing records with related records and they shall make the field searchable and included in the API. They shall build the functionality into ELVIS to have suggested related records be populated for newly added records. In the option years they shall ensure that all new records have this field populated and any related records are updated to reflect newly indexed records. Any updates to the field shall be done via Task 10.

Task 13. Develop New Fields (Optional, Time & Materials). ERIC occasionally adds new metadata fields that are designed to give the public new information. The COR will provide requirements for the field(s) and the way that the contractor shall obtain the necessary data for the field. The contractor shall then propose a schedule to add the field to ELVIS, backfill the data, establish procedures for adding the data to future records, and update the documentation for the API. The specific field(s) and additional information will be provided when this task is exercised.

Task 14. Develop Full Text Display in XML (Optional, Hybrid **Note that each subtask is optional and subtask 14.1 has a non-severable period of performance of 24 months). Having full text available in a machine-readable format, that is, as a web page instead of a PDF, has benefits for researchers and IES. Machine-readable full text allows the entire document to be searched by data scientists. This will help researchers find more nuanced information that may not be the main subject of an article. ERIC publishes approximately 13,500 records per year that have full text freely available as a PDF. Of those, approximately 650 are IES funded.

14.1 Develop an Infrastructure (Time and Materials, 24-month period of performance). The contractor shall identify a software approach to convert PDFs into XML to display in ERIC similar to how PubMed Central displays its full text (example). This includes developing the workflow and documentation, developing the software, and piloting the software with approximately 25 IES produced documents. The proposed approach should be minimally burdensome to grantees and program offices. After the pilot is completed, the contractor shall develop communication materials and guidance materials on how users should submit their content to ERIC to be displayed as XML via the online submission system.

14.2. Transforming IES Produced Documents as XML (Firm Fixed Price). The contractor shall transform all existing IES funded documents into XML. The contractor shall transform 3,000 documents produced since 2018 into XML as part of this effort. The documents may be found here. The contractor shall transform these in a manner that requires no additional input from the author or publisher. The contractor shall perform this work in bundles of 500 records with the first bundle submitted no later than 20 weeks after contract award.

Deliverables:

· Bundles 1-6—as agreed to with the COR

14.3. Renegotiate Agreements with Publishers (Firm Fixed Price). The contractor shall prepare communications materials for existing publishers to request that they submit the full text of their material available as XML. The contractor shall develop documentation and guidance to make this process as easy as possible for publishers.

The contractor shall assume that the communication materials include a one-page email explaining the change and a graphicly pleasing one- or two-page document outlining frequently asked questions and how publishers should submit their full text as XML. They shall also prepare addenda with contact information for existing publishers to make their content available as XML.

Deliverables:

· Communication materials—as agreed to with the COR

· Publisher agreements and contact information—as agreed to with the COR

14.4. Producing New Full Text as XML (Firm Fixed Price). The contractor shall develop or transform 1500 documents per year into XML. The contractor shall prioritize IES funded work, including those submitted through online submission, and non-journal content indexed through regular indexing. Journal publishers would submit XML that would be indexed during the traditional indexing process and published through Task 4. The contractor shall produce 125 new XML documents per month and submit a memo confirming the document creation as part of the monthly report.

Deliverables:

· Batch of 125 full text documents—with the monthly report

Task 15. Transition Out (Optional, Firm Fixed-Price). The contractor shall prepare and submit to the COR no more than 6 months following contract award a clearly written, documented Draft Transition-Out Plan. Five days after COR feedback, the contractor shall submit a Final Transition-Out Plan. This plan shall be updated and submitted to the COR no less than 180 days prior to the end of the contract. The transition plan shall address activities and matters associated with assets; provide a proposed schedule; outline procedures to be followed during the transition period; and identify performance measures to monitor transition progress and corresponding key milestones.

The contractor shall address in the Transition-Out Plan all necessary assets associated with conducting ERIC functions, including, but not limited to:

· Electronic bibliographic database files, 1996 – present

· Magnetic 4MM DDS-1 DAT tapes with bibliographic database records, 1966 – present ,…