Attachment F - Standalone Contract SOOs (9-10).docx

Task 9 (Standalone Contract)

Statement of Objectives Program Support Management (PSM)

National Assessment of Educational Progress

(NAEP)

2024-2029

9. Program Support Management The main objective of the NAEP Program Support Management contract (PSM) is to support the National Center for Education Statistics (NCES) by providing a program-wide approach to identify, prioritize, and successfully execute a portfolio of initiatives and projects that are aligned with the National Assessment of Educational Progress’ (NAEP) strategic goals and priorities. The program management function of this contract refers to the management of the many NAEP projects that operate within the NAEP program.

The primary responsibility of the PSM contract is to provide support for managing the NAEP program in accordance with Project Management (PM) best practices about scheduling, quality control, risk management and communication and handoffs across contractors.

The PSM contract comprises of three task categories:

9.1. Project Liaison and Support

For subtasks within task 4.1, the PSM contractor shall work collaboratively with the Program and Operations Coordination (POC) contractor to ensure alignment with the POC contractor’s work of coordinating the technical aspects of NAEP assessment activities. Overall, the PSM and POC contracts coordinate and collaborate across many aspects of PSM’s efforts to ensure that both contractors are effectively working together and to avoid any confusion across the NAEP contractor partners Functional Team and NCES with their activities. Some specific areas where the POC and PSM contracts closely coordinate and collaborate on decisions, NAEP contractor partners’ financials, contractor partners’ risks, and contractor partners’ schedule coordination. The PSM contractor shall not be responsible for NAEP contractor partner members meeting deadlines or completing deliverables. However, it will be necessary for all NAEP contractor partners to supply information relating to the progress of their tasks to the PSM contractor.

9.1.1. Develop and Maintain NAEP Program Schedules

The PSM contractor shall develop and maintain a master schedule of assessment tasks and major deliverables. This master schedule shall be available in electronic form to the NAEP contractor partners, NCES and others as directed by the Contracts Office Representative (COR). The contractor shall work collaboratively with the hosting contractor to make the master schedule available via a collaborative portal within the NAEP System. The PSM contractor shall provide up to four training activities annually on use of the master schedule, for NCES and NAEP contractors.

Following the award of the NAEP contractor partners contracts, the contractor shall obtain from each contractor partner, a contract-specific project schedule for completion of their work that includes identification of key milestones for tasks and products, and associated timelines. The contractor shall verify that these project schedules have been approved by NCES. The contractor shall use these plans to develop the master schedule, to be completed within 60 days of contract award.

As requested by the COR, the contractor shall produce specific project schedules from the master schedule. These project schedules shall include, but not be limited to, schedules of subject-specific and year-specific item development activities, design, and analysis requirements development, and NAEP for special studies. The contractor shall work collaboratively with the NAEP Web Development, Operations & Maintenance Support (WDOMS) contractor to make such project schedules available electronically via a collaborative portal within the NAEP System.

9.1.2. Track the progress of assessment tasks and major deliverables The PSM contractor shall define and execute a process for leading the NAEP contractors in all phases of project tracking, from project initiation to project closeout. The process shall emphasize procedures for tracking and monitoring NAEP work that requires the integration of several contractors. The process shall include the ability to track and report on handoffs and critical tasks between the NAEP contractors and NCES. As specified in the deliverables, the contractor shall produce tracking reports and recommendations for improving efficiency.

The PSM contractor shall alert NCES within 24 hours to any scheduling or workflow issues that could jeopardize the timely execution of NAEP operations. The PSM contractor shall not be responsible for the other NAEP contractors’ meeting deadlines or completing deliverables.

9.1.3. Provide Scope Management Support

The PSM contractor shall monitor the overall scope of the NAEP program and support NAEP contractor partners and NCES in managing program scope. To that end, the PSM contractor shall be fully engaged in NAEP activities and major NAEP meetings, such as, but not limited to, National Assessment Governing Board (NAGB), NAEP Validity Studies Panel (NVS), NAEP Design and Analysis Committee (DAC), and the NAEP Design Summit, for the purpose of becoming aware of policy and management decisions that have implications for the overall program scope and schedule.

While the POC contractor shall be primarily responsible for recommending and executing specific actions resulting from policy and management decisions, the PSM contractor shall contribute to the identification of policy and management decisions that have implications for the overall program scope and schedule and for advising NCES and the NAEP contractors on the scheduling feasibility of NAEP contractors proposed specific scope changes.

9.1.4. Provide Risk Management Support

The contractor shall provide risk management support, to include the identification and management of risks to NAEP operations, recommendations for risk mitigation, and the analysis and costing of such risks.

The contractor shall maintain a risk register, updated in real-time, to include industry- standard elements such as risk description, category, owner, probability, impact, mitigation responses and costs. In addition, the risk register must also identify risks which are IT-related to support the Educational Assessment IT investment (see Task 4.2).

9.1.5. Maintain a calendar of all Assessment Division activities The PSM contractor shall develop and maintain a calendar of all Assessment activities. The contractor shall work collaboratively with partner NAEP Functional Team contractors and NCES to update the calendar daily. The contractor shall work collaboratively with the WDOMS contractor to make the master schedule available via a collaborative portal within the NAEP System.

9.2. Contract and Financial Management Support Tools

The contractor shall provide project liaison and support service, including provision of the following:

· NAEP Program/Project Charters

· Schedule Management Plan, including Program/Project schedules and sub- schedules

· Memorandum of Understanding outlining relationships of Functional Team contractors

· Yearly Report providing checklist and review of Functional Team NAEP project charters, performance measures for each project, budgets for the year with cost and schedule baseline

· Scope Management Plan

· Monthly budget vs expenditures reports with performance metrics tracking

· Risk Management Plan to include process for identifying, mitigating risks

· Risk Register, updated continuously, to include mitigation responses and costs

· Calendar of Assessment Division Activities

9.3. Project Analytics

This task outlines the contractor’s role in providing support via financial management data reports that provide an enterprise view of NAEP’s overall performance by integrating key indicators centered on task scheduling, contract deliverables, and costs by assessment cycle. It is imperative that other NAEP contractor partners, collaborate efforts with the PSM contractor in forecasting the latest financials needs and projections, since this data is what NAEP will rely upon for determining and providing the latest funding requirements.

NCES prefer the PSM contractor to work with the other NAEP contractors to work towards a milestone schedule template and to reduce the variations of the schedule from operational year to year (e.g., consistent number of milestone reviews, common understanding of when NCES reviews are needed).

9.3.1. Provide Data Entry Support Services

The PSM contractor shall develop and maintain a contract management report (CMR) for use by NCES and other NAEP contractor partners in managing contract financials.

The contractor shall provide on-going technical support to NCES COR and NAEP contactor partners related to CMR data submissions (deliverable status and contract invoices) and reporting.

The contractor shall work collaboratively with NAEP contractor partners to ensure receipt of data needed for the CMR. Such data includes, for example, actual cost, percent complete data, and event history dates. The PSM contractor shall train partner contractors Functional Team members and NCES in data upload processes and in report generation.

The contractor shall produce CMR reports as requested by the COR.

9.3.2. Provide CMR Operating Support Services

The contractor shall provide CMR operations and maintenance support via phone, email, and in person (as needed) during normal business hours (defined as 8 AM – 5 PM Monday through Friday, excluding federal holidays and closures).

Contractor personnel shall record CMR problems and help requests received from CMR users and capture these requests in a MS Excel spreadsheet (log). As problems are resolved, or assistance is provided, contractor staff shall update the problem status and resolution in the system. Contractor shall review CMR problem logs monthly with NCES personnel. Contractor shall resolve all problems and answer user questions in a timely manner. If issues require more than one week to resolve, the contractor shall review the problem and prepare a resolution plan with the respective COR.

9.3.3. Conduct CMR Training

Contractor shall provide NCES with the necessary training on the CMR for NCES staff and NAEP contractors. Contractor shall provide annual refresher training. Contractor shall ensure that CMR training materials are available via the NAEP System, and Contractor shall conduct live training sessions as requested by the COR.

9.3.4. Provide IT Investment Support

The contractor shall support the Capital Planning and Investment Control (CPIC) process for the Educational Assessment IT (EA IT) investment, which includes all NAEP’s IT spending activities. [“IT” is defined as services or equipment that are used in the automatic acquisition, storage, analysis, evaluation, manipulation, management, movement, control, display, switching, interchange, transmission, or reception of data or information by the agency. For purposes of this definition, such services or equipment if used by the agency directly or is used by a contractor under a contract with the agency that requires its use, or to a significant extent, its use in the performance of a service or the furnishing of a product. The term “information technology” includes computers, ancillary equipment (including imaging peripherals, input, output, and storage devices necessary for security and surveillance), peripheral equipment designed to be controlled by the central processing unit of a computer, software, firmware and similar procedures, services (including cloud computing and help-desk services or other professional services which support any point of the life cycle of the equipment or service), and related resources. The term “information technology” does not include any equipment that is acquired by a contractor incidental to a contract which does not require its use (40 U.S.C. § 11101).] The PSM contractor shall provide technical support, quality assurance/quality control, as well as project management consultation so that the underlying planning, processes, and performance results are of high quality and experience continuous improvement. The contractor shall also have fundamental understanding of the IT initiatives included in the investment, such as the development, modernization, enhancement, operations, and maintenance of the IT projects in the NAEP program. Support with the IT investment activities will allow NCES to:

1. assess any changes or modifications to the investment that may require baseline changes,

2. assess the IT investment’s impact on NCES/Department of Education strategic performance, and

3. revise the investment management process based on lessons learned.

In addition, how the Institute of Education Sciences (IES) manages this investment, is tracked on the federal government’s IT Dashboard. It is essential that IES maintains a positive score on this dashboard—a “green” or low risk rating for all projects and an overall Chief Information Officers (CIO) rating of at least 4 or 5.

The PSM contractor shall provide support for the IT investment activities listed below. Office of the Chief Information Officer (OCIO) and the Office of Management and Budget (OMB) periodically change requirements, so this list is subject to change.

IT Funding Request The PSM contractor shall assist the EA PM with gathering and organizing information for the annual IT Funding Request to OMB from other designated NAEP contractors. This may include attending the Investment and Acquisitions Management Team (IAMT) IT Funding Request Training meeting, IAMT IT Funding Request Office Hours (as needed), up to two planning meetings with PM and an individual meeting with CORs/Project Directors to assist with determining spending and narrative for upcoming fiscal years, compiling a functional budget spreadsheet to include all projects and summary, and consolidation of narratives from CORs/Project Directors on goals, objectives, and project activities. This activity typically takes place in the late winter/early Spring of each year, a year and a half prior to the year which is being planned. So, for example, in March of 2025 (mid fiscal year 2025), the IT Funding Request for Fiscal Year 2027 is prepared.

Annual Submission The PSM contractor shall assist the EA PM with gathering and organizing information for the Annual Submission to OMB. The Annual Submission is the process of creating an instance in FOLIO, the Government system which houses IT investment information for the upcoming fiscal year and providing detail such that monthly updates to projects and activities may be made throughout the year. Preparing for this shall include attending the IAMT IT Annual Submission Training meeting, IAMT Annual Submission Office Hours (as needed), up to 2 meetings with PM (as needed) and 1 meeting with CORs/Project Directors (if needed) to clarify their project’s spending and activities for the upcoming fiscal year, reviewing all this with the PM for inputting into the FOLIO system.

Monthly Reporting and Tracking The PSM contractor shall track IT spending on all projects and activities within EA by working with the contract CORs/Project Directors to obtain the monthly actuals for each project activity, performance metrics (as requested by the PM) and project risk. The contractor shall input the information into FOLIO. Information must be provided by the 14th day of each month for PM approval and input into FOLIO.

Artifact Development To comply with OMB IT Investment requirements, various Project Management Book of Knowledge (PMBOK®) standard plans and documents must be maintained on a yearly basis for each project and the overall investment. The PSM contractor shall support the EA PM by creating/modifying/maintaining artifacts for the EA IT investment and related FOLIO data and reports for IT projects. In some cases, documentation will be fully developed by the contractor responsible for the project (e.g. a Project Charter for a web tool redesign project or Requirements Specifications Documentation for development projects under that contract) and supplied to the PSM contractor and PM, and some documentation will be developed by the PSM contractor with information from other NAEP contractors (e.g., a Work Breakdown Structure (WBS) for all contract tasks, Risk Register, etc.). Artifacts developed by the PSM contractor will include annual IT budget, and overall annual resource and cost-loaded Work Breakdown Structure (WBS).

Examples of project management documentation that PSM contractor shall be expected to create or maintain include:* EA Project Charters, EA Operational Analysis Plan, EA Operational Analysis Report (may be created at the end of a fiscal year to evaluate operational events of the year), EA Risk Register, EA Risk Management Plan, Alternatives Analysis Report (if needed), EA Quality Management Plan, EA Scope Management Plan, EA Requirements Specification Documentation (for new initiatives), EA Requirements Traceability Matrix, etc.

Note that it is possible for IAMT or OMB to request a document or report not currently listed. However, this is the most complete list to our knowledge currently. IAMT requirements change yearly, based on OMB guidelines.

Facilitate Risk Change Management Process for IT The PSM contractor shall develop and maintain a Risk Management Plan for the EA investment, as stated above. The contractor shall organize and facilitate a process for monitoring risk, shall maintain a detailed Risk Register, with assigned risk owners and updated at least monthly, and shall notify the PM and investment owner of any high risks. The contractor shall have sufficient knowledge of the NAEP program to assist in identifying risks and developing mitigation plans with risk owners.

When necessary, the PSM contractor shall facilitate meetings of the overarching Educational Assessment Change Control Board (CCB) which shall be comprised of the PM, CORs/Project Directors and/or key project staff. The CCB will meet when necessary to review all major changes to IT projects and track and document those changes. Major changes are those which involve a reallocation of resources or funds. The CCB will resolve any conflicts, set priorities for staff available to work on the changes, and assist in the schedule planning for the work. The PSM support contractor shall include change management procedures and how proposed changes shall be evaluated in the Educational Assessment Project Management Plan. This document must be approved by PM and signed by appropriate stakeholders.

9.3.5. Project Analytics

The contractor shall produce automated project status reports, including those based on pre-determined metrics and varied frequency. Information in these reports may include but are not limited to the status of the projects, cross-projects, and program level; on-track status, status of action items and next steps, project deliverables and quality of those deliverables, possible obstacles or risks, and budget status. The contractor shall make this information available to NCES via customizable dashboards within the NAEP System.

9.3.5.1. Provide CMR Analysis

Contractor shall monthly analyze NAEP contractor members’ CMR data submissions (monthly, quarterly, and yearly cumulative) and identify any variations between the proposed project schedule/budget and the actual schedule/budget.

9.3.5.2. Support NCES Integrated Baselines Reviews

Contractor shall provide support for conducting annual Integrated Baselines Reviews (IBR) and a supplemental review for non-Functional Team contractors. This support shall include planning and scheduling review meetings, note taking during the meetings, and submitting a follow up report after each review.

9.3.5.3. Performance Monitoring and Reporting

The PSM Contractor shall provide processes and tools to enable the monitoring of the overall NAEP enterprise. This monitoring shall include availability and performance-monitoring to provide real-time visibility into the status and performance of the NAEP program. These monitoring and reporting activities shall include developing risk reports and specialized reports. The PSM Contractor shall establish a real-time dashboard(s) for the monitoring of all NAEP activities.

The PSM contractor shall meet with NCES and other NAEP contractors annually to provide a program support management report. In this annual state-of-the-program report the contractor shall report on successes and identify areas for improvement in the program management support, and make recommendations for improvement as needed.

9.3.5.4. Dashboards

The dashboards shall identify and report out metrics across the NAEP activities. The dashboard shall capture all critical hand-offs and status of key milestones.

9.3.5.5. Risk Reports

The PSM contractor shall develop reports to communicate risks. Program and project risk reports are essential communication tools to initiate risk management and achieve program/project outcomes according to expectations. Risk reporting should include understanding existing risks, opportunities, and trade-offs.

NCES requires several specialized reports to communicate with other Government offices, senior level management, and other stakeholders.

Task 10 (Standalone Contract)

Statement of Objectives Web Development, Operations and Maintenance Support (WDOMS)

National Assessment of Educational Progress

(NAEP)

2024-2029

10. Web Development, Operations and Maintenance Support (WDOMS) Purpose The National Assessment of Educational Progress (NAEP) Web Development, Operations and Maintenance Support (WDOMS) contract supports the public and private (extranet) websites and applications of NAEP and the business processes associated with those websites.

NAEP public websites currently include the NAEP website (http://nces.ed.gov/nationsreportcard/) which provides information about the NAEP program, including information about assessment subjects, assessment schedules, resources for parents, schools, students, researchers, and policymakers and much more. It is a website about the business of NAEP.

The Nation’s Report Card website (http://nationsreportcard.gov/) provides interactive data results and tools, dashboards, questions tool of released NAEP items, an interactive sample assessment, and a NAEP data service Application Programming Interface (API).

The NAEP private sites, located on an “extranet” called the NAEP System (https://naep.ed.gov), are primarily used as collaboration tools for NAEP staff, contractors, and state coordinators. These applications include the NAEP Integrated Management System (IMS), NAEP Network, and other sites for facilitating state coordinator communication, internal publication content management reviews, and other internal-use applications.

The WDOMS contract focuses on objectives that are aligned with NAEP’s strategic vision and include:

· Developing and maintaining tools and processes that support communication, collaboration, and business processes for use within the NAEP program.

· The management, scheduling and collaboration of multiple contractors and partners working in dynamic, quality-conscious, fast-paced, data-driven environment.

· Improvements in operational processes using cloud-based systems and enhancing systems to allow for more efficient reviews and better reporting capability.

· Quality control and accountability.

· Public websites that support data-driven design and architecture, thereby supporting NAEP stakeholders with efficient access to information and data.

· Incorporation of application programming interfaces, (APIs), which will allow NCES data to be more widely shared.

· Linking NAEP to other assessments and surveys.

The task activities required in this performance work statement are:

1. Task 10.1 Develop and Enhance Public and Private Websites and Applications

1. Task 10.2 Provide Web Content, Coordination and Maintenance of Websites and Applications

2. Task 10.3 Provide Hosting, Operations, and Cybersecurity Support

3. Task 10.4 Support NAEP Business Processes

10.1 Develop and Enhance Public and Private Websites and Applications NAEP’s public sites are currently managed, maintained, and operated within the Institute of Education Sciences (IES) Data Center, through a third-party cloud web service. The NAEP private sites shall be managed and maintained on servers hosted and managed by the WDOMS contractor through a third-party cloud web service.

Currently NAEP operates two primary public sites. These two public sites are (1) a general site for providing public access to a broad range of NAEP information, data, tools, and documentation—the “NAEP website” (http://nces.ed.gov/nationsreportcard/); and (2) a targeted site for presenting initial releases of NAEP data—the Nation’s Report Card site (NRC) (http://nationsreportcard.gov). NCES releases reports in a digital-only interactive format available on the NRC website, and customized data through the NAEP Data Explorer (NDE) web tool.

NAEP’s private sites are used for a variety of NAEP administrative functions. Collectively, these private sites and applications are hosted in what the Department’s Office of the Chief Information Officer (OCIO) refers to as the NAEP System. The NAEP System is hosted and managed by the WDOMS contractor and is maintains an Authority to Operate (ATO) for security by the Department of Education. Refer to Task 10.3 and Instructions to Offerors for information about the current NAEP System.

10.1.1 Develop and Execute a Strategy to Integrate the NAEP Public Websites For many years, the NAEP program has maintained two separate websites for the program as described above. Through usability studies and user inquiries, we know that having two websites—one that describes how NAEP operates and one that reports NAEP results can often be confusing to the public user. Therefore, the contractor shall propose an integration of the two NAEP websites such that the sites shall be seamless to the user, focus on organization of information with a user-friendly architecture, contain static and basic dynamic content, as well as global elements, adhere to established NAEP branding guidelines and support the priorities of the NAEP program.

Recognizing that these two websites are large and merging them will take some time, the detailed plan to the Government should include a timeline (iterative development schedule) and wireframes which will include both design and architecture. Once the plan is approved, it is expected that the transition to merge the websites will take no longer than one year. See other enhancement considerations in Section 10.1.2 below.

10.1.2 Enhance the NAEP Public Website Interfaces

The WDOMS contractor shall develop, design, and enhance the public websites. As stated above, the contractor will be proposing an integration of the two public sites. Prior to the integration, the sites shall be supported separately as the transition takes place. This includes the following current sites and any future websites proposed:

· NAEP website at http://nces.ed.gov/nationsreportcard/

· Nation’s Report Card (NRC) website (http://nationsreportcard.gov Enhancing the NAEP public websites both before and after integrating them shall include:

· Data-driven design and architecture—The contractor shall plan on refreshing website design, architecture, and content on a continual basis, beginning in the first year of the contract. Related tasks shall include implementing eye-catching graphics, intuitive visual hierarchy to draw user focus to NAEP’s prioritized content, reorganizing menus and navigation as needed, build-out and testing. Data from the following sources shall be used to drive design and architectural enhancements (See also Section 10.2.5):

· Web trend data using Google Analytics 4 (GA4), results from metadata tagging, Search Engine Optimization (SEO) ranking, navigation patterns, etc.

· Usability studies conducted with various audience groups

· Analysis of public inquiries and FAQs

· Frequent releases—The contractor shall propose a schedule of approximately 8-10 releases per calendar year of new features, upgrades to content functionality (e.g., a more usable way to present certain types of information), content sections, or new major graphics. At the beginning of each fiscal year (October 1) and midway through the fiscal year (April 1), the contractor shall propose a 6-month work plan and schedule to be approved by the COR. That work plan will be reviewed monthly by the COR and presented to the Public Site Enhancements Working Group (PSE WG).

· Weekly Meetings of the Web Content Team (WCT) and COR—The WCT, consisting of WDOMS and other contractors involved in NAEP releases shall meet weekly to discuss status of upcoming releases, consensus building on content refresh, and other web content issues.

· Monthly Presentations to the Public Site Enhancements Working Group (PSE WG)—The PSE WG shall meet monthly to discuss recent and upcoming enhancements to the NAEP public websites. The WG is open to all stakeholders—NAEP staff and contractors—who have a vested interest in improving the websites for our user community. The WDOMS contractor shall schedule this meeting and present status of enhancements to the public sites. During the transition process of integrating the sites, this WG and meeting will be particularly critical to the process as ideas and issues may arise that will need to be discussed in this forum, and then taken to NCES for final decision. More frequent meetings of this group (or subgroups formed if necessary) during the site integration project are expected.

· Implementation of a universal design library of standards to create consistency and adherence to these universal design elements that comply with the US Web Design System for 21st Century Integrated Digital Experience Act (IDEA). In addition, ensure all web development and enhancements meet accessibility and compliance standards (i.e., 508, W3C Web Content Accessibility Guidelines (WCAG) 2.0 Level AA, https://accessibility.digital.gov/, https://accessibility.18f.gov/).

10.1.2.1 Collaborate with Design, Analysis and Reporting on Web Reports for Release on Public Websites The WDOMS contractor shall establish and monitor standards for both the timely delivery and quality of content and applications. It includes releasing of all relevant data and reports within established timeframes, while maintaining the high quality of all NAEP web activities and conducting quality control checks on all web content and links.

The contractor shall negotiate a schedule with the DAR contractor that meets the stringent timelines necessary for the adjudication process and reporting requirements of NCES.

The WDOMS contractor shall provide graphics and/or announcement banners on the NCES NAEP website to announce upcoming data releases, as well as graphics/banners with links to announce that data has been released.

The contractor shall deploy a preview/development web site for every NAEP release. This preview site will contain embargoed data that can be used by the NAEP State and District Coordinators to enhance their training experiences.

10.1.2.2 Release All Data and Reports within Established Time Frames The WDOMS contractor shall establish a standard process for setting a delivery schedule with project milestone targets. The contractor shall make schedules available to all NAEP content providers, to include the necessary lead times from delivery to the WDOMS contractor, to posting on the NCES development server, and allotted review time (typically 3-5 days) by IES/NCES Webmaster.

The contractor shall support NCES by scheduling and facilitating all web reviews conducted by NAEP stakeholders on the NAEP websites through the final deployment of the web pages. This will also include the change processes conducted after each review cycle with the appropriate Reporting contractor staff.

10.1.2.3 Place Data on the Web with Tools Suitable for Public Use NCES has several powerful, accessible online tools for making the wealth of NAEP data available to a variety of audiences. One tool is the NAEP Data Explorer (NDE; available at https://nces.ed.gov/nationsreportcard/nde). This is a powerful table and chart generator that allows users to cross tabulate results by the array of NAEP variables and to develop regression models, as well. Other tools include the NAEP Questions Tool (NQT), NAEP Item Maps, and the NAEP State/Districts Tool. These tools can be viewed at https://nces.ed.gov/nationsreportcard/about/naeptools.asp. Review of these tools involve Assessment Division staff and the NCES Chief Statistician.

Under this task, the WDOMS contractor shall be responsible for and manage the overall NAEP website(s), while working closely with the DAR contractor to support the development, placement and overall page designs of all NAEP data tools.

10.1.2.4 Develop Application Program Interfaces (APIs)

The US Department of Education currently provides APIs for several of its surveys. This helps address the NAEP audience of developers wishing to draw on NAEP data and reformat for their own uses. See here https://usedgov.github.io/ for more information about the Department’s efforts around this endeavor. It is important that NAEP API data be shared with the public to allow the public to use these data to develop their own products.

Under this task, the contractor shall coordinate with the DAR contractor to provide necessary support in the development of shareable APIs suitable for public access and place the APIs on the Nation’s Report Card website. The contractor shall support DAR efforts of API documentation and place documentation on the Nation’s Report Card website to facilitate use of the API by developers.

10.1.3 Develop and Enhance Private Sites and Applications

The contractor shall manage development and enhancement activities for private sites and collaborative portals residing on the NAEP System.

During the first 60 days of the contract, the contractor shall meet with the COR and other key members of the NAEP staff to discuss a plan for updating the private sites. The contractor shall review all private site applications and the contractor shall propose new solutions to the offering which incorporate new technologies, expand usability, and make sure the NAEP tool set is utilizing cutting-edge technologies.

The contractor shall support the required business analysis for all NAEP development. This business analysis effort will include analyzing the IT needs of the NAEP program to ensure that these requirements are met and that process improvements are realized.

The contractor shall include the following for all new development activities:

· Perform, as requested by NAEP, feasibility assessments for potential applications. These may be the evaluation of systems provided to NAEP or for the development of design prototypes to assess the feasibility of supporting a particular need. The feasibility assessment shall include documenting the initial idea to be supported; assessing the available infrastructure and data to support the need; study of information security and information assurance of new application; where the application will be hosted; backup and disaster recovery plans, and integration and interoperability issues with other applications and evaluation of solutions. The contractor shall deliver a final feasibility assessment report on the idea for implementation.

· Conduct business analysis--evaluating and documenting requirements specifications for each new feature or new application and defining business logic and interfaces required for the application.

· Evaluate alternatives for implementation of the new feature or application, including customizable commercial off-the-shelf (COTS) software or open-source code. This may include development of a formal Analysis of Alternatives document.

· Create a Project Charter containing the business need the project/application will meet and the goals of the project. The contractor shall also include the makeup of the project team, scope, a budget summary, schedule with project milestones noted, assumptions, constraints, and risks. The COR shall approve milestones upon completion. An example of a previous project charter can be found in Attachment X, WDOMS technical exhibit.

· Adopt a standard Agile framework for completion of all development that includes regular intervals/sprints to engage with appropriate NAEP staff and stakeholders. The contractor will propose their Agile approach for review and approval by the COR. This will include cadence and approach to planning. The contractor will need to detail how they propose to create, manage, and prioritize the backlog of features, set goals, and estimate task duration.

· Include wireframes for system design architecture and databases supporting the applications. The contractor shall follow industry best practices for Software Development Lifecycle (SDLC), providing demos at least monthly during the process and documentation of the system design, information architecture, and physical data models for all systems except as indicated by NAEP.

· Conduct testing for quality assurance, usability, and user acceptance. During all development efforts, the contractor shall follow the quality assurance processes approved by NAEP in the Quality Management Plan. (See Section 10.2.4) The WDOMS contractor shall develop all code following IES Web Standards, NAEP branding guidelines and Department of Education development and enterprise cybersecurity requirements.

· Develop training and demonstrations (either onsite or remote) for hosted applications and for other new NAEP web-related activities. Training may include descriptions of all user functions, features available to each user class, interactive demonstrations, in-lesson tests to assess understanding and a survey about the training for participants. The contractor shall develop the training applications for each new private site application (or each new major feature change) as requested by COR to support business-critical private site applications for both local and remote users. All training developed under this contract must comply with Department Section 508 web accessibility guidelines. The contractor shall collect feedback from training attendees and provide summary in monthly report to NCES.

10.1.3.1 Develop a Content Management System Solution

The contractor shall propose a new content management solution (CMS) for the newly integrated NAEP public website to manage the creation, tagging, modification and versioning (review) of digital content, create structural continuity on the website backend, automate workflows and connect related products and content across the newly integrated website. Requirements may include:

· Inventory existing NAEP website content, define the content types and use cases needed for the redesigned website, and create the go-forward plan and schedule for iteratively standing up the CMS

· Improve organization of content to help NAEP more efficiently build, tag, preview, and publish content with greater speed, ease, and control

· Optimize NAEP website performance, security, and reliability through eliminating site vulnerabilities and page errors, and maximizing page load speeds

· Create content templates aligned to universal web design library in the CMS and enable the selection of reusable content for automated, easy, and consistent webpage creation

· Develop a plan for and migrate all agreed-upon content, web pages, and data, from the current-state NAEP websites/content management platforms into the future state architecture via the approved CMS solution based on the website redesign strategy

10.2 Provide Web Content, Coordination and Maintenance of Websites and Applications The contractor shall be responsible for the overall management and maintenance of the public and private websites to include content creation and updates, coordination of content from other contractors, and maintenance of the public sites regarding quality, accuracy, and security.

10.2.1 Create Web Content

The web contractor shall create most of the web content on the NAEP websites, to support branding and communication goals of the NAEP program. This content should be optimized and efficient, reducing duplication whenever possible, while maximizing impact of key information to targeted audiences. Content shall be created through familiarity with the NAEP assessment and event schedules, report releases, the National Assessment Governing Board (NAGB) framework, NAEP data tools and by working with NAEP staff and other contractors in the NAEP program. The WDOMS contractor shall ensure products are web-ready and compliant and be responsible for converting materials to HTML (coding), PDF and other end-formats, as appropriate.

In addition to creating new content, the WDOMS contractor shall perform other typical web content design and maintenance tasks to include updating outdated content already on the site, the creation of HTML tables, posting of graphics and videos, and the creation of graphics and navigational icons.

NCES also has occasional need for conversion of multimedia content from one format to another while conforming to accessibility and NCES web standards; for example, Microsoft PowerPoint presentations converted to interactive web content. The contractor shall provide such multimedia presentations and content on NAEP websites as needed and convert multimedia content from one form to another on an ad hoc basis.

Reviews of new web content shall be conducted using the CMS. The contractor shall work with NAEP to coordinate reviews of web content and shall facilitate such review sessions. The contractor shall collect comments to content, provide the comments for review, and capture the decisions of the review sessions. Along with content and guideline reviews, the contractor shall be responsible for providing support for accessibility reviews to comply with accessibility and compliance standards (i.e., 508, W3C Web Content Accessibility Guidelines (WCAG) 2.0 Level AA, https://accessibility.digital.gov/, https://accessibility.18f.gov/) and NAEP branding, communications and quality guidelines for accuracy and completeness.

10.2.2 Provide Web Coordination and Guidance—Develop/Update Web Publication Manual Some web content will inevitably need to come from other sources, such as information from the data collection contractor. For this reason, the contractor shall develop/update a Web Publication Manual which outlines processes and provides technical assistance, support and/or infrastructure for such content providers, and other NAEP partners, including NAEP State Coordinators and NCES Statistical Standards Program team. The contractor shall develop and maintain the Web Publication Manual as a dynamic document with updates as needed to describe new processes, procedures, and content. The contractor shall develop the manual to clearly delineate the responsibilities or tasks of the WDOMS and the responsibilities or tasks of other content providers and specify areas of collaboration. The contractor shall coordinate changes and approvals with the COR. The contractor shall post an electronic format of the most current manual on the NAEP collaboration portal.

The contractor shall invite, in collaboration with the COR, web content contributors to the weekly web content meeting, and share discussion notes for web content meetings, approved by the COR, on the NAEP collaboration portal.

The contractor shall serve as the hub through which all web products must pass to meet content, quality and timeliness requirements. The WDOMS contractor—working closely with NAEP staff—will facilitate web content reviews and with other NAEP contractors and the States.

As needed, the contractor shall provide technical assistance by proposing updates to the current NCES Statistical Standards to include, for example, more expansive and state-of- the-art data visualization tools.

10.2.3 Manage Web Content Process for Contributed Content

The contractor shall design processes to support the creation of other approved content on the NAEP public websites. This includes content creation by any of the NAEP contractors, content updating, quality control, and maintenance of software applications for publishing and staging the content prior to posting on NCES servers. The contractor shall manage web content and processes to include but not be limited to:

· Providing systems to manage and control the content being created for all NAEP reporting and publication sites, including but not limited to the NAEP site at http://nces.ed.gov/nationsreportcard/; and the Nation’s Report Card (NRC) site at http://www.nationsreportcard.gov/.

· Providing operational support to NAEP, to the management and coordination of content reviews.

· Providing technical support for the coordination of NAEP web content with NAEP content providers and NAEP personnel.

· Providing updates to current CMS software until new CMS has been developed to support content creation, content review, and content coordination.

· Providing content coordination and review support to NAEP contractors and NAEP staff.

· Providing necessary technical assistance and support to NAEP’s existing private sites and new sites that may be developed to support the program.

The contractor shall review electronic documents, products and tools in various formats to ensure their quality and compliance with NAEP quality standards and guidelines and shall place the materials on a development server for NAEP review.

Content contributors should, in most cases, provide materials, such as PDFs, web-ready and tagged for 508-compliance. However, the WDOMS contractor shall be prepared to provide direct support in making products web-ready and compliant by converting materials to HTML, PDF, and other end-formats, as appropriate.

The contractor shall also provide technical assistance to other NAEP partners in identifying and incorporating new cutting-edge technology, approaches, and tools to be used on NAEP’s websites.

10.2.4 Maintain Quality of all NAEP Websites

The Contractor shall develop a Quality Management Plan (QMP) for review and approval by NCES, which outlines performance standards, a process for maintaining standards and conducting quality assurance reviews, identification of persons/positions who will perform quality assurance reviews, measurable quality metrics stated in quantitative terms that all NAEP content providers can use as internal targets, and defined items to be measured for quality control and acceptable standards. These standards, posted on the NAEP collaborative portal, shall communicate the level of editing and quality control expected of NAEP content before delivery to the contractor for final checking and before posting on the development server for IES/NCES approval. The QMP will propose metrics (strategic outcome, tactical output, and/or delivery) to measure performance, implementation success, adoption, and quality. Other goals of the QMP include:

· Maintain clear government visibility into schedule, technical performance, cost, risk, and issues resolution

· Provide meaningful reporting and analytics that provide NAEP with up-to-date and comprehensive information regarding performance

· Include approach to working collaboratively, as a single unit, with other contractors promoting transparency and cooperation across the organization The WDOMS contractor, working with the NAEP Webmaster, the NAEP staff, and with other NAEP contractors, shall prepare templates in the CMS for new content collections and reformat existing pages to current and evolving NAEP and NCES design standards. Typical web content design tasks shall include, but are not limited to converting content to HTML; updating outdated content already in HTML on the site, the creation of HTML tables, posting of videos, and the creation of graphics and navigational icons.

The contractor shall provide an exclusive Quality Assurance Team to closely examine existing and all new content to ensure that it is grammatically and stylistically sound and in keeping with the NCES Web Standards Guide. The team shall conduct in-depth testing of software applications developed by the contractor and for web page navigation according to industry standards and the requirements of the NAEP and NCES Webmaster and provide Quality Assurance Reports on all public website redesign releases and software development.

10.2.5 Use Web Analytics to Inform Web Content, Design and Architecture The WDOMS contractor shall develop an overall strategy/plan to inform web content based on metrics from web trend data using Google Analytics 4 (GA4) results from metadata tagging, SEO ranking, navigation patterns, etc., usability studies conducted with various audience groups, and analysis of public inquiries and FAQs. Metrics should be updated monthly via an accessible dashboard.

The contractor shall collaborate with the Publications, Outreach, Operations and Dissemination Support (PO2DS) contractor to maximize user engagement through targeted social media efforts. An example of historical site trend data can be found in Attachment X, WDOMS technical exhibit.

10.2.6 Conduct Usability Studies

The contractor shall conduct usability studies on all applications developed and hosted within the NAEP infrastructure, as well as the NAEP websites, as requested by the COR. The usability studies shall assess the proposed site designs to determine whether the site provides proposed users with sufficient access and ease of use of the system and shared throughout all NAEP partners. The usability studies will review the information architecture and site design of each system, evaluating all aspects of system use by the proposed user set. Focus groups of the user set will be convened to support this. Focus group meetings may be conducted in person or using electronic means. The studies may be performed using on-screen prototypes or using mock-ups.