Attachment A - SOW.pdf

1 NASA Federal Acquisition Regulation Supplement 1835.016 and 1872.403 specifies that peer review will be the method used to evaluate and select research for funding. More information on policies for peer review can also be found in NPR 1080.1C, “Requirements for the Conduct of NASA Research and Technology (R&T).”

2 For more information on solicitation mechanisms refer to NASA Procedural Requirement 1080.1C, “Requirements for the Conduct of NASA Research and Technology (R&T).”

NASA Research and Education Support Services-II (NRESS-II) Statement of Work

February 20, 2020

INTRODUCTION

NASA’s aeronautics, human exploration, science, space technology Mission Directorates, as well as the Office of Science, Technology, Engineering, and Mathematics (STEM) Engagement (OSE), depend on NASA Research and Education Support Services (NRESS) to facilitate peer review program management on behalf of the Agency’s basic and applied research, technology development, and STEM engagement programs.

NASA funds intramural and extramural research and technology development, along with competitive STEM engagement program opportunities, primarily through solicited, peer-reviewed1 proposals that result in grants, cooperative agreements, contracts, or intergovernmental agreements. Solicitation mechanisms include NASA Research Announcements (NRAs), Announcements of Opportunity (AOs), and Cooperative Agreement Notices (CANs). NASA also receives and considers a small number of non-competitive proposals, such as unsolicited and sole-source proposals each year.2 Finally, NASA uses the NRESS contract to invite responses that do not result in financial awards via methods such as Requests for Information (RFI).

The life-cycle of the Agency’s peer review program, for purposes of NRESS requirements, consists of the following general processes: (1) development and solicitation of proposals, including related outreach and engagement programs in support of research; (2) proposal review; and (3) disposition of proposals following proposal review and prior to award. All decisions about proposals selected for award are made by NASA selection officials and all awards are made by authorized NASA procurement and federal assistance personnel with appropriate warrants.

Peer review processes and methods differ among the Mission Directorates. Small or Large reviews consisting of one or many (sub)panels, mail-in reviews, permanent committees or one-time panels are all features of the NASA peer review program. Peer review events occasionally are conducted via teleconference; peer review events are customarily conducted as face-to-face panel meetings, often in the Washington DC area, but also across the country. Panels usually consist of seven-to-ten individuals.

Rarely a large panel might consist of as many as 40 individuals. Student and engagement related research efforts may involve participation in conferences or similar events. Despite the variations, NASA peer review processes and methods have evolved over time to meet the predictable, repeatable preferences of their respective Mission Directorate communities.

Despite variations in methods and approach all NASA peer review program activities enforce a common

Agency-wide standard that peer review planning activities, events, deliberations, recommendations, and documentation shall remain confidential and free from conflicts of interest. NASA is committed to accomplishing timely peer reviews consistent with NASA’s goal of 240 days or fewer for the entire solicitation to award process. NASA is also committed to proposer, reviewer, and panelist diversity.

The NASA Solicitation and Proposal Integrated Review and Evaluation System (NSPIRES), a web-based information technology system, including software, enables NASA program officers or other officials, as well as contractor support personnel, to post research announcements or opportunities. NSPIRES enables intramural and extramural researchers and other stakeholders to view announcements and submit proposals. NSPIRES is designed and maintained to support solicitation of opportunities, proposal review, analysis, and evaluation, as well as proposal disposition. Refer to http://nspires.nasaprs.com for an orientation to the NSPIRES system. NSPIRES also interfaces with the Federal e-government system, grants.gov.

REQUIREMENTS SUMMARY

This Statement of Work (SOW) describes NRESS requirements, consisting of professional support services and information technology services as introduced in the following summary. In general terms NRESS and the related NSPIRES information system consists of the following:

Solicitation planning support, solicitation posting (advertising), and related communications; proposal receipt, retention, summarization, and distribution to program officers, panelists, and reviewers; peer review planning and meeting planning; managing and administering the logistics for NRESS events; panel member selection; panel member and reviewer support, including reimbursement of allowable travel and per diem expenses; summarizing and reporting on panel and review activities; preparation and generation of documentation needed to support procurement actions; and assisting with post-evaluation and post-selection communications to proposers and other stakeholders.

Comprehensive information technology program management and operations for the NSPIRES system, its grants.gov interface and any future interfaces specifically designed to enhance peer review programs and activities. Information technology program management and operations services includes: the sustainment, maintenance, operation, enhancement, of the NSPIRES web-based information technology system; the management and delivery of user access services and account provisioning; help desk services and related customer support; system documentation, user training materials and job aids;

comprehensive information technology security including appropriate protective controls, risk management planning and execution; and maintaining and assuring continuous operations in the event of system outages. Information technology support also includes workflow management for work requests under this scope.

The work breakdown structure that follows in the next section provides details about the summary requirements presented above. NASA customer organizations for this contract consist principally of NASA Mission Directorates, the Agency’s Office of STEM Engagement, the Space Technology Program under the Office of the Chief Technologist, and these components’ Center-based program and project offices. Additional information about the programmatic activities of the principal component users of this contract’s services can be found in the following links:

NASA’s Aeronautics Research Mission Directorate (ARMD) see https://www.nasa.gov/aeroresearch.

NASA’s Office of STEM Engagement (OSE), see https://www.nasa.gov/stem/

NASA’s Human Exploration and Operations Mission Directorate (HEOMD), see https://www.nasa.gov/directorates/heo/ .

NASA’s Science Mission Directorate (SMD), see http://science.nasa.gov/.

NASA’s Space Technology Mission Directorate, see https://www.nasa.gov/directorates/spacetech/home/

1.0 RESEARCH ANNOUNCEMENT AND SOLICITATION PLANNING AND

SUPPORT

The Contractor is responsible for the efficient administration of peer review program-related pre-solicitation and solicitation processes and activities, to include documenting the workflows, processes, and procedures for administering pre-solicitation and solicitation activities. As directed by NASA the contractor shall:

1.1. Support (virtual or in person) peer-review panels, as well as workshops and pre-solicitation meetings that lead to the development of NASA Research Announcements, Announcements of Opportunity, and Cooperative Agreement Notices. Support outreach activities for related student research programs, including conferences and presentations. Ensure the confidentiality and information security of information, notes, discussions, and decisions reached during such events.

1.2. Support the preparation of solicitation schedule and milestone charts and provide reporting tools to track progress of activities against schedules and milestones.

1.3. Plan, organize, and deliver communications to support scheduled events; provide all logistics support, including support for outreach events, student research events, solicitation planning meetings, pre-proposal meetings, and any similar planning workshops and operational meetings related to peer review planning.

1.4. In-person meetings are often in the Washington DC area, but may be anywhere in the country.

When not at NASA HQ or another known facility, make planning-related site visits to assure that venues are appropriate and conducive to upcoming events.

1.5. Assist in the development of solicitation documentation such as but not limited to: user-specific questions, FAQs, copyediting documents and checking for internal consistency.

1.6. Assist in drafting and editing solicitations and Federal Business Opportunities Announcements for subsequent announcement to stakeholder communities and post announcements, synopses, and solicitations and on NSPIRES (or subsequently developed information technology system), and/or via grants.gov and/or by other means suitable for reaching targeted communities of recipients, e.g., FBO.gov.

1.7. Maintain, and enhance a searchable web-based database, with attendant software, containing approved synopses, solicitations, and other announcements about research opportunities, proposals (organizations, team members and their roles and affiliations) reviews and reviewers (their review history and affiliations) for example via NSPIRES or subsequently developed information technology system). Search functionality software shall optimize search results to facilitate efficient ease of use and user acceptance by external and internal (NASA) users.

1.8. Promptly post solicitations to the searchable information technology system and Synopsis on Grants.gov on release dates and distribute notifications of solicitations (or other announcements) via existing directorate or division specific email distribution lists and/or by other means suitable for reaching targeted communities of recipients.

1.9. Support or assist in the preparation of reports, lessons learned, improvement ideas, or minutes resulting from these activities in order to enhance the management and operations of future events.

1.10. Support the analytics, including software, related to proposal evaluation prior to or during peer review by, for example, providing for key word search (tags) to enable more effective retrieval and categorization of proposals, analysis of and comparison of distribution of scores by different reviewers for a given proposal, for a given reviewer vs. the other scores given to that proposal, proposals selected vs final grade etc.

1.11 Support mid-term reviews, briefs, and informational presentations related to research awards, to include logistical and travel support.

1.12 Contractors supporting this scope that are required to go on official travel to designated countries, Russia and other high-threat locations, on behalf of or at the direction of NASA, shall receive pre- and post-travel counterintelligence threat briefings/debriefings per NPR 1660.1.

2.0 NOTICE OF INTENT TO PROPOSE

Solicitations generally require potential offerors to submit a preliminary statement such as a Notice of Intent (NOI) to propose or a Step-1 proposal. When NOIs or a Step-1 proposals are requested or required the contractor shall:

2.1. manage, operate, and maintain a user-facing web-based NOI (or Step-1 proposal) submission capability, including software, for example, via NSPIRES.

2.2. Receive and log NOIs (or Step-1 proposals) and any accompanying information that offerors submit in response to Program Announcements.

2.3. Create, manage, operate, and maintain search capabilities on the submission tool software that includes an ability to dynamically organize, summarize, sort, and group on designated NOI (or Step-1 proposal) characteristics, for example, using NSPIRES.

2.4. Within three working days after NOI submission deadlines specified by a solicitation, prepare and deliver NOI summary reports containing key characteristics of NOI data sets, e.g., itemize the names of proposing institutions, name the Principal Investigators, co-investigators, their titles, sub-disciplines, or other information as may have been specified in the instructions in the solicitation or Program Announcement.

2.5 Upon request, respond to NOIs (or Step-1 proposals) either by web-based tool or email, to encourage/discourage (or invite/disinvite) submission of subsequent proposals based on the NASA assessment of the Step-1 proposal, or to provide information to proposers, such as updates to the solicitation or posting of FAQs on the NSPIRES page for the program element.

3.0 PROPOSAL DOCUMENTS, PANEL, REVIEW, AND SELECTION DECISION

SUPPORT

Each research, exploration, technology, or STEM engagement solicitation contains detailed proposal preparation and content instructions. The contractor shall receive, maintain, record information on, and process proposals submitted to the government as follows:

3.1. Manage, operate, and maintain a web-based system (including software) that enables proposers to submit materials required by a solicitation, including proposers’ responses to Requests for Information (RFI), proposal cover pages, proposal summaries, and all other documents specified in proposal announcements.

3.2. Assure that the web-based system assigns a unique identification code to each proposal package to enable retrieval, edit, and review by proposers and by authorized NASA program officials, reviewers, and panelists.

3.3. Assure that the user-facing web-based system provides for submission of partial proposals or that enables proposal information to be submitted incrementally. Proposers shall be able to save incomplete proposals; retrieve, review, edit, and complete over time and enable proposers to verify proposal status via the user-facing web interface.

3.4. Manage, operate, and maintain the provided web-based system (including software) to enable NASA program managers and those whom they designate to assign reviewers to proposals and enter those assignments into the system for them when requested by NASA. Support sharing of documents and spreadsheets (e.g., of proposals and potential reviewers) using existing NASA tools (e.g., g-suite) and/or establish and deliver superior tools for this purpose. Maintaining records of organizational affiliations of all team members allows self-certification by peer reviewers and thus taking into account personal and organizational conflict of interest during peer review.

3.5. Assist NASA in screening each incoming proposal to determine compliance with eligibility criteria and consistency with general program solicitation requirements.

3.6. Establish and maintain the software capability to electronically sort, summarize, group on characteristics of proposal records. Lead, assist, or support the development of reports on characteristics of proposal populations and provide summary information about proposal records as requested.

3.7. Deliver an itemized summary of proposal cover information, formatted according to key characteristics unique to each proposal, within three working days after a proposal’s due date (if fewer than 250 proposals are received) or within five working days (if the number of proposals received exceeds 250).

3.8. Provide each proposal received in electronic format to the appropriate program officer(s) if requested, on an electronic medium requested by the program officer, within 24 hours after the proposal due date; or assist program officers in accessing, summarizing, and analyzing their own proposal data sets. In lieu of electronic documentation, provide for printed versions at the request of panelists.

3.9. Manage, operate, and maintain a software capability to make electronic proposals available to panelists, and/or to (external a.k.a., "mail-in") reviewers. The capability shall support reviewers’ needs to access proposals and evaluate (review) them. The capability shall provide for reviewers’ electronic responses.

3.10. Manage, operate, and maintain an electronic capability, including software, to make proposal reviewers’ evaluations available to panel chairs and panelists prior to and during panel events.

3.11. maintain, support, and enhance and electronic system(s), including software, to support creation, compilation, development, storage, and retrieval of reviewer of panelists findings and recommendations, panel summaries, technical evaluations, panel pre-decision discussion notes, and other artifacts from peer review events.

3.12. Maintain, and support NASA's tools for videoconferencing (e.g., webex) and document editing (g-suite) during peer review and/or establish and deliver superior tools for videoconferencing, and the creation, editing, and finalization of panel summaries during panel events.

3.13. Store panel and review findings, recommendations, and other results or information provided by the review panel (e.g., spreadsheets of scores), mapped to their proposals, in a secure information technology system. Reviewers’ findings and panel results shall be optimized for search and retrieval by authorized personnel.

3.14. Provide electronic copies (or paper copies only upon request) of panel evaluations, technical evaluations, and any other information provided by the review panel (e.g., spreadsheets of scores) to authorized NASA program managers and/or selection officials.

3.15. Support process of providing feedback to all Authorized Organization Representative (AORs) and to Principal Investigators about decisions made about their proposals.

3.16. Prepare and distribute packages containing proposal summaries, proposal review scores and statistical summaries, reviewers’ comments, and outcomes from review panel meetings to appropriate NASA officials within two working days after the conclusion of the panel meeting, unless directed otherwise by NASA.

3.17. Manage, operate, and maintain a web-based capability, including software, to compile, store, and make available to authorized NASA program managers, selection officials, and procurement officials all panel summaries and other panel artifacts (as requested by NASA) related to NASA’s selection decisions related to proposals reviewed during supported peer review program activities.

3.18. Within three working days after NASA announces its selection decisions to the contractor, support and assure the preparation and release of principal investigator packages (PI packages) which shall include such things as decision letters, panel or individual evaluations, debriefing letters, and requests for further information. Release approved PI packages, to Principal Investigators, and to AORs.

3.19. Support the preparation and release of electronic technical requirements (TR) packages to procurement personnel. TR packages include such things as the following: solicitation cover page, proposals; award letters, technical evaluations, transmittal information, proposal status summary, any revised documents from the proposer, and, in some cases, selection statements; summaries of the review process; justifications for awards, and certifications required for award. Support shall be supplemented, at a minimum, by reports to affected stakeholders about the completeness of TR packages and conformance of TR packages with NASA funding decisions.

3.20. Create, manage, operate, and maintain a quality assurance program for the preparation, assembly, release, and distribution of PI and TR packages.

3.21. Support post-review document management activities, such as supporting the submission of post-review and post-selection documents to an electronic peer review management system.

3.22. Maintain electronic records of the review history of each proposal consistent with NASA records management policy (e.g. which proposals were sent to whom, which reviews were submitted, and which requests for reviews were declined or ignored).

3.23. Maintain electronic records of the demographic information of each proposal, proposer, reviewer, and selection/awardee in support of NASA diversity efforts.

4.0 PANEL AND PEER REVIEW PLANNING, OPERATIONS, LOGISTICS, AND

CLOSE-OUT

The contractor shall support planning, operations, logistics, and close-out of peer review events, such as virtual panels, face-to-face panels, and mail-in reviews, as follows:

4.1. Maintain a web-based application and attendant software that contains records of and data from previous peer reviewers to support NASA solicitation managers, usually program officers, for the selection of prospective reviewers. Manage, operate, maintain, and enhance a prospective reviewer module to enable the addition of new (prospective) reviewers to the established population of known reviewers. Ensure that one feature of the application is to enable access and data entry by interested external researchers, engineers, educators and civil servants.

4.2. Support the recruitment of panelists and reviewers, in collaboration with NASA’s solicitation manager. Obtain each reviewer’s commitment to participate, to review designated proposals, to report and avoid conflicts-of-interest. Verify that each reviewer is registered in the web-based peer reviewer database indicated above.

4.3. Develop, manage, operate and maintain a conflict-of-interest program to avoid conflict-of-interest during selection of reviewers and during proposal solicitation and review. Alert NASA officials to potential or actual conflicts-of-interest.

4.4. Obtain conflict-of-interest and non-disclosure agreements electronically from all reviewers;

maintain a record of all conflict of interest and non-disclosure certification forms for all peer reviewers;

and ensure that all conflict of interest and non-disclosure agreements are valid and current. Within twenty-four hours of learning that a potential conflict of interest may exist, the contractor shall notify the appropriate NASA Official in accordance with conflict-of-interest avoidance plans. NASA will make any and all decisions related to conflicts of interest.

4.5. Develop and provide brief initial and refresher training to all appropriate personnel, including reviewers, on the implications of International Traffic in Arms Regulations (ITAR), Export Administration Regulations (EAR), and any other relevant foreign-national restrictions related to the handling of proposals.

4.6. Obtain ITAR/EAR agreements from all foreign-national participants and restrict applicable content as required.

4.7. As required in order to ensure an independent peer review process, provide a respected and technically qualified non-civil service peer review administrator who will be responsible for: (1) supporting the selection of a panel chair; (2) together with the chair, selecting the panel members; and

(3) facilitating the overall peer review process, including assignment of proposals to reviewers and the preparation of accurate and detailed critiques.

4.8. Produce and issue electronically participation requests to panelists and reviewers and, under NASA’s direction, develop electronic review instructions and evaluation forms that are consistent with the evaluation criteria in the Program Announcement.

4.9. Produce for each panelist and reviewer all necessary review information to include, but not limited to, pertinent forms, instructions, list of primary and secondary reviewer assignments, summary pages of proposals.

4.10. Contact panel members and make arrangements for their travel, per diem and honoraria, as appropriate. All travel arrangements shall be made by the contractor and shall comply with rate and fee limits authorized by the contract, per the Federal Travel Regulations (FTR); the Joint Travel Regulation (JTR), Volume 2, Department of Defense Civilian Personnel, Appendix A; or the Department of State Standardized Regulations (Government Civilians, Foreign Areas), Chapter 925. Special accommodations and reasonable exceptions to FTR guidelines shall be provided with authorization by the COR and/or CO, as required. Other explicit guidance to the contractor regarding reimbursement of travel and lodging costs to peer review panelists is summarized in Appendix B.

4.11. Organize, manage, and deliver all logistical and related onsite administrative support necessary to the conduct of panel meetings and events. Plan, contract for, and provide space for panel meetings.

Panel meetings are often outside of the Washington, D.C. area.

4.12. Provide computer support, presentation equipment, audio and video conferencing and file-sharing capabilities as requested to support external reviewers and panel members.

4.13. Facilitate the responsibilities of panel chairs and the activities of reviewers during peer review meetings and events. Sustain and facilitate debate, assist panel chairs in mitigating conflicts. Respond to panelist and panel chair requirements for real-time data gathering and support needed to keep evaluation and debate moving forward.

4.14. Support the activities of panelists during the creation of panel summaries and technical evaluations. Support user acceptance of electronic tools used during panel events (especially those that the contractor provides and maintains specifically to support the production and finalization of panel summaries and technical evaluations).

4.15. Facilitate user acceptance of electronic tools, including software, via pre-event user training, troubleshooting during panels, via periodic evaluation of the user experience, and continuous change and improvement.

4.16. Prepare interim reports during the panel reviews to assess conflicts-of-interest and the need for additional external reviewers.

4.17. Upon request, provide qualified Executive Secretaries to support panel meetings, and provide for minutes of the peer review process.

4.18. At the conclusion of each panel event or other review activity (such as mail-in reviews) administer a customer survey about the quality of the logistics provided. Solicit responses from panelists and reviewers as follows: (1) from the NASA official(s) responsible for panel events; and (2) from the participating panelists. Customer surveys shall evaluate the contractor’s performance relative to scheduling the event, pre-event support, logistics support, event management, event documentation, and post-event support. Customer surveys shall evaluate the contractor’s performance in terms of quality of accommodations, timeliness of disseminating information and written materials, and overall services provided to the attendee. Compile and deliver customer survey results semi-annually. A customer survey is required to be solicited from all participants described above. Contractor will exercise best professional effort to assure that responses are obtained. Maintain the records, including the survey responses, for each six-month period in a form and manner that is available for Government verification.

4.19. Identify non-respondents and calculate the response rate as a component of each semi-annual customer survey report. Provide an itemized list of non-respondents, along with a narrative explanation of the contractor’s effort to acquire the completed survey responses from non-respondents. There shall be at least three attempts to obtain each completed and signed form.

4.20. Conduct infrequent expedited peer reviews on accelerated schedules.

5.0 PROGRAMMATIC DOCUMENTATION, ANALYTICS, AND REPORTING

The contractor shall support the following documentation maintenance requirements, analytics support, and reporting requirements:

5.1. Produce Annual Program Task Books and Bibliographies that contain contract histories and technical information, NRA/AO information with respect to the proposal submitted, PI affiliation and address data, research description abstracts, etc., and which may be made available in hard copy or a standard electronic format and must be available in a searchable database available on the World Wide Web.

5.2. Support activities to facilitate and promote availability of space life sciences scientific and technical knowledge and expertise by collecting, creating, and disseminating space life sciences bibliographic data issuing from NASA and non-NASA sponsored research, and that augment and validate bibliographic material captured in the appropriate enterprise Task Book and bibliography.

5.3. Facilitate and support electronic search, retrieval, and analysis (analytics) of solicitations and proposals, panelists’ evaluations, panelists’ and proposers’ past participation in peer review events, and other system data objects as requested by the government. Examples of support provided and emphasis areas shall include, but shall not be limited to, the following examples:

5.3.1 Select and report proposals, grouped according to selected or not selected; categorize according to proposals submitted in response to NASA NRAs, CANs or AOs. Display or list at a minimum, name of PI, institution, identification of selection announcement, name of task, and related science discipline. For proposals that were endorsed for funding maintain and report information about the requested and recommended funding amounts by funding periods/years.

5.3.1. Provide post-award demographic statistics (e.g., awards by state; awards to minority businesses; to minority universities, faculty, and students).

5.3.2. Summary reports for NOIs received that shall include, at a minimum, data on Institution, Principal Investigator, and Discipline.

5.3.3. Report all proposals received in response to a solicitation and summarize the status and findings for each criterion on each proposal, as well as any overall summary ratings as may have been determined.

5.3.4. Prepare summary reports on all external reviews and panel meetings for appropriate NASA review (e.g., of panel composition).

5.3.5. For each solicitation, provide a summary report showing the proposals accepted and rejected.

5.3.6. Upon request ensure that selections are accurately and correctly entered into in NASA information systems to which the contractor has access, such as RAPTOR; report observed errors and mischaracterizations between peer review decisions and information contained in NASA information systems; report observed errors and mischaracterizations using an emphasis that the contractor deems to be in proportion to the nature and degree of errors or mischaracterizations observed; assist with reconciliation as requested by NASA.

5.3.7. All records, electronic or hardcopy, associated with solicitation planning, NOIs, proposals, peer review planning meetings, and peer review panel events, including notes about panel deliberations, and compiled selection decisions shall be stored and disposed of in accordance with NASA Records Management (NPD 1440.6I) and NASA Records Retention Schedules (NPR 1441.1E). Backup and redundant storage shall be utilized as appropriate.

5.3.8. Assist in compiling prompt, compliant answers to information requests such as FOIA inquiries and congressional inquiries (urgent and routine).

5.3.9. As part of the close-out for each solicitation, prepare a report for the Office of Legislative and Intergovernmental Affairs, including: a brief description of the solicitation, number of proposals received and number selected, total potential value of the selected proposals, period of performance covered by the grant, and a list of AORs and PIs.

5.3.10. Prepare on a quarterly basis two reports on the status of recent NASA Research Announcements (NRA) and Announcements of Opportunity (AO). One report shall be on Announced Competitions; the other shall be regarding the Selected Proposals. For each NRA and AO, the Announced Competitions report shall list: the NASA Center originating the requirement, a description of the research to be competed, type of competition (NRA or AO), solicitation number, and announcement date (day/month/year). The Selected Proposals report shall include: the solicitation number, Selection Date (day/month/year), calculation of calendar days from the announcement date to the selection date, total number of proposals, number of proposals containing NASA civil servant Full Time Equivalent (FTE) positions, and the number of civil servant FTEs included in each proposal. For each selected proposal, the report shall provide: the proposed cost, the period of performance, and the number of civil servant FTEs broken out into the following categories: scientists, engineers, medical personnel, and support personnel.

6.0 PROCESS OR SYSTEMS IMPROVEMENTS AND USER ACCEPTANCE

The contractor shall propose and solicit NASA’s endorsement of initiatives and activities to improve or enhance the quality, effectiveness, or efficiency of the peer review program, its processes and its underlying information technology systems, software capabilities, and services. Endorsed initiatives and activities will be carried out as mutually agreed in the following topic areas:

6.1. Meet with NASA discipline scientists, engineers, and STEM engagement specialists to investigate ways to continuously improve the peer review program and its underlying information technology services. Compile ideas, formulate recommendations, and support implementation activities as requested.

6.2. Investigate new or evolutionary information technology, including software, capabilities and services to enhance the performance, quality, or cost effectiveness of peer review program activities, including solicitation, review, and selection activities as described elsewhere in this statement of work.

Compile ideas, formulate and deliver recommendations. Such investigations shall consider such things as expanded use of teleconferencing, virtual, rather than in-person execution of peer review panel meetings, and the use of evolving social media to share and exchange information among the peer review program’s stakeholder communities.

6.3. Investigate the use of mobile devices to economize during the execution of panel events and peer reviews.

6.4. Investigate new or evolutionary architectures and software to enhance the NSPIRES system’s functionality, performance, security, and capabilities or to economize on the system’s cost of management and operations. Investigations shall lead, at a minimum, to recommendations for redesign or replacement to take advantage of contemporary developments in the information technology marketplace.

6.5. Replace and retire the NSPIRES system and its attendant software as the result of business cases and government decisions, if any, driven by the results of this section (Section 6.0). Archive NSPIRES data and system artifacts as dictated by the NASA’s data retention and retrieval policies in force at the time.

6.6. Initiate, manage, implement, and close-out the projects and initiatives in this section, as endorsed by NASA, deriving from peer review program improvement proposals and information technology improvement or replacement findings and recommendations.

6.7. Develop, manage, and perform communications services to promote or achieve user acceptance of improvement initiatives that affect peer review program activities, processes, procedures, or associated information technologies.

6.8. Manage the workflow lifecycle of work plan requests, authorizations, evaluations, and approvals between the NASA Directorates, COR, Contracting Officer, and the contractor.

7.0 INFORMATION TECHNOLOGY

As described in the introduction to this statement of work this contract requires the management, operation, and sustainment of a government-furnished, web-based, document-management system, including software, that supports announcements, posting of opportunities, the submission, evaluation, and selection of research proposals, the conduct of peer review events, peer review decision-making, and post-selection communications including document management (uploaded by proposers, peer reviewers, and file sharing with procurement). The contractor shall do the following:

7.1. Manage, operate, enhance, and maintain the NSPIRES system and its attendant software, or its successors, and support continuous and effective interfaces with external systems such as grants.gov.

The system shall be accessible to NASA users and other end users authorized by NASA, and the data it contains may only be used for purposes authorized by NASA.

7.2. Maintain comprehensive systems documentation, use cases, and business rules governing the system and its functionality.

7.3. Develop, maintain, improve and administer functional user instructional material, job aids, tutorials, FAQs, and related assistance material to enhance the user communities’ interaction with and use the system.

7.4. Develop, manage, operate, and sustain a comprehensive information technology security program to protect NSPIRES or its successor systems as specified by OMB Circular A-130, Appendix III, “Security of

Federal Automated Information Systems,” and related laws and regulations.

7.5. Management and Operation of the System – The contractor shall manage and operate the system in an efficient and responsible manner. The contractor shall maintain the government furnished and contractor developed system source code, which shall be accessible to NASA at all times. The contractor shall be fully responsible for system data, availability, reliability, security, data transfer and backup, contingency planning/disaster recovery and system risk management (e.g., a functional off-site back-up facility. This includes technical assistance to NASA and other authorized users).

7.6. Change Management - Any changes or enhancements to the system shall be made by the contractor, who shall be fully responsible for life cycle management of the software. It shall be processed under disciplined configuration management and life-cycle management processes, with final change decisions made by a NASA-led Configuration Control Board (CCB). Critical to these processes is accurate and timely documentation that is filed properly and readily available during the life of the system.

7.7. Requirements – All requirements for software change, including problem corrections, shall be documented. The NASA COR shall approve enhancements in cooperation with the CCB. Signatures authorizing completion of this phase shall be obtained and maintained.

7.8. Design – As enhancements to the system or replacements of the system are developed a design review shall be conducted for each release and the overall system design documentation amended to reflect the changes. Signatures authorizing completion of this phase shall be obtained and maintained.

Design documentation includes items such as screen images, report formats, processing logic rules, legal data values, etc.

7.9. Development and Test – The code shall be developed using generally accepted, commercially available tools. Formal test scripts and user scenarios, including expected results, shall be developed and maintained. Signatures for Test Readiness and Test Acceptance shall be obtained and maintained.

7.10. Operational Readiness Review - The system shall pass an Operational Readiness Review for each incremental release. Signatures approving the new software for production release shall be obtained from the COR prior to implementation.

7.11. Version Control – Each software release shall contain a version control number that was approved at the Operational Readiness Review.

7.12. Other Documentation – User guides and data models shall be developed as necessary.

7.13. Problem tracking – Problems with the software shall be formally tracked until resolution and the problem analyzed for any required system changes.

7.14. The system will be maintained in strict conformity to NASA Policy Directive (NPD) 2810.1E – NASA

Information Security Policy, and NASA Procedural Requirements (NPR) 2810.1A, Security of Information

Technology. The system will conform to NPD 2800.1B, Managing Information Technology and NPR

2800.1B, Managing Information Technology w/changes 1, 9/17/04 and Standard for Integrating

Applications into the NASA Access Management, Authentication, and Authorization Infrastructure EA-

STD-0001. This includes NASA provided background checks for personnel with the privileged or limited privileged access to contract systems.

7.15. Minimum Documentation – The contractor shall prepare, deliver and maintain the appropriate amount of documentation to assure system integrity but as a minimum the following:

7.15.1. Requirements Documents

7.15.2. Design Documents

7.15.3. Data Dictionaries

7.15.4. User and Operations Guides

7.15.5. Test Plans and Procedures

7.15.6. Configuration Management Items

7.15.7. System Security Plan as required by NPD 2810.1E and NPR 2810.1A

7.16. The following systems elements have these performance metrics: External NSPIRES availability=7x24x52, 98% of time; Internal NSPIRES availability=7x24x52, 98% of time; Web response time=3 seconds 98% of the time (Defined as the time from when the link is selected to when the full form is displayed (excludes Task Book reports); Client response time= 3 seconds 98% of the time

(Defined as the time from when the enter key is pressed to when the system responds with the requested data); Secure remote access=7x24x52,100% of the time.

7.17. Develop, maintain, or support publicly accessible internet and NASA intranet web sites to facilitate targeted communities’ access to announcements, to convey information about NASA’s research, technology, and STEM engagement programs, and other information useful to internal and external stakeholders in peer review program activities.

7.18. All required delivery of software, whether related to the existing NSPIRES system, its successors, or otherwise, shall be in source code form. Additionally, the source code for all software used or developed hereunder is the property of NASA and shall be maintained by the contractor and made available to

NASA for inspection, use and dissemination, at the discretion of NASA.

7.19 All other information, data, and licenses related to the systems required by this SOW are the property of NASA, and shall be maintained by the contractor and made available to NASA for inspection, use and dissemination, at the discretion of NASA, to include but not limited to: the NASAPRS.com domain, all NASAPRS.com email source code and records, all website source code, and all archives and backups of the above.

7.20The improvements or enhancements of the information technology systems and software capabilities described above are presumed to be incremental given the funding restrictions (i.e., the cost cap and the need for most of the funding to be devoted to the existing peer review process). If another contract is used to implement fundamental changes to the system, then the contractor will work with whomever is selected to carry out those fundamental changes to the system, sharing information, giving access, and making modifications as requested by NASA to facilitate seamless switch over or replacement of the NASA-owned contractor-maintained hardware or software.

8.0 ELECTRONIC COMMERCE

NASA is committed to the automation of manual processes to eliminate redundancy, and improve information and technology sharing among the NASA research community. As directed by NASA, the contractor will coordinate and implement the effective use of the Internet in support of electronic commerce as follows:

8.1. Apply the appropriate technologies to provide a standard “common face” for NASA Research Announcement posting to the Internet.

8.2. Support automated e-mail notification to institutions, through a voluntary subscription service, regarding announcement postings. (Currently done in NSPIRES)

8.3. Enable proposal and electronic certification receipt to include electronic submission of all proposal documents to a secure, private, managed central repository; manage and maintain a quality control process to ensure that all electronic information received is correct and complete.

8.4. Support NASA in its integration with all research and related e-gov initiatives (e.g., Grants.Gov and the Government Management Line of Business).

8.5. Support electronic workflow and related processing of proposals for distribution to internal and external review teams.

8.6. Provide electronic notification of selection for award and a process for assuring the quality of the content.

9.0 ELECTRONIC AND INFORMATION TECHNOLOGY ACCESSIBILITY

STANDARDS (SECTION 508)

The following is from Subpart B – Technical Standards, §1194.21, "Software applications and operating systems":

(a) When software is designed to run on a system that has a keyboard, product functions shall be executable from a keyboard where the function itself or the result of performing a function can be discerned textually.

(b) Applications shall not disrupt or disable activated features of other products that are identified as accessibility features, where those features are developed and documented according to industry standards. Applications also shall not disrupt or disable activated features of any operating system that are identified as accessibility features where the application programming interface for those accessibility features has been documented by the manufacturer of the operating system and is available to the product developer.

(c) A well-defined on-screen indication of the current focus shall be provided that moves among interactive interface elements as the input focus changes. The focus shall be programmatically exposed so that assistive technology can track focus and focus changes.

(d) Sufficient information about a user interface element including the identity, operation and state of the element shall be available to assistive technology. When an image represents a program element, the information conveyed by the image must also be available in text.

(e) When bitmap images are used to identify controls, status indicators, or other programmatic elements, the meaning assigned to those images shall be consistent throughout an application’s performance.

(f) Textual information shall be provided through operating system functions for displaying text. The minimum information that shall be made available is text content, text input caret location, and text attributes.

(g) Applications shall not override user selected contrast and color selections and other individual display attributes.

(h) When animation is displayed, the information shall be displayable in at least one non-animated presentation mode at the option of the user.

(i) Color coding shall not be used as the only means of conveying information, indicating an action, prompting a response, or distinguishing a visual element.

(j) When a product permits a user to adjust color and contrast settings, a variety of color selections capable of producing a range of contrast levels shall be provided.

(k) Software shall not use flashing or blinking text, objects, or other elements having a flash or blink frequency greater than 2 Hz and lower than 55 Hz.

(l) When electronic forms are used, the form shall allow people using assistive technology to access the information, field elements, and functionality required for completion and submission of the form, including all directions and cues.

Subpart B - Technical Standards §1194.22 Web-based intranet and internet information and applications.

(a) A text equivalent for every non-text element shall be provided (e.g., via “alt”, “longdesc”, or in element content).

(b) Equivalent alternatives for any multimedia presentation shall be synchronized with the presentation.

(c) Web pages shall be designed so that all information conveyed with color is also available without color, for example from context or markup.

(d) Documents shall be organized so they are readable without requiring an associated style sheet.

(e) Redundant text links shall be provided for each active region of a server-side image map.

(f) Client-side image maps shall be provided instead of server-side image maps except where the regions cannot be defined with an available geometric shape.

(g) Row and column headers shall be identified for data tables.

(h) Markup shall be used to associate data cells and header cells for data tables that have two or more logical levels of row or column headers.

(i) Frames shall be titled with text that facilitates frame identification and navigation.

(j) Pages shall be designed to avoid causing the screen to flicker with a frequency greater than 2 Hz and lower than 55 Hz.

(k) A text-only page, with equivalent information or functionality, shall be provided to make a web site comply with the provisions of this part, when compliance cannot be accomplished in any other way. The content of the text-only page shall be updated whenever the primary page changes.

(l) When pages utilize scripting languages to display content, or to create interface elements, the information provided by the script shall be identified with functional text that can be read by assistive technology.

(m) When a web page requires that an applet, plug-in or other application be present on the client system to interpret page content, the page must provide a link to a plug-in or applet that complies with

§1194.21(a) through (l).

(n) When electronic forms are designed to be completed on-line, the form shall allow people using assistive technology to access the information, field elements, and functionality required for completion and submission of the form, including all directions and cues.

(o) A method shall be provided that permits users to skip repetitive navigation links.

(p) When a timed response is required, the user shall be alerted and given sufficient time to indicate more time is required.