Attachment 1 PWS DRAFT 02 April 2024.pdf

RFQ# 8298

2/26/24

BLANKET PURCHASE AGREEMENT (BPA)

PERFORMANCE WORK STATEMENT (PWS)

For Internal Revenue Service RFQ # 8298 Enterprise Systems Testing (EST)

Contractor Support Services 02/26/2024

1.0 INTRODUCTION AND BACKGROUND

The Internal Revenue Service (IRS), Information Technology, Enterprise Services, Enterprise Systems Testing (EST), hereinafter referred to as EST, requires contractor support to perform Systems Acceptability Testing (SAT), Integration Testing, Performance Testing, and other types of testing called out in the Performance Work Statement (PWS) for this requirement, and testing-related support services including Test Automation, Test Data Management and Program Management, for a wide universe of IRS-developed tax, administrative, and financial applications and systems. SAT, integration, performance, and other types of testing ensure applications and systems are functioning as intended, are compliant with business requirements, and free of defects prior to production implementation. Thorough, comprehensive, and independent testing of these IRS-developed applications and systems contributes significantly and tangibly to providing the highest possible level of quality and accurate service to the taxpaying public and enhancing the overall taxpayer experience in direct support of the agency's strategic plan. Test Automation support facilitates the efficiency and effectiveness of testing activities by automating manual testing processes and procedures, enabling testing to be performed more rapidly, expediting review of expected results, and conserving human testing band width thereby making it possible for EST to test larger quantities of applications and systems.

The Internal Revenue Service (IRS) has a long-standing commitment to rigorously testing application software prior to production deployment. Established in the 1960s, the IRS ‘Systems Acceptability Testing (SAT)’ methodology provides an independent assessment of a System’s fitness for purpose and traditionally includes elements of User Acceptance as well as white-box and black-box approaches to functional verification and validation. More recently, IRS established the ‘Final Integration Test (FIT)’ which focuses on verification and validation of requirements that span multiple systems by creating primary source inputs (triggers) and tracing them to their final outputs. The FIT is typically conducted annually prior to the beginning of the annual tax filing season and focuses on processing threads that support the Submission Processing and Customer Service business domains at IRS. The IRS is also developing and evolving its approach to Performance Testing. The IRS expects that waterfall or incremental delivery projects will continue to be tested through these existing test methods and processes with some evolution, or continuous improvement over time. The IRS is seeking support in the planning and execution of these tests, as well as with planning for and evolving the methodologies as appropriate.

The IRS leadership is also proactively rethinking how the organization supports modern frameworks, standardization, enterprise cloud capabilities, and is seeking to use agile or iterative development methodologies for new applications. The IRS Information Technology (IT) leaders embrace a more modern, agile pace of innovation and develop new applications using an agile method to achieve an organizational shift to a more modern technological approach to application delivery. The IRS is seeking expertise to assist in

DRAFT

adopting flexible agile methodology to deliver IT projects across the agency in critical areas.

The IRS Enterprise Services (ES) Enterprise Systems Testing (EST) within the IT organization supports the modernization of the tax processing systems. The EST organization is responsible for and supports aspects of software life cycle management, including requirements, design, development, testing, implementation, and transition to operations, as required by the IRS business capabilities associated with the business modernization initiative resulting from the implementation of Legislation.

A significant number of complex technical and business changes will require implementation.

This fast and vigorous approach to development and process engineering requires the IRS to employ the strongest and most aggressive federal workforce as well as competitive vendor resources that can lead the way in supporting requirements, design, development, testing, integration, deployment, and transition to operations of the associated systems. The EST organization is accountable and/or responsible for ensuring all applicable tests are executed, defects are tracked, and test results are communicated to all stakeholders.

The nature of testing, and its incorporation early and often requires a reconsideration of the traditional test processes as well as extensive automation of tests for functionality, performance, and accessibility.

1.1 EST Business Operational Environment

Historically, the EST program has been operating the functional testing, integration testing, and performance testing with contractual arrangements utilizing Work Requests that indicate the required work to be performed, period of performance of each project/system, contractor resources (skill mix and hours), work details, deliverables and due dates within a specific the Period of Performance. These Work Requests were changed when contractor resources required movement from one project to another based on the urgency of the needs or legislation that dictates testing. Deliverable due dates were also often changed due to late delivery of programs or testing environment issues. The intent of this BPA PWS is to issue individual task orders that were previously issued as Work Requests.

2.0 AWARD OBJECTIVES

The IRS requires contractor support to accomplish the following objectives:

i. Perform application and system testing functions and services for EST.

ii. Deliver applications to production with zero defects.

iii. Automate and streamline testing activities to occur earlier in delivery cycles reducing manual labor and increasing coverage.

iv. Promote more efficient Test Data Management (TDM) processes to ensure application software can be validated with minimal dependency on copies of sensitive production data.

v. Help the IRS transition its current testing practices to establish and continuously improve a disciplined practice that increases delivery velocity, quality, and job satisfaction while reducing cost and risk.

The IRS requires contractor support with extensive knowledge of IRS tax administration systems processing, requirements, and timelines. The EST organization is responsible for all aspects of Functional Testing, Integration and Final Integration Testing (FIT), Performance Testing and IT Security related testing. The contractor shall have a comprehensive understanding of testing to anticipate participant behavior and to drive the test strategy/design, test case development, data creation and test execution for validating current and new applications. The Contractor must have proven knowledge of the Test Strategy and Approach and a deep understanding of the complex development, test, and production environment infrastructure. This requires the vendor to be well versed in the centralized test approach for the EST organization.

The contractor shall conform to the prescribed IRS One Software Development Life Cycle (OneSDLC) Development Paths for designated projects, which employ some of the key attributes of the Waterfall and Agile methodology.

The contractor should demonstrate extensive and global experience in enterprise-wide independent verification and validation.

The contractor shall meet or exceed key functions and responsibilities under this PWS which include, but are not limited to, the following:

i. Adhere to IRS OneSDLC Project Management.

ii. Adhere to Program and System Risk Management.

iii. Adhere to security, privacy, financial management, and IT standards.

iv. Serve as the technical expert to review, comment, and develop multi-year work plans, associated resourcing, project objectives and program requirements.

v. Conduct planning, organizing, directing, and reviewing work of vendor staff; to include planning and carrying out vendor staff training and development.

vi. Ensure that the appropriate level of skilled resources is available and assigned to maintain appropriate workflow.

vii. Assign work to contractor staff based on priorities, selective consideration of the difficulty and requirements of assignments, and the capabilities of contractor employees.

viii. Manage cross-system and interagency dependencies and interfaces.

ix. Develop schedules and tasks for test planning, preparation, execution, and requirements validation for assigned system/project.

x. Conduct requirements analysis to include decomposing, analyzing commenting on, and achieving stability for testing against business and technical requirements.

xi. Create and develop program and project work products, artifacts, and deliverables.

xii. Analyze design documentation to surface gaps in clarity, completeness, and requirements coverage and to inform test case development.

xiii. Develop test strategy, user stories/scenarios, test cases, and associated test data.

xiv. Plan, execute, and monitor tests.

xv. Create and develop program and project work products, artifacts and deliverables.

xvi. Conduct quality reviews to ensure processes and work products/deliverables adhere to IT standards.

xvii. Provide test lifecycle activities for projects that are complex and highly technical in nature.

xviii. Evaluate work products and services to ensure narrative and metrics quality, and quality of test administration functions.

xix. Develop test documentation and ensure documents are placed in the designated repository.

xx. Coordinate and collaborate with internal and external stakeholders to deliver professional test products and services.

The contractor shall be a partner that is well versed in IRS business and tax administration with experience in testing and integrating large-scale IT systems to assist the IRS in meeting aggressive schedules and deadlines as mandated by the legislation.

3.0 SCOPE OF WORK AND DESCRIPTION OF TASKS

The Performance-Based requirements under this BPA are Firm Fixed Price (FFP) and Labor Hour (LH).

The IRS requires contractor testing support with an emphasis on test automation in both the traditional waterfall test approaches as well as more modern or agile delivery approaches.

The contractor shall provide performance of a wide range of thorough, comprehensive, and independent testing of tax administration and other critical corporate and financial applications and systems, leveraging testing related automated tools.

The contractor shall provide technical support to evolve test methodologies as the IRS embraces agile, Acceptance Test Driven Development (ATDD), Behavior Driven Development (BDD) and continuous integration including adopting modern DevOps principles. The IRS requires Contractor personnel for planning (including analysis and recommendations of modern industry best practice), and execution to support the delivery of working software, ensuring applications/systems are working prior to delivery of production.

• TASK 1, CONTRACT-LEVEL PROGRAM AND TASK MANAGEMENT SERVICES

(FFP)

• TASK 2, RAMP-UP KNOWLEDGE TRANSFER / TRANSITION-IN /

TRANSITION-OUT (LH)

• TASK 3, FUNCTIONAL TESTING (FFP/ LH)

• TASK 4, FINAL INTEGRATION TESTING (FIT) (FFP/ LH)

• TASK 5, PERFORMANCE ENGINEERING AND TESTING SUPPORT (FFP/ LH)

• TASK 6, TEST AUTOMATION SUPPORT (FFP/ LH)

• TASK 7, TEST DATA MANAGEMENT (TDM) SUPPORT (FFP/ LH)

• TASK 8, SURGE SUPPORT (FFP/ LH)

4.0 TASK 1, CONTRACT-LEVEL PROGRAM AND TASK MANAGEMENT SERVICES (FFP)

The Contractor shall provide dedicated program/project management to direct the support encompassing all work that may be executed under this PWS to include, but not limited to, management of project Contractor personnel; creation, monitoring and coordination of activities; project reporting; attending project related meetings; working collaboratively with the Government; assisting the Government in preparation of supporting documentation;

and producing all deliverables identified under the overall contract. Specific activities shall include, but are not limited to:

i. The Contractor shall provide contract/task management services including program management, task order-level reporting, Capability Maturity Model (CMM), managing Quality Assurance Surveillance Plan (QASP), quality assurance, ensuring adherence to all contract clauses, ensure compliance with IRS security policies, established IRS procedures and task order requirements.

ii. Maintain and provide GFP currently and formally possessed by the contractor at the end of each Base and Optional Periods of Performance.

iii. Provide Monthly Status Reports detailing accomplishments, goals for next reporting period, current and cumulative cost data, and issues and risks at the task order Level.

iv. Provide Period of Performance Summary Report Schedule and conduct a bi-weekly status briefing with EST leadership on the major activities related to this PWS to include progress to date, issues, risks, resource needs, and proposed recommended actions.

v. Provide Contractor Status Reports to the COR, summarizing activities to include dated accomplishments and quantified accomplishments and IRS milestones met.

vi. The Project Manager will establish a technical organization responsive to IRS managers and will direct development and execution of required performance as required by the task order.

vii. The Contractor shall recommend enhancements based on industry leading practices and industry leading innovations that facilitate automation, security, performance, and capacity testing on an on-going basis.

viii. The Contractor shall ensure effective and efficient delivery of technical services to accomplish the work in accordance with the defined Internal Revenue Manuals and IRS OneSDLC policies, processes, and procedures.

ix. The Contractor shall ensure comprehensive planning, execution, and IRS management visibility across all project activities.

x. The Contractor shall ensure new work may promptly commence without delay.

xi. The Contractor shall provide prompt administrative support for acquisition of new system accesses.

xii. The Contractor will provide the Monthly Status Report (MSR), a monthly invoice, and develop plans and forecasts for task resource management at the task order and funding level to ensure timely tracking of all costs.

xiii. The Contractor shall track all costs at the task order and project level and provide input to the IRS’s Earned Value Management (EVM) tracking for all project-funded work.

xiv. The Contractor shall maintain and provide Task Order Organizational Chart of all contract/task order employees, both alphabetically and by Task Order monthly.

xv. The Contractor shall provide a weekly background investigation status report to include all prospective employees. The report shall include descriptions and dates on the status of each prospective employee.

xvi. The Contractor management team shall participate in a Bi-weekly Program Status Review meeting, providing both written and verbal status of all task order, deliverables, issues, and staffing concerns.

xvii. The Contractor shall produce a Requirements Traceability Verification Test Matrix (RTVM) that will track task deliverables monthly. The Contractor shall keep this RTVM updated to include the previous month’s Deliverables.

xviii. The Contractor shall assist in expediting the on-boarding and separation process to include timely submission of required forms and completion of IRS mandatory training, as required.

DESIRED OUTCOME – The contractor will provide dedicated project management necessary to assist EST’s ability to plan and execute testing and plan for and evolve test methodologies. Provide effective management of all work executed under the Task Order.

The Contractor shall provide complete and accurate Period of Performance Summary Reports, Bi-Weekly Status Reports, Monthly Status Reports, Bi-Weekly Meeting, Cost Performance Reports, and Contractor Status Reports.

4.1 ORIENTATION BRIEFING

The contractor shall conduct Orientation Briefing activities as follows:

Within ten (10) business days of award, the contractor shall conduct an Orientation Briefing for the IRS. The IRS does not desire an elaborate Orientation Briefing, nor does it expect the Contractor to expend significant resources in preparation for this Briefing. Rather, the intent of the Briefing is to initiate the communication process between the IRS and Contractor by introducing key participants, explaining their roles, reviewing communication ground rules, and assuring a common understanding of contract requirements and objectives.

The orientation briefing shall be held at the IRS facility or by an alternative method, such as teleconferencing, and both parties shall mutually agree upon the date and time. The completion of this briefing shall result in the following:

i. The Contractor and IRS personnel who will perform work under this award shall be introduced.

ii. The IRS will show the applicable facilities to the Contractor if the Contractor will be performing work at the IRS site.

iii. The IRS may provide Government-Furnished Property (GFP) and Government- Furnished Information (GFI) to the Contractor at this time.

iv. Any issues concerning the Contractor clearances for Contractor personnel shall be discussed.

v. The Contractor shall demonstrate confirmation of their understanding of the work to be accomplished under this PWS.

vi. The Contractor shall provide the accounting period end dates to be used for the term of this award.

5.0 TASK 2, RAMP-UP KNOWLEDGE TRANSFER / TRANSITION-IN / TRANSITION-OUT

(LH)

Ramp-Up Knowledge Transfer is defined as the Contractor acquiring and understanding the necessary documentation, information, and processes related to any work in progress. This task includes meeting with the IRS Subject Matter Experts frequently to gather any tacit knowledge related to the EST tasks or programs in development. The Government shall provide detailed information on the methodologies used to perform testing and testing related support services on tax administration and other critical corporate applications and systems. The Government shall provide the Contractor with all required documentation including but not limited to Internal Revenue Manuals (IRM) governing the performance of application and system testing, as well as specific information concerning the applications and systems for which the Contractor will be required to provide support. The Government estimates this task will take approximately sixty (60) business days to complete after award.

The Contractor shall appoint a knowledge transfer manager to oversee the transfer of knowledge and ensure all areas are accounted for. If desired, this can be the Project

Manager (PM) or Project Scheduler. Prior to the transfer of knowledge, the Contractor shall develop a draft Knowledge Transfer Plan (KTP) and present it to the Government within the first fifteen (15) business days after award. The KTP shall identify the Contractor resources and roles involved in the transfer process, provide a list of risks and mitigation strategies for the transfer, and contain a detailed resource balanced project schedule developed with Microsoft (MS) Project. The project schedule shall identify tasks, dependencies, deliverables, and milestones. The project schedule shall be at a sufficient level of detail to track progress on a bi-weekly basis, (i.e., all tasks will be detailed to a level such that no task has duration of more than ten (10) workdays).

The Government shall have ten (10) business days to review the draft KTP plan and provide comments to the Contractor. Government and Contractor representatives will then meet to discuss the comments after which the Contractor shall incorporate the agreed-upon changes and deliver a final KTP within five (5) business days of the Government’s feedback.

The Contractor shall meet with the IRS bi-weekly (twice a month) to discuss the progress of the knowledge transfer. The Contractor shall show an updated MS Project Schedule and discuss any risks or concerns at this time.

DESIRED OUTCOME - The Contractor shall provide a Knowledge Transfer Plan that is accepted by the Government within thirty (30) business days and successfully complete the knowledge transfer in accordance with the Plan no later than thirty (30) business days after Government acceptance of the Knowledge Transfer Plan.

5.1 TRANSITION-IN (LH)

The Transition-In task permits for an effective transition from incumbent(s) Contractor to a new Contractor. The Government estimates this task will take approximately thirty 90 calendar days to complete from the award date. Successful transition can be defined by its capacity to maintain full continuity of operations, high productivity, and exemplary service quality. To ensure a successful, low-risk transition, the Contractor shall follow the approved Transition-In Plan submitted as part of their proposal and keep the Government fully informed of status throughout the transition-in period. During the transition-in period, the incumbent(s) is responsible for performance. This task would be worked concurrently with Task 2, Ramp-Up / Knowledge Transfer.

In addition to the transition actions in the approved Transition-In Plan, the Contractor shall perform the following activities during the transition-in period:

i. Assign personnel and submit security papers promptly to get the background investigations done quickly and gain access to IRS’s network.

ii. Create a project schedule at a sufficient level of detail to track progress on a weekly basis.

iii. Request access to tools and documentation.

iv. Review and understand all the technical and process documentation.

v. Review and understand the existing data and system security policies in place.

In support of this task, the Contractor shall be responsible for completing the following deliverables:

Deliverable Purpose Transition-In Checklist The Transition-In Checklist provides a vehicle to verify completion of the necessary transition-in activities. The Transition-in Checklist shall be developed by the Contractor and shall include all necessary activities to ensure an effective transition from incumbent(s) Contractor to a new Contractor were performed.

The Transition-In Checklist shall be signed by both, the Contractor and the COR after completion of the task.

5.2 TRANSITION-OUT (LH)

In the final option period of the Call Order, the contractor shall provide transition-out services. The contractor shall facilitate the transition of contracted activities and services to the EST and to the incoming contractor in accordance with the Government Approved Transition-Out Plan. The transition period shall not exceed sixty (60) calendar days.

The Contractor shall:

i. Minimize transition impact to the user community.

ii. Ensure no breaks in service availability.

iii. Maintain existing service quality and performance levels.

iv. Ensure a transparent and seamless transition.

v. Maintain support and meet delivery milestones of ongoing projects.

vi. Ensure that the IT security posture during transition is maintained at current levels without creating gaps and/or vulnerabilities.

In support of this task, the Contractor shall be responsible for completing the following deliverables:

Deliverable Purpose Transition-Out Plan The Contractor shall develop, document, and provide a Transition-

Out Plan no later than one hundred eighty (180) calendar days after the date of award, or earlier if directed by the Contracting Officer, and shall be updated annually. The Contractor’s Transition-Out Plan shall facilitate the accomplishment of a seamless transition from the incumbent(s) to the incoming Contractor and Cybersecurity personnel. The Contractor shall identify transition activities, schedules, and milestones for the turnover of work (e.g., operations, maintenance, engineering, training, asset management, and logistics functions) and identify how it will coordinate with the incoming contractor(s) and Cybersecurity personnel to transfer knowledge.

6.0 TASK 3, FUNCTIONAL TESTING (FFP/LH)

The Contractor shall provide EST with technical support to supplement federal staff in performing Functional testing which may include IT Security Related testing and Integration testing, of tax administration and other critical corporate and financial applications as described in this subsection.

Established in the 1960’s the IRS Functional Testing methodology provides an independent assessment of a System’s fitness for purpose and traditionally includes elements of User Acceptance as well as white-box and black-box approaches to functional verification and validation. Functional testing is typically focused on a single IRS application or system and may be scoped to test only the changes to that system, a subset of changes to the system, changes and some regression testing, or changes and full regression. In the case of new development all requirements are tested.

Functional testing will test the targeted system, and at least one interface test will be conducted with any system that directly interfaces with the targeted system. Interfaces may be bi-directional/real time or may consist of flat files either sent or received. The SAT test team is responsible for ensuring that the interface test is planned, conducted, and that the neighboring system confirms that data processes correctly and that data integrity is maintained.

This test type traditionally focused solely on functional requirements; however, in recent years IRS is expanding the scope to include any non-functional requirements that can be validated by the test team. Performance testing is typically not included, however, performance issues impacting the Functional testing effort are expected to be logged and tracked as defects.

The IRS will be using agile software development methodologies during Functional testing.

These methodologies are characterized by incremental and iterative processes where releases are produced in close customer collaboration with a focus on test driven development, and continuous integration to deliver code and functionality. Small teams will own a feature from start to finish – regardless of how many layers of the architecture it crosses and will develop a ready-to-ship feature package. Team sizes are expected to be between five (5) to eleven (11) people, with the expectation that more teams will be added to handle new products rather than increasing the team size. As teams are added, synchronization techniques (e.g., Common Iteration / Sprint Cadences, Product Release Planning) will be used to identify dependencies and risks.

The IRS Agile methodology is currently in use in EST. EST terminology aligns with the Web Apps Program terminology and team constructs. The Web Apps Program utilizes Federated Delivery Teams (FDTs) which operate under the direction of an IRS Release Manager which will provide all supervisory direction. The Release Manager coordinates activities across multiple FDTs for a single product or release. The Scrum Master will specifically coordinate development activities within the FDT's. Each FDT will be self-directed. Self-directed teams are defined as a group of people working effectively, efficiently, and collaboratively toward a common goal, which the team defines for achieving project results.

The Contractor must perform typical deliverable activities to include:

i. Participation in solution project planning, scheduling, collaboration, communication, and reporting using techniques appropriate for projects using agile methodologies.

a. Refining acceptance criteria that drive test case development.

b. Facilitating/ participating in customer feedback sessions to ensure solution alignment with customer needs/ requirements.

ii. Designing, developing, testing, and deploying test automation utilities or scripts.

a. Integrating with other IRS organizations and Contractors to support integration efforts as well as identifying best practices for facilitating integration for system engineering solutions. Supporting implementation and integration of engineered designs into operational systems using operational (Enterprise Operations (EOps)) processes and procedures.

b. Coordinating and performing demonstrations and conducting unit level, system, and user testing to be performed before the conclusion of each iteration.

iii. Documenting and reporting testing results using agile standards while adhering to IRS requirements relative to product quality.

iv. Identifying agile test results reports that are appropriate for IRS iterative delivery projects.

v. Utilizing a test-driven development approach, as applicable, to support the incremental development of high-quality code.

vi. Leveraging demonstrable, working software (specifically code to support test automation)

The Contractor may be required to provide technical support for some components of IT Security Related testing during the life of the award resulting from this PWS.

EST will:

i. Provide the Contractor with guidance, procedures, and test requirements conveyed via one or more task orders along with all other pertinent documentation required to perform this type of testing or

ii. Request that the Contractor develop and submit for the Government's review and consideration one or more potential methodologies, ideas, and concepts for conducting IT Security Related testing.

The Contractor shall ensure the availability of personnel technically qualified to provide support for IT Security Related testing should EST elect to execute task orders which include this subcomponent.

EST does not anticipate performing formal Security Certification and Accreditation (C&A) or Security Testing and Evaluation (ST&E) activities.

The Contractor may also be required to perform Integration testing (also referred to as End-to-End testing) in a Systemic Processing environment to integrate, test, and accept, software components until the entire system is operational and all agreed upon requirements have been validated. The analysis and approach shall include the development of in-scope business focused scenarios. The test shall be planned and constructed to validate across an integrated platform representing most systems impacted by a given initiative or group of related changes.

DESIRED OUTCOME - The Contractor shall provide EST with technical support for the performance of Functional testing to include Interface, IT Security Related testing and Integration testing of selected systems. The technical support provided by the Contractor for this task shall meet or exceed standards for acceptable performance as described in applicable sections of, and in the relevant attachments to, this PWS. The Contractor will ensure that EST can fully support the agile delivery model where all testing is incorporated as early into the software delivery process as possible and automated testing is incorporated into a Continuous Integration (CI) process.

6.1 REVIEW APPLICATION DOCUMENTATION

The Contractor shall review all application documentation provided to ascertain its consistency, clarity, correctness, and completeness in relationship to the requirements of the task order or other requirements transmission mechanism the Government may elect to adopt as its standard.

Iterative reviews of application documentation shall be performed as required. The application documentation, to be provided as GFI, will describe the functional and non-functional requirements of the application. The mechanism for documenting issues with documentation varies with each project; however, unresolved issues with baselined artifacts/documentation will be documented in the IRS ‘Defect’ tracking system of record.

The review will also include any documentation for any system which directly interfaces with the System Under Test that defines and/or clarifies any interface requirements (typically this is an Interface Control Document (ICD)). Written comments on the application documentation shall consist of the identification of discrepancies between the logical requirements as stated in the UWR (or other requirements transmission mechanism) and the functional requirements as expressed in the application documentation, and any other noted inconsistencies with appropriate technology standards.

DESIRED OUTCOME – The Contractor shall review all application documentation in relationship to the requirements of the UWR or other requirements. The Contractor shall document issues with documentation as defined by each project; however, unresolved issues with baselined artifacts/documentation will be documented in the IRS ‘Defect’ tracking system of record.

6.2 TRAINING

Contractor personnel shall attend training classes determined to be beneficial or required by the Government COR. The training will ensure that Contractor personnel maintain current knowledge and proficiency regarding changes to testing and requirements management methodologies such as changes to the governing IRMs, OneSDLC, etc.

The Contractor shall facilitate technology transfer from Government personnel to Contractor personnel. Training will be conducted on an "as needed" basis. Once the Government COR has notified the Contractor of the need for training, the Contractor will have five (5) working days to provide the Government COR with the names of attendees. The proposed Contractor personnel must be approved by the Government COR prior to attending the training.

Contractor personnel may be required to provide training to government employees on accepted EST methodologies for performing testing that may include testing of tax administration and other critical corporate applications and systems, and leveraging of testing related automated tools functions and services performed and managed by EST.

DESIRED OUTCOME – The Contractor shall ensure that its personnel maintain current knowledge and proficiency regarding changes to testing and requirements management methodologies such as changes to the governing IRMs, OneSDLC, etc. The Contractor shall facilitate technology transfer from Government personnel to Contractor personnel and provide training to government employees.

6.3 PERFORM TEST PLANNING AND DOCUMENT PLAN

The Contractor shall contribute to test planning activities which will result in the creation and subsequent maintenance of a documented Test Plan. The IRS is in the process of migrating to utilizing the Test Plan artifact in Engineering Test Management (ETM) as the official recorded Test Plan. However, the IRS also maintains a Word template, which is referenced from IRM 2.127.2, to create the Test Plan as a Word document.

The Test Plan shall specifically outline how the test is to be conducted, include a schedule of projected events, and identify expected deliverables, entry-criteria and exit criteria. The plan shall consist of detailed specifications, descriptions, and procedures for testing of software and/or processes. It shall identify all personnel, space, equipment, supplies, and hardware/software resources required to complete the test. The Test Plan should also describe automated testing and test verification techniques to be re-used and/or developed and deployed as part of testing. The Test Plan will also outline the approach to Test Data Management (TDM).

DESIRED OUTCOME - The Contractor shall provide a documented Test Plan, as defined in this subsection.

6.4 TEST ENVIRONMENT COORDINATION/MANAGEMENT

The Contractor shall coordinate the establishment and/or maintenance of test environments.

IRS application environments are typically physically designed by architects and engineers outside of EST; however, the Contractor must prepare and present requirements, conduct reviews of logical/physical designs, and occasionally create images that represent as-built environments or the connectivity/integration (normally at a logical level) between related environments.

DESIRED OUTCOME - The Contractor shall coordinate the establishment and/or maintenance of test environments. The Contractor shall prepare and present requirements, conduct reviews of logical/physical designs, and, if necessary, create images.

6.5 PREPARE TEST SCENARIOS/CASES/SCRIPTS/DATA AND PREDETERMINED

RESULTS

The Contractor shall prepare test scenarios (if applicable), test cases and test scripts in ETM.

The Contractor shall create or obtain any necessary data required to validate the requirements allocated to test plan. The IRS will provide the Contractor with examples of data and interactive procedures as well as information and guidance on prevailing IRS standards for creating test data and interactive procedures.

The Contractor shall create predetermined results for each test case and interactive procedure. Based on the requirements, these predetermined results shall define what is expected to occur for each test case or interactive procedure when executed via the application being tested. The predetermined results shall be used during testing and output review to ensure that the application performs as specified in the requirements documentation.

DESIRED OUTCOME – The Contractor shall prepare test scenarios (if applicable), test cases and test scripts. The Contractor shall create or obtain any necessary data required to validate the requirements allocated to test plan. The Contractor shall create predetermined results for each test case and interactive procedure.

6.6 PERFORM TESTING

Upon transmittal of application code to EST the Contractor shall perform the tests and review the output as defined in the Test Plan and Test Cases. This should start with confirmation that the expected version of the software is available in the test environment. The Contractor will conduct the tests from ETM either as a manual test or automated test and ensure that the necessary artifacts to prove the results are captured and stored.

The Test Case Execution Record (TCER) shall be generated to record the test results. The Contractor shall review these outputs to determine whether predetermined results have been satisfied and will record the result in TCER in ETM. When defects are identified the Contractor will perform analysis to determine if any previously ‘passed’ test cases will require regression testing when the correction for the defect is received, and the Contractor will perform the appropriate regression tests. This task includes monitoring the Exit Criteria defined in the Test Plan and periodic reporting of the progress of the test initiative towards meeting the defined Exit Criteria.

DESIRED OUTCOME – The Contractor shall perform the tests and review the output as defined in the Test Plan and Test Cases. The Contractor shall perform analysis when defects are identified. The Contractor shall perform defect analysis to determine if any previously

‘passed’ test cases will require regression testing when the correction for the defect is received, and the Contractor shall perform the appropriate regression tests.

6.7 REPORT DEFECTS

The Contractor shall open Defect work items in Engineering Test Management/ Engineering Workflow Management (ETM/EWM) in accordance with IRS standard procedures which currently include opening a cross-reference ticket via the IRS’s Internal Revenue Workflow Optimization, Requests, and Knowledge System (IRWorks)/ServiceNow (SNOW) tool.

The Defect shall include all information pertinent to the defect being reported and shall conform to applicable EST standards for Defects as detailed in IRM 2.127.2. When notified that the reported defect has been corrected, the Contractor shall revalidate and/or retest the condition identified in the Defect Report to confirm and document that the reported issue has been corrected.

The Contractor shall appropriately update the defect work item after the verification of the correction. A Defect shall be entered for each discrepancy discovered during review of documentation or test execution. Note that documentation defects are typically opened on issues found in base-lined versions of documentation once review cycles are completed.

DESIRED OUTCOME – The Contractor shall open Defect work items in Engineering Test Management/ Engineering Workflow Management (ETM/EWM) in accordance with IRS standard procedures and open a cross-reference ticket in the IRWorks/ServiceNow tool.

6.8 PREPARE WEEKLY TEST STATUS REPORT AND PARTICIPATE IN WEEKLY

STATUS MEETINGS

The Contractor shall submit or provide input to a weekly status report in the required format (which will be provided as GFI) to the designated EST official for each activity against the PWS resulting from this award. When the Contractor performs an activity that impacts multiple test projects, a separate status report may be required for each test.

This report shall identify and summarize all significant activity occurring during the week covered by the report, including status of requirements analysis and system documentation reviews, upcoming transmittals, test cases and/or scripts prepared, percentage of tests automated, meetings, status of Defects, and any other issues impacting the testing effort during that week.

DESIRED OUTCOME – The Contractor shall submit or provide input to a weekly status report and participate in weekly status meetings to present the weekly status report.

6.9 PREPARE END OF TEST REPORT/END OF TEST COMPLETION REPORT

Once testing exit criteria has been met and testing is considered completed, the Contractor shall create or provide input to the End of Test Report (EOTR)/End-Of-Test Completion Report (EOTCR) directly into ETM. The IRS also maintains a Word template, which is referenced from IRM 2.127.2, to create the EOTR/EOTCR as a Word document.

The report shall confirm that the application either does or does not conform to its requirements and documentation as modified before and during testing. The report shall provide the IRS with a comprehensive understanding of the extent of production readiness and quality of the tested application. The report shall include detailed information as to the types of tests conducted, the specific requirements tested, Requirements Traceability Verification Matrix (RTVM), lessons learned, deviations from the Test Plan, the defects identified during testing, how defects identified during testing were resolved, and shall also identify any defects unable to be resolved as well as those waived and/or deferred via the IRS waiver and deferral process which will be provided as GFI.

DESIRED OUTCOME - The Contractor shall provide EST with technical support to supplement federal staff in performing Testing of tax administration and other critical corporate applications. The technical support provided by the Contractor for this task shall meet or exceed standards for acceptable performance as described in applicable sections of, and in the relevant attachments to, this PWS.

7.0 TASK 4, FINAL INTEGRATION TESTING (FIT) (FFP/LH)

The Contractor shall provide EST with technical support for the performance of Final Integration Testing (FIT) of systems.

FIT is a formal test that treats traditional IRS Systems as sub-systems of a larger whole tax processing system and focuses on testing requirements changes that span across multiple systems. The IRS requirements process typically involves the business specifying individual changes to specific Systems and as such the overall requirement that spans multiple systems is often not captured and allocated. Typically, these are changes to support new legislative requirements and may result in changes to Tax Forms and publications.

Thus, the FIT process begins with reviewing changes to existing systems and new systems implementations that impact IRS Submission Processing and Customer Service business processes. FIT includes performing analysis of the changes primarily to systems already defined as in-scope and ensuring any new processing is exercised in the upcoming test cycle.

FIT may also include threads and scenarios that support other IRS business processes;

however, they are typically an integral part of completing the Submission Processing or Customer Service activity (for example some parts of Fraud Detection are included in FIT, but it is very closely tied to processing tax returns).

The FIT environment is typically refreshed annually (in July) with a copy of production data, and processing is executed following IRS cycles. The restoration of the data to the environment is followed with a series of processing steps referred to as ‘validation’ which is run in part to ensure that the restores were completed successfully. Any end-of-year close out and conversion processing is also executed in FIT because the success of that processing (which is typically changed for each annual IRS filing season) is critical to validating the updated tax processing software for the new year. The Annual FIT is performed prior to and during the beginning of the IRS filing season which is the main planned test for each year; however, FIT regularly conducts smaller focused tests or provides environment support for other test initiatives. FIT may also test production corrections based on ad-hoc requests.

The Contractor may be required to evaluate and implement automated tools and utilities designed to enhance and augment the efficiency and effectiveness of FIT. EST uses various automated test support tools and utilities such as the Rational Tool Suite including Engineering Test Management (ETM) for test requirements management, test data creation, automated output review, test progress tracking, and defect tracking for specific tests.

EST will:

i. Provide the Contractor with guidance, procedures, and test requirements conveyed via one or more task orders along with all other pertinent documentation required to perform this type of specialized testing support as part of FIT testing as set forth above or

ii. Request that the Contractor develop and submit for the government's review and consideration, one or more methodologies, ideas, and concepts for leveraging and evaluating automated tools to enhance the efficiency and accuracy of FIT testing performed by EST.

Contractor shall provide qualified staff for testing support, automated tools, and utilities.

The activities included in this task also include:

i. Design and/or review and update processing threads and/or scenarios documenting tests that flow across multiple systems to validate correct functional processing of new or updated business rules.

ii. Prepare a ‘Data Capture List’ to document required data-stores (flat-files, databases) and timing to be saved from production for restoration to the FIT test environment.

iii. Create and/or review Business Scenarios to be validated to ensure requirements that span multiple systems correctly support the desired business results. Typically, this is done in partnership with IRS business analysts.

iv. Create subsystem notebooks to document subsystem testing including UWR/requirements for the upcoming filing season, test environment, test schedule, point of contacts, issues, and action items.

v. Create subsystem analyst guides to describe the test data source, subsystem access, testing tools, test strategy, threads and requirements, input data creation, subsystem processes, Points of Contact (POCs) and lessons learned.

vi. Create a daily status report detailing overall testing status with cycle processing metrics, defect summary by priority and defect status type, and a list of open and closed defects during the FIT annual execution period.

EST does not anticipate performing formal Security Certification and Accreditation (C&A) or Security Testing and Evaluation (ST&E) activities as part of its broader role as the agency's Enterprise Test Authority. Notwithstanding, EST may perform selected aspects of IT Security Related testing as part of FIT, to include testing of login procedures and permissions and access rules established by application and web interface developers, or other aspects of IT Security Related Testing as further defined in individual task orders against the FIT sub-task. IT Security Related Testing, to the extent it may be performed by EST, is compatible with the methodology for FIT testing.

DESIRED OUTCOME - The Contractor shall provide EST with technical support for the performance of the FIT of tax administration systems as set forth in this Section. The technical support provided by the Contractor for this task shall meet or exceed standards for acceptable performance as described in applicable sections of, and in the relevant attachments to, this PWS.

7.1 REVIEW APPLICATION DOCUMENTATION

The Contractor shall review all application documentation provided to ascertain its consistency, clarity, correctness, and completeness in relationship to the requirements of the task order or other requirements transmission mechanism the Government may elect to adopt as its standard.

Iterative reviews of application documentation shall be performed as required. The application documentation, to be provided as GFI, will describe the functional and non-functional requirements of the application. The mechanism for documenting issues with documentation varies with each project; however, unresolved issues with baselined artifacts/documentation will be documented in the IRS ‘Defect’ tracking system of record.

The review will also include any documentation for any system which directly interfaces with the System Under Test that defines and/or clarifies any interface requirements (typically this is an Interface Control Document (ICD)). Written comments on the application documentation shall consist of the identification of discrepancies between the logical requirements as stated in the UWR (or other requirements transmission mechanism) and the functional…