Draft Performance Work Statment.pdf

DRAFT

Performance Work Statement

(PWS)

MODEL VALIDATION:

11/24/2023

Performance Work Statement (PWS)

Model Validation:

Vision Statement

The vision of this requirement is to obtain contractor support for the Department of Housing and

Urban Development (HUD), Federal Housing Administration (FHA), to provide the following critical financial/actuarial models:

1. Annual validation of 5 different types of Single Family Models, and

2. Annual validation of 5 different types of Commercial Models with 18 sub-categories.

1.0 General Information

1.1 Introduction

The Department of Housing and Urban Development (HUD), Office of Risk Management and

Regulatory Affairs (ORMRA) is seeking contractor support to annually validate the financial / actuarial models.

Task A - Single Family Models (MMI Fund)

The purpose of Task A is to validate the Single Family Forward and Reverse Mortgage Models and the TOTAL Model.

Task B - Commercial Models (GI & SRI Fund)

The purpose of Task B is to validate the Commercial Mortgage Models.

1.2 Background

Task A – Single-Family (MMIF)

The MMIF is a group of accounts of the federal government in which transactions associated with FHA’s guarantee programs for single family mortgages are recorded. FHA currently insures roughly 8 million forward mortgages with insurance-in-force of $1.1 trillion and 600,000 reverse mortgages with $112 billion in insurance-in-force under the Home Equity Conversion

Mortgage (HECM) program.

The intended uses of the Single-Family models to be validated are to:

A. Estimate FHA’s liability for loan guaranty for the forward and HECM programs for FHA’s financial statements.

B. Estimate selected elements of the MMIF capital ratio.

C. Develop budget estimates (credit subsidy rates and volume) and re-estimates; and, D. Estimate principal limit factors for HECM.

E. Enhance FHA’s ability to assess and manage risk, monitor trends, and evaluate policy with the capture of credit and loan application data.

Task B - Commercial (GI/SRI)

The GI/SRI is a group of accounts of the federal government in which transactions associated with FHA’s guarantee programs for Multifamily, Hospital, and Nursing Home/Assisted

Living/Board & Care facilities’ mortgages are recorded. FHA currently insures roughly 14,000 commercial mortgages with insurance-in-force of $100 billion.

The intended uses of the commercial models to be validated are to:

A. Estimate FHA’s liability for loan guaranty for the commercial loans for FHA’s financial statements.

B. Develop budget estimates (credit subsidy rates and volume) and re-estimates.

1.3 Constraints

A list of constraints related to this requirement is below.

Constraints include the following:

Document Number Title

Released Mandatory/

Advisory 44 USC §3541 et seq. Federal Information Security Management Act (FISMA, supersedes the Computer Security Act of 1987)

2002 M

44 USC §3601 et seq. E-Government Act of 2002 2002 M

Pub. L. 93-344, 2 USC §661 Federal Credit Reform Act of 1990 http://www.fms.treas.gov/ussgl/creditreform/fcratoc.html

1990 M

Pub. L. 104-106, 40 USC §1401 et seq.

Information Technology Management Reform Act of 1996

(Clinger-Cohen Act)

1996 M

Pub. L. 101-235, 12 USC 1709(f-

16)

HUD Reform Act, amending the National Housing Act 1989 M

Pub. L. 105-277, 44 USC §3504 Government Paperwork Elimination Act http://www.whitehouse.gov/omb/fedreg/gpea2.html

1998 M

Pub. L. 105-220, 29 USC 701 et seq.

Section 508 of the Rehabilitation Act of 1998 1998 M

Pub. L. 97-177, 31 USC §3901 et seq.

Prompt Payment Act 1982 M

Pub. L. 107-300, 31 USC §3321 Improper Payments Information Act of 2002 2002 M

OMB Circular A-125 OMB Circular A-125, Prompt Pay http://www.whitehouse.gov/omb/circulars/index.html

Latest version M

NIST Special Publication 800-18 "Guide for Developing Security Plans for Federal Information Systems"

Feb 1996 A

NIST Special Publication 800-26 "Security Self-Assessment Guide for Information Technology

Systems"

Nov 2001 M

NIST Special Publication 800-26, Rev. 1

"Guide for Information System Assessments and Program Reporting "

A

NIST Special Publication 800-30 "Risk Management Guide for Information Technology Systems" Jul 2002 M

NIST Special Publication 800-34 "Contingency Planning Guide for Information Technology

Systems"

Jun 2002 M http://www.fms.treas.gov/ussgl/creditreform/fcratoc.html http://www.whitehouse.gov/omb/fedreg/gpea2.html http://www.whitehouse.gov/omb/circulars/a127/a127.html

Document Number Title

Released Mandatory/

Advisory NIST Special Publication 800-35 "Guide to Information Technology Security Services" Oct 2003 M

NIST Special Publication 800-37 "Guide for the Security Certification and Accreditation of Federal

Information Systems"

Feb 1996 M

NIST Special Publication 800-47 "Security Guide for Interconnecting Information Technology

Systems"

Aug 2002 A

NIST Special Publication 800-53 "Recommended Security Controls for Federal Information

Systems"

Feb 2005 A

NIST Special Publication 800-53, Rev. 1

"Recommended Security Controls for Federal Information Systems"

Dec 2006 A

NIST Special Publication 800-65 "Integrating Security into the Capital Planning and Investment

Control Process"

Jan 2005 A

NIST Special Publication 800-84 "Guide to Test, Training and Exercise Programs for IT Plans and Capabilities"

Sep 2006 M

HUD Handbook 2400.25, Rev. 1

(or latest revision)

Information Technology Security Policy May 2005 M

HUD Handbook 1325.01, REV-01 Privacy Act Handbook Sept 1995 M

HUD Enterprise Architecture Policy www.hud.gov/offices/cio/ea/index.cfm

Current version

M

HUD Cost/Benefit Analysis Methodology, Volume I –

Methodology

Current version

M

HUD Cost/Benefit Analysis Methodology, Volume II – Workbook Current version

A

‘Project Leader Help Guide’ http://hudatwork.hud.gov/po/i/it/sd/guide/guide.cfm

Current version

A

The services identified in this PWS will adhere to the rules, regulations, laws, standards, and conventions identified by HUD as well as within the Federal Government. Functional constraints include the following:

1. All assumptions, data, methods and techniques used in MMIF models must comply with

12 USC 1708(a) (4), 12 USC 1711(f) (4), 12 USC 1735(f) (16), the requirements of the

Federal Credit Reform Act as implemented through OMB circulars A-11 and A-129, FASAB Handbook of Federal Accounting Standards and Other Pronouncements, as

Amended including TRs 3 and 6; FHA’s Model Risk Management Guidelines as well as model guidance including that issued by Federal financial regulators and OMB.

2. Model Development should begin with the precise definition of the models intended use, a thorough description of the reality to be modeled and a statement of the model’s design objective. The model should then be tailored to achieve its design objective. The resulting model should have three major components inputs, processes, and outputs.

3. Model inputs include data, assumptions, theory / mathematics, operational logic, and specific methodology. Model theory / mathematics describe abstractly the real-world process or system that produces the outcomes being modeled by substituting variables, parameters, constants, etc. for the real-world actions, tendencies, decisions, and events they represent. Model mathematics are the basis for developing model code and include all calculations performed during data preparation which are external to the primary model.

4. Data and assumptions must be evaluated as part of model development to document that they are supportable as model inputs. Models must be based on sufficient relevant and http://www.hud.gov/offices/cio/ea/index.cfm reliable data, if sufficient relevant and reliable data is not available, each deficiency and the range of its potential impacts on the accuracy and reliability of model outputs must be incorporated into the information used to inform decisions. Informed opinion may be used to supplement or extend available data if the basis of the stated opinion is articulated and documented adequately. All data inputs and material assumptions must be identified explicitly along with the evidence and rationale for their use and evidence that they have been approved by the model’s owner.

5. Model design, theory and logic must be supported by documentary evidence of consistency with data inputs, industry practice or published research and an assessment of alternatives leading to the selected methodologies, risk quantification techniques and specifications. Models should then then go through an iterative and documented process of fitting, testing and adjustment which culminates with a final model for which the accuracy and reliability of outputs is known and acceptable. Testing must include significance, in-sample fit, sensitivity, out-of-sample fit to determine the accuracy and reliability of model outputs, and other assessments as may be appropriate. The sensitivity of model results to each assumption must be tested to determine materiality and reasonableness.

6. Each model must have a performance tolerance, which is supported by a documented rationale, and procedures for monitoring model outputs against the performance tolerance and reviewing out-of-tolerance outcomes to identify causes and the necessity for adjustments to the model.

7. Model outputs (reports) must translate the estimates produced by the model into useful business information. Model outputs must be clear and comprehensible to decision-makers and include indications of the accuracy and reliability of the information. Reports should provide a range of estimates (including statistical confidence intervals, sensitivity analyses and alternative econometric and financial model specifications) rather than placing decision-makers in a "take-it-or-leave-it situation.

8. Model documentation should be the only input needed for each independent validation.

Model documentation should include a copy of the model’s annotated computer code, procedures for version control and controlling access to the model and its computer code and a list of personnel responsible for operating and maintaining the model.

9. Model methods and code should be evaluated using formal and quantitative techniques to a maximal extent.

Model Mechanics Integrity Evaluation

Model mechanics assessments are the evaluations of risk model practices results. These evaluations include rigorous tests to assess how well model practices are meeting quality standards. These including ensuring that models are:

1. Clearly documented and that such documentation, in combination with execution methods (to include statistical coding and use of spreadsheet templates), is sufficient to allow independent qualified staff to run current and maintain future model versions.

2. Free from errors in data extraction and transformation, arithmetic, logic, and coding.

3. In complete accordance with the model’s specifications.

4. Produced by efficient means, including optimization of data management and code as well as minimization of human and computing processing times.

5. From decision rules, algorithms and other formulations that can be easily altered for testing of alternative methodologies and program terms.

6. Not statistically vulnerable due to failure to pass significance tests, selection bias, and measurement error.

7. Using econometric specifications that have been tested against alternative data specifications and projections and are not vulnerable to multi-collinearity or fragility of variable choice.

8. Inclusive of diagnostic routines that provide as standard output, statistical summaries, and decision rules to identify material changes in model results and potential errors introduced by model changes.

Task B – Commercial (GI/SRI)

1. The resulting model should have three major components inputs, processes, and outputs.

2. Model inputs include data, assumptions, theory / mathematics, operational logic, and specific methodology. Model theory / mathematics describe abstractly the real-world process or system that produces the outcomes being modeled by substituting variables, parameters, constants, etc. for the real-world actions, tendencies, decisions, and events they represent. Model mathematics are the basis for developing model code and include all calculations performed during data preparation which are external to the primary model.

3. Data and assumptions must be evaluated as part of model development to document that they are supportable as model inputs. Models must be based on sufficient relevant and reliable data. If sufficient relevant and reliable data is not available, each deficiency and the range of its potential impacts on the accuracy and reliability of model outputs must be incorporated into the information used to inform decisions. Informed opinion may be used to supplement or extend available data if the basis of the stated opinion is articulated and documented adequately. All data inputs and material assumptions must be identified explicitly along with the evidence and rationale for their use and evidence that they have been approved by the models’ owner.

4. Model design, theory and logic must be supported by documentary evidence of consistency with data inputs, industry practice or published research and an assessment of alternatives leading to the selected methodologies, risk quantification techniques and specifications. Models should then then go through an iterative and documented process of fitting, testing and adjustment which culminates with a final model for which the accuracy and reliability of outputs is known and acceptable. Testing must include significance, in-sample fit, sensitivity, out-of-sample fit to determine the accuracy and reliability of model outputs, and other assessments as may be appropriate. The sensitivity of model results to each assumption must be tested to determine materiality and reasonableness.

5. Model development must include internal controls on modeling processes to ensure reliable performance. These controls must include: adequate reconciliation of model inputs and outputs to source data systems; verification that computer code is accurate to the models’ theory, mathematics, methodology and logical flow; end-user computing controls including version control, change control, data integrity control, access control and availability control; procedures for operating, maintaining and updating the model and verifying outputs.

6. Each model must have a performance tolerance, which is supported by a documented rationale, and procedures for monitoring model outputs against the performance tolerance and reviewing out-of-tolerance outcomes to identify causes and the necessity for adjustments to the model.

7. Model outputs (reports) must translate the estimates produced by the model into useful business information. Model outputs must be clear and comprehensible to decision-makers and include indications of the accuracy and reliability of the information. Reports should provide a range of estimates rather than placing decision-makers in a "take-it-or-leave-it” situation.

8. Model documentation should be the only input needed for each independent validation.

Model documentation should include a copy of the models annotated computer code, procedures for version control and controlling access to the model and its computer code and a list of personnel responsible for operating and maintaining the model.

1.4 Description of Service

HUD seeks a Contractor to provide Independent Validation support services for the financial / actuarial models. Each model validation and verification report will address the following elements at a minimum:

(For each task (A1 through A5) the contractor shall perform the following 8 steps as needed)

Step 1 Data and assumptions shall be evaluated to ensure that they are the most appropriate model inputs and that adequate controls are in place to ensure their accuracy and integrity.

Step 2 The actuarial, statistical, financial, and economic assumptions underpinning each model shall be evaluated from an empirical, analytical, and best-practice perspective to ensure they are acceptable and supportable considering the models intended business use.

Step 3 Validations shall evaluate the theory and mathematics of each model in terms of its accuracy and completeness with which the theory and mathematics replicate the actual phenomena being modeled and for logical flaws.

Step 4 Model code shall be evaluated to verify its accuracy according to industry best practices and standards, its fidelity to the corresponding theory and mathematics, its avoidance of unnecessary complexity and for processing efficiency. The Contractor will review all model code over a three-year cycle using formal verification techniques and software engineering standards.

Step 5 Performance tolerances shall be evaluated for reasonableness considering the adverse consequences of decisions based on incorrect or unreliable model outputs.

Step 6 All net present value and volume models shall be replicated for the upcoming yearly deliverables, starting with the LLG, using the most recently completed models. (In previous years, validation was done based on the model version aligned with the Budget.

This year and going forward, validation will be done on the current version of the models for the upcoming yearly deliverables, starting with the LLG.)

Step 7 Model outputs (reports) shall be evaluated for their strength at translating estimates produced by the model into clear, comprehensible, and useful business information of known accuracy and reliability.

Step 8 Such other evaluations as needed to identify all sources of and the full extent of model risk, identify model errors and other weaknesses and recommend appropriate action(s).

Detailed descriptions of specific tasks are provided in Section 5 Specific Tasks and Deliverables.

Each validation report shall include findings and/or recommendations together with an assessment of the materiality of each weakness.

Task B - Commercial (GI/SRI)

(For each task (B1 through B18) the contractor shall perform the following 9 steps as needed)

Step 1 Data and assumptions shall be evaluated to ensure than they are the most appropriate model inputs and that adequate controls are in place to ensure their accuracy and integrity.

Step 2 The actuarial, statistical, financial, economic, and other theories and assumptions underpinning each model shall be evaluated from an empirical, analytical, and best-practice perspective to ensure they are acceptable and supportable considering the models intended business use.

Step 3 Validations shall evaluate the theory and mathematics of each model in terms of its accuracy and completeness with which the theory and mathematics replicate the actual phenomena being modeled and for logical flaws.

Step 4 Model code shall be evaluated to verify its accuracy according to industry best practices and standards, its fidelity to the corresponding theory and mathematics, its avoidance of unnecessary complexity and for processing efficiency.

Step 5 Internal controls on modeling processes shall be evaluated for their strength in ensuring reliable performance.

Step 6 Performance tolerances shall be evaluated for reasonableness considering the adverse consequences of decisions based on incorrect or unreliable model outputs.

Step 7 All net present value and volume models shall be replicated for the upcoming yearly deliverables, starting with the LLG, using the most recently completed models. (In previous years, validation was done based on the model version aligned with the Budget.

This year and going forward, validation will be done on the current version of the models for the upcoming yearly deliverables, starting with the LLG.)

Step 8 Model outputs (reports) shall be evaluated for their strength at translating estimates produced by the model into clear, comprehensible, and useful business information of known accuracy and reliability.

Step 9 Such other evaluations as needed to identify all sources of and the full extent of model risk, identify model errors and other weaknesses and recommend appropriate action(s).

Each validation report shall include findings and/or recommendations together with an assessment of the materiality of each weakness.

1.5 Non-Personal Services

The Government will neither supervise Contractor employees nor control the method by which the Contractor performs the required tasks. Under no circumstances shall the Government assign tasks to, or prepare work schedules for, individual Contractor employees. It shall be the responsibility of the Contractor to manage its employees and to guard against any actions that are of the nature of personal services or give the perception of personal services.

If the Contractor believes that any actions constitute, or are perceived to constitute personal services, it shall be the Contractor's responsibility to notify the Contracting Officer (CO) immediately. These services shall not be used to perform work of a policy, decision making, or management nature, i.e. inherently Government functions. All decisions relative to programs supported by the Contactor shall be the sole responsibility of the Government.

1.6 Period of Performance

The period of performance for this requirement will be for one (1) base year for 12 months and four (4) 12-month option periods.

1.7 Place of Performance

The services to be performed under this contract shall be performed at the Contractor's facility.

1.8 Hours of Operation

The Contractor is responsible for providing services between the hours of 9:00 am to 5:00 pm, Monday thru Friday except for Federal holidays or when the Government facility is closed due to local or national emergencies, administrative closing, or similar Government directed facility closings. Weekly hours shall not exceed a forty (40) hour work week and a typical workday will be 8 hours each day Monday through Friday. The Government reserves the right to change hours of operation or restrict contractor access. Work outside of these daily hours is prohibited without

Contracting Officer approval. Government agencies will not be available during scheduled holidays, inclement weather, weekends, and after duty hours.

The Contractor shall always maintain an adequate workforce for the uninterrupted performance of all tasks defined within the contract when the Government facility is not closed for the above reasons. When hiring personnel, the Contractor shall keep in mind that the stability and continuity of the workforce is essential.

1.9 Special Qualifications

Subject Matter Experts

The Contractor shall provide the minimum of two SMEs. SMEs shall have three years’ minimum experience in similar positions involving social science research and in one or more of the following fields: social science research, distressed asset disposition, servicing timelines, mortgage measurement, monitoring, mitigation and management, mortgage modeling, model management, mortgage valuation, mortgage insurance valuation, and other fields related to mortgage insurance contract servicing. SMEs must also have specific experience with validating financial and econometric models and validating statistical model coding. The SMEs should also have solid practical knowledge and experience with scientific research principles, methods and practices, as applied to social science research; including a variety of research approaches, tools, and techniques, such as qualitative and quantitative data, statistical analysis, experiments, field surveys, case research and so forth. SMEs are considered Key Personnel by the Government and shall be listed as such in accordance with HUDAR 2452.237-70, "Key Personnel”.

1.10 Post Award/Kickoff Conference

The Contractor shall attend any post award conference convened by the contracting activity or contract administration office in accordance with FAR Subpart 42.5. The Government intends to convene a Post Award Conference with the Contractor within ten business days after contract award. The Contracting Officer will notify the Contractor of the specific date, location, and agenda within five business days after contract award.

1.11 Status Meetings

The Contracting Officer, Contracting Officer’s Representative (COR) and other Government personnel, as appropriate may meet periodically with the Contractor to also review Contractor performance, requirement status, etc. At these meetings, the Contracting Officer will apprise the

Contractor of how the Government views the Contractor's performance or progress of the requirement. The Contractor will apprise the Government of problems, if any, being experienced. Appropriate action shall be taken to resolve outstanding issues. These meetings shall be at no additional cost to the Government. Post Award Conference and subsequent meetings may be held via teleconference.

1.12 Contractor Travel

Prior to travel, the Contractor shall coordinate with and receive Government authorization from the COR for all travel. Reimbursement of travel costs will be in accordance with the Federal

Travel Regulation and in accordance with FAR 31.205-46. The Contractor shall travel using the lower cost mode transportation commensurate with the mission requirements. When necessary to use air travel, the Contractor shall use the tourist class, economy class or similar lodging accommodations to the extent they are available and commensurate with the mission requirements. HUD will not reimburse Contractor's local travel. Local travel is defined as travel within fifty (50) miles of Washington, DC. All other travel will be reimbursed on a cost reimbursable basis; no profit or fee will be paid.

1.13 Transition In

The Contractor shall provide 30 days of transition-In time if required.

1.14 Transition Out

The Contractor shall provide 30 days of transition-out time if required.

2.0 Definitions and Acronyms

2.1 Definitions

Business/Workdays - Every official workday of the week which are days between and including

Monday to Friday. This does not include public holidays and weekends.

Calendar Day - Any day of the week.

Contractor - A supplier or vendor awarded a contract to provide specific supplies or service to the Government. The term used in this contract refers to the prime.

Contracting Officer (CO) - A person with authority to enter into, administer, and/or terminate contracts and make related determinations and findings on behalf of the Government. Note: the only individual who can legally bind the Government.

Defective Service - A service output that does not meet the standard of performance associated with the Performance Work Statement.

Deliverable Anything that can be physically delivered but may include non-manufactured things such as meeting minutes or reports.

Government Furnished Property (GFP) - Government-furnished property means property in the possession of, or directly acquired by, the Government and subsequently furnished to the

Contractor for performance of a contract. Government-furnished property includes, but is not limited to, spares and property furnished for repair, maintenance, overhaul, or modification.

Government-furnished property also includes contractor-acquired property if the contractor-acquired property is a deliverable under a cost contract when accepted by the Government for continued use under the contract.

Secondary-Contracting Officer’s Representative (S-COR) - An individual designated by the

Contracting Officer to assist in providing technical direction and monitoring performance under the contract.

Primary-Contracting Officer’s Representative (P-COR) - An employee of the U.S.

Government appointed by the Contracting Officer to perform contract administration activities regarding technical issues. This individual has authority to provide technical direction to the

Contractor if direction is within the scope of the contract, does not constitute a change and has no funding implications. This individual does NOT have authority to change the terms and conditions of the contract.

Performance Requirements Summary (PRS) - A listing of the performance requirements under the contract that are to be evaluated by the Government on a regular basis, performance indicators for these requirements, performance standards for these requirement and surveillance methods to be used to determine if performance standards are met.

Performance Standard - The Contractor's performance level required by the Government.

Performance Work Statement (PWS) - A statement of work for performance-based acquisitions that describe the required results in clear, specific, and objective terms with measurable outcomes.

Physical Security - Actions that prevent the loss or damage of Government property.

Quality Assurance (QA) - Policies and procedures adopted by the Government to ensure that supplies and services acquired under Government contracts conform to the contracts quality requirements.

Quality Assurance Surveillance Plan (QASP) - A plan describing how the agency will survey, observe, test, sample, evaluate and document the Contractor's performance in meeting critical performance standards identified in the contract.

Quality Control (QC) - All necessary measures taken by the Contractor to assure that the quality of an end product of service shall meet contract requirements.

Service Contract - A contract that directly engages the time and effort of a Contractor whose primary purpose is to perform an identifiable task rather than to furnish an end item of supply.

Subcontractor - Any person, other than the prime Contractor, who offers to furnish or furnishes any supplies, material, equipment, or services of any kind under a prime contract or a subcontract entered into in connection with such prime contract, and any person who offers to furnish or furnishes general supplies to the prime contractor or a higher tier subcontractor. The

Government does not have privity of contract with a subcontractor.

Work Week - Monday through Friday, unless specified otherwise.

2.2 Acronyms

AQL - Acceptable Quality Level

BAA - Business Area Analysis Study

CFO or OCFO - Office of the Chief Financial Officer of HUD

CFR - Code of Federal Regulations

CIO or OCIO - Office of the Chief Information Officer of HUD

CO - Contracting Officer

COR - Contracting Officer Representative

CPO - Office of the Chief Procurement Officer of HUD

GAO - U.S. General Accounting Office

S-COR – Secondary-Contracting Officer’s Representative

P-COR – Primary- Contracting Officer’s Representative

HUD - U.S. Department of Housing and Urban Development

HUDAR - HUD Acquisition Regulation

HUD/ISG - Internet Services Group within the Telecom Processing Division

HUD Web - HUDs Intranet Web Site and related WEB pages.

IG or OIG - Inspector General (Office of)

IT - Information Technology

IV&V - Independent Validation and Verification

OCPO - Office of the Chief Procurement Officer

OIG - Office of Inspector General

OMB - Office of Management and Budget

PM - Project Manager Management

QA - Quality Assurance

QASP - Quality Assurance Surveillance Plan

QCP - Quality Control Plan

TBD - To be determined

U.S.C - United States Code

3.0 Government-Furnished Property and Services

The Contractor shall ensure accurate control and accountability of all Government-Furnished

Property in accordance with the terms and conditions of this contract. The Government will furnish, at no cost to the Contractor, the GFP shown below.

3.1 Government Services

No Government services will be provided.

3.2 Facilities

No facilities will be provided by the Government.

3.3 Equipment

The Government will provide laptops for individuals who need to interact with HUD’s systems and data to perform the tasks under this contract. The laptops will be returned to HUD at the end of the 30-day transition out period in the final year of the contract. Security for any Government-furnished equipment is the sole responsibility of the contractor and equipment must be returned in serviceable condition.

No other equipment will be provided by the Government.

3.4 Materials

Task A – Single Family (MMIF)

The Government (HUD FHA) will provide baseline data, including historical REO, DASP loan sale pools, Short Sales, CWCOT and Sale to Third Parties at Foreclosure for the fiscal years under examination. Some of this information may be provided to the Contractor by other HUD contractors.

Copies of required materials may be provided to the Contractor in hard copy or soft copy. All materials, whether provided directly by HUD or by contractors, will remain the property of the

Government and will be returned to the COR upon request or at the end of the contract period and all electronic copies must be deleted.

The Contractor may not share or otherwise use any of the materials for activities outside of this contract.

Task B - Commercial (GI/SRI)

The Government (HUD FHA) will provide baseline data for historical REO, loan sales and recoveries for the fiscal years under examination.

Copies of required materials may be provided to the Contractor in hard copy or soft copy. All materials will remain the property of the Government and will be returned to the COR upon request or at the end of the contract period.

3.5 Quality Assurance (QA)

The Government shall evaluate the Contractor's performance under this contract in accordance with the Quality Assurance Surveillance Plan. This plan is primarily focused on what the

Government must do to ensure that the Contractor has performed in accordance with the performance standards. It defines how the performance standards will be applied, the frequency of surveillance, and the minimum acceptable quality level.

4.0 Contractor-Furnished Items and Services

The Contractor shall furnish all facilities, equipment, and supplies required to perform the work under this contract that are not listed under Government-Furnished Property and Services.

4.1 Facilities

The Contractor shall furnish all facilities, equipment, and supplies required to perform the work under this contract that are not listed under Government-Furnished Property and Services.

4.2 Equipment

The contractor shall furnish all equipment, except as noted in 3.3 above.

4.3 Material

The contractor shall furnish all materials.

4.4 Contractor Responsibilities

The Contractor shall only conduct business with designated Government personnel listed as points of contact (POCs). Names of authorized personnel shall be provided to the Contractor by the Government, in writing, and updated as necessary throughout the contract period.

U.S. Government records, copies of original results and reports, verified original data, corrected data, and corrected supporting final reports which are maintained by the Contractor remain the property of the U.S. Government. These files/results must be surrendered to the COR.

4.5 Contractor Personnel

The Contractor shall provide a Contract/Project Manager who shall be responsible for the performance of work. An alternate shall also be designated to act in the absence of the

Contract/Project Manager. These Contractor personnel (main point of contact and alternate point of contact) are considered Key Personnel by the Government and shall be listed as such in accordance with HUDAR 2452.237-70, "Key Personnel. The Contract/Project Manager and alternate shall have full authority to act on all contact matters relating to daily operations of this contract. Accordingly, at a minimum, the points of contact shall have the technical knowledge of the requirement and be in the position to receive assignment, guidance, and direction from the P-

COR, S-COR and CO per HUDAR 2452.237-73, "Conduct of Work and Technical Guidance" and shall be allocated enough hours to the requirement to ensure successful performance. These points of contact shall each be an employee of the Contractor. An employee of a subcontractor is not acceptable for either of these positions. The Contract/Project Manager or alternate shall be available between the hours of 9:00am to 5:00pm Monday through Friday EST, except Federal

Holidays or when the Government facility is closed for administrative reasons.

4.6 Identification of Contractor Employees

All Contractor/subcontractor personnel shall wear company picture identification badges as to distinguish themselves from Government employees. When conversing with Government personnel during business meetings, over the telephone, or via electronic mail, Contractor

/subcontractor personnel shall identify themselves as such to avoid situations arising where sensitive topics might be better discussed solely between Government employees.

Contractors/subcontractors shall identify themselves on any attendance sheet or any coordination documents they may review. Electronic mail signature blocks shall identify their company affiliation. Where practicable, Contractor/subcontractors occupying collocated space with their

Government program customer shall identify their workspace area with their name and company affiliation, or at a minimum, "Contractor" after name.

4.7 Quality Control

The Contractor shall establish and maintain a complete quality control program that shall ensure services are performed in accordance with this contract. The Contractor shall develop and implement procedures to identify, prevent, and ensure non-reoccurrence of defective services. The Contractor's quality control program is how he assures himself that his work complies with the requirements of the contract. The Contractor shall provide the associated

Quality Control Plan to the Government as directed.

4.7.1 Quality Control Plan

The Contractor shall develop, maintain, enforce, and document a Quality Control Plan

(QCP). The QCP shall ensure the Government receives the level of quality that is consistent with the requirements specified in this contract. The QCP shall be sufficiently detailed to provide the Contractor's methodology for identifying and recruiting qualified personnel. The

QCP shall also provide the Contractor's methodology for resolving problems identified by the

Government during reviews conducted in accordance with its Quality Assurance Surveillance

Plan (QASP). The QCP shall also provide the Contractor's methodology for establishing an internal feedback system for support personnel, and for resolving problems identified by that feedback system. The QCP shall demonstrate and validate that the services or deliverables to be provided under the contract are completed with a level of quality that meets the minimum performance threshold established in the Government's QASP. The QCP shall address Quality

Management Approach, Quality Assurance, Quality Control and Quality Standards.

5.0 Specific Tasks

5.1 Task A – Single Family (MMIF)

For the Base Year, all three Single Family models will be validated (Forward + HECM +

TOTAL). Thereafter, the Commercial models below, plus up to 2 model validations/year for Single Family validation scopes, to be determined at the beginning of each performance year (Commercial + Forward + HECM, Commercial + Forward + TOTAL, Commercial + HECM + TOTAL).

A1 - The HECM Annual Review Agenda. FHA uses a risk-based approach to review of its single-family models that provides full coverage over a three-year cycle. Higher risk model methods and components receive higher attention than lower risk elements.

Consequently, the focus of model validation will change from year to year. FHA will work with the Contractor to develop an Annual Agenda by the beginning of each calendar year.

A2 - The FORWARD Mortgage Annual Review Agenda. FHA uses a risk-based approach to review of its single-family models that provides full coverage over a three-year cycle. Higher risk model methods and components receive higher attention than lower risk elements. Consequently, the focus of model validation will change from year to year. FHA will work with the Contractor to develop an Annual Agenda by the beginning of each calendar year.

A3 - The HECM Mortgage Insurance model is used to estimate HECM liability for loan guarantees, the HECM elements of the MMIF capital ratio, HECM budget re-estimates and the credit subsidy elements of budget development.

A4 - The FORWARD Mortgage Insurance model is used to estimate the forward liability for loan guarantees, the forward elements of the MMIF capital ratio, forward budget re-estimates and the credit subsidy elements of budget development.

A5 - The TOTAL-CHALLENGER Model (TCM) Review & Validation is used as a statistically derived algorithm developed by HUD to evaluate borrower credit history and application information. TCM is accessed through an Automated Underwriting System

(AUS) and is not an AUS itself. TCM provides a rank-ordered score of the relative risk of borrower default during the first 4 years of the mortgage resulting in claim. TCM provides AUSs with the “Accept/Refer” decision based on the borrower’s score relative to a policy cut point and other policy rules. Validation of the TOTAL model will be done in increments as information is made available. The scope of the TOTAL model validation task will be determined at the beginning of each performance year and will be developed by working with the vendor to determine specific tasks to be performed that year.

5.2 Task B - Commercial (GI/SRI)

For the Base Year, all three Single Family models will be validated (Forward + HECM +

TOTAL). Thereafter, the Commercial models below, plus up to 2 model validations/year for Single Family validation scopes, to be determined at the beginning of each performance year (Commercial + Forward + HECM, Commercial + Forward + TOTAL, Commercial + HECM + TOTAL).

FHA applies 5 different types of modeling with 18 sub-categories, frameworks to the commercial (multifamily and healthcare) portfolios:

I Logistic Regression Models

II Financial Simulation Model

III Monte-Carlo Simulation Model

IV Cash Flow Model

V Recovery on Assets Model

Within the logistic regression models, there are various sub-models. The differences between the sub-models are the variables and the types of loans or Risk Categories specified for each equation. But they are all structurally the same type of model, just with different variables.

The list of the model categories and the sub-categories are:

I. Logistic Regression Models

1. Multifamily Apartment Claim Model

B1 - Unassisted New Construction/Substantial Rehabilitation Default Model

B2 - Unassisted Refinance Default Model

B3 - Assisted New Construction/Substantial Rehab. and Refinance Default Model

B4 - Claim Transition Model

2. Multifamily Apartment Prepayment Models

B5 - Unassisted New Construction/Substantial Rehabilitation Prepayment Model

B6 - Unassisted Refinance Prepayment Model

B7 - Assisted New Construction/Substantial Rehab. and Refi. Prepayment Model

3. Residential Care Facilities (Nursing Home/Assisted Living/Board & Care fac.) Models

B8 - Section 232 New Construction/Substantial Rehabilitation Claim Model

B9 - Section 232 Refinance Claim Model

B10 - Section 232 New Construction/Substantial Rehabilitation Prepayment Model

B11 - Section 232 Refinance Prepayment Model

4. Hospital Loan Prepayment Model

B12 - Section 242 Prepayment Model

II. Hospital Loan Financial Simulation Model

B13 - Section 242 Claim Model

III. Monte-Carlo Simulation Model

B14 - Stochastic Simulation Model with data input from all the previous models.

IV. Cash Flow Models

B15 - Cash Flow Models (CFM) for Liability for Loan Guarantee (LLG)

B16 - Cash Flow Models (CFM) for Loan Loss Reserve (LLR)

V. Recovery on Assets Models

B17 – Recovery on Assets Models (ROA) for Liability for Loan Guarantee (LLG)

B18 – Recovery on Assets Models (ROA) for Loan Loss Reserve (LLR)

6.0 Deliverables

The Contractor shall complete all work and submit all deliverables as specified herein. The

Government reserves the right to make changes to delivery dates. For the deliverables listed below, completion is often hampered by the need to wait for inputs coming from external entities where schedules are beyond HUD’s control. It is understood that for the Single Family Forward and HECM models, completion is 10 weeks from the date the Government provides all required inputs to the vendor. For the Commercial models, completion is 16 weeks from the date the

Government provides all required inputs to the vendor. For the TOTAL model validation, the parties will work at the beginning of each performance year to define the scope of work, based on the information available and the amount of time remaining in the performance period not used for Single Family or Commercial model validations.

Task A – Single Family (MMIF)

The model validation reports will be delivered by the dates listed below. One report is due for each of the following tasks (A1 through A5):

Number Name Frequency Quantity Format Submit to

Due Date

A1 HECM Annual

Review Agenda

IAW Contract

Requirements

1 Electronic COR No later than

March 25 of each contract year the

HECM models are to be validated

A2 Forward Mortgage

Annual Review

Agenda

IAW Contract

Requirements

1 Electronic COR No later than

March 25 of each contract year the

Forward models are to be validated

A3 HECM Mortgage

Insurance Liability for Loan Guarantees

(LLG), Credit

Subsidy, and Loan

Volume Model

Mechanics

IAW Contract

Requirements

1 Electronic COR May 24 of each contract year in which the HECM models are validated, or 10 weeks from date of receipt of input from the

Government

A4 Forward Mortgage

Insurance (LLG)

Credit Subsidy, and

Loan Volume Model

Mechanics

IAW Contract

Requirements

1 Electronic COR June 9 of each contract year in which the Forward models are validated, or 10 weeks from date of receipt of input from the

Government

A5 Total-Challenger

Model (TCM)

IAW Contract

Requirements

1 Electronic COR December 9 of each contract year in which the

TOTAL models are validated

Each model validation report will, at a minimum provide the following:

a) A clear and comprehensive statement of the findings of the validation,

b) The support for each finding,

c) Assessment of the finding’s materiality, and

d) Recommendations for correcting each weakness.

Task B – Commercial (GI/SRI)

The model validation reports will be delivered by the dates below. One report is due for each of the following tasks (B1 through B18) throughout the Base Years and all Option Years:

Number Name Frequency Quantity Format Submit to

Due Date

B1 Unassisted New

Construction

/Substantial

Rehabilitation Default

Model

IAW Contract

Requirements

1 Electronic COR August 15 of each contract year in which the

Commercial models are validated, or 16 weeks from date of receipt of input from the

Government

B2 Unassisted Refinance

Default Model

IAW Contract

Requirements

1 Electronic COR August 15 of each contract year in which the

Commercial models are validated, or 16 weeks from date of receipt of input from the

Government

B3 Assisted New

Construction

/Substantial

Rehabilitation Default

Model

IAW Contract

Requirements

1 Electronic COR August 15 of each contract year in which the

Commercial models are validated, or 16 weeks from date of receipt of input from the

Government

B4 Claim Transition

Model

IAW Contract

Requirements

1 Electronic COR August 15 of each contract year in which the

Commercial models are validated, or 16 weeks from date of receipt of input from the

Government

B5 Unassisted New

Construction

/Substantial

Rehabilitation

Prepayment Model

IAW Contract

Requirements

1 Electronic COR August 15 of each contract year in which the

Commercial models are validated, or 16 weeks from date of receipt of input from the

Government

B6 Unassisted Refinance

Prepayment Model

IAW Contract

Requirements

1 Electronic COR August 15 of each contract year in which the

Commercial models are validated, or 16 weeks from date of receipt of input from the

Government

B7 Assisted New

Construction

/Substantial

Rehabilitation and

Refinance Prepayment

Model

IAW Contract

Requirements

1 Electronic COR August 15 of each contract year in which the

Commercial models are validated, or 16 weeks from date of receipt of input from the

Government

B8 Section 232 New

Construction/

Substantial

Rehabilitation Claim

Model

IAW Contract

Requirements

1 Electronic COR August 15 of each contract year in which the

Commercial models are validated, or 16 weeks from date of receipt of input from the

Government

B9 Section 232 Refinance

Claim Model

IAW Contract

Requirements

1 Electronic COR August 15 of each contract year in which the

Commercial models are validated, or 16 weeks from date of receipt of input from the

Government

B10 Section 232 New

Construction/

Substantial

Rehabilitation

Prepayment Model

IAW Contract

Requirements

1 Electronic COR August 15 of each contract year in which the

Commercial models are validated, or 16 weeks from date of receipt of input from the

Government

B11 Section 232 Refinance

Prepayment Model

IAW Contract

Requirements

1 Electronic COR August 15 of each contract year in which the

Commercial models are validated, or 16 weeks from date of receipt of input from the

Government

B12 Section 242

Prepayment Model

IAW Contract

Requirements

1 Electronic COR August 15 of each contract year in which the

Commercial models are validated, or 16 weeks from date of receipt of input from the

Government

B13 Section 242 Claim

Model

IAW Contract

Requirements

1 Electronic COR August 15 of each contract year in which the

Commercial models are validated, or 16 weeks from date of receipt of input from the

Government

B14 Stochastic Simulation

Model with data input from all the previous

Models

IAW Contract

Requirements

1 Electronic COR August 15 of each contract year in which the

Commercial models are validated, or 16 weeks from date of receipt of input from the

Government

B15 Cash Flow Models

(CFM)

Liability for Loan

Guarantee LLG

IAW Contract

Requirements

1 Electronic COR August 15 of each contract year in which the

Commercial models are validated, or 16 weeks from date of receipt of input from the

Government

B16 Cash Flow Models

(CFM)

Liability for Loan

Reserve LLR

IAW Contract

Requirements

1 Electronic COR August 15 of each contract year in which the

Commercial models are validated, or 16 weeks from date of receipt of input

B17 Recovery on Assets

Models (ROA) for

Liability for Loan

Guarantee (LLG)

IAW Contract

Requirements

1 Electronic COR August 15 of each contract year in which the

Commercial models are validated, or 16 weeks from date of receipt of input from the

Government

B18 Recovery on Assets

Models (ROA) for

Loan Loss Reserve

(LLR)

IAW Contract

Requirements

1 Electronic COR August 15 of each contract year in…