FINAL_CCMI_BRD_5834_vf_5-29-2009.txt

US Department of Housing and Urban Development Office of Public and Indian Housing Real Estate Assessment Center

Business Requirements Document for the Asset Management Review (5834) Module of the Comprehensive Compliance and Monitoring Initiative (CCMI)

FINAL

System:

PIH-REAC

Subsystem:

CCM

Component:

Asset Management Review (5834) Module

Releases:

Database Release:

REACS

Doc Type:

Business Requirements Document

Doc Date:

May 29, 2009

Doc Author:

Abt Associates Inc.

Doc Number:

2.0

Business Requirements Document for the Asset Management Review (5834) Module of CCMI.

Approved by:

Stephen Murphy Date Daniel Wilkerson Date

T

ABLE OF

C

ONTENTS

1. General Information

1.1 ABOUT THIS DOCUMENT

1.2 APPROACH TO DEVELOPING THE BRD

1.3 CCMI ENVIRONMENT

1.4 CUSTOMERS

1.5 REFERENCES

2. Development of the Asset Management Review (5834) Module

2.1 MANAGEMENT REVIEW FOR PUBLIC HOUSING PROJECTS (HUD-FORM 5834)

2.2 INITIAL MS EXCEL-BASED 5834 PROTOTYPE AND MS ACCESS VERSION

2.3 ASSUMPTIONS, CONSTRAINTS AND GUIDANCE

3. 5834 Checklists, Business Rules, and Protocols

3.1 5834 CHECKLISTS

3.2 EXAMPLE CHECKLIST

3.3 WORKSHEETS

3.4 BUSINESS RULES AND PROTOCOLS

3.5 SCORING AND GRADING

3.6 5834 REVIEW TYPES

4. Sampling Methodology

4.1 CCMI METHODOLOGY

4.2 5834 MODULE METHODOLOGY

5. Core Review Requiremen ts

5.1 CCMI

5.2 5834 MODULE

6. Analysis of CUFF Systems

6.1 APPLICABLE CUFF SYSTEMS

6.2 CUFF SYSTEMS REQUIREMENTS INCORPORATED INTO CCM

6.3 ADDITIONAL REQUIREMENTS INCORPORATED INTO CCM

6.4 COMPARISON OF CCM & CUFF SYSTEMS

Appendix A: Checklists Appendix B: Worksheets Appendix C: Protocols Appendix D: Business Rules

1. General Information

This chapter provides general information on the content of this document, the approach the team used to develop the Business Requirements Document (BRD), the source documents, and the anticipated users of the Asset Management Review (5834) Module.

1.1 A

BOUT

T

HIS

D

OCUMENT

This document describes the business process, business rules and business requirements for the interim, Microsoft Access-based 5834 prototype model and the system-based 5834 Module of HUD’s Comprehensive Compliance and Monitoring Initiative (CCMI). The 5834 module is the electronic version of the assessment form, Management Review for Public Housing Projects (HUD-5834) which will be used to evaluate a project’s management performance.

The interim 5834 Model is a prototype of the business requirements, logic, and functionalities that are currently being developed in the Comprehensive Compliance and Monitoring (CCM) system. Since the prototype is an interim solution, there are certain requirements that pertain only to the prototype that would not necessarily be found as part of the CCM system build for the 5834 Module. In this BRD, the Abt team has identified those requirements that pertain only to the 5834 prototype model versus those requirements that apply to both the prototype and the CCM system.

Prototyping the system design provides the following advantages: 1) allows the CCMI business area the flexibility to test and re-design with greater ease; 2) demonstrates via a working model how functionality could work in CCM/MAPS, when fully functional; and 3) provides Field Offices with a mechanism to test the 5834 checklists and protocols for accuracy and completeness.

The BRD provides the following information on the 5834 Module:

HUD data sources and data elements required for the 5834 review.

The sampling methodology and core review requirements for the 5834 review.

Standard checklists, worksheets, business rules and protocols that encompass the 5834 review.

Note: Certain requirements associated with Pre-Review preparation (e.g., scheduling the review, notifying the PHA, assignment of HUD staff to team, and travel fund obligation) and Post- Review reporting (e.g., sanctions, corrective actions, and final report issuance) have already been addressed as part of the CCM system and are outside the scope of the 5834 prototype model.

Adjustments that need to be made to the CCM system to address the 5834 requirements are detailed in a separate document, Asset Management Review (5834) Prototype and CCM System Requirements.

1.2 A

PPROACH TO

D

EVELOPING THE

BRD

For each module, the CCMI Business Support contract requires a number of smaller deliverables that have similar due dates and the same audience as the BRD deliverable. In the development of the BRD for the 5834 Module and the development of the other CCMI Modules, the Abt team has proposed to group all required deliverables as follows:

Deliverable 1: Business Requirements Document (BRD) Deliverable 2: Process Efficiency and Effectiveness Analysis [Due 50 days after assignment] Deliverable 3: Prototype and CCM System Requirements [Due 50 days after assignment] Deliverable 4: Reports, Tools, and Analysis [Due 20 days after delivery of the final prototype model]

For Deliverable 1 — BRD, all requirements (formal deliverables and additional information) are combined in the BRD so that the end users – HUD, the Work Groups/Focus Groups, and HUD’s IT development partner – have the requirements for each Module in one document. The table below identifies the contract deliverables contained in the BRD, the BRD chapter(s) where the deliverable is addressed, and the due dates for each deliverable as specified in the contract.

The formal contract deliverables transmitted for April 14, 2009 as part of the Draft BRD are:

Contract DeliverableBRD ChapterTimeframeDraft DueFinal Due1Business RequirementsChapters 1 thru 3, Appendices30 days after assignment4/14/20095/12/20092Core Review RequirementsChapter 430 days after assignment4/14/20095/12/20093Sampling MethodsChapter 550 days after assignment5/12/20096/2/20094Analysis of Cuff SystemsChapter 620 days after final process model7/22/20098/19/2009Deliverable 1: 5834 Module -- Business Requirements Document (BRD)

Chapter 1: Business Requirements Document Chapter 2: Development of 5834 Module Chapter 3: 5834 Checklists, Business Rules, and Protocols Appendices A thru D: Checklists, Worksheets, Protocols, and Business Rules

1.3

CCMI

E N

VIRONMENT

HUD’s Compliance Monitoring Initiative (CMI) sets forth the overall framework and requirement for the Department’s compliance monitoring as published in the HUD Monitoring Desk Guide. The Guide covers the basics for compliance monitoring, from risk assessment to conducting routine, remote, and on-site monitoring through the correction of findings and concerns arising from compliance reviews.

The Office of Public and Indian Housing, Comprehensive Compliance and Monitoring Initiative (PIH-CCMI) complements the Department’s CMI. PIH-CCMI provides a framework to standardize the monitoring processes employed for the major PIH program areas and sub-areas.

The CCMI process consists of an on-site and/or remote review and pre- and post-review reporting that make use of data already available in PIH systems to assist in the review analysis.

For each monitoring area, the review procedures will reflect the movement towards an asset management model. That is, the review procedures will reflect those procedures that are at the PHA level, the project level, and at both the PHA and project level. Once fully implemented, PIH-CCMI will consist of the following modules and review areas. The Abt team is tasked with developing prototypes for each module for the base and option year period of the contract.

Base Year

1) 5834-A and 5834-B: Occupancy Monitoring I (RIM, EIV/UIV)

2) 5834-D: Procurement

3) 5834-E: Development

4) 5834: Asset Management Reviews

5) 5834-A2: Occupancy Monitoring II (Tenant Selection and Assignment, Leasing, Lease Enforcement, Grievances, etc.)

6) 5834-H: Demolition/Disposition

7) HOPE VI Monitoring

8) Capital Fund Program Monitoring

Option Period

9) Grant Monitoring Reviews (for 1 generic and up to 3 specific grant monitoring types)

10) Fair Housing and Equal Opportunity (FHEO)

11) Housing Choice Voucher: Occupancy Monitoring (RIM, EIV/UIV)

12) Section 8 Management Assessment Program (SEMAP) Certification

13) Section 8 Management Assessment Program (SEMAP) Quality Control

Each year, HUD establishes annual compliance priorities for each program area. Through a risk assessment process and as documented in the Annual Management Plan, PIH identifies both the PHA and the compliance areas for review. With this information, field office staff is able to schedule, complete, and track these reviews in the CCM system. The CCM system allows the field office review team to set-up the review and alert the PHA of the upcoming review. Once scheduled, the review team will conduct the review, which normally requires a site visit to the PHA. The review results are finalized and approved by HUD, and the PHA is then notified of the results.

Based on the outcome and monitoring area, the PHA may receive different reports. For example, for an asset management review, the PHA would receive a report from the CCM system that shows its scores in the assessed areas along with a report that shows the underlying results of the review checklist. However, if there is a specific compliance review (e.g., procurement review), the PHA would receive a narrative report that summarizes the review process and the results and lists any concerns or findings that need to be corrected. Both types of reports will be generated from the CCM system.

If findings or deficiencies are noted, corrective action is normally required and agreed upon between HUD and the PHA. Once both parties are in agreement, the review stays open until the corrective action has been completed. The outcomes of these reviews are then used to inform senior management in setting future compliance priorities and the annual management plan.

The diagram below depicts the overall environment and flow in which a compliance review is completed.

Figure 1: CCMI Environment

1.3.1 The CCM System

The center box of the diagram above labeled CCM system depicts the high level functionality of the CCM system. The CCM system can be viewed as having three main processes: (1) Pre-review, (2) Review, and (3) Post-review. Pre-review and Post-review are common and available to all compliance reviews, while the Review itself is unique to the compliance area being reviewed. Shown within the CCM System box is the 5834 module which represents the .Review. process as described.

Pre-review. In CCM, pre-review allows the review team to select the PHA/project, the compliance areas for review and associated checklists, and notify the PHA of the upcoming review.

Review. The review process consists of the actual work completed in order to assess compliance, including work completed prior to, during, and after the on-site review.

CCMI uses sets of standardized questions and procedures, which are commonly referred to as checklists, that are completed by the review team. The questions and procedures are specific to the monitoring area being reviewed.

Post-review. This functionality allows the review team and HUD management to finalize the review report, notify the PHA of the results of the review, and track the review to closeout. In addition, this process also allows for a certain level of management reporting.

The table below shows the expected inputs and outputs from the 5834 Module.

Inputs to Asset Management Review (5834) Module

PIC

Building and Unit Module – PHA profile, development, and unit information.

PHA

Information on each PHA’s 5834 review will be entered into the system by the reviewer.

REACS

PASS data for a project.

Outputs from Asset Management Review (5834) Module

PHA

Notification Letters and Review Reports

CCM

PHA review data (i.e., checklist answers), Report Data, such as findings and corrective actions.

HUD

PHA specific and national review reports, summaries, and analysis.

1.4

USTOMERS

The principal 5834 Module users and owners include:

Individual Field Office Review Team leaders and members;

Cognizant Field Office staff and management; and

Cognizant HUD Headquarters PIH staff and management.

Other users or entities that will interface or have a vested interest in the 5834 Module or its products include:

Headquarters Offices:

Office of the Assistant Secretary for PIH o PIH Assistant Secretary Staff o Office of Field Operations o Office of Public Housing & Voucher Programs o Office of Policy, Program and Legislative Initiatives o Office of Public Housing Investments

Office of the Inspector General

PHA Chairman of the Board of Commissioners

Public Inquiry

1.

R

EFERENCES

HUD guidance on the assessment methodology to be used for evaluating a project’s management performance is described in Subpart D – Management Operations Indicator of the PHAS Proposed Rule published in the Federal Register on August 21, 2008. This proposed rule amends 24 CFR 902, the Public Housing Assessment System (PHAS) to reflect the implementation of asset management. Subpart D was revised to more comprehensively address the management operations of a project and also references the review form, Management Review for Public Housing Projects (HUD-5834 form) that will be used by HUD’s field office personnel to perform the management operations review.

The PHAS proposed rule should be consulted for background and detail information regarding the management sub-indicators and components that will be used to evaluate a project and is the basis for the Business Rules document (Appendix D). Similarly, the Asset Management Protocols document along with the protocols found in the Excel-based 5834 prototype was used to develop the new protocols (Appendix C). The electronic version of the 5834 form is an Excel-based prototype (i.e., Dan’s Tool) that was used as the basis for the 5834 prototype built in MS Access. Dan’s Tool served as the blueprint for the 5834 Access-based prototype in terms of the layout of the prototype, the checklists, scoring rules for each sub-indicator and component, worksheets, and protocols.

The following table lists the documentation that was used in the preparation of this BRD.

References

Date Issued

1. Public Housing Evaluation and Oversight: Changes to the Public Housing Assessment System (PHAS) and Determining and Remedying Substantial Default (i.e., Proposed PHAS Rule):

Subpart D – Management Operations Indicator

Appendix C – Management Operations Scoring

August 21, 2008 (Federal Register, Vol.73, No. 163)

2. Dan’s 5834 Excel Tool – File Name:

AMPChecklist_Test_102408_Blank.xls

October 24, 2008

3. Asset Management Protocols (BearingPoint 5834 Training Materials)

November 25, 2008

1.6 D

EFINITIONS

Acronym

Definition

CCMI

Comprehensive Compliance and Monitoring Initiative

CCM

Comprehensive Compliance and Monitoring (System)

PIH

Office of Public and Indian Housing

WG

5834 Working Group

Term

Definition

Prototype

Refers to the Microsoft Access-based process model for the 5834 Module.

Dan’s Tool

Refers to the MS Excel-based process model for the 5834 Module.

Module

Refers to the 5834 review area to be built within the CCM system.

PHA Review

The 5834 review can only be performed at a project level. No PHA level review exists for the 5834 monitoring area.

Project Review

All 5834 reviews are at the project level.

Scored Review

A 5834 review where, at a minimum, all scored component questions are answered so that a score can be generated at a component, sub-indicator, and overall 5834 level. The reviewer has the option of answering a question(s) from a non-scored component.

Scored Comprehensive Review

A 5834 review where all questions were answered for both the scored and non-scored components.

Non-Scored Review

A 5834 review where the reviewer does not answer a sufficient number of questions to generate a score at the component, sub-indicator, and/or overall 5834 level. While an overall score may not be generated because an insufficient number of .scored. questions are answered, the 5834 prototype will calculate a component and indicator score as data is entered into the prototype.

5834 Module

Refers to Asset Management reviews that are conducted only at the project level and based on the following review types: (1) scored, (2) scored comprehensive, or

(3) non-scored.

Report Term

Definition

Finding

Conditions that are not in compliance with handbooks, regulatory or statutory requirements, or other HUD guidance. Findings cited in the report must include the condition (description of the problem), criteria not met (regulation, statute, required policy), cause (why the condition occurred), consequence (what happened because of the condition), and corrective actions required or recommended to fix the condition.

Concern

Deficiencies in performance that are not based on a regulatory or statutory requirement that should be brought to the attention of the PHA. Concerns must include the condition, cause, consequence and recommended corrective actions, if appropriate.

Comment

Conditions deserving recognition that do not meet the definitions of findings, errors or concerns. Comments may be positive or negative but do not require any corrective actions.

Corrective Action

Actions taken to correct findings, concerns or other errors. Corrective actions are required for findings.

2. Development of the Asset Management Review (5834) Module

This Chapter provides a chronology of the development of the 5834 Module to date and the approach and assumptions used to develop the 5834 prototype in MS Access.

2.

M

ANAGEMENT

R

EVIEW FOR

P

UBLIC

H

OUSING

P

ROJECTS

(HUD

F

ORM

5834)

The previous Management Operations assessment (MASS) was conducted at a PHA level and allowed for PHAs to self-report their performance on each management sub-indicator and component. However, under the PHAS proposed rule, the Management Review of Public Housing Projects (HUD-5834) form will be used to evaluate a PHA’s performance at the project level. In addition, the management operations (i.e., asset management) assessment will be based on an on-site review that is to be performed by HUD field office personnel.

The proposed version of the HUD-5834 form assesses a PHA’s project based on the following seven (7) sub-indicators, most of which are statutorily required under Section 6(j) of the U.S.

Housing Act of 1937, as amended.

1. General Appearance and Security

2. Follow-up and Monitoring of Project Inspections

3. Maintenance and Modernization

4. Financial Management/Procurement

5. Leasing and Occupancy

6. Tenant/Management Relations

7. General Management Practices

Within these seven sub-indicators, the 5834 form is further divided into 21 component areas; 12 scored components (in accordance with the proposed PHAS rule) and 9 non-scored components.

The seven sub-indicators and their associated components are listed in Table 3.1 on page 22.

I

NITIAL

MS

E

XCEL

B

ASED

P

ROTOTYPE

AND

MS

A

CCESS

V

ERSION

HUD has developed an electronic version of the 5834 form in addition to the paper version. The electronic version of the 5834 form was initially built in MS-Excel (i.e., Dan’s Tool). The MS Excel-based prototype contains the following information and functionalities:

5834 summary form including scoring/grading functionality;

Supporting checklists for each sub-indicator and component area;

Protocols; and

Summary reports.

This MS Excel prototype has been field-tested by Field Office personnel and further refined based on the test results.

The Abt team was tasked with further developing the existing 5834 prototype in MS Access so that this 5834 module will have a similar .look and feel. to the other modules that will be developed using MS Access. The MS Access version of the 5834 prototype was developed in accordance with the PHAS Proposed rule published August 21, 2008 and mirrors the MS Excel version of the 5834 except as noted:

1. The MS Access 5834 prototype diverges from the guidance provided in the PHAS proposed rule to reflect more recent decisions by HUD management on certain sub-indicators and components. For example, HUD management may also decide to assign each category a different weight rather than assigning each category the same weight.

2. The MS Access 5834 prototype incorporates additional guidance on certain topics that were not addressed as part of the 5834 Excel build. The additional guidance does not conflict with the current guidance but provides additional requirements so that the 5834 prototype conforms to the requirements for the other modules that will subsequently be prototyped. For example, the current MS Excel 5834 prototype does not contain business rules, and business rules will be incorporated into the MS Access version.

3. The Abt team also identified areas for improvements that would improve the efficiency and/or usability of the MS Access 5834 prototype. For example, the team will review and revise the protocols so that guidance is provided at both a general level and at a sub-indicator and component level in the prototype though separate protocol buttons.

4. The final point of departure is that, unlike the Excel version, the component worksheets will not be built side-by-side with the checklists. Just like the Excel version, the checklists will feature buttons guiding the user to the worksheets, however these worksheets are built in Excel, not Access. The Access prototype still reads from those worksheets and offers connectivity, but the worksheets are built elsewhere.

Changes from the 5834 Excel-based version to the MS-Access prototype version will be summarized in the Asset Management Review (5834) Prototype and CCM System Requirements document.

A

SSUMPTIONS

C

ONSTRAINTS AND

G

UIDANCE

In the development of this BRD, a number of assumptions, basic constraints, and other guidance were identified. With respect to assumptions, some are based on HUD guidance while others are based on the team’s own thinking in the development of this BRD. The following assumptions have been addressed with CCMI management. The resolution of each item is provided below.

1. Data to NASS System. Under the current PHAS assessment methodology, each PHAS subsystem creates a PHA score for its respective PHAS indicator; the Integrated Assessment Subsystem (NASS) assembles the PHAS subsystem data from PHA into a single, comprehensive assessment score and is responsible for releasing the PHAS score and designation to the PHAs. For the 5834 review, the following questions were posed to CCMI management:

Will the 5834 prototype and subsequently the CCM system be responsible for generating a project’s management operations score?

What information from the 5834 prototype/CCM system will be sent to NASS for the management operations score?

Are there non-scored data (e.g., average number of unit turnaround days) that will also need to be sent from the 5834 prototype or CCM system to NASS?

Resolution: CCMI Management states that only a project score will need to be generated from the 5834 prototype and CCM system. The project score will also reflect the Physical Condition and Neighborhood Environment (PCNE) adjustment, as applicable.

2. Rounding of Scores. The Abt team discussed with CCMI management the rounding rules that should be used for the 5834 prototype. Specifically, what rounding rules should be applied at the component, sub-indicator, project score, and overall Management Operations indicator level?

The Functional Requirements Document (FRD) for the 5834 CCM system build states that .scores will not be rounded until the final PHAS score is calculated.. However, according to Appendix C (Management Operations Scoring) of the PHAS proposed rule, the Management Operations score is computed as follows and the following round rules are applied.

Scoring Elements

Rounding Rules

Step 1: Component. Scores are calculated for each component, where applicable.

No rounding.

Step 2: Sub-Indicator. A score is calculated for each sub-indicator, where applicable.

No rounding.

Step 3: Project score. All components and sub-indicators scores are then summed to calculate a 5834 form score -- the Project score

Round to 2 decimal places.

Step 4: PHA score. All project scores are summed based on a weighted-unit average to produce an overall Management Operations Indicator score – PHA score.

Round to the nearest whole number.

Resolution: Based on discussions with CCMI management:

1. The 5834 prototype will be coded based on the rounding rules found in the scoring notices. Any rounding rule not covered in the notice or that needs clarification will be discussed with HUD prior to coding.

2. Regardless of rounding rules as it relates to scoring, any developed screen and report will display two decimals points for any scoring component. Any percentage will display two decimal points (e.g. 67.12%).

3. Sample Based on Projects Only. The Abt team sought confirmation that the 5834 form is to be used only for a project level review. In other words, a PHA level review will not be conducted using the 5834 form and hence, the sample of 5834 reviews to be performed would consist only of projects and not PHAs.

Resolution: CCMI Management confirmed that the 5834 module is at the project level only. There is no PHA-level review.

4. Core and Non-Core Requirements. In the development of the prototypes for the CCMI modules, the Abt team is also asked to identify core and non-core requirements. Core requirements are those module areas, sections, or checklist questions that are required to be reviewed. .Non-core. are optional areas that may be selected for review but are not mandated. For the 5834 module, the Abt team has proposed to define the core and non-core requirements as follows:

Core is all scored areas of the 5834. Within this category, core is all questions that must be answered in order to calculate a component.

Non-core is all non-scored areas of the 5834.

Worksheets are non-core.

Core and non-core requirements for the 5834 Module are discussed in more detail within Chapter 5.

Resolution: CCMI Management concurs with the definition of core and non-core requirements for the 5834 Module.

5. Review Types. In the development of the Procurement module, the following review types were developed: 1) Streamlined, 2) Comprehensive, and 3) Targeted. For the 5834 Module, the following questions were posed:

Should a similar concept of different review types be applied to the 5834 Module?

In addition, similar to the Procurement prototype, would the reviewer have to select up front, what type of review will be conducted?

Resolution: The following agreement was reached on the review types to be used for the 5834 Module.

1. The review type concepts of streamlined, comprehensive, and targeted reviews will not be applied to the 5834 Module. Instead, the 5834 will have the following review types.

a. Scored. All scored components questions are answered so that a score can be generated at a component, sub-indicator, and overall 5834 level. The reviewer has the option of answering a question(s) from a non-scored component(s).

b. Scored Comprehensive. A 5834 review where all questions are answered for both the scored and non-scored components.

c. Non-Scored. A 5834 review where the reviewer does not answer a sufficient number of questions to generate a score at the component, indicator, and/or overall 5834 level. While an overall score may not be generated because an insufficient number of .scored. questions are answered, the 5834 prototype will calculate a component and indicator score as data is entered into the prototype.

2. The reviewer will NOT see a menu screen in the 5834 showing the above review types. Instead, the review type will only be defined after a 5834 review is conducted so that:

a. If all questions on the scored components are completed so that a score can be generated, the module will generate a score.

b. If the reviewer does not answer enough questions to generate a score, the module will not generate a score (Review Type: Non-Scored).

c. While an overall score may not be generated due to lack of questions answered, the prototype will calculate indicator and sub-indicator scores as data is entered into the prototype.

3. If the reviewer completes enough questions to generate a score, the module will ask the reviewer whether the review is official and thus the score will be retained. Note:

The CCM/MAPS system should be able to distinguish between the last .official.

score generated versus an .unofficial. generated score. Need to discuss whether this functionality is needed for the 5834 prototype.

4. Reports. The 5834 reports should count and group the above listed review types.

(Where a review was completed that results in a score but the reviewer does not want to issue the score, the review will be counted as a .Non-Scored. review..)

6. Relationship of 5834 Module to .Touchpoint. Modules. As part of the Abt team’s work, we will be developing three (3) distinct separate prototypes that are also covered as component areas on the 5834 form and module. The three areas are:

Component 3.5: Modernization Component 4.4: Procurement

Component 5.3: Occupancy Review

One of the issues raised was what is the relationship of a review that is conducted independently of the 5834 review versus a review of the same area under a 5834 review and what module(s) would the reviewer access to conduct the review. We posed the following two scenarios:

Scenario #1: A reviewer performs a 5834 scored review for Project A (i.e., includes section 4.4 procurement) and chooses to conduct a 5834 Procurement module review for Project A. Would the reviewer be conducting a Procurement review within the 5834 Module or would the reviewer have to access the Procurement Module?

Scenario #2: A reviewer conducts a 5834 Procurement review for Project A and decides to conduct a Comprehensive review for Project B and a Streamlined Review at the PHA level.

Which module(s) will the reviewer access to conduct these reviews? How will these reviews be displayed on a report?

This topic generated several lengthy discussions. At one point, the thought was that functionality would be provided so that the reviewer could access the Procurement Module from the 5834 Module using a link button to conduct a Streamlined or Comprehensive Procurement review.

One ground for dismissing this possibility is the complexity of adding such a feature. However, the greater issue is the difficulty we would incur in displaying this feature and training users.

The other alternative is to limit entry to any module from the Main Portal only. That is, the reviewer would not be able to create/launch a procurement review from inside the 5834 review.

Resolution:

1. The final determination was that entry to any module would be from the main portal. This approach would simplify the development process for the prototype and CCM system build. In addition, this approach would be easier for the reviewer to navigate between the various modules. That is, the reviewer would be trained to always return to the Main Portal to launch a review.

2. The module in the compliance area set-up of the main portal should be renamed so as not to lead with 5834 B (for example a RIM EIV review can be done as a standalone review).

7. Grading – Component 3.4 Energy Audit checklist. For Component 3.4, the following grading system applies:

Grade

Criteria

Grade A

Audit conducted in past 5 years All recommendations from the audit have been completed

Grade C

Audit conducted in past 1 year

OR

Audit conducted in past 5 years Have a comprehensive implementation plan

On schedule with the plan

Grade F

No audit in past 5 years OR No implementation plan OR Not on schedule with the plan OR

Have outstanding recommendations from the audit

The intent of the grading rules was that the project would be awarded the highest grade for which it qualifies based on the above. However, the criteria for Grade F is not sufficiently clear to distinguish between a Grade of C and F. This item was discussed with CCMI management and the criteria for a Grade of F was revised as follows and is reflected in the prototype:

No audit in past 5 years OR Audit in past 5 years, have outstanding recommendations, BUT no implementation plan.

OR

Audit in past 5 years, have outstanding recommendations, BUT implementation plan is not on-schedule.

Resolution: CCMI management concurs with the change to the criteria for Grade F.

8. PIC Data. The Excel-based 5834 prototype contains certain data which is stored in PIC.

Specifically two worksheets in the Excel-based 5834 module contain information:

AMPS_TBL: Projects associated with each PHA, including development name and addresses.

PHA_DATA_TBL: PHA profile information.

These worksheets were created by downloading certain data sets from the HUD’s internal website and through files created by the PIC team. These data sets were then manipulated to create these two worksheets (Dan’s methodology). Further certain information was not available, especially as it relates to PASS data. In these cases, the reviewer is asked to key information directly into the prototype from screen shots and reports contained in PASS.

Current questions revolve around the best approach to ensure that the prototype has the most current and complete data.

Currently, in the CCM system, when a reviewer sets up a review, the CCM system then accesses the data warehouse for the required PIC data. The full PIC data is not stored in tables in the CCM system.

Based on discussion with HUD and the CCM IT team, there are four options for accessing the PIC data using the 5834 prototype:

1) Use Dan’s methodology, i.e., to store the PIC data as static files within the 5834 prototype.

2) Query the PIC data warehouse directly, which includes reproducing the queries that CCM is already coding.

3) Have the data put in a table in the data warehouse or CCM system; a query could be used to periodically download this data into the prototype.

4) Have CCM create the data set and place it in an area which the prototype can access and automatically import the data. This functionality would be completed each time a CCMI review is set-up.

Resolution: This item is still open and being discussed.

9. Use of N/A and N/R in Checklists. The 5834 Excel prototype was not developed with N/A (not applicable) and N/R (not reviewed) as universal options. Should these options be included:

Universally for each question, Only for those questions which they are applicable, or

Coded exactly as shown in the Excel prototype?

Resolution: HUD has provided the following guidance. Overall N/A and N/R answers should be added to the form where applicable.

1. Currently N/R is only available currently for the Procurement checklist (4.4).

2. N/A is not an applicable response for most questions. Where N/A is applicable, it is usually based on the response to previous questions that renders the given question as not applicable. In these cases, the N/A radial button is locked but will be automatically marked as N/A through dependency logic coding.

3. The reviewer may select the exclude option for any component that is not reviewed or not applicable.

10. Dependency Questions.

Like other checklists in other prototype modules, some questions are dependent upon the answer of others. Should the prototype be built using the same dependency logic as developed under the procurement module.

Resolution: HUD agrees with the following approach The dependencies should be coded using the same logic as developed under the procurement module, with the following comments:

1. For those components whose leading question starts with .Does the project have an adequate system….; if the reviewer checks .No., the PHA will receive a grade of .F. for that area, as required by the scoring notice, but the reviewer will still be allowed to input data for the underlying questions from this .suspect. system.

2. For inferred dependent questions (e.g. 3.4 Energy Conservation, which infers that an energy audit has been completed or 4.3 Budget Management, which infers an approved budget exists), the sub questions are based on the premise that the primary focus in the preceding questions exists (in this example - an energy audit or an approved budget). If the primary focus does not exist, then the sub-questions are automatically marked with a N/A (not applicable).

3. For those questions that are N/R (not reviewed) all sub- questions will be automatically marked as N/R.

11. Development of Worksheets in MS Excel. The team discussed with CCMI management the option of developing the worksheets for the 5834 module and the worksheets for the other prototypes in MS Excel instead of building the worksheets in MS Access as part of the prototype. The rationale for building the worksheets in MS Excel is that the worksheets can be built more quickly in Excel than in MS Access, particularly those worksheets that require substantial math calculations (e.g., worksheets associated with RIM reviews). The reviewer will not be able to access the MS Excel worksheets from the prototype but the worksheets will be sent with the zip file that contains the prototype. The reviewer will need to separately open the Excel file as well as the prototype.

Resolution: CCMI Management agreed with the recommendation to develop the worksheets for each prototype using MS Excel. The MS Access-based prototype will (1) reference the specific tab of the worksheet, (2) map the worksheet to the specific checklist item in the prototype, and (3) indicate by a check mark in the prototype that the worksheet was used.

12. Protocols. The protocols in Dan’s tool currently have one protocol button on each screen that contains general guidance as well as guidance for the sub-indicator and components displayed on the screen. The Abt team proposes to break down the protocols into smaller pieces that address specific areas of the checklist, sub-indicator, and components. This change to the protocols will be reflected in the Final BRD and in the Draft and Final Prototype models.

Resolution: CCMI management concurs with this recommendation to revise the protocols so that each protocol button in the prototype addresses specific areas of the checklists.

3. 5834 Checklists, Business Rules, and Protocols

After various HUD meetings, work groups and discussions, a proposed rule was published in the Federal Register on August 21, 2008 (24 CFR Parts 901, 902, and 907 [Docket No. FR-5094-P- 01]) called the Public Housing Evaluation and Oversight; Changes to the Public Housing Assessment System (PHAS) and Determining and Remedying Substantial Default. This proposed rule also contained the scoring notices in its appendices. Specifically, this proposed rule contains the requirement of the Management Operations Indicator (Subpart D) and the associated Management Operations Scoring (Appendix C to part 902), which form the basis of this 5834 module. Equally important, the rule assesses at the project level, not by PHA. This approach means that only projects are sampled or reviewed under this module.

Based on this proposed rule but under separate efforts, two other items specifically related to the 5834 module were created by HUD: (1) A prototype in Excel and (2) CCM business requirements.

The goal of our prototype is not to define the business requirements anew. Rather, the goal is to re-develop the Excel prototype as an Access prototype with the same look and feel as the other modules being built under this CCMI effort. In addition, the Abt team was asked to incorporate the latest developments and thoughts into the Access prototype. Abt was instructed to use the checklist questions and information in the Excel 5834 prototype as the starting point for this effort. The Excel 5834 prototype was based on the latest HUD views and information at the time of the build. The only changes from the Excel based prototype are as follows:

1. Changes to the general look and feel of the other modules being built (i.e. light blue background, font size, navigation, protocol and business rule links, etc).

2. Changes based on comments received under the proposed rule that HUD believes will likely result in a change in the final rule.

3. Updates to business rules and logic that may have been missing in previous efforts.

.1

HECKLISTS

The primary goal of the 5834 module is to assess certain key elements of a project’s management operations. The checklists, a series of questions developed based on the requirements in the proposed rule, provide a systematic approach to assessing the project. Many of the requirements stem from specific questions or required information necessitated by the implementation of the proposed rule. All the questions and related user input can be found in:

Appendix A: 5834 Asset Management Review Checklists or

Appendix B: 5834 Asset Management Review Worksheets (optional).

Appendix C and Appendix D contain the associated Protocols and Business Rules.

The following table shows the Management Operation Assessment areas under the proposed rule. The table identifies the associated checklists, whether the component is scored, and if a supplemental worksheet is offered to help the reviewer complete the checklist.

Table 3.1 Management Operation Assessment Areas

Monitoring Area

Checklist

Scored Area

Worksheet (How Many)

General Appearance and Security

1.1

Appearance and Market Appeal

Yes

No

1.2

Security

Yes (2)

Follow-up and Monitoring of Project Inspections

2.1

Exigent Health and Safety Issues Deficiencies

No

Yes (1)

2.2

Lead-based Paint Inspection Deficiencies

No

Maintenance

3.1

Unit Inspections

Yes (2)

3.2

Work Orders

No

3.3

Preventive Maintenance

No

3.4

Energy Conservation / Utility Consumption

Yes (1)

3.5

Modernization

No

Financial Management / Procurement

4.1

Accounts payable

No

4.2

Rent Collection

No

4.3

Budget Management

Yes (1)

4.4

Procurement

No

Leasing and Occupancy

5.1

Vacancy Rate

Yes (1)

5.2

Turnaround Time

Yes (1)

5.3

Occupancy

No

Tenant Management Relations

6.1

Economic Self-sufficiency

No

6.2

Resident Involvement in Project Administration

No

General Management Practices

7.1

Prior Management Review Findings

No

7.2

Other Prior Review Findings

No

7.3

Insurance

Components 3.5 Modernization, 5.3 Occupancy, and 7.3 Insurance were not developed in the 5834 Excel prototype; therefore the Abt team used the questions as shown in the draft Form HUD-5834 Management Review for Public Housing Projects. Likewise, component 4.4 Procurement was not developed within the 5834 Excel prototype. There was an accompanying checklist in Form HUD-5834 Management Review for Public Housing Projects, but this was not used. Instead, four recommended questions from the Procurement prototype serve as the basis for this checklist. Appendix C does not yet contain protocols for these four checklists.

Certain components of the 5834 module, such as Procurement and Occupancy, only provided limited questions. Therefore, HUD is developing in-depth monitoring modules for these areas under CCM. It has been decided that the 5834 will not provide direct access to these modules / checklists. Rather, the reviewer must access these modules from the main Portal or the main CCM screen.

E

XAMPLE

HECKLIST

The CCM and prototype portal concept is constructed such that the reviewer selects all pertinent entities (projects and maybe the PHA itself) and review modules (e.g., 5834, Occupancy, Procurement) from a main Portal. At this stage, however, the system has not linked a specific entity to a specific review module or review type. The system filters subsequent menu options based on what has been selected. Thus, the 5834 module is set up so that the reviewer can start a 5834 review by simply selecting a project in the Management Review screen. All projects that were selected in the initial portal / CCM main selection will be available. Once the reviewer has selected the project of interest, the reviewer reaches the checklists by clicking the associated .Go. button.

It is possible that the reviewer will not wish to complete a 5834 review for each of the selected projects. Therefore, the 5834 module has its own Setup screen that features the following:

An option to de-select a project from the 5834 review. De-selecting the project means that it will not be available in the Management Review screen.

The type of review completed (see Section 3.6 below) will be displayed.

Finally this screen will aid in reporting of the 5834 review progress by having the reviewer provide the universe of project level 5834 reviews that will be completed at that PHA.

The following screenshot shows a portion of the Project Management Review Screen, which serves as the access point to the individual checklist and provides status (such as Complete, In Progress, or Excluded), feedback and the results of each checklist.

Figure 3.2.1 Project Management Review Screen

The following screenshot shows the checklist for the 5.2 Unit Turnaround Time components as developed within the module.

Figure 3.2.2

5.2 Unit Turnaround Time

As the screenshot above shows, the look and feel of this module strongly resembles that of the Procurement module.

The following table provides clarification of specific issues that were discussed.

Table 3.2 Checklist Question Response Options

Issue

Resolution

The 5834 Excel prototype was not developed with N/A (not applicable) and N/R (not reviewed) as universal options.

Should these options be included:

universally for each question only for those questions which they are applicable, or be coded exactly as shown in the Excel prototype?

Overall N/A and N/R answers should be added to the form where applicable.

Currently N/R is only available for the Procurement checklist (4.4) and only because one may skip those questions in favor of completing them in the Procurement module. The reviewer may select the exclude option for any component that is not reviewed.

In most—but not all—cases, the N/A radial button is locked. In those cases, N/A is automatically filled in when an answer to a prior question renders the given question not applicable. (See checklist 3.4, Energy Conservation, for a good example.)

Like other checklists, some questions are dependent upon the answer of others.

How should these dependencies be coded?

These dependencies will be coded using the same logic as the Procurement module, with the following comments:

For those areas whose leading question is .Does the project have an adequate system….;

if the reviewer checks No, the project will receive a grade of .F. for that area as required by the scoring notice. However, the reviewer could still input data for the underlying questions from this .suspect. system.

For inferred dependent questions (e.g., 3.4 Energy, which infers that an energy audit has been completed, therefore all sub-questions relate to this audit, or 4.3 Budget Management, whose sub-questions relate to an approved budget for the assessed fiscal year), the prototype will be developed with the appropriate dependency answers (i.e. N/A the sub-questions). See the resolution to Issue 1, above, for more detail.

For those questions that are N/R, all sub-questions will be N/R.

3.3 W

ORKSHEETS

Because of the complexity of certain checklist questions, separate worksheets were created to help the reviewer organize and answer the questions. These worksheets, which are optional, break the questions down into clearer and more manageable components. Currently, nine worksheets exist. The table below provides a list of the worksheets.

Table 3.3 Worksheets

Component

Worksheet

1.2

Security

1. Applicant Screening Worksheet

2. Lease Enforcement Worksheet

2.1

Exigent Health and Safety Deficiencies

3. EHS Worksheet

3.1

Unit Inspections

4. Unit Inspection Exclusion Worksheet

5. Unit Inspection Sample Worksheet

3.4

Energy Conservation / Utility Consumption

6. Energy Conservation / Utility Consumption Worksheet

4.3

Budget Management

7. Budget Management Worksheet

5.1

Vacancy Rate

8. Vacancy Exclusion Worksheet

5.2

Turnaround Time

9. Unit Turnaround Worksheet

The worksheets will not be developed in the Access prototype. Instead, the Access prototype will provide notice to the reviewer that an Excel worksheet exists.

Figure 3.3 Sample Worksheet Instructions All worksheets will be prototyped and deployed in one Excel file. The Access prototype will reference the specific tab of the worksheet. In addition, a check-box in the Access prototype will denote whether the reviewer used the worksheet. The Excel worksheet will provide mapping of the worksheet results back to the Access prototype.

B

USINESS

R

ULES A

ND

P

ROTOCOLS

The Excel 5834 model was built with both the business rules and the protocols contained in one tab for each component. While accurate, this leads to cases where the reviewer must search through pages of information to find the answer to a specific question. The protocols and business rules will be broken down, where appropriate, into smaller pieces that address specific areas of the individual checklists. For example, the portion of the protocols and business rules that deals with a component’s grading will be captured separately with business rule and protocol buttons (links) provided in the grading portion of the checklist. Currently, Appendix C – Protocols contains the protocols as they are found in the 5834 Excel prototype and the current version of the Access prototype. Appendix D - Business Rules contains information from the proposed rule as it relates to the management operations indicator and scoring.

S

CORING AND

G

RADING

Twelve of the 21 components that are reviewed can be scored and graded. The scoring and grading rules for all components, which can be found in Appendix C of the proposed rule, are the basis for the scoring of the prototype. The following table provides clarification of specific scoring and grading issues that have been discussed.

Table 3.4 Scoring and Grading Issues

Issue

Resolution

How and when should values be rounded for the purpose of scoring and display?

Components and Sub-Indicators will not be rounded for the purposes of scoring.

Regardless of rounding rules as it relates to scoring, any developed screen and report will display two decimals points for and scoring value (e.g. , 13.73). Percentages will be display with two decimals points (69.43%).

Under the 5834, Sub-Indicators and Components can be excluded from scoring. How should the prototype handle an exclusion?

Each checklist will have an Exclusion check box in the top right corner. If the reviewer checks the Exclude check box, the component will not be scored and the points associated with that component will be…