EP_CSC_Evaluation_Tool.docx

EVALUATION TOOL WORKSHEET

APPLICANT:

EP CSC Program Location:

Date Application Received:

Review Committee Review Members:

Dates of Review:

|_| Applicant is already on the Master Agreement for the following Program(s):

SUMMARY OF EVALUATION

Minimum Submission: |_| Passed |_| Failed Program Proposal Requirements: |_| Passed |_| Failed Cost Proposal Requirements: |_| Passed |_| Failed Qualified for Master Agreement: |_| Yes |_| No EP CSC Status: |_| Fully Qualified |_| Provisionally Qualified GRANT PRIORITIES (Section 9.3):

Existing Program

a. |_| Existing, operational EP CSC Program Staffing

b. |_| Applicant proposes use of DMH funds to hire dedicated staff (i.e., staff performing a single function on the CSC Team) who cannot be reimbursed through 3rd party billing, in one or more of the following roles ideally for 1.0 FTE and no less than 0.5 FTE: Supported Employment/Education Specialist; Peer Specialist; and/or Care Coordinator.

c. |_| All EP CSC Program staff, including the prescriber, exceed the minimum FTE requirements, e.g., are all 0.5 FTE or more dedicated to the program.

d. |_| The Peer Specialist position is 1.0 FTE or equivalent.

EP CSC Program

e. |_| The proposed EP CSC program is sited in one of the following communities: Pittsfield; Greenfield; Fitchburg/Leominster; Peabody/Salem; Gloucester; Quincy/Cohasset; Norwood; Attleboro; Plymouth; Lawrence; or Hyannis. DMH has assessed the needs of Massachusetts communities related to EP CSC and will adjust this list as needs change over time

f. |_| The EP CSC program is located in a community-based clinic with a history of providing services delivered in community settings “outside the walls” of the clinic.

g. |_| Applicant proposes use of DMH funds to provide out of the clinic programming for EP CSC youth and families.

h. |_| The EP CSC Program provides services to individuals and their families regardless of ability to pay.

Applicant Organization

i. |_| Applicant agency and the EP CSC staff are licensed/credentialed to serve individuals across the life-span, including youth and adults.

j. |_| Applicant has a history of successfully integrating peer and/or employment services into clinical programming.

PART A MINIMUM SUBMISSION REVIEW

|_| Signed letter of commitment and signed attestation form.

Part II Application Format:

1. Cover Letter: Cover Letter provided in COMMBUYS is used and completed

2. The Program Proposal must be in the following format (Program Proposal Requirements):

|_| type-written, 12-point Times New Roman font |_| 8 ½ x 11-inch paper |_| fully justified (print fully abuts both margins) |_| one-inch margins |_| single-spaced |_| Table of Contents

3. The Cost Proposal must be in the following format (Cost Proposal Requirements):

|_| First FY Budget |_| Second FY Budget (if different from First year FY Budget) |_| DMH Budget Form and Staff titles used |_| Budget Narrative |_|in the format described above of the Program Proposal (except no Table of Contents)

4. Page Limits and Attachments (RFA Subsection 7.5) |_| Program Proposal within the stated 10 Page limit (plus 2 pages for the Executive Summary) OR |_| page limit was exceeded, excessive pages removed from the end of the Proposal.

|_| Budget Narrative within the stated 2 Page limit OR |_| Page limit was exceeded, excessive pages removed from the end of the Proposal |_| Attachments limited to those on the List of Permitted Attachments, OR ___ Extra attachments included but removed by DMH.

Early Psychosis Coordinated Specialty Care Program - Staff Enhancement and Evidence Based Practice Implementation – Master Agreement Request For Applications DMH Document Number: 2023-DMH-3045-01 April 2023

Part III Subsection 4.1 Minimum Applicant Requirements:

Clinic has relevant accreditation/certification to provide outpatient services and to optimize billing of 3rd party payers including Medicaid and Medicare.

|_| Meets |_| Not Meet (explain)

Part IV Action Taken as a Result of Minimum Submission Findings:

|_| Clarification sought in the following items:

|_| Rejected, determined not to meet Minimum Submission Requirements; (Specify) |_| Determined meets Minimum Submission Requirements. If this is checked continue to Part B, below.

IF THE REVIEW COMMITTEE ELECTS TO HAVE PRELIMINARY REVIEWS DONE BY ANY EXPERTS PRIOR TO THE REVIEWS REQUIRED BY PARTS B OR C, A SUMMARY OF THE PRELIMINARY REVIEWS MUT BE ATTACHED THIS WORKSHEET, DATED AND SIGNED BY THE EXPERT.

PART B PROGRAM PROPOSAL EVALUATION

The Bidder’s Program Proposal will be reviewed in four Parts:

(1) The Applicant organization;

(2) The Existing or Proposed EP CSC Program;

(3) The Proposed Use of Grant funding; and

(4) Race, Equity, and Inclusion.

To be considered eligible for a Master Agreement, the following items in the Program Proposal Evaluation must be met:

Group 1:

· item 2

Group 2

· items 5-7

· and 20 of the other 24 items set forth in this Part B must be met (approximately 80%), provided, however, that if any one of the remaining 24 items is not met, the identified weakness must be nonmaterial, such that it can be corrected as part of contract negotiations.

In addition, the Cost Proposal must be found to be reasonable as set forth in Part C.

Group 1. Applicant’s Organization & Executive Summary

Requested Information

Applicant’s Organization & Executive Summary

1. |_| It is reasonable to believe that the Applicant has the capacity to support and maintain an EP CSC Program. This is because the Applicant demonstrates strengths in following areas: (a) the Applicant has sufficient experience implementing specific EBPs and assuring on-going fidelity to expect that it will be able of implementing and maintaining fidelity to EP CSC standards; (b) the Applicant has sufficient experience providing outpatient and community-based services to adolescent and young adults with early psychosis and their families to understand and implement the EP CSC Standards. This includes experience providing services delivered in community settings including “outside the walls” of the clinic (Subsection 9.3.e); and (c) the Applicant has staff with sufficient competencies to support an EP CSC Program.

2. |_| the Applicant submitted a Letter of Commitment from the Agency’s Leadership that details how Agency leadership is or will be involved with the administration and clinical operations of the EP CSC Program; how the EP ESC Program will be supported by the Agency; and how the Agency will provide supervision to the EP CSC Program team leader. The provided descriptions illustrate that the EP CSC Program will be integrated and fully supported by Agency.

3. |_| The Applicant completed and Submitted the DMH Environmental Preferable Product Form and at least one item in the check list will be accomplished if the Applicant is awarded grant funds.

4. |_| Self-Assessment Tool. The Applicant submitted a completed Self-Assessment Tool that contains a sufficient amount of detail that reflects that the Applicant has a good understanding of the RFA requirements, and its EP CSC Program meets or is being designed to meet such requirements.

Group 2. EP CSC Program or Proposed EP CSC Program

Self-Assessment & EP CSC Program

5. EP CSC Program Core Required Staff and Roles (RFA 5.2; Self-Assessment Tool Section 2). To pass this requirement one of the following boxes must checked:

|_| All minimum staffing requirements and required roles are fully met (a minimum of four (4) team members who fulfil the six (6) core functional roles), including that all staff have completed EP CSC model training and role specific EBP training(s); OR, |_| All minimum required staffing is nearly met; key leadership roles are filled (Team Leader; Agency Lead); most required roles are filled with trained staff; and there is a clear plan to address any current staffing vacancies by providing coverage for vacant positions and identifying a reasonable timeline for addressing vacancies and assuring completion of role-specific EBP training; OR, |_| Specific individuals who will fill all Key Leadership Roles (Team Leader; Agency Lead) and all minimally required roles that are 3rd party reimbursable (I.e., individual and family clinicians, prescriber) have been identified for the designated EP CSC program.

6. DEDICATED TIME TO EP CSC. To pass this requirement one of the following boxes must checked:

|_| All current staff have 0.2 FTE or more dedicated time with the EP CSC program; OR

|_| All identified individuals for minimum required roles (e.g., 3rd party reimbursable) will have 0.2 FTE or more dedicated time with the EP CSC program within 3 months of receipt of funding; AND applicant provides an implementation plan that demonstrates a realistic understanding of tasks and challenges necessary to assure full minimal staffing is achieved within 1st year of funding.

7. EP CSC CORE SERVICE COMPONENTS. To pass this requirement one of the following boxes must checked:

|_| The Self-Assessment Tool and the Program Proposal demonstrate the Applicant is committed to and has the ability to provide the seven (7) required EP CSC Core Service Components and Operations as described in Subsections 5.3, 5.5 and 5.6 in a way that is consistent with EP CSC Core Principles; OR, |_| Applicants proposing a new EP CSC Program or currently operating EP CSC Program that does not yet meet all of the specifications of Section 5 of this RFA, provide a clear plan for utilizing funding under this RFA to meet this requirement fully within one year.

EP CSC JOB DESCRIPTIONS

8. |_| Proposed job descriptions demonstrate understanding of the specific EP CSC role and associated responsibilities and reflect the staff skills and cultural and linguistic competence necessary to implement the EP CSC Program for which the Application is being submitted, consistent with the requirements of Subsection 5.2.

9. |_| The job description for the Team Leader sufficiently addresses the role and responsibilities required for coordinating the EP CSC program’s multidisciplinary team and assuring the EP CSC program operates with fidelity.

10. |_| Submitted resumes reflect the applicable job descriptions and, if applicable, evidence is provided of completed EBP trainings.

EP CSC STAFF EBP TRAINING

11. |_| The Applicant plan for supporting EP CSC Program staff as they complete on-going EBP training requirements and supervision is reasonable and should ensure that the training will be timely completed with limited impact on the Program’s ability to provide services.

EP CSC MODEL

12. |_| The Applicant’s selected EP CSC Program model is NAVIGATE, or the selected EP CSC model which the Applicant proposes to use aligns with the EP CSC Core Principles and Practices set forth in RFA Subsection 5.1 and the Applicant has identified applicable resources for training for purposes of implementing the model.

EP CSC Operational Requirements

13. |_| The programmatic, administrative, and fiscal actions that are needed for the EP CSC Program to meet the requirements of Section 5 of the RFA within a reasonable timeframe are supported by a Self-Assessment, which is completed with details that reflect a good understanding of the RFA and the MAPNET Fidelity Tool. If the EP CSC has not started or is not currently serving a minimum of 25 persons, the timeline provided is reasonable considering the extent of the actions that need to be taken.

14. |_| Multi-disciplinary team meets weekly and EP CSC staff participate in weekly EP supervision.

15. |_| Workflows and job descriptions demonstrate understanding and commitment to use of EP Assessment Battery for treatment planning.

Ongoing Growth of EP Program

16. |_| The Applicant agency provides internship opportunities for BH trainees.

17. |_| The Applicant agency partners with researchers.

Quality Management (QM)

18. |_| The Applicant has a reasonable plan to use information, including specifically the EP Assessment Battery and Self-Assessment, to monitor and ensure that the EP CSC Program for which you are seeking a grant consistently meets the requirements for Section 5 of the RFA.

19. |_| The Applicant has a reasonable plan to use information, including specifically the EP Assessment Battery and EP Program Self-Assessment, to monitor the effectiveness of EP CSC services. It is reasonable to believe that the Applicant agency provides resources to support EP CSC activities to assess and improve program performance.

Group 3 Proposed Use of Grant Funds

20. |_|The Applicant identifies the amount of grant funds it is seeking; how it intends to use grant funding in its EP CSC Program and the extent to which the funds will (1) enhance staffing or assist with the implementation of Evidence Based Practices; and/or (2) address gaps/goals identified in the completed Self-Assessment; and (3) supplement and not supplant staff time and roles that are third party reimbursable.

Note to reviewers: if the above box is checked, one or more of the following below must be marked as applicable and/or an ‘Other’ purpose is identified and a comprehensive description of how funds will be used is provided:

___Staff roles in services not supported by third party payments including, but not limited to: care coordination, supported employment, supported education, peer support and family support/education.

___Staff time to participate in the DMH-sponsored training and implementation support activities, including but not limited to NAVIGATE model of CSC, Individual Resiliency Treatment, MacFarlane Multi-family Groups, Supported Employment/Education Group Supervision, or Cognitive Enhancement Treatment, and with DMH-approval, other EBP trainings.

___Staff time to participate in multi-disciplinary team meetings, supervision, and other EP CSC activities necessary to maintain EBP fidelity;

___Staff time to administer, review and use the EP Assessment Battery at baseline and 6-month intervals during a person’s enrollment in EP CSC, for the purposes of treatment planning and quality improvement.

___Staff time to administer, review and use the EP Assessment Battery at baseline and 6-month intervals during a person’s enrollment in EP CSC, for the purposes of treatment planning and quality improvement.

___Other.

21. |_| If the Applicant has proposed to use grant funds for staff positions, the Applicant identified the positions that will be supported; the percentage of time each identified staff position will be supported, and provided a reasonable explanation for how the positions and percentages were determined based on the EP CSC model.

22. |_| The Applicant’s start-up plan is reasonable and reflects the purpose of the grant and the EP CSC Program requirements.

23. |_| The Applicant’s timeline is reasonable based on the actions needed for start-up of an EP CSC Program.

NOTE: If “Other” is checked, reviewers must verify and describe below (a) how these services are not supported by third party billing (b) how the service will enhance staff; support EBP or support a gap/goal identified in the self-Assessment.

Group 4 Race, Equity, and Inclusion

Group 4 Race, Equity and Inclusion must be addressed throughout the bid. The Committee will rate the proposal as a whole when considering how the bid meets the Race, Equity and Inclusion needs of the individuals and families that will be served by the applicable EP CSC Program

Applicant’s Organization

24. |_| The Applicant demonstrates with specific details that its commitment to Race, Equity and Inclusion is similar to DMH’s and that its EP CSC Program is or will be designed to reflect that commitment.

EP CSC Program

25. |_| It is reasonable to believe that the culture of care of the applicable EP CSC Program will meet the requirements of Subsections 5.1 and 5.2. of the RFA.

26. |_| The Applicant’s Self-Assessment and Program Response reflect an awareness of the linguistic and cultural needs of their client/prospective client population and have a specific plan to address these needs.

27. |_| The response identifies historically underserved or marginalized communities within the area to be served by the EP CSC Program and the Applicant has a reasonable plan for providing outreach to these communities.

28. |_| The Applicant addresses race, equity, and Inclusion in its proposed job descriptions and plans for hiring, seeking staff reflective of the cultures, languages, and backgrounds of the communities to be served.

PART C

To be considered for eligible for being awarded a Master Agreement, the Cost Proposal must be determined to be reasonable as set forth in this Part C. In addition, the Program Proposal must be determined to meet the requirements as set forth in Part B provided, however, that if any requirement is not met, the identified weakness must be considered minor such that it can be corrected through contract negotiations.

COST PROPOSAL REVIEW

The Cost Proposal must be reasonable.

The Cost Proposal will be considered to be reasonable if it is:

|_| Reflective of and limited to the actual costs of the activities for which the grant is being sought;

|_| Consistent with how similar costs are currently covered by the Applicant; and |_| The administrative and other costs not directly related to the activities for which the grant is being sought are explained and justified.

If any of the above boxes are not checked, the Cost Proposal shall not be considered reasonable. The Review Committee must explain why a box was not checked.

PART D CONCLUSION. The Applicant will be awarded a Master Agreement for its proposed EP CSC Program Project if (1) The Program Proposal was determined to meet or exceeds the minimum requirements set forth in Part B (85%) and those requirements that are not met, are as being nonmaterial. and (2) The Cost Proposal was found to be reasonable in Part C.

|_| Eligible for Master Agreement |_| Not Eligible for Master Agreement

The Project will be listed on the Master Agreement as being either “Fully Qualified” or “Provisionally\Qualified” based on the criteria listed below (See Subsection 7.6 of the RFA for the ramifications):

|_| Fully Qualified. The applicable EP CSC Program has been operating for 1 year or more, fully meets the requirements of Section 5 of the RFA and is serving a minimum of 25 people.

|_| Provisionally Qualified. The applicable EP CSC Program is new (to be implemented or has been implemented for less than 1 year) will be determined to be “Provisionally Qualified.” Provisional status will also apply to existing EP CSC Programs that (1) are found not to fully meet Section 5 of the RFA, or (2) are not currently serving a minimum of 25 persons.

If an Applicant currently has a Master Agreement for a project under this RFA, the Master Agreement will be amended to include the new project. Award of a Master Agreement does not commit DMH to awarding a Grant. That is determined separately by a process developed by the awarding Authority pursuant to Section 9 of the RFA. That process will take into consideration the priorities set forth in Section 9.