Appendix B rev 07142025.docx

APPENDIX B

County of Santa Clara RFSQ-BHS-FY24-0084: Program Evaluator Services

EVALUATION CRITERIA AND QUALIFICATIONS

Section 1: Organizational Structure and Experience[footnoteRef:2] Total Points: [2: For Section 1, Offeror shall either:

Select applicable experience, specialties, training, and/or treatment practices; or Select Yes/No; and Provide evidence/supporting information for each criterion in the Response Packet miscellaneous documents.]

Experience serving clients OR organizations with complex client risk factors:

☐ Abuse/Violence ☐ Youth and/or Foster Care ☐ High Utilization of Emergency/Institutional Care ☐ Homelessness (high-risk; chronic/repeated support needed) ☐ Intergenerational Impacts of Incarceration ☐ Involuntary Clients ☐ Justice-Involved Populations ☐ Serious Mental Illness (SMI) ☐ Substance Use ☐ Psychological Trauma

☐ Behavioral health services programs including mental health (e.g., prevention and protection interventions, services, policies, and/or educational programs) ☐ Access and Unplanned Services ☐ Cross Systems Initiatives ☐ Forensic, Diversion, and Reintegration (FDR) ☐ Mental Health Services Act (MHSA)-funded programs (e.g., prevention and early intervention; innovation projects) Specialties ☐ Lesbian, Gay, Bisexual, Transgender, Queer or Questioning (LGBTQ+) ☐ Underserved, Underrepresented, and Unserved Ethnic and Cultural Populations ☐ Other:

☐ Adult and Older Adult System of Care (e.g., outreach services to culturally diverse and treatment resistant populations) ☐ Children, Youth, and Family System of Care (e.g., clinical treatment plans) Specialties ☐ Other:

☐ Substance use disorder or prevention services ☐ Prevention Services ☐ Detoxification and Recovery Services ☐ Intervention Services and Treatment Services (e.g., medication assistance) ☐ Other:

☐ Data coordination and analytics projects (e.g., statistical methodology) ☐ Quality Assurance and Improvement Practices (e.g., County compliance and targets with accrediting agencies) Specialties ☐ Qualitative evaluation methods including Participatory Action Research (PAR) and Community-based Participatory Action (CBPAR or CBPR) methods and approaches ☐ Quasi-experimental evaluation methods (e.g., propensity score matching, difference in differences, and regression discontinuity) ☐ Experimental evaluation methods (randomized control trials [RCTs]) ☐ Statistical analyses of observational data ☐ Dashboard and decision-making tool design and development ☐ Other:

Specialties ☐ Master Leasing Models ☐ Supportive Housing Services ☐ Other:

· Five (5) years of experience conducting evaluations:

· For community or governmental agencies;

· That are culturally sensitive or culturally appropriate for programs serving populations as described in Appendix G, Cultural Competence Form.

· Five (5) years of experience preparing reports and presenting complex technical information to municipal and governmental agencies and community stakeholders.

· Five (5) years of experience in gathering qualitative and quantitative data, ability to gather and work with large data sets, and the ability to develop data collection/evaluation tools.

· Quantitative skills include, but are not limited to:

· Complex sampling and weighting techniques;

· Designing and implementing multi-site evaluations;

· Merging and analyzing large data sets;

· Multivariable analysis and modeling, including logistic regression, spatial, hazard, mixed effects, and/or hierarchical modeling;

· Quasi-experimental statistical techniques including propensity score matching, difference in differences, and regression discontinuity.

· Longitudinal data analysis, including health forecasting and time series analysis;

· Descriptive and inferential statistical analysis, including multivariable analysis for intervention studies/trials/designs;

· Data visualization using data visualization software;

· Questionnaire design and validation; and/or

· Other statistical techniques such as propensity score methods.

· Qualitative skills include, but are not limited to:

· Facilitating focus groups and transcribing data;

· Structured, semi-structured, and unstructured interviewing;

· Field observation;

· Environmental scans;

· Social network analysis;

· Social media analysis; and/or

· Analysis of audio files and visual data such as videos and images.

· Quality (e.g., content, results, interpretation, etc.);

· Correct grammar usage; and

· Clear and professional language.

For client-based evaluation samples (when applicable), the additional requisites shall apply:

· Diagnostic impressions;

· Assessment knowledge; and

· Setting experience (e.g., court-ordered, outpatient, etc.).

If Offeror is also applying for behavioral health services programs, at least one of the (3) sample evaluations shall include system of care. Submissions shall be evaluated for criteria described above in addition to the following:

· Clinical Interview (e.g., addresses referral reasoning, limits to confidentiality, language(s) used, location of evaluation);

· Relevant history/background information (e.g., etiology of behavior and support base(s) appropriately);

· Appropriately addresses risk issues; and

· Assessment tool experience/use (e.g., frequency and mastery) related to:

· Orientation/Mental Status Examination;

· Attention/Concentration;

· Estimate of Intellectual Functioning;

· Immediate Memory;

· Delayed Memory;

· Visuo-Spatial/Constructional;

· Language Functioning;

· Reasoning and Problem-Solving (Executive Functioning); and

· Emotional Functioning.

· Organization’s capacity and process to administer and collect the required outcome evaluation data (e.g., assessment/outcome tools and performance measures currently utilized by staff);

· Process for staff to monitor and evaluate data (e.g., client/family satisfaction surveys, pre-/post-service tests, and culturally responsive services/practices); and

· Plan for ongoing evaluation and outcome improvement for the program (e.g., data collection and analysis); and

· How their organization’s data results inform clinical or program practices and service quality.

☐ Brief Psychodynamic Therapy ☐ Cognitive-Behavioral Therapy (CBT) ☐ Dialectical Behavioral Therapy ☐ Exposure Therapy ☐ Family Therapy ☐ Integrated Co-Occurring Disorder Practices

☐ Critical Time Intervention, Assisted Outreach Treatment ☐ Seeking Safety ☐ Solution Focused Therapy ☐ Stages of Change ☐ Trauma-Informed Care ☐ Wellness Recovery Action Plan (WRAP) ☐ Other:

1. Personnel Retention Plan which addresses the following topics:

· Organizational strategies for ensuring that continuity of staffing and care is provided to the target population(s); and

· How the program will be staffed appropriately and in a timely manner, including plan for clinical supervision of professional and paraprofessional staff, and ability to provide services according to Medi-Cal standards.

2. Staffing plan which addresses the following topics:

· Organizational details (including staffing chart, leadership and management structure, reporting structure, and lists for staff providing supportive housing, clinical services);

· Resumés of existing staff that indicate position titles, linguistic and cultural skills, education, experience, and certifications;

· Staff training, recruitment, and retention process;

· Organizational practices related to recruiting, hiring, and promoting multi-lingual staff and/or a culturally or ethnically diverse workforce; and

· Organizational training requirements related to diversity, equity, and/or inclusion (e.g., LGBTQ+, cultural and linguistic capacity, racial equity).

3. Additional required plans, including:

· Provision of quarterly staffing reports to the BHSD using County-provided or other tools.

Section 2: Proposed Program Description[footnoteRef:4] Total Points: [4: A written response for each criterion in Section 2 is required. Refer to the RFSQ Section IV, “Offeror’s Submission” for additional details.]

· Organization methods to addresses barriers when staff are not available to meet the County threshold languages;

· Current and future actions and changes to ensure diversity and inclusion in service delivery; and

· Capacity to obtain collateral information for the underrepresented, underserved, unserved, and/or multicultural populations.

· County Health System (primary care physicians, County clinics), Adult Custody Health, Pharmacy, and Social Services Agency;

· Justice System (judges, public defender, district attorney, and probation/parole officers);

· Housing organizations [subsidy administrators, property managers];

· Other County-contracted providers responsible for clients’ behavioral health treatment (mental health, substance use or co-occurring services) and placement upon release; and

· Organizational experience working successfully with system partners.

Section 3: Historical Performance Analysis Total Points:

Historical data analysis will be reviewed by BHSD Program Subject Matter Experts (SMEs) and rated according to the criteria laid out in this RFSQ.