PWS_Attachment_4-YCP-Quarterly_Assessment_Summary_Report.pdf

ChalleNGe Operational and

Resource Effectiveness (CORE) Program

Quarterly Assessment Summary Report

June 2016

STATE NAME

Program/State:

Program Director's Contact Information:

Analyst's Contact Information:

Assessment Date: 01/14/2016 CAP Due Date: 04/11/2016 CAP Submission Date: 04/07/2016 On‐time submission: Yes

Operational Compliance and Performance Summary

FINAL

NO GOs

FINAL

N/As

Significant Findings

Systemic Issues

Addressed in CAP

Category Rating Addressed in CAP

4 13 0 2 4 Overall Rating OUTSTANDING

Graduation Target

OUTSTANDING Not Required

Placement Rate 6 months

OUTSTANDING Not Required

Placement Rate 12 months

EXCELLENT Not Required

Contact Rate 6 months

OUTSTANDING Not Required

Contact Rate 12 months

OUTSTANDING Not Required

Significant Findings:

No significant findings were found.

Operational Compliance Operational Performance

Program: XXXX YCA /State Name

Operational Compliance Final NO GOs

Topic Question CORE Comment from Checklist CAP Synopsis/Recommendations for State, NGB Significant Finding Systemic ROI Number

CAP

Number

Corrective Action Plan 25d. Did each strategy from the Corrective Action Plan resolve the issue of non‐compliance?

Corrective actions implemented to resolve noncompliance issues associated with mentor requirements did not result in compliance.

The Case Managers continue to develop strategies to be in compliance with Mentor/Mentee requirements on the Post Residential Phase. The program has established an aggressive contact process in order to comply with placement requirements. In order to correct this systematic finding, the Director has the RPM department producing a bi‐weekly report that reflect the status of Mentors/Mentees contact and the actions made by each Case Manager in order to successfully comply with the graduates' Placement. Case

Managers are aware that if mentors do not meet their responsibilities, they must take over contacting the graduates and verifying placement activities. After the 15 days of each month, Case Manager will identify those that have not been contacted with the purpose to conduct a visit to the Home of Record, to obtain new contact information that will help the Case Manager and the Mentor.

Case Managers will write a note for every action taken and also ensure that the parents get involved in the placement process.

1 1

Director Self Evaluation 26c. Did the Director’s Self‐Evaluation identify all areas of non‐compliance?

Program did not identify noncompliance issues in the area of mentors meeting their monthly contact and placement verification requirements.

DSA submission deadline will be established in order to comply with suspense date provided by NG‐ Jl‐A Y. All non‐compliance issues will be clearly assess during the next DSA, scheduled for year 2017. The Director will address the systematic finding of non‐compliance with Mentor/Mentee requirements on her next DSA on year 2017.

2 2

Program: XXXX YCA/State Name

Topic Question CORE Comment from Checklist CAP Synopsis/Recommendations for State, NGB Significant Finding Systemic ROI Number

CAP

Number

Post‐Residential Requirements

85b. During the 12‐month Post‐Residential Phase, does a minimum of four contacts, four hours of contact, or a combination of both occur each month between the Mentor and Cadet, including two face‐ to‐face contacts each month if possible?

Systemic: Not all mentors are meeting the monthly contact requirements.

Yes 3 3

Post‐Residential Requirements 85c. Have Mentors verified all placement activities each month in the Mentor report?

Systemic: Not all mentors are verifying placement activities each month in the mentor reports.

Yes 3 3

The RPM section will continue to explore new processes that will assist with mentors in honoring their commitments. Case Managers will continue to monitor the Mentors responsibility in orders to comply with the monthly contact report. The RPM section will familiarize themselves with their responsibilities as described in the RPM Manual in order to improve placement activities. At the end of the Post‐Residential Phase, everyone that completed this phase will be invited to next graduation activity of the ongoing cycle. The intent is to promote the commitment to the Program of those that are culminating the Post‐Residential

Phase and also to create a sense of responsibility of those ready to graduate. Each graduation ceremony will have the class that is graduating, the new incoming class and the cycle that will be completed the Post Residential Phase. During the Residential Phase, each student will have a

Facebook account that will be use during the Post‐ Residential Phase for the purpose to get in contact with their Mentor and Case Manager, provide contact information changes, provide important changes on lifestyle and provide information regarding placement. MIS personnel will monitor this social media for appropriate use by students.

Operational Performance Overall Rating: OUTSTANDING

Topic Explanation for Unsatisfactory Rating CAP Synopsis/Recommendations for State, NGB Score Rating Addressed in CAP

Graduation Target 150.17% OUTSTANDING

Placement Rate 6 months 84.81% OUTSTANDING

Placement Rate 12 Months 86.77% EXCELLENT

Contact Rate 6 months 97.35% OUTSTANDING

Contact Rate 12 months 97.70% OUTSTANDING

Training T‐Level T‐4 NOT RATED Not Required

Not Required

Not Required

Not Required

Not Required

CAP Submission Date: 4/07/2016 On-time submission: Yes

Resource Management Compliance and Financial Performance Summary

FINAL

NO GOs

FINAL

N/As

Significant

Findings

Systemic

Issues

Addressed in CAP Category Rating Addressed in CAP

1 47 0 0 0 Overall

Rating OUTSTANDING

Dollar Cost

Per Cadet OUTSTANDING Not Required

Budget

Execution EXCELLENT Not Required

Significant Findings:

Resource Management Compliance Financial Performance

Resource Management Compliance Final NO GOs

Topic Question CORE Comment from Checklist CAP Synopsis/Recommendations for State, NGB Significant Finding Systemic

ROI

Number

CAP

Number

Program audit and risk assessment

82. Has the USPFO-IR conducted an audit of the

Program or review (full or partial) of the Program at least every three years?

The USPFO-IR did not conduct a full or partial audit within the allowable time. The last USPFO-IR audit was conducted in 2012.

XX YCA did not address this finding. 1 Not

Addressed

Financial Performance Overall Rating:

Topic Explanation for Unsatisfactory Rating CAP Synopsis/Recommendations for State, NGB Score Rating Addressed in CAP

Dollar Cost per Cadet 70.42% OUTSTANDING

Budget Execution 98.88% EXCELLENT

Program/State: XXXX YCA/State Name

Analyst's Contact Information:

Assessment Date: 01/28/2016 CAP Due Date: 06/06/2016

CAP Submission Date: 06/05/2016 On-time submission: Yes

Operational Compliance and Performance Summary

FINAL

NO GOs

FINAL

N/As

Significant

Findings

Systemic

Issues

Addressed in CAP Category Rating Addressed in CAP

23 30 0 5 23 Overall

Rating

MARGINAL

Graduation

Target OUTSTANDING Not Required

Placement Rate

6 months MARGINAL Not Required

Placement Rate

12 months UNSATISFACTORY No

Contact Rate

6 months MARGINAL Not Required

Contact Rate

12 months UNSATISFACTORY No

Significant Findings:

Topic Question CORE Comment from Checklist CAP Synopsis/Recommendations for State, NGB Significant Finding Systemic ROI Number

CAP

Number

Mandatory minimum staffing levels

4. Does the Program meet the mandatory minimum

Staffing levels?

Systemic: Program does not meet the mandatory minimum staffing level of 54. Program currently employs 52.

The Director has requested additional base budget funding from NG-J1-AY to support the hiring of additional personnel to meet mandatory minimum staffing levels. (Ref memo: 01 Jan 16 – Budget

Increase Request to NGB). The Director has requested increased funding from the Michigan

State Legislature, increasing the base budget by

$710,000 to support hiring of additional personnel.

(On-going: 01 Feb 16). The Director requested an exception to policy to mandatory minimum staffing levels (01 Jun 16). The Director will conduct an analysis and determine if graduation target should be adjusted.

Yes 1 1

Staff training 22a. Have Staff members completed CPR/first aid training?

Nineteen staff members require CPR/First Aid training.

2 2

Staff training 22b. Have Staff members completed gang awareness training?

Twenty-four staff members require Gang

Awareness training.

2 2

Staff training 22c. Have Staff members completed conflict resolution training?

Twenty-four staff members require Conflict

Resolution training.

2 2

Staff training

23. Is the Program in compliance with State policies requiring mandated reporting?

Nineteen staff members require Mandated

Reporter training.

2 2

Corrective Action Plan

25c. Did the Corrective Action Plan address each area of non-compliance from the Report of

Evaluation with specific strategies, updated policies, detailed procedures, etc?

Program did not address the operational performance standards that were "Not in

Compliance."

3 3

Corrective Action Plan 25d. Did each strategy from the Corrective Action

Plan resolve the issue of non-compliance?

Corrective actions implemented to resolve noncompliance issues associated with mentor requirements did not result in compliance.

3 3

All staff will be required to attend mandatory staff training during summer 2016 cycle break. The teaching staff will receive mandatory training during Marshall School professional development days during Cycle 35. (nlt: 18Jan16)

The Director provided guidance to the Academy staff to review 2013/2016 CORE results and to begin a line by line review with corrective action. Will conduct monthly updates until complete. (nlt:

18Jan16)

Topic Question CORE Comment from Checklist CAP Synopsis/Recommendations for State, NGB Significant Finding Systemic ROI Number

CAP

Number

Director Self Evaluation 26c. Did the Director’s Self-Evaluation identify all areas of non-compliance?

Mentors not meeting their requirements was not identified as a noncompliance issue in the DSA.

Comments on item #26c states, “Mentors not meeting their requirements was not identified as a non-compliance issues in the DSA.” The Director missed this element during the DSA and will use greater care in future reviews in order to not miss elements in the future.

4 4

Confidential information

32g. Have all staff, mentors, and volunteers been trained on the specifics of the Confidentiality SOP to include examples of breaches of confidentiality?

Twenty-one staff members require Confidentiality training.

2 2

Hands-off leadership guidance 35a. Have staff members been trained on the Hands-

Off Leadership program?

Nineteen staff members require Hands-Off

Leadership training.

2 2

Acclimation Period Requirements

51. Is the Program’s Acclimation Period pool of prospective Cadets sufficient to select enough qualified Cadets to equal the Program’s Cadet graduation target plus its historical attrition rate over the 22-week Residential Phase?

Program's Acclimation Period pool is not sufficient to meet graduation target for the last two classes.

The Director has hired a second recruiter with additional Social Media and Public Affairs skills to improve recruiting efforts. (Completed: 01May16) The

Director has re-prioritized recruiting efforts and recruiters are now focused on recruiting to bigger populations of at-risk youth and supporters of at-risk youth. New recruiting priority as follows: At-Risk

Youth, Parents of At-Risk Youth, Youth Supporting

Organizations, Educator Association, Veterans Support

Organizations, State of Michigan Department of

Health and Human Services, Court Involved Youth, and

Juvenile Court Judges and Staff. Recruiting has scheduled twice as many orientations and increased agency/partner program briefings by 200%. (Complete nlt: 30Nov16) The Director is refining the educational curriculum and cadet course work to increase Cadet loss percentage from 26.8% to below 20%. (nlt: 5Jul17)

The program is limited to 150 beds. Acclimation phase is conducted at Ft Custer and accommodates

160 Cadets, plans are being conducted to move back to Ft. Custer in July 2017. A new application process that includes both the GRIT and Locus of Control tests that provides a quantifiable measure of commitment to succeed is being used.

Yes 5 5

All staff will be required to attend mandatory staff training during summer 2016 cycle break. The teaching staff will receive mandatory training during Marshall School professional development days during Cycle 35. (nlt: 18Jan16)

Topic Question CORE Comment from Checklist CAP Synopsis/Recommendations for State, NGB Significant Finding Systemic ROI Number

CAP

Number

Residential Phase TABE

68e. Are the administered Pre-TABE test scores recorded into a data management system NLT Week

4 of the Residential Phase?

Pre-TABE scores are not entered into the data management system.

6 6

Residential Phase TABE 68i. Are the administered Post-TABE test scores recorded into a data management system?

Post-TABE scores are not entered into the data management system.

6 6

CORE Curriculum

69a. Has the Program Director developed and approved curriculum for each of the seven non-academic core components?

The Program Director did not have a fully developed or approved curriculum for the seven non-academic

Core Components..

The Director will review the 8-core component curriculum and approve no later than 17July2016.

7 7

Mentor Recruiting

77c. At the conclusion of Week13 of the Residential

Phase does the Program have all of the required prospective Mentors recruited?

Reviewed 106 files from NGB Class 44/XX YCA Class

32. Four files (3.7 %) showed that not all mentors were recruited by the end of Week 13.

8 8

Mentor Requirements

83. By Week 13 of the Residential Phase, are

Mentors and Cadets matched in a formal event that, when geographically feasible, includes a joint meeting with the case manager, Mentor and Cadet, and the signing of a written Mentoring agreement?

Reviewed 106 files from NGB Class 44/XX YCA Class

32. Four files (3.7 %) showed that not all mentors were matched by the end of Week 13.

8 8

Post-Residential Requirements

85a. Beginning in Week 14 of the Residential Phase, are cadets who are matched with a mentor making weekly contacts with their mentor?

Systemic: Not all Cadets are making weekly contact with their mentor beginning in Week 14.

Yes 8 8

Post-Residential Requirements

85b. During the 12-month Post-Residential Phase, does a minimum of four contacts, four hours of contact, or a combination of both occur each month between the Mentor and Cadet, including two face-to-face contacts each month if possible?

Systemic: Not all mentors are meeting the monthly contact requirements.

Yes 8 8

Post-Residential Requirements 85c. Have Mentors verified all placement activities each month in the Mentor report?

Systemic: Not all mentors are verifying placement activities each month in the mentor reports.

Yes 8 8

NGB currently does not provide a digital management system to enter TABE scores into. As the NGB Cadet tracker comes on line the summer of

2017, Michigan will migrate TABE spreadsheets to the Cadet Tracker database.

The RPM staff continues to emphasize mentor contact and training during all phases of the Cadet training cycle. Monthly contact with Mentors and

Mentees is being pursued by RPM staff with continued negative contact by a portion of Mentors and Mentees. There are no consequences for non-compliance of this requirement and RPM staff does not have the ability to force compliance. Michigan currently does not have an action plan the effectively works to resolve this issue. The Director will continue to work new solution for compliance to these requirements until an effective solution is found.

Operational Performance Overall Rating: MARGINAL

Topic Explanation for Unsatisfactory Rating CAP Synopsis/Recommendations for State, NGB Score Rating Addressed in CAP

Graduation Target 100.00% OUTSTANDING

Placement Rate 6 months 61.97% MARGINAL

Placement Rate 12 Months Average placement rate over the last four class was

59%.

CAP plan did not address the standard. Talked to the Director on 21 June 2016 in reference to not addressing the Unsatisfactory rating in his CAP.

59.66% UNSATISFACTORY

Contact Rate 6 months 65.13% MARGINAL

Contact Rate 12 months Average contact rate over the last four class was

49%.

CAP plan did not address the standard. Talked to the Director on 21 June 2016 in reference to not addressing the Unsatisfactory rating in his CAP.

49.79% UNSATISFACTORY

Training T-Level T-4 NOT RATED Not Required

Not Required

Not Required

No

No

CAP Submission Date: Did Not Submit On‐time submission: NO

FINAL

NO GOs

FINAL

N/As

Significant Findings

Systemic Issues

Addressed in CAP

Category Rating Addressed in CAP

14 44 0 0 0 Overall Rating UNSATISFACTORY

Dollar Cost Per Cadet

UNSATISFACTORY No

Budget Execution

OUTSTANDING Not Required

Significant Findings:

Topic Question CORE Comment from Checklist CAP Synopsis/Recommendations for State, NGB Significant Finding Systemic

ROI

Number

CAP

Number

Budget submission 1b. Did the proposed budget include an updated annual goal‐focused State Plan?

The State Plan was not included in the budget submission.

Not Addressed

Quarterly reports

7. Are Quarterly Reports submitted not later than 30 days after the end of each quarter?

Submission of quarterly reports could not be validated for FY14 3rd and 4th quarter and FY15 1st and 2nd quarter.

Not Addressed

Budget management 12b. Does the Program ensure that lodging and per diem reimbursements do not exceed the Joint Travel Regulation (JTR) limits?

Documentation could not be provided to validate JTR rates had been verified.

Not Addressed

Budget management

26. Does the Program manage a petty cashfund in compliance with applicable State regulations?

XX YCA Director does not have oversight on PTSA or Student Council activity.

Not Addressed

Corrective Action Plan 37c. Did the Corrective Action Plan address each area of non‐compliance from the Report of Evaluation?

XX YCA did not address their financial performance strategies in the CAP.

Not Addressed

Corrective Action Plan 37d. Did each strategy from the Corrective Action Plan resolve the issue of non‐compliance ?

XX YCA did not address their financial performance strategies in the CAP.

Not Addressed

Director Self Evaluation 38b. Was the Program Director’s Self‐Evaluation submitted within the required timeframe included in the Memo from Chief, NG‐J1‐AY?

DSE due date was 15 Apr 14; DSE received 17 Apr 14.

Not

Addressed

Director Self Evaluation 38c. Did the Director’s Self‐Evaluation identify all areas of non‐compliance?

DSE did not address non‐compliance findings.

Not

Addressed

Matching funds

41. If the Grantee’s (State) share of cash funding for the current year is from sources outside the respective Grantee's State appropriation, did the State Comptroller/Treasurer, or designated responsible individual, certify that the funds are available and are exclusively committed for the express purpose of funding the Grantee's (State) share of the Program?

Grantee (State) certification letter did not meet NGYCP‐CA requirements when fund sources are outside the respective Grantee's State appropriation, per the AY Chief.

Not Addressed

Compliance with internal control reporting procedures

55d. Did the consolidated, detailed listing of all un‐ cleared obligations include a projected timetable (date) for their liquidation and disbursements to the

USPFO?

FY13 unliquidated detailed listings did not include a projected timetable for liquidation/disbursements to the USPFO.

Not Addressed

CAP was not submitted for the RM Compliance Inspection. Emailed the Director on 16 June 2016 in reference to submitting a CAP to address the RM non‐ compliance issues.

Topic Question CORE Comment from Checklist CAP Synopsis/Recommendations for State, NGB Significant Finding Systemic

ROI

Number

CAP

Number

Joint Reviews

63. Did the USPFO conduct joint reviews no less than tri‐annually with the YC Program Director, Federal Program Manager, Grants Officer Representative and Grantee (State) Financial Representative to ensure contractual and financial compliance?

Reviews are being conducted, but have not included XX YCA Director nor a State Finance Office representative.

Not Addressed

GOR Responsibilities

72. Does the Grants Officer Representative use the DoD Assistance Award Action (DAADS) Report System within 15 days of award?

GOR is TDY in Germany and evidence of posting awards into DAADS within 15 days could not be located.

Not Addressed

CA Modifications

80a. Have all requests for funding modifications of a cooperative agreement been initiated by the Cooperative Agreement Program Manager and provided to the GOR for action?

Documentation could not be provided to validate FPM had been formally requesting modifications to MYCA's cooperative agreement.

Not Addressed

Program audit and risk assessment

82. Has the USPFO‐IR conducted an audit of the Program or review (full or partial) of the Program at least every three years?

USPFO‐IR's last audit of XX YCA was 28 Sep 2011.

Not

Addressed

CAP was not submitted for the RM Compliance Inspection. Emailed the Director on 16 June 2016 in reference to submitting a CAP to address the RM non‐ compliance issues.

Topic Explanation for Unsatisfactory Rating CAP Synopsis/Recommendations for State, NGB Score Rating Addressed in CAP

Dollar Cost per Cadet

CAP was not submitted for the RM Performance Inspection. Emailed the Director on 16 June 2016 in reference to submitting a CAP to address the RM

Unsatisfactory ratings.

117.44% UNSATISFACTORY

Budget Execution 99.85% OUTSTANDING

Yes

Program/State: XXXX YCA/State Name

Analyst's Contact Information:

Assessment Date: 02/11/2016 CAP Due Date: 06/06/2016 CAP Submission Date: 05/26/2016 On‐time submission: Yes

Operational Compliance and Performance Summary

FINAL

NO GOs

FINAL

N/As

Significant Findings

Systemic Issues

Addressed in CAP

Category Rating Addressed in CAP

7 30 0 2 7 Overall Rating SATISFACTORY

Graduation Target

OUTSTANDING Not Required

Placement Rate 6 months

SATISFACTORY Not Required

Placement Rate 12 months

SATISFACTORY Not Required

Contact Rate 6 months

SATISFACTORY Not Required

Contact Rate 12 months

SATISFACTORY Not Required

Significant Findings:

Topic Question CORE Comment from Checklist CAP Synopsis/Recommendations for State, NGB Significant Finding Systemic ROI Number

CAP

Number

Mandatory minimum staffing levels

4. Does the Program meet the mandatory minimum

Staffing levels?

Program currently employs 50 employees and is required to have 58 for a target of 125. Program does not meet the mandatory minimum staffing level.

OYCP is currently working below the mandatory minimum staffing level with an approved waiver from NG-J1-AY. OYCP is in the process of requesting additional Full-time employees (FTE) from the

Oregon State Legislature. If the additional FTE are approved the hiring process will begin during the next biennium.

1 1

Staff training 22a. Have Staff members completed CPR/first aid training?

Three staff members need CPR/First Aid training. 2 2

Staff training 22b. Have Staff members completed gang awareness training?

Three staff members need Gang Awareness training.

2 2

Corrective Action Plan 25d. Did each strategy from the Corrective Action

Plan resolve the issue of non-compliance?

Corrective actions implemented to resolve noncompliance issues associated with mentor requirements did not result in compliance.

Program concurs with the finding, but will take no action for the following reasons. This problem was addressed in the CAP and OYCP does it’s best to ensure mentors send in their monthly reports;

however, it is virtually impossible to get 100% of the cadet mentors to comply with this unrealistic requirement. Every program / academy in the nation should be failing to meet the 100% requirement. NG-J1-AY needs to address this.

3 3

Program concurs with the finding, but will take no action for the following reasons. It is impossible to train a new employee before they are an employee.

The new employees had been working at OYCP for less than 2 weeks and were going through their orientation and training at the time of the inspection. The new NG-J1-AY Training Policy Letter that is now in place gives a new employee 1 to 6 months to complete the mandatory in-house training requirements.

Topic Question CORE Comment from Checklist CAP Synopsis/Recommendations for State, NGB Significant Finding Systemic ROI Number

CAP

Number

Director Self Evaluation 26c. Did the Director’s Self-Evaluation identify all areas of non-compliance?

Program did not identify noncompliance issues in the areas of the daily assessment of Cadets in the

Acclimation Period, and mentors meeting their monthly contact and placement verification requirements.

Program concurs with the finding, but will take no action for the following reasons. This problem was addressed in the CAP and OYCP does it’s best to ensure mentors send in their monthly reports to verify contacts and placement; however, it is virtually impossible to get 100% of the cadet mentors to comply with this unrealistic expectation.

Every program / academy in the nation should be failing to meet the 100% requirement. NG-J1-AY needs to address this. Also states the Program is receiving two NO GOs for one finding.

4 4

Post-Residential Requirements

85b. During the 12-month Post-Residential Phase, does a minimum of four contacts, four hours of contact, or a combination of both occur each month between the Mentor and Cadet, including two face-to-face contacts each month if possible?

Systemic: Not all mentors are meeting the monthly contact requirements.

Yes 5 5

Post-Residential Requirements 85c. Have Mentors verified all placement activities each month in the Mentor report?

Systemic: Not all mentors are verifying placement activities each month in the mentor reports.

Yes 5 5

Program concurs with the finding, but will take no action for the following reasons. There is no corrective action to achieve compliance for this unrealistic requirement. Mentors are unpaid volunteers and 100% compliance for 100% of the mentors is out of the Program Director’s control.

This is another area that NG-J1-AY needs to address.

Reasonable levels of reporting should be determined and placed in all of the Youth Challenge guiding documents. In my opinion, 7 out of 10 mentors reporting each month would be a success

(70%).

Operational Performance Overall Rating: SATISFACTORY

Topic Explanation for Unsatisfactory Rating CAP Synopsis/Recommendations for State, NGB Score Rating Addressed in CAP

Graduation Target 108.54% OUTSTANDING

Placement Rate 6 months 64.02% SATISFACTORY

Placement Rate 12 Months 70.99% SATISFACTORY

Contact Rate 6 months 71.18% SATISFACTORY

Contact Rate 12 months 80.46% SATISFACTORY

Training T‐Level T‐4 NOT RATED Not Required

Not Required

Not Required

Not Required

CAP Submission Date: 5/26/2016 On-time submission: Yes

FINAL

NO GOs

FINAL

N/As

Significant Findings

Systemic Issues

Addressed in CAP

Category Rating Addressed in CAP

5 43 0 0 5 Overall Rating OUTSTANDING

Dollar Cost Per Cadet

OUTSTANDING Not Required

Budget Execution

OUTSTANDING Not Required

Significant Findings:

Topic Question CORE Comment from Checklist CAP Synopsis/Recommendations for State, NGB Significant Finding Systemic

ROI

Number

CAP

Number

Cadet living allowance 16c. Are field trip expenses limited to no more than $25 per month, per cadet?

The Program conducted 10 field trips in FY14, 3 of those filed trips exceeded the funding limits. The Program conducted 12 field trips in FY15, 2 of those filed trips exceeded the funding limits.

The process OYCP was using to determine the monthly costs for cadets on field trips was being figured in the aggregate. OYCP has since changed program policy and SOP so the budget officer gets a list of cadets attending and now tracks total costs per cadet, per month.

1 1

Director Self Evaluation 38c. Did the Director’s Self-Evaluation identify all areas of non-compliance?

The GOR did not input all modifications in DAADs with in 15 days of the modification.

No action taken. The OYCP Director's response on the DSA was information received from the USPFO.

Again the program receives two "Go Go's" for one finding.

2 2

Matching funds

41. If the Grantee’s (State) share of cash funding for the current year is from sources outside the respective Grantee's State appropriation, did the State Comptroller/Treasurer, or designated responsible individual, certify that the funds are available and are exclusively committed for the express purpose of funding the Grantee's (State) share of the Program?

The current years State certification letter does not disclose the funds used to support the required 25% State match is a combination of USDA School Lunch Program reimbursement, State basic school support and State general revenue fund.

The 2016 State Match Certification letter was signed by the Oregon TAG in Sep 2015 and accepted by NG- J1-AY during the budget approval process. The letter has had the same language in it for many years.

Since the evaluators found the letter unacceptable, program staff and OMD Finance staff drafted a new letter that was acceptable to the evaluators. This took place while the evaluation team was on-site.

The letter was signed by the TAG a few weeks later and forwarded to NG-J1-AY and the evaluation team.

3 3

Joint Reviews

63. Did the USPFO conduct joint reviews no less than tri-annually with the YC Program Director, Federal Program Manager, Grants Officer Representative and Grantee (State) Financial Representative to ensure contractual and financial compliance?

Joint reviews are being conducted but Program staff have not been included. Program staff would perform a reconciliation with the State and the State representative would take that information to the review.

Triennial reviews (now quarterly reviews) are conducted between the State Comptroller's office and the USPFO Comptroller's office. The USPFO reviews and signs the quarterly documents reported to DFAS. However, the USPFO's GOR will work closely with the OYCP Federal Program Manager to ensure all appropriate representatives are present during the review.

4 4

GOR Responsibilities

72. Does the Grants Officer Representative use the DoD Assistance Award Action (DAADS) Report System within 15 days of award?

OYCP DAADS reporting was reviewed for FY2015.

There were 5 modifications for 2015. Two of the 5 were not reported within the 15 day requirement.

GOR could not explain the non-compliance except both modifications were at year end and he was very busy.

The GOR has set up one day during the week used solely for entering information into DAADS. Also, with the implementation of the USPFO signing all modifications after TAG signature this will expedite the entering of modification information into DAADS.

5 5

Topic Explanation for Unsatisfactory Rating CAP Synopsis/Recommendations for State, NGB Score Rating Addressed in CAP

Dollar Cost per Cadet 92.05% OUTSTANDING Budget Execution 99.18% OUTSTANDING

Analyst's Contact Information:

Assessment Date: 03/17/2016 CAP Due Date: 05/31/2016 CAP Submission Date: 05/24/2016 On‐time submission: Yes

Operational Compliance and Performance Summary

FINAL

NO GOs

FINAL

N/As

Significant Findings

Systemic Issues

Addressed in CAP

Category Rating Addressed in CAP

6 28 0 3 6 Overall Rating SATISFACTORY

Graduation Target

UNSATISFACTORY Yes

Placement Rate 6 months

SATISFACTORY Not Required

Placement Rate 12 months

SATISFACTORY Not Required

Contact Rate 6 months

EXCELLENT Not Required

Contact Rate 12 months

EXCELLENT Not Required

Significant Findings:

Topic Question CORE Comment from Checklist CAP Synopsis/Recommendations for State, NGB Significant Finding Systemic ROI Number

CAP

Number

Corrective Action Plan 25d. Did each strategy from the Corrective Action

Plan resolve the issue of non-compliance?

Corrective actions implemented to resolve noncompliance issues associated with mentor requirements and an insufficient Acclimation Period pool to meet graduation target did not result in compliance.

XX YCA will continue to identify methods to improve compliance with Mentor reporting requirements.

Program also recognize complete compliance with program requirements is dependent on the commitment of individual volunteers who may fail to, or have difficulty, achieving compliance requirements. Additionally, cadets on active duty in the military are exempt from mentor contact and reporting requirements, which is not accounted for

(filtered) in the current data management system used to measure reporting percentages. The program has, and will continue to enhance its outreach efforts to ensure achievement of program recruitment, retention and graduation goals.

1 1

Topic Question CORE Comment from Checklist CAP Synopsis/Recommendations for State, NGB Significant Finding Systemic ROI Number

CAP

Number

Director Self Evaluation 26c. Did the Director’s Self-Evaluation identify all areas of non-compliance?

Mentors not meeting their monthly mentor report requirements was not identified as a noncompliance issue in the DSA.

2 2

Director Self Evaluation

26d. Have all areas identified as non-compliant in the Director’s Self-Evaluation been brought into compliance?

Strategies implemented to resolve the mentor contact requirements did not lead to compliance.

2 2

Program non-concurs with this finding. The

Program did not rate Operational Performance standard 26c, "Mentors submit monthly reports to verify the Cadets placement activity" as a "NO GO".

This compliance item was rated a "GO" on the DSA because mentors do submit monthly reports to verify Cadet placement activity. During the on-site evaluation, we were given a "NO GO" on failure to correctly rate compliance standard on the DSE. The

CORE Team stated that "not ALL mentors submit reports EVERY month" and therefore we did not rate program compliance appropriately. Program leadership refutes the interpretation of the standard. The standard does not state "Do ALL mentors submit monthly report to verify Cadet placement EVERY month", the DSA would have rated the standard a "NO GO." XX YCA will continue to identify methods to improve compliance with

Mentor reporting requirements. Mentors will continue to be trained on the importance of establishing and maintaining the cadet/mentor relationship for 14 months during mentor training.

Additionally, monthly letters and e-mails will be sent to the mentors during the Post Residential

Phase stressing the importance of sustaining 4 contacts a month, 4 hours of contact a month, or a combination of both with cadets for the duration of the Post Residential Phase.

Topic Question CORE Comment from Checklist CAP Synopsis/Recommendations for State, NGB Significant Finding Systemic ROI Number

CAP

Number

Acclimation Period Requirements

51. Is the Program’s Acclimation Period pool of prospective Cadets sufficient to select enough qualified Cadets to equal the Program’s Cadet graduation target plus its historical attrition rate over the 22-week Residential Phase?

Systemic: Program's Acclimation Period pool was insufficient to meet graduation target for the last three classes. Using historical data from the last four classes, XX YCA needs to register 128 candidates on Day 1 of Acclimation.

We have continued to encounter barriers in our efforts to achieve this goal which include: high turnover, military obligations, employee pay considerations, increase in high school retention efforts, completion of new alternative educational facilities, and etc... The Program continues to create more a diverse set of educational opportunities for our cadets. Cadets are able to: I) test for their HiSet diploma (formerly GED); 2) participate in the high school credit recovery option that allow them to return to high school and graduate with their class; 3)enroll in the HiSet

Option program and receive a high school diploma;

4) earn college credits through the University of

State XX-Western; 5) earn college level certifications that result in credentials that may be applied toward eligibility for Tier One status for application into the military; 6) exploring vocational education opportunities through the Butte

Technical Institute that can be completed during the residential phase of the program.

Yes 3 3

Post-Residential Requirements

85b. During the 12-month Post-Residential Phase, does a minimum of four contacts, four hours of contact, or a combination of both occur each month between the Mentor and Cadet, including two face-to-face contacts each month if possible?

Systemic: Not all mentors are meeting the monthly contact requirements.

Yes 4 4

Post-Residential Requirements 85c. Have Mentors verified all placement activities each month in the Mentor report?

Systemic: Not all mentors are verifying placement activities each month in the mentor reports.

Yes 4 4

Mentors will continue to be trained on the importance of establishing and maintaining the cadet/mentor relationship for 14 months during mentor training. Monthly letters and e-mails will be sent to the mentors during the Post Residential

Phase stressing the importance of sustaining 4 contacts a month, 4 hours of contact a month, or a combination of both with cadets for the duration of the 12 month Post Residential Phase. Mentors will be given a variety of options to submit mentor reports, including reporting via e-mail, Survey

Monkey on-line reporting, phone calls, texting, Facebook and US mail.

Topic Explanation for Unsatisfactory Rating CAP Synopsis/Recommendations for State, NGB Score Rating Addressed in CAP

Graduation Target Program's Acclimation Period pool was insufficient to meet graduation target.

CAP did not address the standard. Emailed the Director on 27 June 2016 in reference to not addressing the Unsatisfactory rating in her CAP.

Received an updated CAP from the Director on 28

June 2016. The Director stated the program continues to encounter barriers in their efforts to achieve graduation target due to high turnover, military obligations, employee pay considerations, increase in high school retention efforts, completion of new alternative educational facilities, and etc... The Program continues to create more a diverse set of educational opportunities for our cadets. Cadets are able to: I) test for their HiSet diploma (formerly GED); 2) participate in the high school credit recovery option that allow them to return to high school and graduate with their class;

3)enroll in the HiSet Option program and receive a high school diploma; 4) earn college credits through the University of State XXWestern; 5) earn college level certifications that result in credentials that may be applied toward eligibility for Tier One status for application into the military; 6) exploring vocational education opportunities through the Butte Technical Institute that can be completed during the residential phase of the program.

83.29% UNSATISFACTORY

Placement Rate 6 months 69.27% SATISFACTORY

Placement Rate 12 Months 69.54% SATISFACTORY

Contact Rate 6 months 92.18% EXCELLENT

Contact Rate 12 months 90.03% EXCELLENT

Training T‐Level T‐4 NOT RATED Not Required

Yes

Not Required

Not Required

CAP Submission Date: 5/24/2016 On‐time submission: Yes

FINAL

NO GOs

FINAL

N/As

Significant Findings

Systemic Issues

Addressed in CAP

Category Rating Addressed in CAP

0 55 0 0 0 Overall Rating MARGINAL

Dollar Cost Per Cadet

UNSATISFACTORY Yes

Budget Execution

EXCELLENT Not Required

Significant Findings:

Topic Question CORE Comment from Checklist CAP Synopsis/Recommendations for State, NGB Significant Finding Systemic

ROI

Number

CAP

Number kseery Typewritten Text Program received 100% on their RM Compliance Inspection.

kseery Typewritten Text

Topic Explanation for Unsatisfactory Rating CAP Synopsis/Recommendations for State, NGB Score Rating Addressed in CAP

Dollar Cost per Cadet Program's Acclimation Period pool was insufficient to meet graduation target.

CAP did not address the standard. Emailed the Director on 27 June 2016 in reference to not addressing the Unsatisfactory rating in her CAP.

Received an updated CAP from the Director on 28

June 2016. The Director stated the program continues to encounter barriers in their efforts to achieve graduation target due to high turnover, military obligations, employee pay considerations, increase in high school retention efforts, completion of new alternative educational facilities, and etc...

The Program continues to create more a diverse set of educational opportunities for our cadets. Cadets are able to: I) test for their HiSet diploma (formerly GED); 2) participate in the high school credit recovery option that allow them to return to high school and graduate with their class; 3)enroll in the HiSet Option program and receive a high school diploma; 4) earn college credits through the University of State XXWestern; 5) earn college level certifications that result in credentials that may be applied toward eligibility for Tier One status for application into the military; 6) exploring vocational education opportunities through the Butte Technical Institute that can be completed during the residential phase of the program.

105.60% UNSATISFACTORY

Budget Execution 99.00% EXCELLENT

Yes

Analyst's Contact Information:

Assessment Date: 03/31/2016 CAP Due Date: 05/19/2016

CAP Submission Date: 05/18/2016 On-time submission: Yes

Operational Compliance and Performance Summary

FINAL

NO GOs

FINAL

N/As

Significant

Findings

Systemic

Issues

Addressed in CAP Category Rating Addressed in CAP

19 34 3 2 19 Overall

Rating

UNSATISFACTORY

Graduation

Target OUTSTANDING Not Required

Placement Rate

6 months UNSATISFACTORY Yes

Placement Rate

12 months UNSATISFACTORY Yes

Contact Rate

6 months UNSATISFACTORY Yes

Contact Rate

12 months UNSATISFACTORY Yes

Significant Findings:

There were three significant findings. XXXX YCA accepts applicants into the Program who do not meet all eligibility requirements. XXXX YCA has a conflict of interest by accepting applicants into the Program as part of interagency agreements between both the Department of

Juvenile Justice (DJJ) and Department of Children and Families (DCF) in return for financial considerations. XXXX YCA does not meet all

Post-Residential requirements.

Topic Question CORE Comment from Checklist CAP Synopsis/Recommendations for State, NGB Significant Finding Systemic ROI Number

CAP

Number

Participant eligibility 1e. Are participants unemployed or underemployed?

Application could not validate the employment status of the participants.

Yes 1 1

Participant eligibility

1f. Are participants of the Program not currently on parole or probation for other than juvenile status offenses, not awaiting sentencing, not under indictment, charges, or convicted of a crime that is considered a felony if charged as an adult?

Application could not validate that all participants were not charged with or on probation for a crime that would be considered a felony if charged as an adult.

Yes 1 1

Participant eligibility 1g. Are all participants of the Program free from use of illegal drugs or substances?

In XX YCA Class 29 three Cadets had positive results on the required initial drug screening on Day 35 and on the subsequent confirmatory drug screening.

Yes 1 1

Corrective Action Plan 25d. Did each strategy from the Corrective Action

Plan resolve the issue of non-compliance?

Corrective actions implemented to resolve noncompliance issues associated with mentor requirements did not result in compliance.

The new corrective actions identified in this plan as a result of the 2016 CORE evaluation are planned to be implementable to improve compliance in this area. The strategies provided in this CAP will be monitored by Program staff to ensure whether desired results are being achieved.

2 4

The Cadet application has been adjusted to specifically and clearly ask if applicants are unemployed or underemployed. Cadets that fail their initial drug screening will be given the option for a confirmatory test at their expense and any positive reading over the required cutoff will result in immediate termination. The Cadet application has been adjusted so that applicants understand that the Program is drug-free and drug screenings will be administered during the Program. The

Program will not accept any applicants into the program that have been accused of or charged with a felony offense or a juvenile offense that would be considered a felony when charged as an adult. The

Cadet application has been updated to reflect this.

XX YCA does not actively recruit DJJ participants into the program and funding has not been adjusted either more or less as a result of the number of DJJ participants enrolled in the program.

The reality that the target demographic the program is designed to serve, will also intersect with DJJ and DCF clients is unavoidable. The inclusion of specific numbers in future agreements with the aforementioned agencies will be avoided.

The XX YCA Director will request that State

Quartermaster review the interagency agreement with DJJ to address this finding.

Topic Question CORE Comment from Checklist CAP Synopsis/Recommendations for State, NGB Significant Finding Systemic ROI Number

CAP

Number

Director Self Evaluation 26c. Did the Director’s Self-Evaluation identify all areas of non-compliance?

Cadet eligibility requirements, prospective mentor recruitment deadlines and interviews, and mentors not meeting their monthly mentor report requirements were not identified as a noncompliance issue in the DSA.

In relation to not accurately assessing fulfillment of mentor requirements, XX YCA will ensure on future

DSA’s that all requirements are properly assessed to include all related source documentation.

However, the ROI stated “During this inspection, the Operations Analyst found the Program to be out of compliance in the areas of Cadet eligibility, not terminating Cadets with positive drug test results…”

This aspect of the finding is incorrect as the issue of non-compliance occurred in August 2015 while the

DSA was conducted in April 2015.

3 5

Cadet early release

40h. Have all positive required Initial Drug Tests

(Screening Tests), Optional Confirmatory Drug Tests, For-Cause Drug Tests, or Reasonable-Suspicion Drug

Tests resulted in immediate dismissal?

Three cadets from XX YCA Class 29 with confirmatory test results greater than 15 ng/ml were not immediately terminated from the program.

Cadets that fail their initial drug screening will be given the option for a confirmatory test at their expense and any positive reading over the required cutoff will result in immediate termination.

4 6

Mentor Screening

77b. By the end of Week 6 of the Residential Phase, does the Program have 95% of the required prospective Mentor applications on file?

Ninety-five percent of the prospective mentors were not recruited by the end of Week 6.

5 3

Mentor Requirements 81c. Did the screening process include an interview conducted by Program Staff?

During the screening of prospective mentors for

XX YCA Classes 26 and 27, the RPM staff did not conduct interviews.

5 3

Post-Residential Requirements

85b. During the 12-month Post-Residential Phase, does a minimum of four contacts, four hours of contact, or a combination of both occur each month between the Mentor and Cadet, including two face-to-face contacts each month if possible?

Systemic: Not all mentors are meeting the monthly contact requirements.

Yes 5 3

Post-Residential Requirements 85c. Have Mentors verified all placement activities each month in the Mentor report?

Systemic: Not all mentors are verifying placement activities each month in the mentor reports.

Yes 5 3

RPM mentor database will include the date when the completed application is received and the date when the mentor is screened, to include when the criminal and sex offender background check is submitted and report returned, along with when the mentor is interviewed, trained, and matched.

Local background checks will be requested for the last five years along with checking against the national sex offender registry. Applicants will be encouraged to have a completed mentor application on intake day / day one.

Topic Question CORE Comment from Checklist CAP Synopsis/Recommendations for State, NGB Significant Finding Systemic ROI Number

CAP

Number

Post-Residential Requirements

86. Are Mentor- Cadet contacts made during the

Residential and Post-Residential Phases reported in a data management system?

Not all contacts made during the Residential and

Post-Residential Phases could be verified in the data management system.

5 3

Post-Residential Requirements

89b. Do case managers monitor, document, and record mentor relationship activities in the database, including contact between the mentor and mentee?

FLYCA's data base did not contain all mentor relationship activities.

5 3

Post-Residential Requirements

89c. During each month of the Post-Residential

Phase, are all Graduates accounted for in one of the three following categories: placed, not placed, or unknown?

FLYCA's current data base does not contain the required categories of Placed, Not Placed, or

Unknown.

5 3

Post-Residential Requirements

89e. Do case managers verify placement activities using one or more of the following: mentor or parent contact/report; employment supervisor;

school registration/registrar; military paperwork/recruiter; or, other documents verifying placement?

Not all placement activities are verified using the required reporting methods.

5 3

Post-Residential Requirements 89f. Do case management files contain copies of documents verifying placement activities?

Not all case management files contained documents verifying placement activities.

5 3

Post-Residential Requirements

89g. If the mentor does not fulfill the requirements or terminates the formal mentoring relationship during the Post-Residential Phase, have the case managers (or other Program Staff) verified the

Cadets’ placement activities?

Case managers are not utilizing the Memorandum for Record in lieu of a Mentor Report to validate placement activities.

5 3

Post-Residential Requirements

89h. Have case managers (or other Program Staff) documented mentoring relationship and cadet placement activities in the information management system no later than the 15th of the month following the reporting period?

Not all of the data recorded on the case managers' documentation record were entered into the data base.

5 3

RPM contacts database will include date, type of contact, and length of contact and will be updated within required timeframes. Case managers will verify Cadet placement using required source documentation which will be placed in the Cadet file.