REL_-_Needs_Sensing.pdf

Assessment of Needs by Regional Educational

Laboratory Region

These summaries of regional needs were prepared by the 2012-2017 Regional Educational Laboratories Contractors as part of their Updated Annual Plans. They are being shared so that potential offerors for the 2017-2022 Regional Educational Laboratories competition can better understand the perceived needs of each region. It is possible that offerors will have a different assessment of regional needs based on their own needs sensing process. This summary includes needs assessment data collected from July 16, 2014 to July 15, 2015.

Contents REL Appalachia

Introduction and Regional Profile Discussion of Regional Needs Needs Sensing Tools and Strategies References

REL Central

Introduction and regional profile Discussion of regional needs Needs-sensing tools and strategies References

REL Mid-Atlantic

Introduction Regional Profile Needs-Sensing Tools and Strategies

REL Midwest

Regional Profiles Discussion of Regional Needs Regional Profiles Discussion of Regional Needs Needs Sensing Tools and Strategies References

REL Northwest

Introduction and Regional Profile Needs-Sensing Tools and Strategies References

REL Pacific

Introduction Regional Profile Discussion of Regional Needs Needs-Sensing Tools and Strategies Appendix A: Sources Used for Regional News Scans

REL Southeast

Selected Student Subgroups Discussion of Regional Needs Need Sensing Tools and Strategies

REL Southwest

Introduction Regional Profile Regional needs

Needs-sensing tools and strategies Appendix. Source list for environmental scanning

REL West

Introduction and Regional Profile Discussion of Regional Needs Needs Sensing Strategies and Sources

REL Appalachia1

Introduction and Regional Profile In the following section, we detail our needs assessment findings and process for the REL Appalachia (REL AP)region by providing a regional profile of education in Kentucky, Tennessee, Virginia, and West Virginia; discussing regional needs by state; and detailing our needs sensing tools and strategies.

Across the Appalachian region, states vary in the total number of students served, with Tennessee and Virginia serving the largest student populations (Table 1). In addition to serving the most students, Virginia has the greatest percentage of ELL/LEP students. Tennessee has the greatest percentage of students eligible for free or reduced-price lunch. By contrast, Kentucky serves fewer students, but has the largest numbers of migrant and homeless students. Finally, West Virginia serves the fewest students, but has the greatest percentage of students with an individualized education program (IEP).

Table 1. State-level data for selected student subgroups

State

Total number of students1

Percentage of students receiving free or reduced-price lunch1

Percentage of

ELL/LEP

students1

Percentage of students with an IEP1 Migrant students2

Homeless students2

KY 677,389 54.3 2.9 14.1 3,861 31,210

TN 993,566 58.2 3.5 13.1 1,996 17,272

VA 1,273,825 39.6 7.4 12.6 737 18,486

WV 280,958 52.53 0.7 15.8 594 7,430

Source: 2013-2014 Common Core of Data (CCD)1; 2013-2014 Consolidated State Performance Report 2; 2012-13 Common Core of Data (CCD), 2013-2014 data do not meet NCES data quality standards3; 2009-2010 Consolidated State Performance Report, 2009-10.4

States in the region also vary in their metropolitan profile (Table 2). For example, Virginia has the greatest percentage of urban school districts; West Virginia has the greatest percentage of rural school districts; and Kentucky has the greatest percentage of suburbs/towns in the region.

1 This was prepared under contract ED-IES-12-C-0005 with CNA Corporation.

Table 2. Percentage of school districts by metro status

State� Urban Suburban Rural

�KY 3.4 46.3 50.3

TN 12.1 41.1 46.8

VA 13.6 35.6 50.8

WV 7.3 38.2 54.5

Source: 2013-2014 Common Core of Data (CCD)

States also vary across socioeconomic indicators (Table 3). In Kentucky, Tennessee, and West Virginia, 13 to 14.4 percent of all families are below the poverty level, with 21.4 to 22.3 percent of families with children below it. In these three states, about a third of children have at least one parent with a postsecondary degree, compared with nearly half of children in Virginia.

Table 3. Socioeconomic indicators

State Total number of families

Percentage of families below the poverty level

Percentage of families with children below the poverty level

Percentage of children with at least one parent with a postsecondary degree

KY 1,137,576 14.4 22.6 37

TN 1,654,579 13.3 21.4 35

VA 2,037,361 8.0 12.6 49

WV 483,367 13.0 22.2 33

Source: 2009-2013 American Community Survey (ACS).

Student achievement and education standards illustrate similarities and differences across the region (Table 4). All four states have similar graduation rates. Tennessee and West Virginia have the smallest percentage of students performing at high levels on Advanced Placement (AP) exams. Kentucky, Tennessee, and West Virginia require 22–24 credits to earn a standard diploma and they do not have high school exit exams, compared with Virginia’s 28 credits plus an exit exam. Only Tennessee does not offer an alternative credential for students who don’t meet graduation requirements.

Table 4. State-level data on student achievement and education standards

Indicator/Standard KY TN VA WV High school graduation rate for school year 2013/141

87.5 87.2 85.3 84.5

Percentage of the class of 2013 scoring a 3 or higher on an AP exam during high school2

16.3 10.1 28.3 9.4

Total number of credits required to earn standard diploma

223 224 285 246

Alternative credential for not meeting all standard requirements

Yes3 No4 Yes5 Yes8

Indicator/Standard KY TN VA WV Basis for alternative credential Students with disabilities can receive an Alternative High School Diploma (based on individual student needs) 3

Na� Students with disabilities can receive a Modified Diploma if they meet IEP but do not meet other diploma requirements

Other students not meeting diploma requirements can receive a Certificate of Program Completion, General Achievement Adult High School Diploma, or

GED®5

Students not meeting requirements can complete a high school equivalency assessment

(HSEA)

Students who drop out can receive a high school equivalency diploma for passing the

HSEA

All other students who pass the HSEA receive a standard diploma8

State has exit exam No3 No4 Yes5 Noa6,7 State finances remediation for students failing exit exams na Na Yes5 na

a. Putnam County has an algebra exit exam.

Source: .EdFacts/2013-14 Consolidated State Performance Report1; The College Board, 20142; Kentucky Legislature, n.d. 3;

Tennessee Department of Education, 20154; Virginia Department of Education, 2015a5; West Virginia Department of Education, 20126; West Virginia Department of Education, 20137; West Virginia Department of Education, 2015a8

Table 5 shows performance by state on the National Assessment of Educational Progress (NAEP). On the 2015 NAEP assessments, West Virginia students performed below national averages in grades 4 and 8 in math and reading. Tennessee students performed within three percentage points of the national average in these exams; Kentucky students performed above national averages in reading for grades 4 and 8, as well as in grade 4 math; and Virginia students performed above national averages on all four assessments (U.S. Department of Education, 2015).

Table 5. NAEP performance, state versus national

State

Percentage at or above proficient in grade 4 reading

Percentage at or above proficient in grade 8 reading

Percentage at or above proficient in grade 4 math

Percentage at or above proficient in grade 8 math

KY 40 36 40 28

TN 33 33 40 29

VA 43 36 47 38

WV 30 27 33 21

U.S. 35 33 39 32

Source: 2015 National Assessment of Educational Progress.

Teacher evaluation systems have several similarities across states and a few key differences (Table 6).

All four states include a measure of student growth/academic achievement and some form of teacher observation. Three of the four states include some form of student feedback, such as a survey.

However, states differ in the total number and weighting of evaluation system components. For example, student growth accounts for 15 percent of a teacher’s evaluation score in West Virginia, but for 35 percent in Tennessee.

Table 6. State-level information on key characteristics of teacher evaluation systems

State Year implemented System components Weight of each component KY2 2014 Four evidence indicators:

1. Student growth

2. Observations

3. Self-reflection and professional growth

4. Student voice survey

Indicators not currently weighted

TN1,3 2011 Three components:

1. Student growth

2. Student achievement

3. Qualitative measures (e.g., observations, student surveys, review of past evaluations)

Teachers with growth scores:

1. Student growth: 35%

2. Student achievement (other measures): 15%

3. Qualitative measures: 50%

Teachers without growth scores:

1. School-wide student growth: 25%

2. Student achievement (other measures): 15%

3. Qualitative measures: 60%

VA1,4 2012 Seven performance standards:

1. Professional knowledge

2. Instructional planning

3. Instructional delivery

4. Assessment of and for student learning

5. Learning environment

6. Professionalism

7. Student academic progress

1-6. 10% each

7. 40% (20% student growth, 20% alternative measures)

WV1,5 2009 Five evidence indicators:

1. Self-reflection

2. Evidence (e.g., assessment data, lesson plans, student feedback)

3. Observations

4. Student growth

5. School-wide growth in reading and math

1. Self-reflection, Evidence, and Observations: 80%

2. Student growth: 15%

3. School-wide growth: 5%

Source: Center on Great Teachers and Leaders at American Institutes for Research, n.d.1; Kentucky Department of Education, n.d.2; Tennessee Department of Education, 20133; Virginia Department of Education, 2011a4; West Virginia Department of Education, 2015b5.

All states in the region require that schools link teacher evaluation scores to student achievement (Table 7). In three of the four states, teacher evaluation occurs annually for a select group of teachers (e.g., untenured, probationary). Three states require that evaluators receive formal training in the skills of evaluating teachers, as determined by state departments or boards of education.

Table 7. State-level information on evaluation of teacher performance

State

State requires all teachers to be formally evaluated

Teacher evaluation is tied to student achievement

Teacher evaluation occurs on an annual basis

State requires all evaluators to receive formal training KY2 Yes

Yes Tenured: No

Untenured: Yes Yes

TN1,3 Yes Yes Yes Yes VA1,4 Yes Yes Continuing teachers:

No Probationary: Yes

Yes

WV1,5 Yes Yes If teacher has more than five years of experience: No

Yes

Source: Center on Great Teachers and Leaders at American Institutes for Research, n.d.1; Kentucky Department of Education, n.d.2; Tennessee State Board of Education, 2015b3; Virginia Department of Education, 2011a4; West Virginia Legislature, n.d.5.

All four states have formal standards for professional development and all finance professional development for all school districts (Table 8). Three of the four states also require districts to align professional development with local priorities and goals.

Table 8. State policies on professional development

State Formal professional development standards Finances professional development for all districts

Requires districts to align professional development with local priorities and goals KY1 Yes Yes Yes TN2,3 Yes Yes Yes VA Yes4 Yes5 Yes6 WV7 Yes Yes No

Source: Kentucky Legislative Research Commission, 704 KAR 3:035, n.d.1; Tennessee State Board of Education, 20122;

Tennessee State Board of Education, 2015a3; Virginia Department of Education, 20144; Virginia Department of Education, 2015b5; Virginia’s Legislative Information System, n.d.6; West Virginia Board of Education, 20157.

REL Appalachia consists of a diverse region with varied demographics, education performance, and education standards. The similarities and differences exhibited in Tables 1–8 offer some context to education needs in the region, which are detailed in the next section.

Discussion of Regional Needs We identified nine key needs across the four states and REL Appalachia’s three high-priority topic areas (Table 9).

Table 9. Key needs by state and high-priority topic area

Kentucky Tennessee Virginia West Virginia

Ensuring College and Career Readiness

1. Technical assistance and research related to college and career readiness interventions;

career readiness resources for different grade levels; and career readiness definitions and measurements.

6. Technical assistance in using and interpreting early warning system data, specifically classroom data related to student learning difficulties.

8. Technical assistance on career readiness, specifically related to the Simulated Workplace initiative.

Supporting Effective Teachers and Leaders

2. Technical assistance related to student growth goals and providing principals with support on the state teacher evaluation system.

4. Technical assistance related to using data for instructional decisionmaking, within and outside of school data teams, and creating a culture of data use.

9. Technical assistance on best practices in instructional leadership for new and emerging leaders in the state.

Improving Low-Achieving Schools

3. Technical assistance and data support related to closing achievement gaps in focus schools and focus districts.

5. Technical assistance in identifying and implementing effective, school-specific turnaround strategies for focus and priority schools.

7. Technical assistance on using data to turn around priority and focus schools and close achievement gaps, particularly for English learners and students with disabilities.

The following sections present key findings for each state, with supporting statements backed by evidence from the needs sensing data analysis.

Kentucky Need #1: Technical assistance and research related to college and career readiness interventions;

career readiness resources for different grade levels; and career readiness definitions and measurement.

• Description: Stakeholders request support in multiple areas pertaining to college and career readiness. First, multiple stakeholders cite a lack of access to effective college and career readiness interventions and resources linked to the Kentucky Core Academic Standards. Next, within the Kentucky College and Career Readiness Alliance (KyCCRA)�2, members cite a need for career readiness resources for elementary, middle, and high school students. Specifically, alliance members request personalized learning approaches that direct different age groups of students toward potential career pathways

2 For more information on this research alliance, see https://ies.ed.gov/ncee/edlabs/regions/appalachia/KCCRalliance.asp and dropout prevention programs that keep students on a career and college pathway.

Finally, multiple stakeholders desire a better understanding of career readiness through the development of a common definition, greater understanding of career readiness measurement/assessment, and a needs assessment on in-demand jobs in various parts of the state.

• Context: Kentucky is making great strides toward improving college and career outcomes for all students. According to the Education Commission of the States (Glancy et al., 2014), Kentucky has achieved 9 of the 10 recommended steps for improving college readiness by integrating K-12 and postsecondary education. One example of the state’s emphasis on college and career readiness is the Infinite Campus section of the Kentucky Student Information System, a statewide database, which allows for tracking success of college and career readiness interventions.

Since the summer of 2015, Kentucky Department of Education staff has been able to create reports of effective interventions and associated best implementation practices using the Infinite Campus tool. This information will then, ultimately, be available for school and classroom use (U.S.

Department of Education, 2015a). Another example is Kentucky’s iLead Academy, a new high school-to-workforce program beginning in Fall 2015, that will prepare students for high-paying careers in Advanced Manufacturing, Engineering, and Technology (Hughes, 2015; Rodriguez, 2015).

Need #2: Technical assistance related to student growth goals and providing principals with support on the state teacher evaluation system.

• Description: Stakeholders at all levels request support with the state Professional Growth and Effectiveness System (PGES) in the areas of student growth goals and principal training.

For student growth goals, stakeholders request support in interpreting and implementing student growth goals, including measuring across content areas and interpreting the meaning of growth goals at classroom, district, and state levels. With respect to principal training, stakeholders believe principals need support in multiple facets of PGES, including planning, implementation, and follow-through. For example, principals would benefit from training in maintaining consistency across teacher

What Kentucky stakeholders say they need

“[We] need training. What works with what and what’s most appropriate for what? To match that to what’s available to us to help students. Training on most effective intervention strategies and matching with a particular college and career readiness intervention.” (Local Education Agency Staff)

“Students receive industry-recognized certificates, but are these in high demand? … Idea that if you go into welding, you’ll make six figures, but not if there aren’t any welders needed in your area.” (State Education Agency Staff)

“I’m not feeling real comfortable about the growth part, because I wasn’t trained very well on the growth part. Where we had to measure our students’ growth and what they’re looking for and what we’re looking for and how to do that.”

(Teacher) evaluations, creating an overall teacher rating from multiple data points, and using PGES data to improve teacher instruction.

• Context: Nationally, principals believe that they could use greater support on teacher evaluation systems, particularly in assessing student growth, determining need-related professional development for teachers, and identifying whole school professional development needs (Schuermann et al., 2014). Kentucky’s reported needs are similar to national needs. According to the 2015 Kentucky TELL survey, 42 percent of principals need additional professional development on the PGES system (TELL Kentucky, 2015).

In response to this need, the Kentucky Department of Education currently provides an updated set of PGES resources on its website (n.d.). For example, school staff can access webcasts, newsletters, and a toolkit of resources for understanding PGES components, including student growth goals.

Need #3: Technical assistance and data support related to closing achievement gaps in focus schools and focus districts.

• Description: Multiple stakeholders reference a need for data support related to closing achievement gaps in focus schools and focus districts. Specifically, stakeholders express interest in best practice strategies for closing achievement gaps for minorities and other underrepresented groups. Additionally, they request support in interpreting achievement gap data from state and district databases. Stakeholders view existing databases as cumbersome and overwhelming.

• Context: Kentucky has a distinct process for identifying and supporting focus schools and districts. Focus schools in Kentucky are (a) schools ranked in the lowest 10 percent of all schools in the state according to K-PREP3 gap scores for student subgroups (e.g., English learners, students with disabilities); (b) schools at a specific level (elementary, middle, or high school) ranked in the bottom 5 percent of that cohort according to proficient/distinguished scorers in any student subgroup within any K-PREP assessment subject area (e.g., reading); or (c) a high school with a graduation rate of 80 percent or lower for two consecutive years. Similarly, Kentucky identifies focus districts by ranking all districts and selecting those that perform in the bottom 10 percent according to K- PREP student subgroup gaps. Focus schools and districts also receive support. For example, districts are required to support focus schools in drafting and implementing their comprehensive school improvement plans. Focus schools and districts also receive resources from the Kentucky Department of Education, Office of Next Generation Schools and Districts to address the needs of student subgroups (U.S. Department of Education, 2015a).

Closing achievement gaps in focus schools and districts remains a difficult and ongoing area of focus in the state. Teachers highlight its importance on the 2015 Kentucky TELL survey, with approximately 58 percent of teachers requesting more professional development on closing the gaps. Closing achievement gaps also remains a concern of the Kentucky

3 Kentucky uses the K-PREP (Kentucky Performance Rating for Educational Progress) as a school accountability measure in grades 3–8 and 10–11 (Kentucky Department of Education, 2014).

Department of Education, which is investing in new gap closure initiatives. The Department is developing a plan to reduce the number of students with disabilities performing at or below Proficient and to reduce the number of students achieving at the Novice level (i.e., lowest level on the K-PREP), especially in middle school math. The current goal is to decrease the percentage of all students performing at Novice levels on reading and mathematics tests by 50 percent within five years (Kelly, 2015; U.S. Department of Education, 2015a).

Tennessee Need #4: Technical assistance related to using data for instructional decisionmaking, within and outside of school data teams, and creating a culture of data use.

• Description: Various stakeholders report that teachers and principals have limited capacity to use data for instructional decisionmaking within and outside of school data teams.

Within data teams, stakeholders request assistance in (a) structuring the time and format of meetings, (b) facilitating and leading data teams, (c) creating team goals and an action plan for improvement after the data interpretation or analysis phase, and (d) progress monitoring of data team implementation efforts. Outside of data teams, stakeholders request support for teachers in data use best practices (e.g., understanding and interpreting student achievement data). To make data use a reality, stakeholders believe that there needs to be a schoolwide culture of data use. As a result, stakeholders believe all school staff would benefit from training and support on best practices.

• Context: As a state, Tennessee prioritizes data use but has room for continued growth.

The Data Quality Campaign (2014) reports that Tennessee currently exhibits eight actions indicative of effective data use, including training teachers and principals to use longitudinal data for instructional decisionmaking and to interpret reports.

Various state-led initiatives also illustrate the state’s focus on building stakeholder capacity. For example, Tennessee is providing data use technical assistance to districts through TNCore offices.4 The state also provides training modules, from the Tennessee Department of Education and Battelle for Kids, on using data for instructional decisionmaking (U.S. Department of Education, 2015b). Even with this assistance, TELL Tennessee survey results suggest a need for continued support. According to the 2013

4 TNCore is Tennessee’s Centers of Regional Excellence. Eight TNCore offices serve educators in the different regions across the state through professional development and best practices.

What Tennessee stakeholders say they need

“I don’t think most teachers know how to use data. It’s given to us, but something you put on your desk and stuff gets packed on top of it. We weren’t taught how to use it.”

(Teacher)

“I think we need to treat individual teachers and individual schools the same way we treat individual students. I don’t think that there’s one prescription that will make it better for everybody. Every school/community is different. I think you have to quit being judgmental and think people being told they are bad is hurtful and I think that doesn’t cause them to want to get better. So I think that you have to go into each individual school and to each individual teacher and say, ‘Let’s identify your strength and support you where you need support.’” (Teacher) survey, 38 percent of principals reported that they would benefit from professional development in data-driven decisionmaking (TELL Tennessee, 2014).

Need #5: Technical assistance in identifying and implementing effective, school-specific turnaround strategies for focus and priority schools.

• Description: According to stakeholders, school turnaround continues to be an issue in Tennessee. Stakeholders believe that teachers and school staff need individualized, hands-on support in identifying and implementing school-specific turnaround strategies for focus and priority schools. For example, stakeholders request support in using state data to determine why schools are identified as low performing and to make an action plan for change.

• Context: The Tennessee Department of Education offers clear guidelines for the identification and support of priority and focus schools. Priority schools are those schools performing in the bottom 5 percent of schools statewide according to TCAP scores.5 Tennessee identifies focus schools in three ways: (a) schools ranked in the bottom 10 percent of all schools in the state according to TCAP student subgroup gaps (i.e., schools with the largest student subgroup gaps according to TCAP); (b) schools ranked in the bottom 5 percent of schools for TCAP student subgroup performance (i.e., schools with an aggregated proficiency rate of below 10 percent for a student subgroup across TCAP assessments); or (c) high schools with graduation rates of less than 60 percent (Tennessee Department of Education, n.d.; U.S. Department of Education, 2015b). To support priority and focus schools, the Tennessee Department of Education currently provides turnaround support, information sharing, and technical assistance through TNCore locations and quarterly focus school convenings (starting in 2015/16). The convenings will provide focus school principals and staff with the opportunity to learn from one another, share resources, and collaborate to solve problems (U.S. Department of Education, 2015b).

Even with existing supports, turnaround strategy assistance remains an area of need for various stakeholders. For example, the 2013 TELL Tennessee survey results reiterate teacher and principal requests for turnaround strategy support: 60 percent of teachers believed that they would benefit from professional development on closing achievement gaps and 29 percent of principals thought it would be helpful to receive additional training in school improvement planning (TELL Tennessee, 2014).

5 TCAP is the Tennessee Comprehensive Assessment, administered annually to students in grades 3–11 (My Tennessee Public Schools, n.d.).

Virginia Need #6: Technical assistance in using and interpreting early warning system data, specifically classroom data related to student learning difficulties.

• Description: Stakeholders at multiple levels request data use support, specifically related to early warning systems and uncovering student learning difficulties in middle school.

Stakeholders shared that Virginia teachers would greatly benefit from the sharing of data use practices in other schools, assistance in analyzing and interpreting student learning data, and training in monitoring progress in student learning.

• Context: In recent years, Virginia developed an early warning system for high school students at risk of dropout, in collaboration with the American Institutes for Research (AIR), the Appalachia Regional Comprehensive Center (ARCC), and the National High School Center at AIR. In this system, schools use data to identify potential dropouts in grade 9 and incorporate progress monitoring to track intervention efforts (AIR, 2015;

Virginia Department of Education, VEWS, n.d.).

Virginia stakeholders’ interest in early warning systems and data use has been addressed previously by REL Appalachia and the Virginia Department of Education. REL Appalachia has provided Virginia stakeholders with assistance in identifying and supporting middle school students at risk for dropout through the Virginia Middle School Research Alliance (VMSRA). The Virginia Department of Education also provides staff with professional development in using student data. One example was the six-week online course “Thinking Critically with Data” in Fall 2015 (Virginia Department of Education, SOL Events, n.d.).

Need #7: Technical assistance on using data to turn around priority and focus schools and close achievement gaps, particularly for English learners and students with disabilities.

• Description: Virginia stakeholders request support related to priority and focus schools, specifically research-based strategies and data use support for turning around schools and closing achievement gaps, particularly for English learners and students with disabilities. Principals, state education agency staff, and policymakers frequently mention data use as it relates to turnaround assistance, specifically requesting support in (a) examining data to improve classroom interventions and instruction and (b) using data to understand school, classroom, and individual student performance and associated areas of need or improvement. Virginia stakeholders also request more information and technical assistance on identifying and implementing effective instructional strategies for English learners and students with disabilities.

What Virginia stakeholders say they need

“Learning how to take the data and converting it into an action plan. Okay, here is our [teacher/classroom] data, now what do we do with it?” (Assistant Principal)

“There needs to be a lot of training for ELL folks. We have virtually no staff or money to help support training for ELL populations of kids. We have plenty of funding to help with SPED, but our ELL funds are very limited with hardly any support being able to go towards providing any systemic interventions for that population of kids. It is our fastest growing population of kids.”

(Superintendent)

• Context: Priority and focus schools are identified differently and supported in varied ways in Virginia. Priority schools are Title I schools, ranked across the state in the bottom 5 percent in graduation rate or the bottom 5 percent in achievement on Standards of Learning (SOL) Reading or Math assessments. Focus schools are Title I schools, ranked across the state in the bottom 10 percent according to student subgroup (e.g., English learners, students with disabilities) in graduation rate or subgroup SOL Math or Reading performance (U.S. Department of Education, 2015c).

Virginia currently supports priority and focus schools in several ways, including (a) providing resources and webinars related to school improvement; (b) offering technical assistance and professional development workshops for teachers and schools working with English learners and students with disabilities; (c) providing state support and technical assistance for focus schools related to closing achievement gaps; and (d) offering access to Indistar®, a school improvement planning tool that provides online tutorials and research on effective school turnaround strategies Virginia Department of Education, 2012a, 2015c). Furthermore, the Virginia Department of Education began working with ARCC in 2014 on a five-year project to support teachers of English learners and students with disabilities on best practice instructional strategies, particularly in the area of dropout prevention for these student subgroups (ARCC, n.d.; U.S. Department of Education, 2015c).

Of the four states in the REL Appalachia region, Virginia has the most English learners (91,051 students) and a large population of students with disabilities (157,494 students). These large populations might explain why supporting these student subgroups through focus and priority school efforts continues to be an area of need.

West Virginia Need #8: Technical assistance on career readiness, specifically related to the Simulated Workplace initiative.

• Description: The West Virginia Department of Education (WVDE) is partnering with the WVWRA to provide training for in-school coaches who will guide Simulated Workplace teachers in implementing the new pedagogy associated with the program.

• Context: The WVDE notes, “The demand for a more work ready employee among West Virginia business and industry leaders continues to grow.”

(http://wvde.state.wv.us/simulated-workplace/about.php). The state has thus initiated an innovative new program, Simulated Workplace, which replaces a traditional classroom training model with a teaching model that immerses students in an actual workplace environment. In addition to technical skills, the Simulated Workplace initiative promotes soft skills (e.g., timeliness, reliability, and teamwork), business skills, and problem-solving skills all essential to the success of both workers and business. As the program rolls out across the entire state, the WVDE asked REL AP to assist with the training of in-school coaches who will scale the effort, and with an evaluation plan to assess longer-term effects of the Simulated Workplace initiative.

Need #9: Technical assistance on best practices in instructional leadership for new and emerging leaders in the state.

http://wvde.state.wv.us/simulated-workplace/about.php

• Description: According to West Virginia stakeholders, a large percentage of superintendents and principals are nearing retirement, increasing the likelihood of shortages across the state in future. As a result, stakeholders report a need for leadership training; specifically, overall and instructional leadership training for principals, superintendents, and teachers.

• Context: Leadership is a topic of concern in West Virginia for three key reasons. First, a recent report by Imagine West Virginia offered five recommendations for changing leadership practices in the state: (a) revising leadership roles–emphasis on instructional leadership; (b) training–revising postsecondary preparation programs to include instructional leadership; (c) screening–establishing an independent group to screen applicants; (d) salaries–offering higher salaries when individuals attain their degree and assume the principal position; and (e) career pathways–offering a career pathway for teachers to become classroom leaders (Imagine West Virginia, 2014; West Virginia Department of Education, n.d.).

Second, in response to the Imagine West Virginia report, West Virginia proposed House Bill 2387, which would reform school leadership practices in the state (West Virginia Legislature, n.d.). Finally, the West Virginia Board of Education started in March 2015 to reform leadership preparation programs and redefine leadership in West Virginia (U.S. Department of Education, 2015d).

Initially, West Virginia is providing some technical assistance through the West Virginia Center for Professional Development (n.d.). Through this center, new principals can participate in a Principals’ Leadership Academy for New Principals, which is designed to support the role of principal as an instructional leader.

Needs Sensing Tools and Strategies The needs of the REL AP region drive the REL’s work, both within and beyond the research alliances. There are various sources for identifying regional needs, which include local educators, state association and education officials, research alliance members, REL AP Governing Board members, and the Ask REL Appalachia service�6. As part of a broader scan of issues and needs surfacing in the region, staff also conduct ongoing reviews of regional news, publications, and state policies and legislation.

The design and data collection methods for the Annual Assessment of Analytic Needs are guided by the following overarching questions:

• What data use and analytic needs exist in the region?

6 For more information on the research alliances, Governing Board members, and Ask a REL service for the REL Appalachia region, see https://ies.ed.gov/ncee/edlabs/regions/appalachia/index.asp

What West Virginia stakeholders say they need

“A root cause analysis revealed that we have a leadership crisis—harsh word, but many of our leaders, principals, and superintendents are eligible for retirement.… We are looking for how to build leadership capacity. Initially, we looked at teacher leadership. This is much broader than that. Overall leadership development. What can we do to support teachers and leaders?” (State Education Agency Staff)

“One of the things we are struggling with in our county is instructional leadership, getting our principals to understand what true leadership looks like in the building.” (Principal)

• What is the nature and extent of data use and analytic needs in the region?

• What are the implications of these needs for the high-priority topic areas?

To address these questions, we conducted the needs assessment activities shown in Table 10, presented as they occurred during year 4.

Table 10. Needs assessment activities as of September 1, 2015

Data source Data collection method and samplea Data collection frequency REL AP Governing Board

Needs prioritization activities at meetings;

all attending Governing Board members participated

October 2014, May 2015

Ask REL Appalachia requests

Review of Ask REL Appalachia requests Ongoing

Non-alliance stakeholders

State and role-specific interviews with 76 stakeholders (i.e., policymakers/advocacy groups, state education agencies, local education agencies, teachers); convenience sample followed by snowball approach

March–May 2015

Alliance researchers and members

Review of 285 entries in internal REL AP database

Analysis of 739 total stakeholder feedback surveys from 24 events

Ongoing

Ongoing

Regional and national news, publications, and policy scans

Review of 139 articles; reviewed websites of national and regional newspapers, policy organizations, nationally recognized magazines and journals, state departments of education, and state legislation in high-priority topic areas

Ongoing

a. U.S. Department of Education (2011). State Education Data Profiles. Retrieved August 19, 2015 from https://nces.ed.gov/programs/stateprofiles.

Source: Compilation by REL Appalachia staff.

To synthesize the various types of data from multiple sources and determine how needs should be prioritized, we developed a three-step process that involves both qualitative and quantitative procedures:

1) Conduct analysis of data from each source by REL Appalachia high-priority topic area (Ensuring College and Career Readiness, Improving Low-Achieving Schools, and Supporting Effective Teachers and Leaders) and by state (Kentucky, Tennessee, Virginia, and West Virginia); and identify and code themes revealed in the analysis by subtopic categories (e.g., early warning systems, dual enrollment/dual credit, data use, turnaround strategies).

2) Establish and apply prioritization criteria to key findings for each source (Altschuld & White, 2010).

3) Conduct qualitative analyses across prioritized, key needs.

https://nces.ed.gov/programs/stateprofiles

Our process for analyzing data from each data source primarily involved qualitative analyses, including content analysis and analytic induction (Erickson, 1986). We used ATLAS.ti®, a qualitative data analysis software program, to analyze the data, first by source (e.g., interviews, stakeholder feedback surveys, internal REL AP database entries), and then across data sources for a final synthesis.

We used a combination of predetermined and emergent codes to create a hierarchical coding framework for the analyses.

Data segments, such as passages from the interviews, literature scans, and the internal REL AP database entries, were organized and indexed in the ATLAS.ti database. We gave particular attention to confirming instances generated by mixed data collection methods. This follows the assumption that needs based on multiple data sources are more robust and persuasive than those based on a sole form of data. In this method, one works toward coherence of data, assertions, and findings.

References Altschuld, J. W., & White, J. L. (2010). Needs assessment: Analysis and prioritization. Thousand Oaks, CA: Sage.

American Community Survey. (2009-2013). American fact finder [website]. Retrieved July 1, 2015, from http://factfinder.census.gov/faces/nav/jsf/pages/searchresults.xhtml?refresh=t.

American Institutes for Research [AIR]. (2015). Early warning systems [website]. Retrieved August 19, 2015, from http://www.earlywarningsystems.org/our-work/early-warning-systems.

Appalachia Regional Comprehensive Center [ARCC]. (n.d.). Virginia plan [website]. Retrieved August 21, 2015, from https://www.arccta.org/va-state-plan.

Center on Great Teachers and Leaders at American Institutes for Research. (n.d.) Databases on state teacher and principal evaluation policies [website]. Retrieved September 18, 2015, from http://resource.tqsource.org/stateevaldb/

College Board, The. (2014). The 10th annual AP report to the nation. Retrieved September 18, 2015, from http://media.collegeboard.com/digitalServices/pdf/ap/rtn/10th-annual/10th-annual-ap-report-to-the-nation-single-page.pdf.

Common Core of Data. (2014). Elementary/secondary information system [website]. Retrieved July 1, 2015, from http://nces.ed.gov/ccd/elsi/default.aspx?agree=0.

Consolidated State Performance Report (2009-2010). ED data express. Retrieved July 1, 2015, from http://eddataexpress.ed.gov/index.cfm.

Consolidated State Performance Report. (2013-2014). ED data express. Retrieved July 1, 2015, from http://eddataexpress.ed.gov/index.cfm.

Data Quality Campaign. (2014). State analysis: Tennessee. Retrieved August 13, 2015, from http://dataqualitycampaign.org/wp-content/uploads/files/pdf/stateprofiles/TN.pdf.

Erickson, F. (1986). Qualitative methods in research on teaching. In M. C. Wittrock (Ed.), Handbook of research on teaching (3rd ed.). New York: Macmillan.

http://factfinder.census.gov/faces/nav/jsf/pages/searchresults.xhtml?refresh=t http://www.earlywarningsystems.org/our-work/early-warning-systems https://www.arccta.org/va-state-plan http://resource.tqsource.org/stateevaldb/ http://media.collegeboard.com/digitalServices/pdf/ap/rtn/10th-annual/10th-annual-ap-report-to-the-nation-single-page.pdf http://media.collegeboard.com/digitalServices/pdf/ap/rtn/10th-annual/10th-annual-ap-report-to-the-nation-single-page.pdf http://nces.ed.gov/ccd/elsi/default.aspx?agree=0 http://eddataexpress.ed.gov/index.cfm http://eddataexpress.ed.gov/index.cfm

Glancy, E., Fulton, M., Anderson, L., Zinth, J. D., Milliard, M., & Delander, B. (2014, October).

Blueprint for college readiness. Denver, CO: Education Commission of the States. Retrieved August 10, 2015, from http://www.ecs.org/docs/ECSBlueprint.pdf.

Hughes, B. (2015, February 10). Students attending iLead Academy will be targeted to northcentral Kentucky’s high-demand career fields. Retrieved August 10, 2015, from http://www.ksba.org/protected/ArticleView.aspx?iid=5U0BPI&dasi=3G32.

Imagine West Virginia. (2014). Recommendations for transforming school leadership in West Virginia. Retrieved August 17, 2015, from http://static.k12.wv.us/tt/2014/IWV-EducationStudy2014- Final.pdf.

Kelly, B. R. (2015, March 10). State will focus on Novice reduction to close achievement gaps.

Kentucky Teacher. Retrieved August 20, 2015, from http://www.kentuckyteacher.org/features/2015/03/state-will-focus-on-novice-reduction-to-close-achievement-gaps/.

Kentucky Department of Education. (n.d.). Professional growth and effectiveness system (PGES).

Retrieved July 27, 2015, from http://education.ky.gov/teachers/pges/pages/pges.aspx.

Kentucky Legislative Research Commission. (n.d.). 704 KAR 3:035. Annual professional development plan [website]. Retrieved July 1, 2015, from http://www.lrc.ky.gov/kar/704/003/035.htm.

Kentucky Legislative Research Commission. (n.d.). 704 KAR 3:370. Professional Growth and �Effectiveness System [website]. http://www.lrc.state.ky.us/kar/704/003/370.htm.

Kentucky Legislative Research Commission. (n.d.). 704 KAR 3:305. Minimum requirements for high school graduation [website]. Retrieved July 31, 2015, from http://www.lrc.ky.gov/kar/704/003/305.htm.

My Tennessee Public Schools. (n.d.). Academics and accountability, Tennessee Comprehensive Assessment Program (TCAP) testing [website]. Retrieved August 28, 2015, from http://mytennesseepublicschools.net/academics-and-accountability/.

National Assessment of Educational Progress. (2015). State profiles [website]. Retrieved December 10, 2015, from http://nces.ed.gov/nationsreportcard/states/.

National Council on Teacher Quality (2011). State of the states: Trends and early lessons on teacher evaluation and effectiveness policies. Washington, DC: National Council on Teacher Quality.

Retrieved September 18, 2015, from http://www.nctq.org/dmsView/State_of_the_States_Teacher_Evaluation_and_Effectiveness_Policies_ NCTQ_Report

Rodriguez, N. (2015). KY first regional academy offers students a competitive advantage in global economy. Kentucky Teacher. Retrieved August 10, 2015, from:

http://www.kentuckyteacher.org/news/2015/02/ky-first-regional-career-academy-offers-students-a-competitive-advantage-in-global-economy/.

http://www.ecs.org/docs/ECSBlueprint.pdf http://www.ksba.org/protected/ArticleView.aspx?iid=5U0BPI&dasi=3G32 http://static.k12.wv.us/tt/2014/IWV-EducationStudy2014-Final.pdf http://static.k12.wv.us/tt/2014/IWV-EducationStudy2014-Final.pdf http://www.kentuckyteacher.org/features/2015/03/state-will-focus-on-novice-reduction-to-close-achievement-gaps/ http://www.kentuckyteacher.org/features/2015/03/state-will-focus-on-novice-reduction-to-close-achievement-gaps/ http://education.ky.gov/teachers/pges/pages/pges.aspx http://www.lrc.ky.gov/kar/704/003/035.htm http://www.lrc.state.ky.us/kar/704/003/370.htm http://www.lrc.ky.gov/kar/704/003/305.htm http://mytennesseepublicschools.net/academics-and-accountability/ http://nces.ed.gov/nationsreportcard/states http://www.nctq.org/dmsView/State_of_the_States_Teacher_Evaluation_and_Effectiveness_Policies_NCTQ_Report http://www.nctq.org/dmsView/State_of_the_States_Teacher_Evaluation_and_Effectiveness_Policies_NCTQ_Report http://www.kentuckyteacher.org/news/2015/02/ky-first-regional-career-academy-offers-students-a-competitive-advantage-in-global-economy/ http://www.kentuckyteacher.org/news/2015/02/ky-first-regional-career-academy-offers-students-a-competitive-advantage-in-global-economy/

Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: A systematic approach. Thousand Oaks, CA: Sage.

Schuermann, P., Goldring, E. B., Cannata, M., Drake, T. A., Grissom, J. A., Neumerski, C. M., & Rubin, M. (2014, May). Supporting principals to use teacher effectiveness data for talent management decisions. Nashville, TN: Vanderbilt Peabody College. Retrieved August 11, 2015, from http://www.principaldatause.org.

TELL Kentucky. (2015). Reports for TELL KY 2015 [website]. Retrieved August 11, 2015, from http://www.tellkentucky.org/results.

TELL Tennessee. (2014). Reports for TELL TN 2013 [website]. Retrieved August 13, 2015, from http://www.telltennessee.org/results.

Tennessee Department of Education. (2013). Teacher Evaluation. Retrieved August 18, 2015, from http://team-tn.org/evaluation/teacher-evaluation/.

Tennessee Department of Education. (2015). Graduation Requirements. Retrieved July 27, 2015, from http://www.tn.gov/education/topic/graduation-requirements.

Tennessee Department of Education. (n.d.). 2015 School Accountability [website]. Retrieved August 19, 2015, from https://www.tn.gov/education/article/2015-school-accountability.

Tennessee State Board of Education. (2012). Standards for Professional Learning. Retrieved July 31, 2015, from http://www.tn.gov/assets/entities/sbe/attachments/4-20-12- III_F_Standards_for_Professional_Learning.pdf.

Tennessee State Board of Education. (2015a). Tennessee State Board of Education Professional Development 5.200. Retrieved July 31, 2015, from http://state.tn.us/sbe/Policies/5.200%20Professional%20Development.pdf.

Tennessee State Board of Education. (2015b). Tennessee State Board of Education Teacher and Principal Evaluation Policy 5.201. Retrieved August 4, 2015, from https://tn.gov/assets/entities/sbe/attachments/5201_TeacherandPrincipalEvaluationPolicy_1-30- 2015.pdf.

U.S. Department of Education. (2015a.). Kentucky: ESEA flexibility request [website]. Retrieved August 20, 2015, from http://www2.ed.gov/policy/elsec/guid/esea-flexibility/map/ky.html.

U.S. Department of Education. (2015b). Tennessee ESEA flexibility request. Retrieved August 21, 2015, from http://www2.ed.gov/policy/elsec/guid/esea-flexibility/flex-renewal/tnrenewalreq2015.pdf.

U.S. Department of Education. (2015c). Virginia: ESEA flexibility request [website]. Retrieved August 21, 2015, from http://www2.ed.gov/policy/elsec/guid/esea-flexibility/map/va.html.

U.S. Department of Education. (2015d). West Virginia: ESEA flexibility request [website]. Retrieved August 21, 2015, from http://www2.ed.gov/policy/elsec/guid/esea-flexibility/map/wv.html.

Virginia Department of Education. (2011a). Guidelines for uniform performance standards and evaluation criteria for teachers. Retrieved July 1, 2015, from http://www.principaldatause.org/ http://www.tellkentucky.org/results http://www.telltennessee.org/results http://www.tn.gov/education/topic/graduation-requirements https://www.tn.gov/education/article/2015-school-accountability http://www.tn.gov/assets/entities/sbe/attachments/4-20-12-III_F_Standards_for_Professional_Learning.pdf…