26MACISCS5 - Attachment A - Final.docx

135 Santilli Highway, Everett, MA 02149 | 781.338.3000 | TTY: N.E.T. Relay (800).439.2370

Attachment A

Alternate ID #: 26MACISCS5 COMMBUYS Bid #: BD-26-1026-DOE02-DOE01-127733

Background:

As a result of recently updated regulations allowing the Massachusetts Department of Elementary and Secondary Education (DESE) to accept approved alternative assessments to the Massachusetts Tests for Educator Licensure (MTEL), DESE is seeking sponsoring organizations to offer educator preparation attestations. The information collected in this master agreement (MA) will assist DESE in selecting approved sponsoring organizations to offer attestations comparable to the relevant subject matter knowledge (SMK) MTELs.

The regulations state:

Any candidate who passes an alternative assessment approved by the Commissioner as comparable to the MTEL Communication and Literacy Skills test or a subject matter knowledge test will be deemed to have satisfied the requirements of 603 CMR 7.04 (2) (a) (2), or (a) (3), or 603 CMR 7.04 (2) (b) (2) or (b) (3), or 603 CMR 1.06 (4) (a), as applicable. The Commissioner will publicly report annual data on all approved alternative assessments, including the number of candidates taking the alternative assessments and assessment pass rates. The Department will issue Guidelines for the approval and sustainability of all alternative assessment options.

All submitted attestations must adhere to the Guidelines for Alternative Assessments to the MTEL; the requirements from these Guidelines are reflected in this RFR.

Attestations are approved alternative assessments to the MTEL. They should not replace any aspect of an approved educator preparation program; they are not alternative programs or alternative pathways. Attestations are solely a replacement for the MTEL. Attestations are accepted for purposes of licensure in lieu of the relevant MTEL. Scores on attestations are submitted to the Department separately from the endorsement for completing the requirements of an approved educator preparation program. A candidate who completes and passes an attestation must still meet all additional licensure requirements in order to be endorsed for an initial license.

Figure 1 defines the criteria used to consider all alternative assessments, including attestations, at different points in the approval and implementation process.

Figure 1.

Attestations approved through this master agreement will be made available to candidates for the purposes of meeting the assessment requirement for educator licensure in Massachusetts through June 30, 2033.

If the Department terminates the contract, the sponsoring organization’s attestation(s) will no longer be eligible to meet Massachusetts’ licensure assessment requirements. If the Department terminates the contract for cause, the sponsoring organization must not register any new candidates. The Department reserves the right to suspend the implementation of any attestation as necessary, based on the results of an audit or for any other emergent evidence, including a change in program approval status for a sponsoring organization and/or their program(s).

Scope of Service:

Subject matter knowledge attestations are designed to be administered by sponsoring organizations based on evidence of a candidate’s functional content knowledge. The attestation allows sponsoring organizations to attest that teacher candidates have demonstrated the required functional content knowledge for specific assessments as aligned to the MTEL objectives. Subject matter knowledge attestations will be approved to be offered to candidates enrolled in an initial educator preparation program(s) that correspond to MTELs that are required for that licensure area. General Information: Subject Matter Test Requirements - Massachusetts Tests for Educator Licensure (MTEL)

Viability

Sponsoring Organization Eligibility Sponsoring organizations that meet the following requirements are eligible to apply:

· Have received an Approved or Approved with Distinction designation based on their most recent formal review;

· Offer initial teacher licensure programs in the licensure areas of the attestation(s) that are approved or approved with distinction;

· [For attestations available through Elementary, Early Childhood, and Moderate Disabilities PK–2/PK–8 initial licensure programs only] Implement evidence-based early literacy programs aligned to the Early Literacy Program Approval Criteria;

· Have submitted Subject Matter Knowledge Statement of Assurance indicating programs have been revised in support of ensuring candidates meet expectations as outlined in the Subject Matter Knowledge Guidelines;

· Dean or Dean-equivalent sign-off on all attestations (one individual identified per organization); and

· One designated content expert per MTEL who will sign off on the relevant attestations in accordance with their content expertise (see below for additional information on content expertise).

Attestation Eligibility Eligible sponsoring organizations can apply to attest to any of the content MTELs (with the exception of those listed below) for the corresponding initial teacher licensure programs that are approved or approved with distinction.

All MTELs are eligible for attestations except for the following:

· Adult Basic Education

· Reading Specialist

· Sheltered English Immersion

· Vocational Technical Literacy Skills

· Bilingual Education

· Communication and Literacy Skills Test

If an organization offers multiple programs for a license (e.g., English 5-12 at both the baccalaureate and post-baccalaureate level), they must indicate if one or both programs will provide the proposed attestation.

For example, an organization attesting to functional content knowledge covered on the English MTEL could use this option for five of the 20 candidates enrolled in the English 5-12 baccalaureate licensure program. In the BID, organizations will specify which MTELs they wish to include in the attestation and will submit a written MTEL Attestation Policy that articulates how they will implement the attestation process at their organization in accordance with the guidance below. Candidates who complete the attestation will be part of program approval accountability alongside completers who did not participate in the attestation.

Candidate Eligibility At a minimum, for attestations, candidates must:

· be enrolled in an approved initial teacher licensure preparation program at the institution offering the attestation, as documented in ELAR, and

· be in good standing with the program, and

· demonstrate they have taken and failed the required MTEL for which they are seeking an attestation.

The intent behind requiring that candidates meet an eligibility threshold is twofold. First, using the results of one failed MTEL as a baseline streamlines the focus of the attestation to the components of the assessment where the candidate did not demonstrate proficiency. Second, it is essential that sponsoring organizations do not preemptively make assumptions about an individual’s ability to pass an MTEL.

Sponsoring organizations may choose to add additional candidate eligibility requirements, as long as those requirements ensure consistent and fair opportunities for candidates to engage in the attestation. Articulating clear and consistent eligibility criteria at the outset will ensure that candidates understand the options available to them so that: 1) programs are not completely restructured when not all candidates will participate in this attestation, and 2) there is fair and consistent access to this attestation.

Attestations may not be required components of a licensure program. Candidates enrolled in a program with an approved attestation are not required to participate in the attestation and may elect to take and retake the relevant MTEL instead. An organization’s implementation of an attestation does not automatically signal that enrolled candidates must gain access to the attestation option.

Similar to policies for passing the MTEL, insufficient demonstration of functional content knowledge through this attestation should not, on its own, preclude a candidate from progressing through their licensure program.

Alignment

Standards for Evidence Collection using MTEL Test Objectives MTEL attestations must measure the depth and breadth of content knowledge outlined in the SMK Guidelines and found within the Frameworks in the manner that the SMK MTELs are currently designed to do, e.g., including content two grade levels above and below the grade span of the license and full coverage of all MTEL objectives. This level of functional content knowledge is articulated through the relevant test objectives for each MTEL.

MTEL Tests Scheduled for Revision In the case where the MTEL test objectives are undergoing revision, the proposed attestation can either be aligned to the current test objectives, or to the revised test objectives, where available. When revised test objectives are still under development, sponsoring organizations submitting attestations for MTEL tests scheduled for revision must include a plan in their submission to revise the attestation to align to the revised test objectives by the MTEL release date. A list of tests currently under revision can be found here. Once a revised test becomes operational, the attestation must be aligned to the operational version of the test. If an organization does not update and get approval of their attestation by the launch of the new test, it will need to cease operation until approved by DESE.

Sources of Evidence In the application, sponsoring organizations must identify the sources of evidence they will use to authentically assess candidate functional content knowledge at the same level of rigor as the MTEL, as well as how they will evaluate that evidence (see Appendix A, evidence grid). It is likely that multiple pieces of evidence will be needed to comprehensively assess functional knowledge of each test objective, given the depth of content embedded within each objective. Evidence may be evaluated from work previously completed by the candidate, assessed by the content expert from the sponsoring organization, and/or evaluated from newer demonstrations of content knowledge through assignments, oral defense questions, or other performance-based assessments developed by the sponsoring organization and assessed by the designated content expert.

Examples of what may and may not be used as evidence of functional content knowledge for the purposes of attestations are listed below.

· Example sources of evidence may include, though are not limited to:

· Specific assignments with grading rubric and grade

· Essays with clear rubric/expectations and grade

· Pre-practicum Gateway Tasks with grading rubric and grade

· An interview or content-defense review

· Graded, written test

· Work products or experience from other industries where the candidate was the sole contributor

· An analysis of a student work sample focused on addressing student misconceptions

· Observation of a lesson where demonstration of functional content knowledge is clear

· Non-examples include:

· GPA

· Course grade in subject area

· Lesson Plans

· The Candidate Assessment of Performance

Sponsoring organizations must also indicate in their application how they will determine the originality of the work submitted by a candidate, designing the process in such a way that ensures a candidate provides documentation that is solely a reflection of their own knowledge.

Test Objectives The sponsoring organization must, at a minimum, assess and attest to knowledge aligned to any MTEL test objective(s) within subareas for which the candidate did not answer “many” or “most” of the multiple choice items and/or provide an “adequate” or “thorough” response to open-response items. Sponsoring organizations cannot use results from one or multiple MTEL attempts to account for more than 50% of the MTEL subareas.

Content Expertise The individual(s) evaluating evidence to attest to candidates’ functional content knowledge must have expertise in the Massachusetts Curriculum Frameworks and the license area subject matter knowledge requirements outlined in the SMK Guidelines. While multiple individuals can collectively evaluate evidence, a single content expert (one per MTEL attestation) must sign the attestation for all candidates going through this attestation route within an organization to ensure consistency and rigor. Sufficient content expertise per proposed attestation will need to be described in the BID.

Utility and Impact

Documentation Sponsoring organizations must maintain attestation documentation submitted for each candidate, including the attestation submissions, for a period of at least seven years, even if the contract is terminated before that time. Scores must be maintained permanently. At minimum, sponsoring organizations are required to collect and report the following data for each candidate participating in an attestation:

· Personal information, including first and last names, MEPID, date of birth, and last four digits of their social security number, as they appear in the Educator Licensure and Renewal (ELAR) system;

· Educator preparation program in which the candidate is enrolled;

· Race/ethnicity (if the candidate chooses to disclose);

· Primary language (if the candidate chooses to disclose);

· Email address;

· Scores, including Pass/Fail status for the attestation for each attempt; and

· Documentation of candidate eligibility (as applicable).

Sponsoring organizations will provide the Department with a copy of each candidate’s score report in a manner determined by the Department and will provide any of the above information to the Department at its request.

Annual Reporting Sponsoring organizations will also be expected to submit the following annually by June 30th:

· Annual reporting requirements, including:

· At the candidate level:

· Candidate identifiers (name, MEPID, email, etc…)

· Candidate attestation dates

· Candidate attestation field(s)

· Candidate overall results

· At the aggregate level

· Pass rates for first attempt and best subsequent attempt

· Pass rates by race/ethnicity, primary language, and gender as provided in the MTEL Annual Pass Rate report

· Additional information as needed based on sponsoring organization implementation and annual data collected by the Department

· Total participation counts and counts by demographic groups;

· Pass rates and retake rates;

· Differential pass rates and retake rates between demographic groups; and

· Any changes to the implementation of approved attestations, including but not limited to approved changes to eligibility requirements for candidates, approved changes to requirements for demonstration of skills or content knowledge, relevant staffing changes for scorers/content-experts, or changes to the scoring process for attestations.

The Department will work with the sponsoring organizations on reporting statewide results in a format that can be reported separately as well as aggregated with the Department’s data systems.

Interim Audit Requirements The Department will monitor the implementation of all attestations in an ongoing manner. The Department reserves the right to conduct audits of sponsoring organizations providing attestations at any time. Audits will likely occur within the first or second year of the contract period but may occur at any time. Audits may occur virtually or at sponsoring organization’s primary location. Audits may involve, but are not limited to:

· A review of data gathered by each sponsoring organization,

· Interviews with candidates, evaluators, and other staff,

· A review of candidate documentation, including, if applicable, a documented list for each candidate of the evidence types and actual artifacts that were submitted (e.g. assignments, a recorded oral defense)

· A review of the tools used to evaluate the attestation (scoring rubric, etc…), including how the attestation evaluates the breadth of content knowledge for each test objective.

Budget:

Sponsoring organizations may propose a fee-based model, in which candidates pay an additional fee to access the attestation option, OR they may propose a no-fee model for eligible candidates to access the attestation. The submitted budget should reflect the model. For proposed attestations that will require a separate fee for candidates, the sponsoring organization must submit a description of anticipated costs, including but not limited to:

· The anticipated fee per candidate to access the attestation

· The cost of advertising the sponsoring organization’s alternative assessment option

· Costs associated with determining candidate eligibility based on the requirements found in the Scopes of Work section

· The estimated cost of development of all aspects of the alternative assessment

· The estimated cost to administer the alternative assessment The Department of Elementary and Secondary Education will not pay fees directly to the awarded vendor(s). All associated fees must be obtained directly between the vendor and the candidate.

Requirements:

All applicants must meet the following eligibility criteria:

· Have received an Approved or Approved with Distinction designation based on their most recent review;

· Offer initial teacher licensure programs that correspond to the attestation(s) that are approved or approved with distinction;

· Implement evidence-based early literacy programs aligned to Early Literacy Program Approval Criteria for relevant programs;

· Submitted Subject Matter Knowledge Statement of Assurance indicating programs have been revised in support of ensuring candidates meet expectations as outlined in the Subject Matter Knowledge Guidelines;

· Have a Dean or Dean-equivalent who will sign off on all attestations (one individual identified per organization); and

· Have one designated content expert per MTEL who will sign off on all the corresponding attestation in accordance with their content expertise (see below for additional information on content expertise).

Submission Requirements:

· Cover Letter: Identifies contacts, roles, and emails of those from your organization who can be contacted if questions arise pertaining to this bid. The Cover Letter must indicate which subject matter knowledge test(s) for which the sponsoring organization intends to provide an attestation(s). Sponsoring organizations must indicate that they are eligible to offer attestations and indicate which approved educator preparation program(s) through which the attestation(s) will be offered. Sponsoring organizations may submit one BID that includes information about each content MTEL for which they would like to establish an attestation.

· Section 1 – Viability: Address the capacity of the sponsoring organization to administer the assessment at the relevant scale (e.g., nationally, statewide, educator preparation organization). The narrative should:

· For each proposed attestation, provide a description of the sponsoring organization’s MTEL attestation policy, including:

· Candidate eligibility requirements to attempt the attestation

· The process for opting into and completing the attestation, including what will happen if a candidate does not meet the expected bar of the attestation

· Where in the program an attestation would become available to a candidate, and if there will be different approaches for Baccalaureate and Post-Baccalaureate candidates

· How the program will ensure equitable access to attestation(s) for all candidates

· Describe the sponsoring organization’s ability to securely report assessment results to DESE via the Educator Licensure and Renewal (ELAR) portal or other Department designated secure portal, as well as in a secure manner to individual candidates;

· Provide a description of the sponsoring organization’s commitment to and methods for reviewing bias in the attestation design, including a detailed description of how the sponsoring organization will review bias in the attestation on an ongoing basis, and the bias evaluation process (sponsoring organizations must consistently monitor the reviews of evaluators and performance of candidates for evidence of bias in the attestation design);

· Provide a description of the sponsoring organization’s ability to provide alternative testing arrangements in compliance with state and federal law, as applicable;

· Provide a description of the sponsoring organization’s monitoring process for the originality of submissions (including possible use/misuse of generative artificial intelligence).

· Section 2 – Alignment:

For each proposed attestation, the sponsoring organization must submit:

· A description of the attestation design.

· A list of allowable evidence or other documentation that candidates may submit for each attestation, with an explanation of how the sponsoring organization selected that evidence, and how it will evaluate the evidence submitted. The list may include, but is not limited to, sponsoring organization-developed assessments (assignments, oral defense questions, etc.) or artifacts that candidates may have previously created. The sponsoring organization should use the list of acceptable and unacceptable evidence described in the Scope of Work but may expand that list with a compelling rationale. If applicable, please indicate how past MTEL results may be used as evidence as described in the Scope of Work section.

· A crosswalk between the relevant subject matter knowledge MTEL test objectives and the proposed attestation(s) to demonstrate alignment. For a template, see Appendix A. In the case where the MTEL test objectives are undergoing revision, the proposed attestation can either be aligned to the current test objectives, or to the revised test objectives, where available. Once a revised test becomes operational, the attestation must be aligned to the operational version of the test. A separate evidence grid should be submitted for each MTEL included in the application.

· Organizations submitting attestations aligned to current test objectives must include a plan, with a timeline to revise the attestation to align to the revised test objectives by the MTEL release date including at least 30-days for DESE to review the updates to confirm alignment. Final revisions must be reviewed by DESE prior to implementation. A list of tests currently under revision can be found here.

· A description of the process that will be used for capturing and reviewing evidence of a candidate’s functional content knowledge. Considerations include: Will you allow the inclusion of evidence from outside of your sponsoring organization? Will you allow the inclusion of evidence captured before the individual became eligible for this route? How will you determine the originality of the work submitted by a candidate?

· A description of the procedure that will lead to attestation of candidate knowledge. This attestation must include sign-off from at least a relevant content expert and program leader (Dean or Dean-equivalent) - please name the individuals who will serve in this role and explain why they are the appropriate signers.

· Section 3: Utility and Impact

· Describe the sponsoring organization’s ability to provide data reporting on assessment results including disaggregated pass rates;

· Indicate the sponsoring organization’s commitment and capacity to complete requirements outlined in the Annual Reporting section of the Scope of Services

· Section 4: Project Plan

· A project plan with milestones, dates, and leads identified for the Scope(s) of Work as described above.

· Section 5: Budget

· Please submit a description of how the sponsoring organization will sustain the implementation of the alternative assessments. If implementing a fee-based structure, include the cost to candidates and budget for the assessment. This description might address the following:

· The cost of advertising the sponsoring organization’s alternative assessment option

· Costs associated with determining candidate eligibility based on the requirements found in the Scopes of Work section

· The estimated cost of development of all aspects of the alternative assessment

· The estimated cost to administer the alternative assessment

· Attachment B – Contact information form (complete and upload with COMMBUYS submission

Responses MUST be uploaded to COMMBUYS. Responses not posted to COMMBUYS will not be considered for evaluation. Do not Email and/or mail responses to the department.

To respond to this bid, please register on the COMMBUYS website. If you need assistance contact the OSD Help Desk at OSDHelpDesk@mass.gov or (8am - 5pm ET Monday – Friday) at 1-888-627-8283 or 617-720-3197.

image1.gif image2.jpeg image3.png