27MATALMB1 - Scorecard - Final.docx
DOCX document 118 KB Posted
- Attached to
- 27MATALMB1 | Early Literacy Screening Assessments in Grades K-3 | Departmental Master Agreement (MA)
- Solicitation number
- BD-26-1026-DOE02-DOE01-130126
- Issued by
- Massachusetts
About this file
This is a scorecard evaluation document for the Massachusetts Department of Elementary and Secondary Education (DESE) reviewing early literacy screening assessments submitted for a Departmental Master Agreement in grades K-3. The scorecard establishes the criteria and evaluation methodology for assessing publisher submissions of English-language literacy screening tools. The evaluation process focuses on alignment with Massachusetts standards, technical quality of assessments, ease of administration and interpretation, and overall utility for classroom teachers identifying students requiring additional literacy support. Publishers submitting assessments for approval must demonstrate that their products meet rigorous benchmarks across multiple dimensions including validity, reliability, and practical applicability in educational settings. The Master Agreement aims to provide schools across Massachusetts with approved screening options that educators can confidently implement to support early literacy instruction.
The scorecard utilizes a point-based rating system to systematically evaluate each submitted assessment against established criteria. Evaluators assign scores across categories measuring psychometric properties, cultural responsiveness, technical documentation, user accessibility, and alignment with evidence-based literacy frameworks. The structured evaluation approach ensures objective comparison among competing assessments and maintains consistency in the approval process. Approved assessments will be available to school districts statewide as authorized options for screening students in kindergarten through third grade, enabling educators to select tools that best fit their specific instructional contexts and student populations. The Master Agreement framework allows DESE to maintain quality standards while offering schools flexibility in choosing from multiple vetted assessment options.
View the file
Other files
| File | Type | Posted |
|---|---|---|
| FY26 State Forms - April 2026~13.pdf | ||
| DESE Required Language 10-2025.docx | DOCX document | |
| 27MATALMB1 - Bid - Final.docx | DOCX document | |
| IP Language.docx | DOCX document | |
| Attachment B - Contact Form~1.docx | DOCX document | |
| 27MATALMB1 - Attachment A - FINAL_.docx | DOCX document | |
| RFR - Required Specifications from OSD 10.1.21.docx | DOCX document | |
| IT RFR Required Specifications.doc | DOC document | |
| 27MATALMB1 - Evidence Form - Final.docx | DOCX document |
On GovTribe
Work with this file
- Download the original file
- Contacts named in this file
- Similar government files
- Ask GovTribe AI about this file
Text version
135 Santilli Highway, Everett, MA 02148 | Main Telephone: (781) 338-3000 | TTY: N.E.T. Relay 1-800-439-2370
EVALUATION CRITERIA SCORECARD
Agency Staff use only (This document will be used to evaluate all responses submitted to this Solicitation)
| Name of Bidder: |
| Name of Reviewer(s): |
| Date Reviewed: |
Required Qualifications Criteria (Bidder must receive a “yes” on all the following criteria to move on to the next evaluation section)
| 1 |
| The constructs available at each assessment period include, at a minimum, the following constructs in the corresponding grades: |
1. Phonemic Awareness
a. Kindergarten and Grade One (1):
- Phoneme Segmentation (BOY, MOY, EOY) OR Elision (BOY, MOY, EOY) OR Blending (BOY, MOY, EOY)
2. Alphabetic Knowledge
a. Kindergarten
- Letter Identification OR Letter Sound Knowledge (BOY/MOY)
- Letter Sound Knowledge (EOY) AND Decoding Nonsense Words (EOY)
b. Grade One (1)
- Letter Sound Knowledge (BOY) AND Decoding Nonsense Words (BOY, MOY, EOY)
c. Grades Two (2) and Three (3)
- Decoding Nonsense Words (BOY, MOY, EOY)
3. Word Reading
a. Grade One (1)
- Word Identification (BOY) AND Passage Reading Fluency (MOY, EOY)
b. Grades Two (2) and Three (3)
- Passage Reading Fluency (BOY, MOY, EOY)
4. Reading Comprehension
a. Grades Two (2) and Three (3) -Reading Comprehension (BOY, MOY, EOY)
5. Oral Language
a. Kindergarten, Grade One (1), Grade Two (2)
- Listening Comprehension Measure (BOY, MOY, EOY): Retell OR Passage level listening with follow-up questions OR Sentence Repetition
6. Rapid Automatized Naming (RAN)
a. Kindergarten
- Objects OR Letters (BOY/MOY) AND Letters (EOY)
b. Grade One (1)
- Letters (BOY)
| Yes |_| |
| No |_| |
| 2 |
| Classification Accuracy: provides affirmation that the level "significantly below" or its equivalent for grades K-3 meets the following criteria: |
a. Sensitivity: greater than or equal to 0.70
b. Specificity: greater than or equal to 0.70
c. Area under the Curve (AUC): greater than or equal to 0.70
| Yes |_| |
| No |_| |
| 3 |
| Assessment results in a composite percentile rank for each student and a composite "risk level" is identified for each student (e.g. risk/requires intervention) as part of the reported score. |
| Yes |_| |
| No |_| |
| 4 |
| Evidence of a user-friendly report of student results for families. |
| Yes |_| |
| No |_| |
| 5 |
| Technical Manual is included in accordance with specifics listed in Attachment A and the Evidence Form |
| Yes |_| |
| No |_| |
| 6 |
| Assessment is designed to be administered at least twice per school year year in kindergarten and at least three times per school year in grades 1, 2, and 3. Statement of willingness to cooperate with the Department to collect screening data three times per year from selected schools. |
| Yes |_| |
| No |_| |
| 7 |
| Offers professional development, technical support, and customer service to public school educators using the assessment. |
| Yes |_| |
| No |_| |
Scoring rubric:
| Non-Responsive |
| Qualifications is not applicable to request; elements missing / non-responsive to request |
| Unsatisfactory: |
| Little evidence of qualification. Many required elements are missing. |
| Below Average: |
| Evidence of expertise or qualification in some areas is unclear or unsubstantiated by supporting documentation. Some required elements are missing. |
| Average: |
| Application materials are complete and suggest adequate qualifications, providing clear evidence of expertise. |
| Above Average: |
| Application materials are complete and suggest solid qualifications, providing clear and well-documented evidence of expertise. |
| Outstanding: |
| Application materials is complete, suggesting exemplary qualifications, providing compelling, clear, and well-documented evidence of expertise. |
Alternate ID: 27MATALMB1 - English COMMBUYS Document #: BD-26-1026-DOE02-DOE01-130126 Non-Responsive 0 pts.
Unsatisfactory 1 pt.
Below Average 2 pts.
Average 3 pts.
Above Average 4 pts.
Outstanding 5 pts.
| Multiplier |
| Maximum |
Points Available (5 x Multiplier) Points Awarded (0 – 5 x Multiplier)
Additional Evaluation Criteria:
| 1 |
| Preferred: Evidence that the assessment includes a measure of Rapid Automatized Naming (RAN) for grades 2 and/or 3. If RAN is measured as a part of a subtest such as letter naming fluency, evidence demonstrates the validity of this as a measure of RAN OR RAN, if included as a standalone measure, features |
a. Timed Administration
b. A set of at least 36 familiar objects or letters (stimuli) presented serially and pseudorandomized
c. Five or six token items (stimuli)
d. Requires student to read continuously from left to right, and engage in a "return to sweep" to read on the next line
e. Norming sample identified and criterion referenced scoring based on total naming time.
| (x1) |
| 5 |
| 2 |
| Preferred: Evidence that the assessment includes a measure of encoding or speling at any/all grade levels K-3. |
| (x1) |
| 5 |
| 3 |
| B1: Classification Accuracy: Evidence of meeting the criteria for classification accuracy listed on the Evidence Form: numbers 3- 6. |
| (x5) |
| 25 |
| 4 |
| B2: Reliability: Evidence of meeting the criteria listed on the Evidence Form: numbers 7-12. |
| (x3) |
| 15 |
| 5 |
| B3: Validity: Evidence of meeting the criteria listed on the Evidence Form: numbers 13-18. |
| (x3) |
| 15 |
| 6 |
| B4: Bias Analysis: Evidence of meeting the criteria listed on the Evidence Form: numbers 19-21. |
| (x2) |
| 10 |
| 7 |
| Evidence that the assessment is brief with clear directions for administration. Evidence Form: number 22 |
| (x1) |
| 5 |
| 8 |
| Evidence of including Progress Monitoring Tools to quantify student's rate of improvement or responsiveness to instruction, and to evaluate instruction. For each progress monitoring tool, provides information pertaining to administration frequency and minimum data points for a stable slope that can be use to make instructional decisions. Evidence Form: number 23. |
| (x1) |
| 5 |
| 9 |
| Evidence of supportive resources to guide school based administrators and educators in effective data assessment administration, data access, and data based instructional decision-making. Evidence Form: number 24. |
| (x1) |
| 5 |
| 10. |
| Cost: The assessment and support materials are affordable and cost effective for schools. (separate document) |
| (x2) |
| 10 |
| Total: |
| 100 |
Comments:
image1.jpeg image2.gif
File details come from the government source that posted it. Updated .