Amendment 0001.pdf

NSN 7540-01-152-8070 STANDARD FORM 30. (Rev. 10-83) Previous Edition unusable Prescribed by GSA FAR (48 CFR) 53.243

Page of Pages AMENDMENT OF SOLICITATION/MODIFICATION OF CONTRACT 1. Contract ID Code

2. AMENDMENT MODIFICATION NO. 3. EFFECTIVE DATE 4. REQUISITION/PURCHASE REQ. NO. 5. PROJECT NO. (if applicable)

6. ISSUED BY CODE 7. ADMINISTERED BY (If other than item 6) CODE

(x) 9A. AMENDMENT OF SOLICITATION NO.

9B. DATED (SEE ITEM 11)

10A. MODIFICATION OF CONTRACT/ORDER NO.

8. NAME AND ADDRESS OF CONTRACTOR (NO., Street, Country, State and ZIP Code)

CODE FACILITY CODE

10B. DATED (SEE ITEM 13)

11. THIS ITEM ONLY APPLIES TO AMENDMENTS OF SOLICITATIONS

The above numbered solicitation is amended as set forth in item 14. The hour and date specified for receipt of offers is extended, is not extended.

Offers must acknowledge receipt of this amendment prior to the hour and date specified in the solicitation or as amended, by one of the following methods:

(a) By completing items 8 and 15, and returning ______ copies of amendment; (b) By acknowledging receipt of this amendment on each copy of the offer submitted;

or (c) By separate letter or telegram which includes a reference to the solicitation and amendment numbers. FAILURE OF YOUR ACKNOWLEDGMENT TO BE RECEIVED AT THE PLACE DESIGNATED FOR THE RECEIPT OR OFFERS PRIOR TO THE HOUR AND DATE SPECIFIED MAY RESULT IN REJECTION OF YOUR OFFER. If by virtue of this amendment your desire to change an offer already submitted, such change may be made by telegram or letter, provided each telegram or letter makes reference to the solicitation and this amendment, and is received prior to the opening hour and date specified.

12. ACCOUNTING AND APPROPRIATION DATA (If required)

13. THIS ITEM ONLY APPLIES TO MODIFICATION OF CONTRACTS/ORDERS. IT MODIFIES THE CONTRACT/ORDER NO. AS DESCRIBED IN ITEM 14.

Check One

A. THIS ORDER ID ISSUED PURSUANT TO: (Specify authority) THE CHANGES SET FORTH IN ITEM 14 ARE MADE IN THE CONTRACT

ORDER NO. IN ITEM 10A.

B. THE ABOVE NUMBERED CONTRACT/ORDER IS MODIFIED TO REFLECT THE ADMINISTRATIVE CHANGES (such as changes in paying office, appropriation date, etc) SET FORTH IN ITEM 14, PURSUANT TO THE AUTHORITY OF FAR 43.103(b).

C. THIS SUPPLEMENTAL AGREEMENT IS ENTERED INTO PURSUANT TO AUTHORITY OF:

D. OTHER (Specify type of modification and authority)

E. IMPORTANT: Contractor is not, is required to sign this document and return _______ copies to the issuing office.

14. DESCRIPTION OF AMENDMENT/MODIFICATION (Organized by UCF section headings, including solicitation/contract subject matter where feasible)

Except as provided herein, all terms and conditions of the document referenced in item 9A or 10A, as heretofore changed, remains unchanged and in full force and effect.

15A. NAME AND TITLE OF SIGNER (Type or print)

16A. NAME AND TITLE OF CONTRACTING OFFICER (Type or print)

15B. CONTRACTOR/OFFEROR 16B. UNITED STATES OF AMERICA

(Signature of person authorized to sign)

15C. DATE SIGNED

(Signature of Contracting Officer)

16C. DATE SIGNED

PAGE 2 OF 7 ED-OSE-12-R-0027/0001

The purpose of this amendment is to respond to questions received regarding Solicitation

ED-OSE-12-R-0027. Set forth below are the Department’s responses to questions received:

1. What is the expected level of effort for this project for the base year and four option years? The estimated level of effort is not being released. Offerors should develop a proposed level of effort based on the requirements set forth in the Performance Work Statement.

2. What is the average amount of payment for review panelists for comparable

ED reviews? There are no ED reviews precisely comparable to the GPRA reviews. However, See response to question 14, below.

3. Is training for reviewers to be conducted in-person, or may reviewers complete the training via remote technology (i.e., webinars)? Training of panelists can be conducted via remote technology (i.e., webinars, teleconference).

4. Do the June and July review panels take place in-person, or may materials be mailed to reviewers? Materials may be mailed to reviewers, or the contractor may establish an electronic reporting mechanism. Panelists may rate products and services at a location of their choice (including at their homes), providing the confidentiality of the reviews be maintained at all times.

5. For budgeting purposes, please confirm the base year (June 30, 2012-June 29, 2013) timeline for data collection from projects and the reviews of these data by panels. According to our reading of Subtask 4, panel reviewers are recruited by June 15, 2013, which is very near the end of the base year.

The panels are active from June through August 2013. It appears to us that the majority of the panels’ work occurs after the base year (in option year

1). Subtask 2 indicates that a draft report based on these panels’ ratings is due September 10, 2013 (also during option year 1). ED confirms the accuracy of the information above. As explained in the PWS, data for the base year’s Substask 2 is to be collected during the Spring and Summer of

2012, prior to the start date of the base year for the current contract. For the base year, data collection and the Draft Annual Report will be completed under the existing contract and will be provided by the Contracting

Officer’s Representative (COR) to the successful offeror for this procurement.

6. The company’s representations and certifications in ORCA are accurate and current. Please confirm that we need only include our response to 52.212.3

PAGE 3 OF 7 ED-OSE-12-R-0027/0001

(b) found on page 23 of the RFP and not respond to paragraphs (c) through

(o) found on pages 24-34. Correct.

7. What are the on-going, routine reporting requirements to the CO for this contract? A copy of each deliverable shall be submitted to the Contract

Specialist (CS) as well as a copy of each invoice. There are no on going routine reporting requirements submitted directly to the Contracting Officer for this contract. All documents are submitted to the COR and the CS.

8. What form is the data in from the four programs referenced on page? And in what type of system and/or software is the data stored in? Data from the four programs is to be retained in both Excel and Word formats.

9. Is the data compatible with the visual performance system? The raw scores and other data collected should be retained in Excel format. The VPS system is a text-based system. Please also see the response to item 10., below.

10. Does OSEP or the contractor enter data into the VPS? The COR will enter the data and relevant text from the Draft Annual Report into the VPS system.

The contractor is not required to interact with the VPS system.

11. Is the Visual Performance System up and running? If not, is there another data system used? Yes, the VPS is currently functional.

12. How does the data get entered into the Visual Performance system or current system (if VPS) is not fully functional? See ED’s response to question 10.

13. Under task 2 there is a mention of incorporating requested additional analysis into the annual reports. Can OSEP expand on what types of analysis it has requested in the past? A review of the quick turnaround requests indicates that most were generated by inquiries from OSEP’s budget liaison staff who oversee OSEP’s Part D GPRA reporting. Budget staff have an interest in better understanding obtained results, in examining the validity or reliability of current measures or, (often in response to the OSEP-initiated requests) in investigating the feasibility of new or alternate approaches to data collection and calculation of measures. Most of these inquiries arise in an attempt to improve the utility of the GPRA performance measures in improving grants management. Additional analyses generally include calculations that can be performed by persons experienced in data analyses using parametric and non-parametric statistical significance testing and analyses of subsets of datasets of ratings in the possession of the contractor.

PAGE 4 OF 7 ED-OSE-12-R-0027/0001

14.What have experts been compensated in the past? ED estimates the average fixed-price honorarium paid in the past to GPRA panelists to range from

$1200.00 to $1500.00 for each panel review, although honoraria paid to expert panelists may increase or decrease depending upon the number of products/services reviewed. Offerors may use the attached 2011 GPRA report which provides the number of products/services reviewed last year for each measure, to estimate the work-load of each panel.

15. Piloting the measures would require input from the expert panels, correct? It is not expected that piloting measures would require substantial additional expert panels input over prior year averages.

16. There is no mention of travel. Are panel members ever convened in DC for training? Contractor travel is expected in conjunction with Subtask 1 of the

PWS (The Contractor should budget for three (3) on-site meetings annually with the COR and other ED officials). However, travel is not required to train panel reviewers.

17. Can the guidance information for experts be accessed now? - (there was a guide put together by AIR but not mentioned in the SOW. A 19 page evidence based track out info protocol developed by AIR under contract with Dept. of

ED.) The guidance referred to, the 19-page EBPs protocol developed by

American Institutes for Research (AIR) under contract with OSEP is attached.

This represents only one of the protocols used by the Personnel

Development Syllabi Review Panel. Note that Leadership grants are evaluated with a separate protocol that is included in the Appendix of the attached

2011 IDEA Part D GPRA Draft Performance Report. In addition, other protocols used in the collection of panel data is included in the appendix of in this report.

18. Are the expert panels concurrent in a year? Yes. All 5 panel reviews are expected to be conducted during June and July of each year.

19. Can individual stakeholders serve on more than one panel? This practice is not encouraged, but may occur occasionally when the COR recommends a reviewer for more than one panel.

20. Does OSERS plan to hold in-person stakeholder panel meetings? If so, will

OSERS consider other ways (e.g. group conference calls, online rating systems) of obtaining stakeholder reviews? OSERS does not hold in-person

PAGE 5 OF 7 ED-OSE-12-R-0027/0001

stakeholder panel meetings for the GPRA Part D review. Other ways of holding panel meetings such as those methods described above are encouraged.

21. What is the process OSERS uses to identify qualified stakeholders? The COR provides lists of stakeholder nominees recommended by GPRA Part D program staff on the basis that they are considered to have a history of experience and training relevant to the work of the IDEA Part D program for which they are serving as experts. Such nominees also must not have a conflict of interest in serving as reviewers of Part D program products and services.

22. Please provide documentation on the Department’s Visual Performance Suite that describes data collection/storage capabilities that OSEP uses to capture

GRPA PART A data, in addition to the system’s technical platform, data requirements, reporting capabilities and raw data file format. OSEP declines to respond to this question because the contractor is not expected to interact with the VPS system. Data files submitted to us in Excel or Word format are acceptable.

23. Subtask 2 requires the contractor to conduct “an evaluation of any new measures or protocols implemented for the first time during the prior year, and including the evaluation of any new measures in the appendix of the report.” Do these evaluations include the design and implementation of primary data collection activities by the contractor? Is so, please describe the methodologies previously used by the Department to conduct these evaluations? Yes, in the past, the Contractor’s evaluation of new measures or protocols implemented for the first time has included both design and implementation of data collection activities in collaboration with the COR. A sample of a proposal for the validation of a new long-term measure used for the parent program is attached.

24. The PWS states that the “contractor shall reimburse the reviewers an equivalent rate of pay for their reviews and shall supply necessary evaluation materials to reviewers.” How does the Department determine the “equivalent pay rate” for reviewers? The statement was made to convey that reviewers are to receive the same daily rate of reimbursement for their reviews, regardless of the Part D Program Area on which the reviewer serves. However, reviewer honoraria may in fact differ depending upon the number of days the contractor estimates that the review will take.

PAGE 6 OF 7 ED-OSE-12-R-0027/0001

25. Is the contractor required to implement a reviewer confidentiality procedure so that the names of individual reviewers cannot be linked to their ratings and narrative explanations? If so, please describe the Department’s current procedure for maintaining the confidentiality of reviewers? Reviewers are selected by the contractor from a list of eligible reviewers provided each year by the COR, and reviewer names are redacted from ratings when files containing final reviewer scores are returned to GPRA Part D Program leads for review each Fall. Reviewers are also provided guidance during the process of training to ensure the absence of a conflict of interest, and of the confidentiality of review proceedings.

26. Please describe the methodology the Department uses for selecting the grantees that participate in IDEA PART D GPRA Performance Measurement activities.

These methods are described, by Part D Program, in the attached 2011 IDEA

Part D GPRA Draft Performance Report.

27. Subtask 6 requires the Contractor to “establish or ensure valid and reliable processes and procedures for the implementation of all performance measures, including any new or revised performance measures.” Does the Department expect the contractor to monitor the implementation of grantee data collection activities to ensure valid and reliable processes and procedures are in place at the grantee level? No. This reference refers only to processes and procedures designed and carried out by the Contractor and within the Contractor’s influence and control .

28. How many grantees on average, will be included in IDEA PART D GPRA

Performance Measurement activities on an annual basis? The exact number of included grantees in 2011 is available, for each Part D Program, in the attached 2011 IDEA Part D GPRA Draft Performance Report. This information can be considered typical of the number of grantees included in the GPRA performance review in an average year.

29. The RFP mentions that a Science Panel will be composed of 6-8 reviewers who will rate the quality of PTI, Technology and Media, and TA&D products and services. Will there be a separate panel for each area or will a single panel review products and services across all areas? Will this apply to the other types of panels and panelists as well? In the past, there has been a single

Science Panel that has rated products across these three Part D programs.

PAGE 7 OF 7 ED-OSE-12-R-0027/0001

30. For budgeting purposes, what are the daily rates of each of the different types of panel members? How many days or hours should we budget for their reviews? Please see the response to question 14.

31. Are there any costs involved when the contractor obtains products or services from the various grantees? The contractor is expected to devote time to preparation of and editing of correspondence required for the collection of products and services from grantees, and to the process of corresponding with grantees during the course of soliciting products and services and related documentation from grantees. Additional costs may be required for the purpose of making copies of products (such as CDs or DVDs) sufficient for panelists to conduct their reviews and for mailing copies of products and service descriptions to panelists.

32. Are there any costs involved with the collection or compilation of data from the agency or from the various grantees? The contractor is not expected to incur costs related to the collection of data from ED other than the costs related to shipping by UPS or FEDEX, or courier service required to pick up approximately 2 10 lb (maximum) boxes of materials from ED per year, as well as costs of shipping back to ED in Fall, after panel reviews conclude, 1 sample copy of each product and related product/ service description collected in the Spring for the each of the four Part D program areas. Typically four boxes of sample grantee materials are returned by UPS or FEDEX to ED in

Fall of each year.

All terms and conditions of Solicitation ED-OSE-12-R-0027 including the solicitation closing date of May 3, 2012, 1:00 pm ET, remain unchanged.

Contracts & Acq Mgt/Group B, 550 12th St SW - 7th Floor Washington DC 20202-4240