School_Choice_Amendment_0001-_ED-IES-15-R-0023.pdf

Attachment Page

PAGE 2 OF 12 ED-IES-15-R-0023/0001

Responses to Clarification Questions

Parent Information and School Choice Evaluation

Solicitation # ED-IES-15-R-0023

1. Do the appendices need to be in Word or just the 50-page technical?

U.S. Department of Education Response: Yes, offerors shall submit the appendices in

Word.

2. Is it required that all experiment participants be parents of school-aged children? Parents of children of any age? Merely adults?

U.S. Department of Education Response: ED believes that participants should be parents of school aged children and that findings based on experiments not using such participants would lack external and face validity. If an offeror believes that a specific research question can be answered using different participants -- without compromising validity or the cohesiveness of the proposed experiments when taken together to answer the set of research questions -- this can be proposed but should be thoroughly justified.

3. Page 55 (section J) of the RFP says the documentation of IRB approval is due 34 weeks after contract award and page 60 says that it is due 38 weeks after contract award – can

IES please confirm the correct due date?

U.S. Department of Education Response: IRB approval is due 38 weeks after the contract has been awarded.

4. Instructions on page 169 (section L) of the RFP indicate the 50-page limit includes letters of commitment, but instructions on p. 170 indicate they should be included in an appendix. Is it acceptable to submit these letters in an appendix?

U.S. Department of Education Response: Letters of commitment can be included in an appendix.

5. Page 8: This page refers to three contract types: fixed price, cost reimbursement, cost reimbursement fixed fee. The PWS designates fixed price and cost reimbursement tasks.

Can a fixed fee be added to the cost reimbursement tasks?

PAGE 3 OF 12 ED-IES-15-R-0023/0001

U.S. Department of Education Response: The chart provided on page 8 includes the column labeled “CR Fixed Fee” in the event that an offeror proposes a fixed fee on the cost reimbursement tasks.

6. Page 8: What is the estimated total value of the contract? What is the estimated level of effort in hours?

U.S. Department of Education Response: Please see the Forecast of ED Contract

Opportunities (http://www2.ed.gov/fund/contract/find/forecast.html) for a rough estimate of the total contract value. Offerors should keep in mind that a primary goal of this evaluation is to carry out low cost, quick turnaround experiments.

7. Page 2: Who fills in the schedule on page 2? If the offeror, does it go in the cost proposal only?

U.S. Department of Education Response: The schedule should be filled out by the offeror and included in the business proposal. No price/cost data should be included in the offerors technical proposal.

8. Page 44: Legislative authority references OII’s charter school program. Will OII have any role in the supervision of this contract? If so, please describe.

U.S. Department of Education Response: OII will not be supervising the contract but will be a primary audience for the evaluation’s findings.

9. Page 46-48: Research questions are focused on the format, amount, and organization of parent information and the district guide will include findings. Can the guide include guidance for districts related to effective strategies for customization to district context and dissemination?

U.S. Department of Education Response: The goal of the guide is to provide evidence based guidance to districts. ED does not believe that answering questions about customization or dissemination will be possible as part of the current evaluation and is not aware of rigorous studies that have examined these issues.

10. Page 49: Is it allowable for the project budget to include monetary incentives for participants in studies?

U.S. Department of Education Response: Yes, however these incentives must be modest and approved by the Office of Management and Budget.

PAGE 4 OF 12 ED-IES-15-R-0023/0001

11. Page 50, recruiting additional participants is suggested “…to assure that subgroups of interest are included in large enough numbers.” Does this mean there is an explicit interest in testing contrasts for specific subgroups or only that these subgroups should be adequately represented in the samples?

U.S. Department of Education Response: Given the evaluation’s cost constraints, ED suspects it will not be possible to sufficiently power all experiments to look for group differences. For validity, it will be important to include parents with limited time, economic resources, and even English language literacy challenges as participants.

However, if an offeror has a specific hypothesis about a presentation strategy that may work well with one group of parents but not another, an experiment powered to look at group differences can be proposed.

12. Page 50: Outcome measures: Please provide a reference to the “already validated measures of outcomes” developed outside the field of education. What are these measures?

U.S. Department of Education Response: Studies of information presentation have been conducted in other fields such as health care and information sciences. Offerors will need to identify and consider which, if any, previously developed measures used in those studies are appropriate for the current evaluation given the contexts in which they were developed and validated.

13. Page 50: Parent/participant recruitment: What are some examples of “appropriate standing panels of survey participants?” What survey is being referred to?

U.S. Department of Education Response: There are multiple standing panels of individuals, maintained by different companies and organizations, available to complete surveys or engage in other online exercises. These standing panels are recruited in various ways and have varying characteristics. The offeror will need to determine which, if any, are comprised of individuals appropriate to participate in experiments about school choice information presentation.

14. Page 53: Are individuals who served on the preliminary contract’s expert panel appropriate TWG members?

U.S. Department of Education Response: Yes.

PAGE 5 OF 12 ED-IES-15-R-0023/0001

15. Page 11: OMB clearance . We see that OMB clearance is to be obtained by the end of the base period of the contract. Would the contract timeline be amended or extended if there were delay in obtaining clearance beyond the contractor’s control?

U.S. Department of Education Response: If events beyond the contractor’s control delay

OMB clearance, ED would make efforts to extend the base period of the contract.

However, the schedule of deliverables was designed to leave enough time for clearance, if earlier deliverables are submitted in a timely manner.

16. Page 57-58, Task 7: Can the information presented in the guide include formats beyond tables, maps, lists, charts, etc. to include paper or interactive tools?

U.S. Department of Education Response: A 508 complaint version of the guide and a print version of the guide will be necessary. However, if possible given the evaluation’s timeline and cost constraints, offerors can propose interactive features for print or online versions of the guide.

17. Page 57-58: Task 7: How will ED disseminate the district guide?

U.S. Department of Education Response: This has not yet been determined but it is likely that OII will provide one avenue of dissemination.

18. Page 169: Do tables have to be DS 12 pt?

U.S. Department of Education Response: Tables can use smaller print but they must be easily legible.

19. Page 174: Offeror labor rates have to broken out; they cannot be fully loaded. Does this refer only to the prime contractor or do subcontractor labor rates also have to be broken out? May subcontractors bid fully loaded rates? If so, are they held to the $800/day consultant cap?

U.S. Department of Education Response: The requirement to break out labor rates applies to both the offeror and any proposed subcontractors. The requirement to provide a justification for any rate higher than $800 per day is for consultants.

20. Page 170 says resumes of no more than four pages each for key staff? Are resumes only required for key staff (not project staff)? What is the definition of key staff?

PAGE 6 OF 12 ED-IES-15-R-0023/0001

U.S. Department of Education Response: Typically, key staff include a Project Director, Deputy Project Director (if there is one), Principal Investigator (or Co-PIs), and an individual charged with overseeing data collection. Given the scope of this evaluation, one or more of these roles may be held by the same individual. As a general rule of thumb, résumés for any staff providing leadership and direction or overseeing a significant aspect of the project (e.g., development and programming of experimental materials) should be included.

21. Page 171: “offerors shall provide a complete list of the title, contract or grant number, award date, end date, and dollar amount of all contracts or grants with the US Department of Education that are current or ended after November 1, 2011” What is the purpose of this list?

U.S. Department of Education Response: This list helps ED understand offerors prior experience, current commitments, and identify any possible conflicts of interest.

22. Page 171: “offerors shall provide a complete list of the title, contract or grant number, award date, end date, and dollar amount of all contracts or grants with the US Department of Education that are current or ended after November 1, 2011” Should we include ED contracts for which we are a subcontractor?

U.S. Department of Education Response: Offerors shall provide the requested information about contracts that are considered to be relevant to this opportunity.

Offerors shall indicate whether they were the prime contractor or subcontractor for each contract listed.

23. Page 171 of the solicitation references Attachment C, Small Business Subcontracting

Plan (SBSP) template, although the attachment was not included with the solicitation. Can IES kindly provide the template or may Offerors use their own?

U.S. Department of Education Response: Attachment C, Small Business Subcontracting

Plan (SBSP) template has now been included for offerors to use.

24. The Small Business Subcontracting Plan is referenced in the instructions for the technical proposal on page 171, but presumably should be included in the Business Proposal because there are typically dollar amounts in such a plan. Can IES kindly confirm that the SBSP should be included in the Business Proposal?

U.S. Department of Education Response: The Small Business Subcontracting Plan should be included in the offerors business proposal

PAGE 7 OF 12 ED-IES-15-R-0023/0001

25. When are the past performance references due? The RFP states July 31 but the form states July 15 (page 143).

U.S. Department of Education Response: All past performance references are due at the same time proposals are due, by 1:00 pm EST, July 31, 2015.

26. Does the Department want three past performance references from each subcontractor in addition to the prime contractor, or three total across all organizations?

U.S. Department of Education Response: ED would like three past performance references from the prime contractor. Additional references can be included for subcontractors but are not required.

27. The number of contracts and grants won or completed after November 1, 2011 (page

171) can be large for some of the larger organization and the contracts and grants listed may not be at all relevant, but take pages to list. Would be possible to either 1) provide this information in an appendix or 2) only provide the requested information about contracts/ grants that are considered to be relevant for this opportunity?

U.S. Department of Education Response: Please provide the requested information about contracts/grants that are considered to be relevant to this opportunity.

28. Page 1 of the solicitation states that proposals are due by 1:00 PM on July 31, 2015 however page 168 of the solicitation states that proposals must be submitted by 11:00

AM, EST on July 31, 2015. Can IES kindly clarify the submission time?

U.S. Department of Education Response: Please submit all proposals no later than

1:00pm EST on Friday, July 31, 2015.

29. When does IES anticipate submission of the OMB package? It appears to be due before the first TWG meeting is held (week 22-24), yet the materials and procedures may be revised based on feedback from the TWG. Does IES anticipate making these changes after the first 60 days of the clearance process? period?

U.S. Department of Education Response: The draft OMB package will include a description of the number and types of experiments that will be conducted, the sample sizes and burden estimates, any expected recruitment activities, and the analytic methods that will be used. If any changes are made to specific plans or data collection instruments

PAGE 8 OF 12 ED-IES-15-R-0023/0001

based on feedback from the TWG, these will be submitted to OMB during the clearance process.

30. The PWS says that the contractor cannot collect any personally identifiable information on people in a sample for any experiments conducted. How does IES define “personally identifiable information” for this contract? Can we collect demographic information as long as we don’t collect name and DOB/SSN? If people are part of a “standing panel” of survey participants and personally identifiable data is already known for them, can this be used to field data collection instruments and track responses?

U.S. Department of Education Response: Demographic information can be collected.

Personally identifiable information cannot be collected for the purposes of this study.

This does not include information (such as contact information) already gathered for members of a standing panel.

31. If we cannot collect or use any personally identifiable information on respondents, how does the Department suggest that we follow-up with or identify people who do not respond to a survey?

U.S. Department of Education Response: It is permissible for the offeror to access an appropriate panel of participants for which contact information has already been gathered. Also, many panels have intermediaries managing communications with potential participants, in which case the offeror would not need to obtain any contact information themselves. The PWS suggests using opportunistic samples and ED does not expect any follow up contact with participants will be necessary after they have participated in an experiment.

32. In Task 5 of the PWS, there is a statement about tracking data collection for each experiment. How does IES define an “experiment” in this case? If there are multiple comparisons of approaches conducted with a sample, is each comparison or round considered its own experiment?

U.S. Department of Education Response: Given the various ways in which experiments addressing the set of research questions may be designed and organized, it is up to offerors to develop a meaningful and logical framework.

33. On p. 46 of the RFP (third bullet in Section C), one of the research questions is: “Is there a way to make the information parents most often report caring about salient?” Can IES clarify the focus of that research question?

U.S. Department of Education Response: Research generally finds that parents list academic achievement among the most important school characteristics but often do nout act on the basis of it when choosing schools (e.g., Schneider et al. 1998). Districts do not consistently provide the same types of achievement information to parents and formats for displaying information varies greatly. Therefore, one question that is within the scope

PAGE 9 OF 12 ED-IES-15-R-0023/0001

of the current study is whether academic information would be used more in decision making if it was easy to understand and more salient (e.g., prominently displayed, presented before other types of information, etc.).

34. Are there any restrictions on responding to this RFP?

U.S. Department of Education Response: Responsible, large and small businesses who are (a) not suspended or debarred from doing business with a state/local/or the federal government, (b) can conduct businesses under the NAICS code for this procurement, and (c) meet the terms and conditions set forth in the solicitation clauses and provisions may submit proposals for this solicitation.

35. Section L direct an electronic submission via email of the proposal, then goes on to specify hard copy submittals. Please clarify the requirement.

U.S. Department of Education Response: In addition to an electronic submission, offerors must submit one hard copy of the technical and business proposal.

36. Do the required cover sheet, abstract, and table of contents count towards the 50 page limit?

U.S. Department of Education Response: Yes, these are included in the page limit.

37. The end of the second paragraph of the “Organizational Capabilities & Experience” section on pg. 171 states that, “This section shall provide the names of the proposed project director, other key personnel, subcontractors, and consultants.” Does this indicate the these individuals and organizations should be included with the summaries of existing/completed contracts directly related to the current procurement? Should these individuals/organizations be listed if they are not proposed as key personnel in the current proposal?

U.S. Department of Education Response: Yes, if the proposed project director, other key personnel, subcontractor or consultant for the current requirement was involved in one of the existing/completed contracts, the offeror shall indicate so.

38. The RFP instructions do not mention where/if references should appear. Should a reference page be included with the technical proposal or as an appendix? Do the references count towards the page limit?

U.S. Department of Education Response: References will not be counted against the page limit and shall be included at the end of the Technical Proposal or as an appendix.

PAGE 10 OF 12 ED-IES-15-R-0023/0001

U.S. Department of Education

SUBCONTRACTING PLAN REVIEW FORM

Requisition No: Multiple Awards? Yes: No: (If yes, attach a copy of each subcontracting plan)

PROJECT INFORMATION

Solicitation/Contract No.: MOD No. (If applicable):

Title of Acquisition:

Contractor’s Name:

Period of Performance: Total Contract Amount (including options): $

Total Modification Amount: (if applicable) $ Base Period (if there are options): $

Option 1 (if applicable): $ Option 2 (if applicable): $

Option 3 (if applicable): $ Option 4 (if applicable): $

Contracting Officer/Specialist Name: Tel&Fax:

Operating Component/Division: Email:

SUBCONTRACT PLAN REQUIREMENTS

1. Type of Plan (check one): Individual Master Commercial

(A=Acceptable; U=Unacceptable) C.O. SBS SBA/PCR

2. Subcontracting Goal Data A U A U A U

a. Total Subcontracting Dollars [(2b+2h=2a), except when subcontract baseline equals contract value]

b. Total Subcontracting Dollars & Percentage with Small Businesses (incl. SDB, WOSB, HUBZone, SDVOSB) - [Percentage of 2a.]

$ and %

c. Total Subcontracting Dollars & Percentage with Small Disadvantaged Businesses - [Percentage of 2.a.]

$ and %

d. Total Subcontracting Dollars & Percentage with Woman-owned Small Businesses - [Percentage of 2.a.]

$ and %

e. Total Subcontracting Dollars & Percentage with HUBZone Small Businesses - [Percentage of 2.a.]

$ and %

f. Total Subcontracting Dollars & Percentage with Veteran Owned Small Businesses - [Percentage of 2.a.]

g. Total Subcontracting Dollars & Percentage with Service-Disabled Veteran Small Businesses - [Percentage of 2.a.]

$ and %

h. Total Subcontracting Dollars & Percentage with “Other” than Small Businesses (i.e., large companies, non profits, etc.) [Percentage of 2.a.]

$ and %

h. Subcontracting Opportunities (description of all principal products/services to be subcontracted to all types of concerns) i.j.k. Methodology used to develop goals & identify potential sources (e.g. historical trends, information on technical and competitive bidding, formula for calculating goals, etc.).

ED-IES-15-R-0023 Attachment C

PAGE 11 OF 12 ED-IES-15-R-0023/0001

U.S. Department of Education

SUBCONTRACTING PLAN REVIEW FORM

SUBCONTRACTING PLAN REQUIREMENTS (con’t) C.O. SBS SBA/PCR

(A=Acceptable; U=Unacceptable) A U A U A U

3. Subcontracting Plan Administrator’s Name (Contractor):

4. Description of efforts to ensure the Small Businesses (incl. SDB, WOSB, HUBZone, SDVOSB) entities has equitable opportunity to compete for subcontracts.

5. Required flow-down clause(s) to be included in prime contractor’s subcontracts (i.e, FAR Clause 52.219-8, 52-219-9, 52.212-5(e)):

6. Reporting and Cooperation:

a. Agreement to submit required reports

b. Agreement to cooperate in studies and surveys

7. Record keeping

8. Timely Payments to Subcontractors

9. Description of Good Faith Effort

CONTRACTING OFFICER DETERMINATION, SBS AND SBA PCR

RECOMMENDATIONS

C.O. SBS SBA/PCR

Yes No Yes No Yes No

1. The proposed plan meets the requirements of FAR 19.704.

2. In accordance with 19.705-4, past performance has been considered when determining acceptability of this plan.

3. The proposed plan requires an additional pre-award review.

Contracting Officer Signature

Contracting Officer: Date:

Additional Comments:

Small Business Signature

Small Business Specialist: Date:

COMMENTS: If any elements are determined to be unacceptable, summarize below:

SBA PCR Signature

SBA PCR: Date:

COMMENTS: If any elements are determined to be unacceptable, summarize below:

PAGE 12 OF 12 ED-IES-15-R-0023/0001

SUBCONTRACTING PLAN REVIEW FORM INSTRUCTIONS

Requisition No.: Insert the requisition number assigned by the

CO/CS.

Multiple Awards: indicate as appropriate. If yes, attach a copy of each subcontracting plan.

PROJECT INFORMATION:

Solicitation/Contract No.: Enter the assigned RFP or Contract Number

Modification: Identify the modification number for the contract if applicable.

Title of Acquisition: Enter the item/service description or project title

Contractor’s Name: Enter Successful Offeror/Contractor’s name

Period of Performance: Enter the estimated performance period, including all options, in the following format (mm/dd/yy – mm/dd/yy)

Total Contract Amount: Enter the total estimated dollar value of the contract, including all options & modifications.

Total Modification Amount: (if applicable)

Base Period & Options 1 through Option 4: Complete these boxes if options are part of the contract.

CO/CS Contact Information: Enter Contracting

Officer/Specialist‘s Name, Room, Telephone, and Fax and e-mail.

SUBCONTRACTING PLAN REQUIREMENTS (ITEMS 1 – 9)

1. Enter type of plan: individual, master or commercial

2. For each of the sub-items 2.a. through 2k., the C.O. must review the plan & determine if the requirement is acceptable or unacceptable.

For each of the sub-items 2.b. through 2.g., the C.O. shall include the subcontracting dollars & percentage for the category. In calculating percentage, use the subcontracting dollars for the sub-item as the numerator & the total subcontracting dollars for the contract as the denominator. (i.e.

2.b. divided by 2.a. = % for 2.b.)

3. 3 through 9: the C.O. must review the plan & determine if the requirements are acceptable or unacceptable.

CONTRACTING OFFICER DETERMINATION, SMALL

BUSINESS SPECIALIST & SBA PCR RECOMMENDATION

(ITEMS 1 – 3)

1. The CO must review the plan to determine if it meets the requirements in FAR 19.7.

2. Indicate whether past performance had been considered when determining acceptability of plan

3. Indicate whether plan requires further pre-award review.

SIGNATURES

The CO who has the authority to bind the government will make a determination, sign and date.

The SBS will sign and date the review form and the subcontracting plan signature page if acceptable.

During the plan review, the SBS may require additional input from the CO and/or contractor. The SBS may also include comments regarding the plan as necessary

The SBA PCR shall sign and date the review form & the subcontracting plan signature page if acceptable.

concurrence or non-concurrence of the acquisition method determined by the CO The SBA PCR may also include comments regarding the plan as necessary.

NOTE: In order for the Small Business Specialist to conduct a comprehensive review of each plan, at a minimum, the documentation forwarded by the CO should include:

1. A completed Subcontracting Plan Review Form signed by the Contracting Official

2. A completed Subcontracting Plan, using the Department of Education Plan, reviewed, signed by the offeror and the CO.

3. The Summary of Proposed Costs from the offeror’s Final Proposal Revision.

4. Any narrative, attachment or supplemental documentation to the offeror’s plan describing efforts to locate small business subcontractors, rationale for using other than small businesses as subcontractors, etc.