Questions_&_Answers.pdf

OMB Control No. 1505-0081

AMENDMENT OF SOLICITATION/MODIFICATION OF CONTRACT

1. CONTRACT ID CODE PAGE OF PAGES

2. AMENDMENT/MODIFICATION NO. 3. EFFECTIVE DATE 4. REQUISITION/PURCHASE REQ. NO. 5. PROJECT NO. (If applicable)

A0002 11/21/2018 5000037222

6. ISSUED BY CODE 12163 7. ADMINISTERED BY (If other than Item 6) CODE

Internal Revenue Service 5000 Ellin Road

Lanham, MD 20706

Lisa S. Martin 240 613-5620

See Item 6

8. NAME AND ADDRESS OF CONTRACTOR (No. Street, county, State and ZIP: Code)

Vendor ID: To All Offerors

(x) 9A. AMENDMENT OF SOLICITATION NO.

2032H8-19-R-00002 x 9B. DATED (SEE ITEM 11)

11/21/2018

10A. MODIFICATION OF CONTRACT/ORDER NO.

10B. DATED (SEE ITEM 13)

CODE FACILITY CODE

11. THIS ITEM ONLY APPLIES TO AMENDMENTS OF SOLICITATIONS

is not extended.

Offers must acknowledge receipt of this amendment prior to the hour and date specified in the solicitation or as amended, by one of the following methods:

(a) By completing Items 8 and 15, and returning _____ copies of the amendment; (b) By acknowledging receipt of this amendment on each copy of the offer submitted; or (c) By separate letter or telegram which includes a reference to the solicitation and amendment numbers. FAILURE OF YOUR ACKNOWLEDGMENT TO BE RECEIVED AT THE PLACE DESIGNATED FOR THE RECEIPT OF OFFERS PRIOR TO THE HOUR AND DATA SPECIFIED MAY RESULT IN REJECTION OF YOUR OFFER. If by virtue of this amendment you desire to change an offer already submitted, such change may be made by telegram or letter, provided each telegram or letter makes reference to the solicitation and this amendment, and is received prior to the opening hour and data specified.

12. ACCOUNTING AND APPROPRIATION DATA (If required)

13. THIS ITEM APPLIES ONLY TO MODIFICATIONS OF CONTRACTS/ORDERS,

IT MODIFIES THE CONTRACT/ORDER NO. AS DESCRIBED IN ITEM 14.

(x) A. THIS CHANGE ORDER IS ISSUED PURSUANT TO: (Specify authority) THE CHANGES SET FORTH IN ITEM 14 ARE MADE IN THE CONTRACT ORDER NO. IN ITEM 10A.

B. THE ABOVE NUMBERED CONTRACT/ORDER IS MODIFIED TO REFLECT THE ADMINISTRATIVE CHANGES (such as changes in paying office, appropriation date, etc.) SET FORTH IN ITEM 14, PURSUANT TO THE AUTHORITY OF FAR 43.103(b).

C. THIS SUPPLEMENTAL AGREEMENT IS ENTERED INTO PURSUANT TO AUTHORITY OF:

D. OTHER Specify type of modification and authority)

E. IMPORTANT: Contractor is not, is required to sign this document and return ____ copies to the issuing office.

14. DESCRIPTION OF AMENDMENT/MODIFICATION (Organized by UCF section headings, including solicitation/contract subject matter where feasible.)

The purpose of this amendment is to: Provide a comprehensive list of vendor questions and answers in response to the Enrolled Agents Special Enrollment Examination solicitation 2032H8-19-R-00002 dated 8 November 2018.

Except at provided herein, all terms and conditions of the document referenced in Item 9A or 10A, as heretofore changed, remains unchanged and in full force and effect.

15A. NAME AND TITLE OF SIGNER (Type or print) 16A. NAME AND TITLE OF CONTRACTING OFFICER (Type or print)

15B. CONTRACTOR/OFFEROR 15C. DATE SIGNED 16B. UNITED STATES OF AMERICA 16C. DATE SIGNED

(Signature of person authorized to sign)

BY _____________________________________________

(Signature of Contracting Officer)

NSN 7540-01-152-8070 30-105 STANDARD FORM 30 (REV. 10-83)

PREVIOUS EDITION UNUSABLE Computer Generated Prescribed by GSA

FAR (48 CFR) 53.243

X The above numbered solicitation is amended as set forth in Item 14. The hour and date specified for receipt of Offers is extended, Lisa S. Martin

X

X

Disclaimer: Answers to many questions are found in Agency policy (IRM, Pubs, standards, etc.) referenced in the solicitation document. As a courtesy, we have answered these questions here as well. However, in the event of any conflict between answers here and Agency policy, Agency policy shall prevail. All answers, including historical data, are subject to the terms and conditions of the RFP.

1. Q: Pg. 14, 11: What percentage of the items item bank is currently affected by statute or law change, on average, per year?

A: All questions are reviewed for accuracy every year. On average, over the last four years, roughly 5% of the questions in the item bank were affected by changes in tax law.

2. Q: Pg. 17, 4.02: Are candidates pre-authorized to take the examinations?

A: Candidates are not pre-authorized by IRS to take the examination but they must have a current Preparer Tax Identification Number (PTIN).

3. Q: Pg. 19, 4.03.1.3: Could the IRS provide the total number of examinations administered in each of the past three years, broken down by month and zip code?

A: The number of tests administered by month is as follows. This information by zip code is not readily available.

Testing Cycle May Jun Jul Aug Sep Oct Nov Dec Jan Feb Total 5/1/17-2/28/18 1,479 1,433 1,503 1,828 1,697 1,924 2,308 2,839 2,473 3,525 21,009 5/1/16-2/28/17 1,253 1,231 1,106 1,848 1,835 2,045 2,370 2,796 2,070 2,890 19,444 5/1/15-2/29/16 1,206 1,495 1,366 1,891 2,019 2,231 2,346 3,321 2,028 2,714 20,617

4. Q: Pg. 185 of the RFP states that vendors must “request past clients’ representatives to complete the questionnaire and submit it directly to the IRS contact identified on the questionnaire no later than 10AM ET 11/05/18.” The Questionnaire on Pg. 190 states that the desired response date is 11/30/2018. We presume that the 11/05/18 date was included in error, and the desired response date is 11/30/2018. Would the IRS consider extending that response date to 12/04/18 to reflect the updated RFP schedule?

A: The 11/05/18 date on page 185 was an error. The IRS will extend the response date for Past Performance Questionnaires from 11/30/18 to 12/04/18.

5. Q: Should 52.212-1 – Instructions to Offerors – Commercial Items, pgs. 156-179 or the RFP, be included with the proposal, or should it be completed by the awarded vendor and submitted post-award?

A: Instructions to Offerors are required to be included with submitted proposals.

6. Q: Should Attachment A, entitled “Letter, Non-Disclosure Agreement” on page 188-189 of the RFP be included in the proposal response, or should it be completed by the awarded vendor and submitted post-award?

A: The required non-disclosure agreements will be completed by the awarded vendor and submitted post-award.

7. Q: 4.01.2: What methods have typically been used for cut-score determination, e.g., Modified-Angoff, Bookmark, other methods?

A: The model that has historically been used to establish a cut score is proprietary. The Contractor will be required to model the cut score determination in accordance with the current edition of The Standards for Educational and Psychological Testing or other legally defensible test development standard.

8. Q: How often are cut-score reviews typically conducted per exam?

A: A cut score determination for all three parts of the exam is normally conducted only once at the beginning of the five-year contract period. However, scoring for each exam is continuously monitored to make sure the exams are performing within their acceptable ranges. In the event a cut score needs to be revised for a specific exam part, another cut score determination may be conducted.

9. Q: Have the job analyses resulted in three separate surveys or one comprehensive survey?

A: One comprehensive survey was administered as part of the job analysis.

10. Q: As a no cost to the government contract, please confirm if Background Investigations and Consumer Credit Scores are paid for by the IRS.

A: The IRS pays for the cost of background investigations, including consumer credit scores and fingerprinting. Any costs for fingerprinting outside an IRS office or approved enrollment center will be borne by the Contractor.

11. Q: Do consumer credit checks (FCRA) apply to contractors who are deemed “Low Risk” or Tier 1?

A: Yes, all background investigations include consumer credit checks, regardless of the position sensitivity risk level.

12. Q: Are there any “Moderate” or “High” risk investigations that would apply to contract staff for this project?

A: The selected vendor will have to complete a Position Designation Survey (PDS) post-award for each category of employee to determine their risk level, subject to IRS review and approval. The IRS expects employees in certain positions to be designated as Moderate risk. A blank PDS is attached for your convenience.

Position Designation Survey.

13. Q: Could you please identify the current breakdown in positions requiring security clearances and their associated clearance level?

A: All Contractor employees, as well as all employees of the contractor’s subcontractors, partners, consultants, SMEs, suppliers, vendors, etc. require security clearance if their assignments expose them, or could potentially expose them, to IRS sensitive but unclassified (SBU) information. See IRS Publication 4812 for the definition of SBU.

14. PRICING – please provide current candidate fee collection per candidate - per test

a. Q: Testing fee from Prometric Bulletin

A: $181.94

b. Q: The testing fee is $181.94 for each part of the examination. This fee is paid at the time you schedule your examination A: This is correct

c. Q: Government User-Remittance Fee Collected $81 A: This is correct

d. Q: Contractor Fee Collected A: $100.94

e. Total Fee Collected from candidate assume $181.94 A: This is correct

15. Q: When was the last job analysis of EAs completed?

A: The last comprehensive job analysis was conducted in August 2015.

16. Q: Was a national survey conducted?

A: Yes.

17. Q: Are the job analysis data and report available to the new vendor?

A: Some of the data may be available on a need-to-know basis, as determined by the IRS.

18. Q: Is an electronic file available containing email addresses of practitioners?

A: An electronic file containing email addresses of practitioners is not available. Contact information for practitioner organizations can be provided by IRS to assist the vendor in recruiting a diversified group of SMEs for test development.

19. Q: What item types are included in the current EA exam?

Attachment A.

A: Each question is multiple choice with four answer options including one single best answer. See 4.01.1(5)(g) of the Performance Work Statement.

20. Q: Does the IRS desire new item types to be developed?

A: Currently there are no plans to change the format of the test questions.

21. Q: What languages are to be supported?

A: English. See 4.01.1(5)(b) of the Performance Work Statement.

22. Q: Are the current exams fixed forms or dynamically generated for each candidate?

A: The current exams are multiple versions of fixed forms. The questions are randomized within the fixed forms upon delivery to the candidate at the test center.

23. Q: Does the IRS have an item bank that can be leveraged for future exams?

A: Yes; however, newly written questions will also be required as described in Section

4.01.1 of the Performance Work Statement.

24. Q: How many items are in the bank?

A: Approximately 1,300 items are in the item bank.

25. Q: Are the items available in electronic format?

A: Yes.

26. Q: What types of metadata and historical information is available?

A: A listing of all candidates who passed one or more examination parts within the preceding two years (including name, PTIN, appointment confirmation number, date of examination, examination part, and examination form), item analyses from the most recent testing cycle, the item bank, test specifications, pass rates by exam part by testing cycle, and the style guide.

27. Q: How many forms of the test are published each year?

A: The number of published test forms is proprietary information. The Contractor will be required to develop test forms and/or test questions so that test integrity is maintained and test question saturation is prevented.

28. Q: What psychometric model has been used to calibrate items? (e.g. CTT, IRT)

A: Item Response Theory (IRT).

29. Q: Is there a preferred model moving forward?

A: The IRS is interested in vendors’ proposals and will review all options.

30. Q: Is the IRS interested in alternatives to fixed form examinations (e.g., Linear on the

FLY -LOFT)?

A: The IRS is interested in vendors’ proposals and will review all options.

31. Q: Can any of the existing items be recycled for use in the future?

A: Yes, after an annual item review with subject matter experts.

32. Q: What psychometric model has been used historically to calibrate the EA items and exams (CTT, IRT 1pl, 2pl, 3pl)?

A: See question 28.

33. Q: Has the Enrolled Agents exam ever been the subject of a legal challenge?

A: No.

34. Q: It states that the session is 3.5 hrs. Including NDA, Tutorial, and survey. Does this time also include time to check-in and check-out the test taker?

A: The session time is 4.0 hours. The exam is 3.5 hours with 30 minutes for the NDA, tutorial and survey. Check-in and check-out time is not included in the 4.0 hours.

35. Q: What cut score setting method(s) have been used in the past with the EA exams?

A: See question 7.

36. Q: Does the IRS impose constraints on item exposure that would indicate the number of items and exams to be developed per year?

A: No, the IRS does not impose constraints; however, the vendor shall develop sufficient questions and test forms to prevent test question saturation and maintain test integrity.

37. Q: Will the IRS provide contact information and assist in soliciting industry Subject Matter Experts (SME) and IRS Technical Experts to support job analysis and test development activity?

A: It is the vendor’s responsibility to solicit and recruit SMEs. See question 18. IRS will provide the IRS Technical Experts.

38. Q: Since there is only one line for each Period of Performance we are assuming that all costs need to be included in this one line item – correct?

A: Yes, vendors are to disclose the fee per exam for each period of performance to include all vendor costs plus a reasonable profit. See Section 5, Pricing, in the Performance Work Statement.