Attachment 2 - QAs.pdf

Questions and Answers Attachment Title Page # Section Vendor Question Government Response

Notice of Solicitation

1333ND26QNB770088

13 III. SPECIFIC

RQUIREMENTS, Task 2:

Usability Testing

To help us provide a more accurate and relevant proposal for the Usability Testing IDIQ, could you please provide representative examples of task orders you anticipate issuing? Since usability testing can span a wide range of hardware and software-based research with different logistical and cost considerations, examples of both would be greatly appreciated.

Yes, we have provided a representative task order as an example.

Notice of Solicitation

1333ND26QNB770088

13 III. SPECIFIC

RQUIREMENTS, Task 1:

Technology Evaluations

To help us scope and accurately target our response, can you please provide more information about the subject matter? For example, will the topics fall under the general domain, or it could include more specialized domains, for e.g. medical, finance, etc.

The subject matter will fall under the general domain, but capabilities in specialized domains in addition to that would not be out of scope.

General N/A Is this a new Opportunity or Rework? This is a new requirement.

General N/A If rework, who is the incumbent and Can the Govt share the Contractor Name, Contract Number, and Contract Value of the Incumbent?

N/A

General How do we submit the quotes? The Solictation states: All correspondence in conjunction with this solicitation must be directed to the Government Contracting Officer: Billy Wong, hingpan.wong@nist.gov, and Contract Specialist: Jennifer Lohmeier, Jennifer.lohmeier@nist.gov.

General Do we need to submit redacted or named resumes? Named resumes.

General Do we need to submit live or sample resumes? Live resumes.

General What is the level of effort on this project? This requirement is for an IDIQ. Task orders will be issued as firm-fixed price. It is up to the individual contractor to propose the level of effort at the task order level.

RFQ (PDF) 24 Instructions to Offerors Can the Government confirm that Phase 1 (Technical Volume I) is strictly limited to a maximum of nine (9) pages, inclusive of all content (project descriptions and narrative responses)?

Confirmed.

RFQ (PDF) 24 Instructions to Offerors If content exceeds the 9-page limit, will the excess pages be disregarded or will the entire submission be considered non-compliant?

Pages in excess of the limit will not be evaluated.

RFQ (PDF) 24 Instructions to Offerors The solicitation states that of the 9 pages, three (3) pages are allocated for project descriptions and six (6) pages for responses to questions. Is this allocation mandatory, or may offerors adjust the distribution as long as the total remains within 9 pages?

Offerors shall adhere to the page limit as described.

RFQ (PDF) 24 Instructions to Offerors If an offeror uses fewer than 3 pages for project descriptions, may the remaining pages be reallocated to the narrative responses?

No.

RFQ (PDF) 24 Instructions to Offerors The solicitation allows up to five (5) projects with a maximum of one (1) page per project, while also limiting project descriptions to three (3) pages total. Can the Government clarify which requirement governs?

(a) Up to 5 pages total for projects (1 page each), OR

(b) Projects must collectively fit within the 3-page limit?

Projects must collectively fit within the 3-page limit. Not to exceed 5 projects total.

RFQ (PDF) 24 Instructions to Offerors If the 3-page limit governs, is it acceptable to include fewer than five (5) projects to remain compliant?

Yes.

RFQ (PDF) 24 Instructions to Offerors Is there a minimum number of projects required for Phase 1 submission? Please refer to the Instructions to Offerors.

General Can the Government clarify how Factor 1 (Phase 1) will be weighted relative to Phase 2 factors in the final award decision?

The evaluation factors are listed in descending order of importance. All non-price factors when combined are significantly more important than price.

General Will Phase 1 scores carry forward into Phase 2 evaluations, or will Phase 2 be evaluated independently?

The Phases will be evaluated independently. However, all evaluation factors will be considered for the award of the

IDIQ.

RFQ (PDF) 24 Instructions to Offerors How will subcontractor experience be evaluated compared to prime contractor experience?

The solicitation states: The offeror may use prior experiences in which they served as a prime or subcontractor; however, experience as a prime may instill higher confidence.

General Can an entity submit one proposal as a Prime Contractor and also participate as a Subcontractor on one or more additional proposals?

Yes.

General Are there any restrictions on the number of teams an entity may participate in as a subcontractor?

No.

General If an entity participates on multiple teams, are there any requirements to disclose such participation?

The entitiy should disclose any conflicts of interest.

Attachment 1 - Pricing Worksheet

— IDIQ Labor Category Rates tab

The IDIQ SOW (pp. 16–19) defines 12 required labor categories (Project Manager, Research Manager, Assessor I, Assessor II, UX Researcher, Usability Test Moderator, Focus Group Moderator, Senior Developer, Developer, QA Engineer, Software Engineer, Computer Science/Electrical Engineer, Senior Writer). The Pricing Worksheet lists only 9 categories, some of which do NOT appear in the SOW (e.g., "Research Associate," "Research Assistant," "Research Analyst"). Should offerors use the SOW labor categories or the spreadsheet categories?

This has been corrected and the pricing worksheet updated.

Attachment 1 - Pricing Worksheet

— IDIQ Labor Category Rates tab

The Pricing Worksheet includes "Project Manager, TREC" as a separate category from "Project Manager." The SOW defines only one "Project Manager" labor category. Is "Project Manager, TREC" a distinct labor category with different qualifications, or is it the same Project Manager applied to TREC task orders?

The pricing worksheet has been updated.

Attachment 1 - Pricing Worksheet

— Mock TO 2 Pricing tab The Mock TO 2 pricing tab lists "Senior Associate" as a labor category.

This title does not appear in the IDIQ SOW's required labor categories.

Should offerors use this category as shown, or substitute a SOW-defined category?

Updated Mock TO 2 Pricing tab in Pricing Worksheet to change Senior Associate to Project Manager. Updated Mock TO 2 labor categories as well.

Attachment 1 - Pricing Worksheet

— Task Order 1–7 tabs The pricing worksheet contains tabs for seven additional task orders (SATE Support, BlockChain, TestFramework MDC, Conformance Lab, Election Guidelines, Image Analytics, SCDCS) that do not appear in the solicitation. Are these legacy tabs from a prior contract, or are they future task orders that offerors should price?

The pricing worksheet has been updated.

RFQ (PDF) 23-31 Instructions to Offerors The solicitation does not specify page formatting requirements (font, font size, margins, spacing) for any volume. What formatting standards apply?

The Technical and Price Quotes shall all be separate and distinguishable from each other. Text shall be single-spaced with a minimum of one-inch margins on all sides. All text shall be 12-point font, Times New Roman. The solicitation has been revised accordingly.

RFQ (PDF) 1 SF 1449, Block 10 The set-aside is 100% Small Business (checked "Small Business" box), NAICS 541990 with size standard $41.5M (NAICS 541519 also listed).

Which NAICS code governs — 541990 or 541519? Are these separate categories (primary vs. specific)?

The applicable NAICS code is 541990 with a size standard of $19.5M.

RFQ (PDF) 24 vs. 27 Quote Instructions / Phase 2 Instructions

The Phase 1 summary table (p. 24) states Factor 3: IDIQ Management Approach has a 10-page limit. The Phase 2 detail table (p. 27) states Factor 3 has a 5-page limit. Which page limit governs?

This error has been corrected to reflect a 5 page limit.

RFQ (PDF) 3 IDIQ Contract Information The solicitation states "up to three contracts for the primary category and up to three contracts for each of the specific categories." What are the distinct specific categories? Are the primary and specific categories defined elsewhere, or is there only one category (primary)?

This is an error and has been corrected.

RFQ (PDF) 32 Evaluation Factors Factors 1, 2, and 3 are stated as "equal in importance" and each more important than Factor 4 (Price). However, p. 26 states "the Phase 1 factor is more important than Phase 2 factors." Do these statements conflict, or does the advisory down-select effectively make Factor 1 a gating criterion even though formally co-equal?

The Phases will be evaluated independently. This error has been corrected. The evaluation factors are listed in descending order of importance. All non-price factors when combined are significantly more important than price.

RFQ (PDF) 33 Factor 1 Evaluation The evaluation notes "the Government's confidence level in the offeror may be increased if elements of the offeror's prior experience included work with disasters or natural hazards." How does disaster/natural hazard experience relate to the TTED technology evaluation mission?

This is an error and has been corrected.

RFQ (PDF) 27 Phase 2 Submission Due DatePhase 2 is due May 7, 2026. The solicitation does not specify when advisory notifications will be issued after Phase 1 (due April 23). What is the anticipated timeline between Phase 1 evaluation completion and advisory notification?

The advisory notice is estimated to be sent within 15 business days of the solicitation due date. Please note this is an estimate and might increase/decrease based on the number of responses received.

RFQ (PDF) 37 Mock TO 1, Task 2 The SOW states the contractor shall "Judge approximately 80 questions with approximately 40 generated paragraphs per question and approximately 200 web page citations per question." What is the expected throughput rate (judgments per hour) for citation judgments, nugget creation, and nugget assignments?

In our experience, citation judgments can take between 30 seconds and 2 minutes per citation, nugget creation can take 1-15 minutes per nugget, and nugget assignments can take 30 seconds to 1 minute per nugget-generated paragraph pair.

RFQ (PDF) 38 Mock TO 1, Key Personnel All key personnel must have "experience in intelligence analysis" with IC agencies (DoD, NSA, CIA, DIA). Does this requirement apply to the Supervisor/Program Manager in their capacity as a key person on this task order, or only if they are also performing assessor duties?

This requirement only applies to the Program Manager in their capacity as key personnel. Individual assessors may have this experience but it is not required.

RFQ (PDF) 37 Mock TO 1, Place of PerformanceWork is off-site at the contractor's location, but the Risk Assessment (p.

38) states the contractor "shall have access to the DOC/NIST local area network." Will NIST provide VPN/remote access credentials, or is network access only relevant for on-site task orders?

The risk assessment text is wrong, and has been updated to read, "The Contractor shall have access to the web-based tool provided by NIST. NIST web-based tools are limited to specific users who are authenticated through login.gov".

RFQ (PDF) 43 Mock TO 2, Key Personnel The Mock TO 2 SOW defines a Program Manager requiring "at least 10 years of professional research or management experience including at least 4 years successful management or technical direction of large and complex projects." The IDIQ SOW's Project Manager requires only a bachelor's and 3 years. Is the Mock TO 2 PM a different/higher-level role than the IDIQ Project Manager?

Updated Mock TO 2 SOW to refer to the IDIQ labor categories for Program Manager, Project Manager, Research Associate, and Research Assistant.

RFQ (PDF) 41 Mock TO 2, Task 1 The sampling strategy must recruit 90 participants "diversely distributed from diverse demographic groups (according to NIST-provided criteria)."

Will the recruiting criteria be provided before or after award? Can NIST share the demographic distribution requirements so offerors can assess recruitment feasibility and cost?

Specific recruiting criteria are provided after award. Unless a specialized demographic is specified, NIST's general sampling strategy seeks a Census-representative sample in terms of basic demographics such as age, education, sex. For online studies like this one, sampling for variability in geographic location will be important since some states support mDLs (mobile drivers licenses) and some do not.

NIST does not need to recruit people who have mDLs, but does want to recruit people from states that issue mDLs (currently 20 states plus one territory, it can be found via https://www.tsa.gov/digital-id/participating-states)

RFQ (PDF) 42 Mock TO 2, Task 4 Participant compensation is $75 each (90 participants = $6,750). Should this be priced as an ODC line item or absorbed into labor? The Mock TO 2 pricing tab includes an "Other Direct Costs (ODCs)" row — does participant compensation belong there?

Participant compensation should be priced separately as an ODC line item and not absorbed into labor.

RFQ (PDF) 43 Mock TO 2, General RequirementsThe SOW states "Work under this requirement may be subject to Human Subjects Clearance" and that the NIST IRB will serve as IRB of record.

What is the anticipated timeline for NIST IRB approval? Is it obtained before or after task order award?

NIST RPO (Research Protections Office) approval typically takes between 2-4 weeks for exempt human subjects research, which is where most of our work falls. RPO approval is obtained after task award.

RFQ (PDF) 42 Mock TO 2, Task 5 Transcripts must maintain "98% accuracy." Is this word-level accuracy, or does it refer to a specific transcription accuracy metric (e.g., WER)?

98% accuracy requirement equates to a maximum 2% Word Error Rate (WER). We are concerned with functional accuracy, ensuring that the content, speaker identification, timestamps, and punctuation are correct. Non-lexical filler words such as "ah" and "um" do not count since strict verbatim was not requested.

RFQ (PDF) 41 Mock TO 2, Period of PerformanceThe POP is September 30, 2026 through September 29, 2027 (12 months). However, Task 2 cannot start until IRB/PRA approvals are received, and Task 3 runs for up to 2 months after recruitment begins. Is the Government willing to extend the POP if IRB/PRA delays consume the available timeline?

Yes, we are willing to extend the POP if needed. We have extended POPs in the past when government approval delays for IRB/PRA consumed too much of the timeline.

RFQ (PDF) 5 Task Order Procedures (f) "Only firm-fixed-price type task orders are anticipated under the IDIQ contract. Hybrids of these three contract types are also allowable." The text references "three contract types" but only describes FFP. What are the other two contract types NIST contemplates issuing?

This is an error. Only firm-fixed-price type task orders are anticipated under the IDIQ contract. The solicitation has been updated.

RFQ (PDF) 29 Phase 2, Business Volume IIIThe solicitation requires pricing to remain valid for 180 days (Volume III instruction) but Phase 1 quotes need only remain valid for 90 days (p. 26).

Mock Task Order pricing must remain valid for the first 12 months of the IDIQ (p. 31). Are these three separate validity periods intentional?

This reference has been removed. Pricing for the base IDIQ and mock task orders must remain valid for 180 days. The solicitation has been updated.

RFQ (PDF) 3, 5 Section (e) The solicitation states that quotation due dates may be as short as 48 hours after issuance of an RFQ "at OCO discretion depending on the urgency of the requirement." The solicitation further states that failure to submit a timely quotation or a no-quote rationale by the response deadline may result in contractor off-ramping from the IDIQ contract. a) Will the Government commit to a minimum response period for non-urgent task order RFQs, and if so, what is that minimum? b) For purposes of off-ramping determinations, will the Government distinguish between a contractor's failure to respond and a contractor's timely submission of a no-quote rationale? Specifically, will a contractor who consistently submits timely no-quote rationales but does not win or quote substantively be considered at risk for off-ramping? c) In the event a 48-hour turnaround is invoked, will the Government provide any advance notice or pre-solicitation communication to awardees prior to formal RFQ issuance, so that contractors can mobilize the appropriate personnel for proposal preparation? (d) Will the Government clarify what constitutes an acceptable "brief rationale" for a no-quote submission? Is there a minimum stated standard, or will any written explanation submitted before the deadlines be considered compliant?

a) 15 calendar days. b) This is not applicable. c) Yes. This would be a rare occurrence. d) Any written explanation submitted before the deadlines be considered compliant.

RFQ (PDF) 17 Section III Is the IC experience requirement for Assessor Level I limited to task orders with a text retrieval and review component as written, or is it intended to apply across all task order types?

It is limited to task orders with a text retrieval and review component. Additionally, the qualifications have been edited to include analytic experience outside the IC such as professional journalists.

RFQ (PDF) 17 Section III For task orders involving usability testing or human subjects research, may the Contractor propose assessment personnel without IC backgrounds, provided they are otherwise qualified for the task order scope?

The requirement for experience in analysis only applies to the Technology Evaluations Project Manager and Assessor labor categories.

General Is the primary need here more focused on hands-on technical execution, or assessment/advisory-type support around evaluation, documentation, and alignment?

For technology evaluations, the primary need is for hands-on technical execution following tasks and procedures specified by NIST staff. For usability testing the primary need is also for hands-on technical execution following tasks and procedures specified by NIST staff; NIST researchers design the test protocols and data collection instruments.