Fiddler_QA_20220420.pdf

HR001122S0016: Fiddler

Questions and Answers

Q0: DARPA received many questions regarding data, training data, image formats, and ATR function. Q0 attempts to answer them all in one place:

• Performers are strongly encouraged to seek sources of data to pretrain and test their methods as needed. While the Government has data to provide, the Government may not be able to deliver in the quantity and quality of data that you would like.

• This effort will work with complex SAR image chips of targets + metadata. Any metadata that was collected by the radar such as geometry, radar parameters, scene parameters, etc. will be provided if it is available. Any metadata on post-processing such as filtering or image formation method will also be provided if available. If target truth data or features is known, it may also be provided.

• If the proposed solution can make use of images in other modalities such as EO, IR, etc, please discuss in the proposal. Currently, images of these types will not be provided in the Government-provided datasets.

• If the proposed solution makes use of amplitude only SAR images, please discuss inthe proposal.

• It is likely that performers will still need to pre-process the images for their methods as needed (for example: centering, background removal etc).

• The amount of training data that will be required is unknown. The metrics in the BAA are fixed and one of the things we expect to learn during execution is how much training data a given method needs to meet them.

• We have intentionally not defined ‘classes’ or features that an ATR would use downstream or for evaluation.

Q1. What is the estimated budget that DARPA has for this opportunity?

A1. There is no set budget per performer. As a software and algorithm effort, it is envisioned that a team size of 3-6 full time personnel plus management and security support may be sufficient, depending on technical approach, if that helps guide your response. Any tasking not directly tied to meeting the BAA metrics should be provided and priced as an option.

Q2. How many awardees does DARPA anticipate?

A2. DARPA anticipates 3 awards in Phase 1. The number of awards will depend on available funding and the quality of the proposals received.

Q3. Should the draft Science and Technology Protection Implementation Plan (S&TPIP), Small

Business Subcontracting Plan and Intellectual Property assertions mentioned in section IV.B.3 of the BAA be contained in separate volumes?

A3. (a) The draft Science and Technology Protection Implementation Plan (S&TPIP) should be included as a separate volume.

(b) The Small Business Subcontracting Plan, if required, should be included as a separate volume.

(c) The Intellectual Property assertion should be included in the cost volume.

Q3. The BAA stated that the Government will provide SAR imagery of maritime objects at the start of Phase 1 and an additional set of SAR imagery of maritime objects at the start of Phase 2.

Does DARPA intend to provide imagery of the same objects at other sensing modalities such as electro optical (EO) or Infrared (IR)?

A3. See Q0.

Q4. Does DARPA anticipate a specific range of collection parameters (e.g. polarization, resolution, frequency)?

A4. Proposers should assume the range of collection parameters that commercial SAR systems are capable of today.

Q5. Does DARPA anticipate that the Phase 1 mission scenario will be focused only on open ocean maritime, or is a general capability for open ocean and littoral together expected?

A5. DARPA has not confined the location or motion state of targets.

Q6. Does DARPA have any interest in digital engineering or model-based approaches and methods?

A6. The proposal should reflect whatever the performer feels is the best approach for addressing the goals and metrics of the BAA.

Q7. Would DARPA deem it acceptable for performers to conduct unclassified work in unclassified facilities, and transition into a classified environment as needed?

A7. Yes. The program was designed to make this possible. The performer should describe their approach for performing unclassified work in unclassified facilities in the Science and Technology

Protection Implementation Plan (S&TPIP). A draft of that plan is requested as part of the proposal package. If a contract is awarded, the performer will have to submit an updated S&TPIP and

DARPA has to approve that plan. Once the plan is approved the performer may conduct unclassified work in unclassified facilities in accordance with that plan. Please consult the

Security Classification Guide.

Q7. Will DARPA sponsor contractors for AFRL Xpatch licenses?

A7. Yes. The proposal should identify any actions requested of the government such as sponsoring the AFRL Xpatch license, and explain why or how the action (obtaining an Xpatch license in this case) relates to this effort.

Q8. Does DARPA expect performers to conduct development on corporate resources or will

DARPA resources such as processing hardware be supplied?

A8. The proposal should identify what corporate resources will be employed in this effort. The cost proposal should include the cost of any processing equipment required for purchase on the contract to support this effort.

Q9. Do tables in the pdf cost volume have to be editable?

A9. The intent of the instructions is to have the Excel spreadsheet, with formulas intact, attached to the cost volume. Any tables contained within the pdf cost volume are not required to be editable.

Q10. Will the government provide any data independent of the sparse set of SAR imagery for an object to support the offline training? For example, if the goal is to use sparse data for creating new images of target X, then will there be data available for offline training on an independent set of targets (A, B, and C)?

A10. See Q0.

Q12. No total page limit for the Technical and Management Volume is stated, but based on the recommended page requirements for each section it appears that Section II (Summary of

Proposal) should be 10 pages max and Section III (Detailed Proposal Information) should be 28 pages max. Was that the intent?

A12. Yes.

Q13. The Technical and Management Volume, Section III.I of the BAA asks the proposer to provide a description of milestone cost and accomplishments. Can you clarify what exactly is being asked for? Proposers may not necessarily propose FFP with milestone payments. Are the accomplishments described meant to be a description of anticipated future accomplishments on the program, which relate somewhat to Section III.J – cost schedules and measurable milestones?

A13. The description of milestone cost and accomplishments (Section III(I)) should be a succinct listing of milestones, associated costs, and associated accomplishments or technical events for each milestone. The cost schedules and measurable milestones portion (Section III(J)) should expound, in detail, on each of the previously listed milestones.

Q14. The BAA for Fiddler states, “The Government requires that proposers use the provided MS

Excel DARPA Standard Cost Proposal Spreadsheet in the development of their cost proposals” and “All proprietary subawardee proposal documentation, prepared at the same level of detail as that required of the awardee’s proposal and which cannot be uploaded with the proposed awardee’s proposal, shall be provided to the Government either by the awardee or by the subawardee organization when the proposal is submitted.” on page 27. If a subawardee proposal value is under $2M, are they expected to submit documentation prepared at the same level as the awardee to include the DARPA Standard Cost Proposal Spreadsheet?

A14. Yes.

Q15. On Page 27, the RFP references FAR 15.403-4, certified cost or pricing data required.

However, the BAA is soliciting a price competition. Wouldn't 15.403-1 (c) Standards for exceptions from certified cost or pricing data requirements - (1) Adequate price competition apply?

A15. No, adequate price competition does not apply as the BAA is not soliciting a price competition, and proposals are expected to present varying technical approaches and are evaluated solely on their own merits against the evaluation criteria listed in the BAA. Proposals are not ranked or compared against other proposals, as there is no common statement of work.

Q16. Are there target RF parameters for phase 1 (bandwidth, center frequency, etc.)? Similarly, are there proposed constraints on RF parameters for phase 2 generated images (example maximum/minimum bandwidth)?

A16. See Q4

Q17. Does the Government have a preference on where in the proposal to include our completed

Section K Representations and Certifications?

A17. Hard copy Representations and Certifications may be attached as an appendix to the Cost

Volume.

Q18. Regarding Intellectual Property, should Offerors seeking a Procurement Contract use the table on page 34 of the BAA, taken from DFARS 252.227-7017, to make any assertions? Does the

Government want this table inserted in the section of the proposal described in paragraph

IV.B.2.a.2.C (bottom of page 23) of the BAA? Should the table be inserted into the narrative proposal or should it be an appendix to the proposal?

A18. Offerors proposing a Procurement Contract should use the table format shown in the

Intellectual Property section of the BAA to make any assertions. As stated in the BAA, the table/form does not count against any page limit, and should be an appendix to the Technical

Volume. Narrative discussing intellectual property is still required in parts of the technical proposal, and do count against any page limits. Please see the requirements for Volume I:

Technical and Management Proposal.

Q19. Does the Government have a preference on where in the proposal to include the draft

Science and Technology Protection Implementation Plan (S&TPIP)?

A19. Science and Technology Protection Implementation Plan (S&TPIP) is a separate addendum, within Volume I.

Q20. May the Cost Volume be submitted via unclassified means as indicated on page 35 via Broad

Agency Announcement site (https://baa.darpa.mil) and the Technical Volume be submitted separately via classified means due to its classification? And if so, may they be submitted on different dates (both before the due date)?

A20. Yes, the Cost and Technical volumes may be submitted separately, unclassified and classified, on different days as long as they are submitted prior to the due date and time.

Remember to encrypt, or password protect, your unclassified proposal and mark with CUI markings.

Q21. Within the solicitation it states that the targets of interest with sparse observations will already be detected within the scene, before the synthetic data generation component, and their target locations in the scene will be passed to the algorithm. Stated another way - we understand that our "sparse training samples" that drive synthetic generation will be chipped out of the scene, and that our algorithm is not responsible for detecting and localizing those targets. Is this understanding correct?

A21. Yes. See Q0.

Q22. Regarding the government provided training data and test data, what is its form and content (e.g., SAR images only, or SAR images + metadata, etc.)? Approximately how many samples will be included, and how many different target classes?

A1: See Q0.

Q23. What metadata will be available for the training imagery that will be provided by the

Government (i.e., for each of the Government provided training images, will we know the azimuth and elevation in Phase 1 and the center frequency, range/cross-range resolutions, and polarization in Phase 2)?

A: Yes, see Q0.

Q24. How much training data will be provided by the Government? Will we only receive data of the three test targets of interest or will we be provided with additional data of other targets/objects? Will the provided data be complex?

A: See Q0.

Q25. Should we assume that the training images provided as input may be collected on an object against different background and motion states?

A: Yes, See Q0 and Q4.

Q26. Are we allowed to pretrain our models on additional SAR data that we have available?

A: Yes and it is encouraged. See Q0.

Q27. According to Figures 5 and 6 in the BAA, motion and background “pentagons” seem to be added after the Fiddler module is run. Is it assumed that the output synthetic imagery from the

Fiddler module do not include motion and background? Are the motion and background blocks provided by the government or developed by the performer and required as part of the deliverable? If provided by the performer, may they be integrated as part of the Fiddler SW module or must they be kept separate?

A: Generating motion effects on the output of the Fiddler SW module is out of scope. Training on images that have motion effects (the input of the SW module) is expected and one of the major challenges. The government has SW modules to add realistic on-water motion effects to images. Those will most likely be provided to the performers.

Q28. Should we assume that the maritime objects being imaged are undergoing motion that is limited so as to make SAR image formation more appropriate to use compared to ISAR image formation processing? Will input images always be SAR image products and not ISAR products?

A: Images will be SAR images. No assumptions are made about the target motion. Performers are free to propose methods such as ISAR processing to further process images if appropriate.

Q29. Is there a page count limit for the full proposal? Which sections are included in the page count limit and which are not?

A29. Each volume has different instructions with regard to the page limits. Please refer to the

BAA.

Q31. Page 9 of the BAA refers to data being “post-detection.” Page 7 refers to SICD imagery.

Will the government provided data be complex chips in SICD format? If so, what meta data fields will be populated within that format?

A: See Q0.

Q32. The Fiddler SCG talks about synthetic data of particular targets being sensitive. Is this only for synthetic data generated via the tools developed under Fiddler?

A32. Yes.

Q33. The BAA states that “proposers are required to submit a draft Science and Technology Protection Implementation Plan (S&TPIP) as part of their security package. Refer to the Fiddler Science & Technology Protection Plan (S&TP2) (part of Classified Addendum) for initial analysis and estimation of critical program protection risks identified for this program.” In the referenced S&TP2, Section 8.8 states that “A S&TPIP will be required from each performer two

(2) months after contract award.” Please confirm that a draft S&TPIP is required to be submitted as part of the proposal package.

A33. Yes, a draft is due with the proposal, and the final submission will be due two months after contract award to incorporate changes required by the Program Security Officer (PSO).

Q34. Will there be an opportunity to ask additional questions once the abstract feedback is submitted?

A34. Yes, but there is no guarantee the questions will be answered. Specific questions on your technical approach will not be answered.

Q35. Will the BAA team consider an extension of the due date so that proposers can incorporate abstract responses into their proposals?

A35. No.

Q36. Is the software source code required to be delivered for the offline training module and the online SAR image generation module or would an executable/container be sufficient?

A36. An executable/container is what is planned for program integration. Source code is also expected as a deliverable. If this is not possible, please identify why in your IP claims section.

Q37. Does DARPA anticipate that all SAR chips will have associated object class labels?

A37. See Q0.

Q38. Does DARPA have a target hardware resource requirement for Fiddler evaluation (e.g.

number of CPU cores, memory, etc.)?

A38. No, address the expected computing needs to meet the program metrics.

Q39. The link to the Terms and Conditions in the BAA contains a list of FAR and DFARS clauses that “may apply if this opportunity results in one or more FAR-based procurement contracts, depending on the circumstances and rules for applicability.” Can the Government provide a listing of terms and conditions that will apply to a FAR based contract if awarded?

A39. A draft contract with all terms and conditions will be provided to the performer for review prior to award execution.

Q40. Can the Government confirm that our completed Section K Representations and Certifications should be included at the end of Volume II of our proposal?

A40. Hard copy Section K Representations and Certifications may be included as an appendix to

Volume II of the proposal.

Q41. Regarding the proposed performance metrics for image generation, is the amount of time allocated to image generation the average time per frame, or do those timeframes need to be met per frame? Are these timeframes average latency or latency benchmarks?

A41. For the proposal, you can assume it is the maximum time required as opposed to average time. Ultimately a latency metric of the SW module measured from an outside request to

“learn/train on a sparse set of inputs” or to “generate a new image” until the SW module says it is finished training or it passes back an image. The GOV+performer team will work together on defining the exact test points and care will be taken to remove overhead latency on either side.

Q42. Will performers be provided any information regarding the timescale of images to be generated?

A42. Yes. The expectation is that generated images will have typical SAR image integration times (but it may differ from the training data).

A42. See Q4.

Q43. ATRs may be used to detect and classify objects, classify objects, or to classify features of objects (e.g. direction or speed). Will the government ATRs be focused on object detection and classification, object classification, or something else?

A43. See last bullet of Q0. This is undefined to generalize the image generation method.

Q44. In training and test data received, will the target object be centered in the frame, or will the position of the primary target object vary (e.g. depending on the phase of the program)?

A44. See Q0.

Q45. The BAA says “The Government will provide SAR imagery of maritime objects at the start of Phase 1 and an additional set of SAR imagery of maritime objects at the start of Phase 2.” a.

How much data does the government anticipate delivering?

A45. See Q0.

Q46. The BAA also says that “All complex SAR imagery used or created by this software will be in Sensor Independent Complex Data (SICD) or a similar common format.” Is this the only format in which data will be provided, or will unprocessed versions corresponding to the SICD images be available?

A46. Currently, only images are intended to be provided. If you require or would like additional data, please specify in your proposal.

Q47. Is it possible to know the processing algorithms that were used to form the complex images which are being provided (i.e. time-domain or frequency-domain methods)?

A47. See Q0.

Q48. Has any autofocus or out-of-plane compensation algorithms been applied to the government-provided imagery , and can the details of those algorithms be provided to the Fiddler performers?

A48. See Q0.

Q49. Will the images provided to the Performer Software Module (BAA, Fig. 3) always be of the same object instance, captured over a short period of time to preclude changes to the object?

A49. The assumption is the object does not change but this is a practical issue that the Government acknowledges will likely come up during execution. Solutions will be explored at that time.

Q50. What ATR scenarios will be performed? How many classes will there be?

A50. See last bullet of Q0.

A51. If images are given of multiple object instances, will there be metadata indicating which images come from the same object instance?

A51. See Q0.

Q52. In Phase 1, the BAA states “In this phase the imagery used in testing will have been collected with the same RF characteristics as the sparse training data, where RF characteristics include center frequency, polarization, bandwidth, and cross-range resolution.” Can more details be given about this, specifically what frequencies and bandwidths were used for generating this initial set of data?

A52. See Q0 and Q4.

Q53. Table 7 in the BAA specifies timing metrics, but does not specify the compute resources that would be available. Does the Government intend to specify this across all performers, or are performers expected to propose a reference system against which these metrics would be measured? For approaches leveraging machine learning, the availability of Graphics Processing Units (GPUs) will have a large impact on timing.

A53. See Q38.

Q54. Figure 3 in the BAA illustrates the Performer Software Module as producing a single synthetic image as output in response to an image request message. Is it expected that there will be a 1-to-1 relationship between the input message and output images? Or will it be allowable to generate multiple outputs corresponding to unknowns (e.g., sea state or receiver noise)?

A54. Yes, 1-to-1. As a starting point for development, the sea state and noise would be an additive effect applied after image generation.

Q55. Will the motion and background variation (BAA, Figure 5) augmentations be provided within the test harness?

A55. Yes.

Q56. Will the test harness functionality extend to ATR training, or will any performer-generated ATR algorithms be subject to other restrictions on how they are trained?

A56. Performer ATRs provide additional test population for the program. The government will work with performers to use or train them within their designed limits. Performer ATRs will not be used for “official” evaluation.

Q57. During the proposer’s day, it was mentioned that motion artifacts in the input imagery might increase throughout the program. What is the intention for the imagery generated by performers? Should we generate imagery without motion artifacts regardless of the inputs, or should we generate outputs with motion artifacts that match the inputs?

A57. See Q54.