VideoLINCS BAA questions Round 1_11 JUL 24_C (1).pdf

VideoLINCS Q&A

Round 1, IARPA BAA 24-01

1. We would like to know if this is a brand new contract OR if there is (was) an incumbent performing these services.

Answer: This is a new Broad Agency Announcement contract opportunity. There are no incumbents currently performing this research. VideoLINCS is an entirely new program.

2. If IP has been developed in advance to the program (for non IARPA funds) can it be utilized in the proposal with commercial rights while additional things developed as part of the program be Government purpose rights?

Answer: Reference BAA section 4.B.1.d. For Data Developed with Mixed Funding, "the Government seeks at minimum “Government Purpose Rights” (GPR) for all noncommercial data and software deliverables; offering anything less shall be considered a weakness in the proposal"

3. Can GSI Technology be considered for the T & E servers?

Answer: This announcement is solely for the VideoLINCS program and T&E partnership including equipment is at the discretion of the selected T&E partners.

4. One of the requirements is the development of a system. Are there specific base software or open APIs that we must adhere to in order to ensure compatibility with existing systems?

Answer: An API will be provided by the T&E team after program kickoff. As stated in the BAA in Section 1.G.3.1, "The program will also require the use of a common processing framework, such as the Open Media Processing Framework [3] for development. The first version of the Video LINCS API and nominal framework will be provided to Performers at the beginning of the program and updated periodically thereafter".

5. Regarding data collection, if I cannot get IRB approval from my school, would it still be feasible for us to participate in the project?

Answer: IARPA does not offer IRB services. It is up to offerors to determine if and what type of data is necessary for their solution. As stated in Section 1.A of the BAA, "All data collections with potential Human Subject Research (HSR) will require Institutional Review Board (IRB) approval".

6. Will the collected data be made available to the public, and if so, will non-participants have access to it?

Answer: At minimum, any data used for training shall be made available to all selected VIDEOLINCS performers and T&E teams. Any further disclosure will be made based on the Government’s negotiated rights with each Performer. Reference Section 1.C of the BAA.

7. Regarding the monthly progress reporting, will it resemble the format of the ImageNet Competition, featuring a leaderboard showcasing the performance of each participating team?

Answer: The intent is to provide visibility of participating team performance across the program.

The format will be instantiated closer to program kickoff. Reference BAA Section 1.A.2 Program Phases.

8. Could I invite new team members or join other teams after the proposal submission?

Answer: Changes to a proposed team made after the selection decision but before contract award, could impact whether the prime offeror will receive an award. The reason being that the original team, including the team's expertise and each team member's contribution to the proposed project, may have influenced the government's final selection decision. Therefore, any significant changes made after selection but before award, will need to be assessed by the contracting officer.

After contract award, any addition of or revision to team members (i.e., subcontractors) including major redistribution of work will require notification of the contracting officer and possible contract modification. See FAR 35.009:

9. Could you provide an estimate of the budget allocation for each proposal?

Answer: IARPA is not releasing overall program information for this program.

10. Does IARPA have any guidance on budgetary considerations or expectations for each of the Phases of the project?

Answer: IARPA is not releasing overall program information for this program.

11. The solicitation outlines the expected computing power to be used during testing (Footnote 2 on page 20). Is that only for Phase 1? Or will that be the computing power for all phases. We are proposing that portions of the computing are edge enabled for certain devices (not using GPU) that already have onboard computing to speed analysis (but not required). Is that allowable?

Answer: The expected computing power, for which general guidelines are provided in the BAA (footnote 2 on page 19) and will be finalized around program kickoff, is expected for all phases.

Some modification of specifications is possible as the program progresses. Edge processing is not an objective of the program and will not be part of the evaluations. IARPA is interested in innovative solutions, and an edge capable solution that meets requirements and is not reliant on such hardware is allowed.

12. For all phases, is there any guidance on the approximate length of the videos that will be used for the testing phase? Will the algorithms be provided 1-minute videos, 1 hour, 24 hours? It will be very helpful for our proposal on how we will obtain R&D test data for our own use and to share with other teams Answer: The length of the test videos will be determined during the program. Individual test videos may vary in length from one another. A rough order of magnitude estimate for the length of the aggregated test dataset is expected to be on the order of hundreds of hours.

13. For Geolocation, is there any guidance on how far apart the test data might be? All within the same general city? Or possibly at very distant locations (different states or countries).

Answer: There is no limitation on proximity of the test dataset (different states or countries are within scope), but the general vicinity of the sensor from which a given video was obtained would often be known at test time (as delineated throughout Section 1.A.2 in the BAA).

14. The BAA specifies at least two locations in the proposal that should contain an RDMP.

According to 4.B.1.c, Section 3L of the proposal should discuss the RDMP. According to 4.B.1.d, Attachment 11 of the proposal should discuss the RDMP. What should be included in each to avoid duplication?

Answer: Section 4.B.1.c, Section 3: Detailed Proposal Information, paragraph L. defines what is a Research Data Management Plan, and what should be addressed in your RDMP. Appendix A.6:

Research Data Management Plan (RDMP) BAA 24-04 on page 60 is the template that should be submitted as an attachment within your proposal.

15. The BAA specifies at least three locations in the proposal where deliverables should be discussed.

According to 4.B.1.b, Section 2B of the proposal should discuss deliverables. According to 4.B.1.c, Section 3A of the proposal should discuss deliverables as per Appendix A.7.6. According to 4.B.1.c, Section 3E of the proposal should discuss deliverables What should be included in each to avoid duplication?

Answer: The BAA outlines the expectations for deliverables in each of these sections, please reference the BAA for guidance. To prevent duplicative information, Appendix A.7 has been marked “Reserved”, and removed.

16. It is our intention to release any and all technical contributions to the public not only through publications and dataset releases, but also through open-source code. Is this permitted?

Answer: See Section 6.B.5 of the BAA that provides guidance on publication approval, and Section 6.B.12 Public Access to Results for what is permitted public releases.

17. The BAA states, "but pose measurements might not be synchronized with the video frames for cameras with variable pose (e.g. panning and moving cameras) and will contain uncertainty." In terms of the provided pose, does this mean we are given just the first frame pose? Or perhaps a list of poses, but just not synced with the corresponding frame?

Answer: Sensor pose measurements that are updated as time progresses will be provided, however they will not necessarily be synchronized with or at the same sampling rate as the video frames.

18. For orientation, while we do not see it specifically defined in the BAA, we assume they are referring to roll, pitch, and yaw of the camera, given in euler angles.

Answer: The format in which orientation will be provided will be specified at program kickoff.

19. What is the rough size of the media from T&E team, both in terms of total storage we will need to handle, and in terms of how much media is associated with a particular evaluation sample (how many cameras, how long is each recording)? We would need this information in order to make a reasonable estimate of storage costs and of compute costs for algorithm training.

Answer: Media from the T&E team will include 1) a sequestered dataset that will be processed exclusively on T&E servers and 2) a self-test dataset that will be processed by performers on their own hardware. A rough order of magnitude estimate for the length of the sequestered test dataset is hundreds of hours. The length of the self-test dataset will be determined during the program.

20. Is there a live template for the Cost Proposal for a sub-contract?

Answer: Appendix B.2 Contractor/ Subcontractor Cost Element Sheet for Volume 2 Cost proposal is the example, should a Cost Proposal be requested after selection. Please Reference Section 4.B.1.b for details regarding subcontractors when submitting Volume 1: Technical and Management Proposal, and Section 4.B.2.c Section 3: Supporting Information Subcontracts if a Cost Proposal is requested. Should a cost proposal be requested, a template will be provided.

21. Where can I find the FAQs for this solicitation please?

Answer: There are no FAQ's provided for this solicitation phase since every Broad Agency Announcements is different in specifications and scope.

22. The BAA says: “In the event that such reference data is made available at test time, it will be provided to all performer teams.” How do performers know what is available at test time? Will this be standardized by the T&E team across all performers?

Answer: Reference data that is made available at test time will be identified by the T&E team and specifications for the reference data will be made available to all performer teams.

23. If we get lat, long, can we use an external datasets to get altitude for the trajectory?

Answer: Section 1.C of the BAA states: "Delivered solutions must include all resources and execute autonomously, not relying on internet access during execution. There may also be the ability to accumulate external data for system reference. If a solution would require external data sources, proposals should describe the necessary data and what would be required to obtain such data for offline use."

24. Can we assume a high speed internet connection will be available upon deployment?

Answer: No. The BAA in Section 1.C states: "Delivered solutions must include all resources and execute autonomously, not relying on internet access during execution."

25. Could the Government please confirm that the 5 pages for Section 2 are outside of the 30 pages allocated to Section 3?

Answer: Section 2 is included in the 30 page limit. Please reference section 4.B.1 that outlines the documents included in the Technical Management Proposal, which are at the top of pg. 29.

26. Would the Government please confirm that the SOW is outside of the 30-page limit set for Section 3 and should be included as an Appendix?

Answer: The Technical and Management Proposal is limited to 30 pages, and the SOW is within the 30 page limit. Only relevant papers will not count towards the 30 page technical volume and is only limited to three. Reference 4.B.1. The statement of work template has been removed and marked as “Reserved”, and follow BAA section 4.B.1.c detailing what is required for the SOW.

27. Could the Government please confirm that a Security Plan is not required with this BAA submission?

Answer: Yes, a security plan is not applicable and not required for this BAA. Reference 4.B, and 4.B.1.d

28. Could the Government please provide an estimated timeline between the BAA submission and the notification of selection for negotiation?

Answer: At this time, the government cannot provide a firm timeline of the notification and selection of potential offerors. Program timelines can change, but we estimate this notification can occur sometime between the 1st or 2nd quarter of Fiscal Year 25.

29. When will Video LINCS be available on iarpa-ideas.gov?

Answer: Video LINCS is now available on IARPA IDEAS as of 26 JUN.

30. "The proposal reflects a mature and quantitative understanding of the program milestones and metrics, and the statistical confidence with which they may be measured." Would you please clarify what the last part of this sentence means? For example, would the metric be F1 and the statistical confidence be F1 values that align with our milestones (i.e., the statistical milestone confidence would increase over time).

Answer: This refers to demonstration of a clear understanding of all program metrics, milestones and components of performance variability.

31. Indoor cameras: Will geospatial coordinate references to room features or indoor features, such as corners/doorways, be available?

Answer: Geospatial coordinate reference will be provided relative to the reference data provided, but it need not be connected to a specific feature. E.g., if a floor plan is provided, coordinates of a specific point in the floorplan could be provided.

32. Georeferencing of the video data: Will we be provided a map or will we be expected to provide our own map? Will we be provided boundaries or regions when developing the algorithm?

Answer: There is no need to georeference the video, just the objects (people, vehicles and generic objects) in the video. Geo-localization will only be required for sensors that contain some level of location information, as stated in Section 1.C, “When no sensor location is available, objects in the video will only be evaluated for reID.” Reference data may be made available at test time to the systems (see Section 1.A.2, “Varying levels of reference data may be made available to systems at test time, especially for later phases when less camera information is provided.”). No boundaries or regions will be provided to performers during algorithm development.

33. Would you define "camera pose" in terms of measurement (i.e., height, angle, zoom)?

Answer: Pose refers to position and orientation. The exact format will be provided by the T&E team at program kickoff.

34. What are the anticipated minimum/maximum length for input video (one minute, one hour, etc.)?

Answer: There are no specific bounds. Videos could be of any duration.

35. Mobile cameras: What is the anticipated maximum area of coverage for those cameras? At what speeds?

Answer: There is no bound on the area of coverage for moving cameras. There are also no specific bounds on speeds.

36. The Video LINCS BAA has not yet been added to the IARPA Distribution & Evaluation System (IDEAS), so proposers are unable to register a new account. Please advise when the Government will load this BAA into the IDEAS dropdown menu.

Answer: The Video LINCS BAA has been added to IDEAS as of 26 JUN.

37. Are there any specific examples on the anticipated frame rate (min, max)?

Answer: Frame-rates may vary throughout the collection, with most being <= 30 frames per second

38. Please confirm that 'a plan for review by an Institutional Review Board (IRB) with the final proposal submission' will meet the needs of the management plan (Reference Section 4 of the

BAA).

Answer: Per BAA 6.B.3 Human Use regarding IRB approval statement " For all proposed research that will involve human subjects, the institution must provide evidence of or a plan for review by an Institutional Review Board (IRB) with the final proposal submission to IARPA. A revision to the BAA will be posted on SAM.gov and IARPA IDEAS.

39. In BAA it is said that in Phase 1, ReID will be performed across videos within the same collection type. Will it still be performed across video feeds of different cameras?

Answer: Yes, reID will be performed across video from different cameras during Phase 1.

40. On page 10 of the BAA for TA2- Object Geo-Localization it mentions that for object geo-localization in Phase 1 and 2, full sensor pose will be available but might not be synchronized with the video frames in case of a moving or panning camera. Please clarify what does synchronization mean? Is the pose available or not? Is a range of poses available?

Answer: 1) The BAA, in Section 1.A.2.b states that for Phase 2, “Camera location will often be available to performer systems during test runs but orientation may not be available”. 2) Synchronization: For moving sensors, where pose changes with time, any pose provided might be at a different sample rate than the video frames and might not be temporally aligned with the frames.

41. Since the BAA discusses software in the solicitation, does IARPA intent to add FAR 52.227-14 Alt III? Similarly, will FAR 52.227-13 Alt II be added for Technical Data deliverables?

Answer: Please reference 6.B.13.b.Clauses statement "This is not a complete list of clauses to be included, nor does it contain specific wording. A full list of applicable FAR clauses, to include full text IARPA Clauses, will be made available for review after selection and prior to contract award." IARPA will review each proposal and add the appropriate clauses as applicable.

42. The BAA requires the proposal to include Data Rights assertion in accordance with FAR 52.227- 14 under Section 4 attachments. Simultaneously, Data Rights assertion is also requested as a part of the SOW in accordance with DFARS 252.227-7013. Please clarify the applicable regulation.

Answer: Only FAR and ICM clauses apply. Appendix A.7 (SOW) template had been removed and marked reserved. The instructions for IP are in 4.B.1.d , description for Attachment 2.

43. Will the government be requiring shared data in a specific format? If so, what format?

Answer: No specific format is required.

44. Does the government have a recommended Privacy Protection Plan (PPP)?

Answer: No

45. We attempted to register in the IARPA Distribution and Evaluation System (IDEAS), so that we can begin preparing our proposal, but the BAA is not available for selection. According to the IDEAS HelpDesk, "IARPA currently does not have any published BAAs on IDEAS. Once at least one is published, it will appear in the BAA dropdown for registration. The timeline for publishing BAAs on IDEAS is determined by IARPA." Could you please advise when the BAA will be published on IDEAS?

Answer: The Video LINCS BAA has been added to IDEAS as of 26 JUN. It has been published.

46. Can you please confirm that multiple files can be uploaded to the IARPA Distribution and Evaluation System (IDEAS)? Can all required proposal attachments be uploaded separately, and should they be uploaded separately?

Answer: Yes, multiple files can be uploaded to the IARPA Distribution and Evaluation System (IDEAS). The proposal attachments should be uploaded in accordance with the prescribed outlined format for Volume 1: Technical and Management proposals as stated in in 4.B. Proposal Format and Content, and also 4.C.2. Proposal Delivery.

47. Page 39 of the BAA in the description of Attachment 9: Three Chart Summary, indicates that it is a "PowerPoint summary." Should this be uploaded to IDEAS as ppx document, or a pdf? Can it be uploaded separately, or does the system only provide for a single upload?

Answer: The Three Chart Summary should be uploaded as a PowerPoint per the BAA instructions for Attachment 9. Now that the BAA is available in IDEAS, there is a function to upload the attachment as a PowerPoint and or PDF please follow the site instructions.

48. How rare do we expect aerial/drone video will be in Video LINCS program datasets? If we expect it to be available most or all of the time, this may influence both our technical approach and how much aerial data collection is proposed.

Answer: There should be no expectation of rarity, frequency or spatial overlap for any type of video in program datasets. As stated in Section 1.D of the BAA, "Nominal video types to be included in evaluations include stationary and mobile sensors that are air, building, and ground based (e.g. drone/UAV, security/CCTV, ground mobile)".

49. For the generic object reID task, the BAA specifies matching of foreground objects that move (in Phase II) or are stationary for the entirety of the video clips (in Phase III). For the stationary case, there is considerable ambiguity in what qualifies as a foreground object and therefore what should be autonomously detected and matched. Will there be some manual annotation of the type of object to be detected in this case, or is it expected that every semantically meaningful object in the scene will be detected and reID'd across views?

Answer: There will not be any manual annotation of objects to be re-identified. All foreground objects would be in scope for re-identification. The final determination of foreground vs.

background will be made during the program. It is likely that elements in the scene which are not permanent (e.g. vehicles, people, bicycles) are foreground, while those which are permanent (ground, buildings, trees, lamp-posts) are background.

50. What can be said about the envisioned spatial limits of typical program datasets that enclose all cameras and reID objects? Whether this is 200m x 200m or 25 sq km will have a big impact on the types of algorithmic solutions and reference data we propose to use.

Answer: There is no spatial limit for the program datasets. As stated in Section 1.A.2 of the BAA, "Varying levels of reference data may be made available to systems at test time..." which would correspond to areas in the test dataset.

51. Is the MEVA dataset (https://mevadata.org/) representative of typical collection density, sensor coverage, scene size, and metadata quality expected under Phase I of the Video LINCS program?

Answer: There should be no expectation that any specific dataset (such as the MEVA dataset) would be representative of collection parameters or quality. As the BAA states in Section 1.A.1, "The objective is for Video LINCS to be effective in the open-world setting where there is no advance knowledge of the sensor types, sensor collection geometries, scenes, objects to be re-identified, etc."

52. The BAA mentions 750 distinct truthed people (increasing in Phases II-III) to be measured for reID performance. What can be said of the timeframe over which this number of people will appear in the video sources (e.g. 750 at once or over the span of an hour+) Answer: There should be no expectation on the time-frame over which the people would appear in in the dataset. As the BAA states in Section 1.A.1, "The objective is for Video LINCS to be effective in the open-world setting where there is no advance knowledge of the sensor types, sensor collection geometries, scenes, objects to be re-identified, etc."

53. Geolocation in indoor scenes without sensor metadata (in Phases II-III) is a challenging problem for which the BAA mentions there may be reference data available such as floorplans. To clarify, can we expect that floorplans will always be available for indoor scenes and with some level of metadata to enable alignment with a geodetic coordinate system?

Answer: For sensors with location information, some level of metadata and/or reference data to facilitate alignment will be available. As stated in Section 1.A.2 of the BAA, "For indoor scenes, such reference data may include floorplans that have accurate proportions but not necessarily absolute scale, and potentially snapshots or short video clips of some areas included in the floorplan. Offeror's may propose solutions that rely on other reference data, provided such data is readily available and/or may be easily obtained for an arbitrary location." If no sensor location is available, only reID (TA-1) will be necessary and not geo-localization (TA-2).

54. Can you provide more information on the F1 metric used in section 1.F. Given that there is no a priori gallery, the proposed algorithm will presumably detect subjects and build a gallery based on subject ID. Each element in this constructed gallery should be mapped to an actual subject in the ground-truth gallery to compute the F1 score. How will this mapping be performed?

Answer: The mapping will be determined by the T&E team and will be specified in the program T&E plan at program kickoff.

55. Will there be more information on temporally and spatially distanced conditions. For instance, will we be required to perform ReID across Video separated by 1 km and 10 minutes? Video separated by 1000 km and 1 day? All of the above?

Answer: There should be no expectation on temporal and spatially distanced conditions other than "temporally proximate" conditions which would have videos taken primarily on the same day.

56. The BAA notes that there might be missing instances as well as errors in sensor pose and image timestamps. Will both pose and timestamp be missing? Is there any information on the error bound in the timestamps?

Answer: It is possible that both pose and timestamps will be missing. When errors are present in timestamps, there should be no expectation of information on an error bound.

57. Generic objects: Are these objects come from a know domain such as urban environment or these can be arbitrarily and unknown objects.

Answer: They may be arbitrary, unknown objects, as stated in the BAA in Section 1.A.2.B, "For generic objects, systems will need to autonomously detect and reID foreground objects throughout the collection, without any prior information regarding the types or characteristics of objects to be re-identified."

58. External data sources: The BAA mentions about an experimental setup without any training data.

Can we use open-source datasets and/or simulated data for training Answer: As stated in Section 1.C of the BAA, "Performers are permitted to use data from external sources provided that the data is consistent with an approved privacy plan, may be shared across all performer and T&E teams, and is not proprietary. " In a similar vein, Section 1.D.1 states "All data collected, curated, annotated and/or simulated by performer teams under Video LINCS funding, as well as any data used by performer teams for Video LINCS R&D must be shared program wide with other performer and T&E teams."

59. In the open-world and hence open-gallery re-ID, are expected to re-ID all the objects? Or there will be instructions of re-IDing specific instances of objects such as ‘blue cars’ or ‘persons with a backpack’?

Answer: All of the objects, as stated in the BAA in Section 1.A.2.B, "For generic objects, systems will need to autonomously detect and reID foreground objects throughout the collection, without any prior information regarding the types or characteristics of objects to be re-identified."

60. From Figure 1 of BAA page 6, it seems that IARPA is interested in at least the following types:

Wall/rooftop mounted fixed, Wall/rooftop mounted PTZ, Overhead mounted on drone, Web cam mounted on mobiles. For geolocating algorithm development purpose, are the above four types of cameras the camera types we have to propose algorithms for, or are there are other types?

Answer: The figure is for illustrative purposes only. As stated in Section 1.D of the BAA, "Nominal video types to be included in evaluations include stationary and mobile sensors that are air, building, and ground based (e.g. drone/UAV, security/CCTV, ground mobile)". Additional details on video types will be provided at program kickoff.

61. On page 14 of BAA, it states that “Delivered solutions must include all resources and execute autonomously, not relying on internet access during execution. There may also be the ability to accumulate external data for system reference. If a solution would require external data sources, proposals should describe the necessary data and what would be required to obtain such data for offline use.” Does it mean we can use online data for development, but describe the methods for accumulating the offline data for testing and execution in IARPA environment?

Answer: As stated in Section 1.A.2, "Varying levels of reference data may be made available to systems at test time". Such data will be provided by T&E. As stated in Section 1.C, "There may also be the ability to accumulate external data for system reference. If a solution would require external data sources, proposals should describe the necessary data and what would be required to obtain such data for offline use." That data, if considered a feasible solution, would be collected by T&E and as stated in Section 1.A.2, "In the event that such reference data is made available at test time, it will be provided to all performer teams." The offline requirement is for execution at test-time, and does not apply to the development / training portion.

62. Since we need large amount of online data for geolocation, for deliverables for testing, will we be given the specific regions where testing of TA-2 is to be conducted, so we can cache the online data of that region for the T&E team?

Answer: Region information for test-set videos will not be provided in the development phase, but associated reference data for the test videos may be available to systems at test time. As stated in Section 1.A.2 of the BAA, "Varying levels of reference data may be made available to systems at test time..." Proposers may also propose additional/alternate reference data as Section 1.C of the BAA states: "If a solution would require external data sources, proposals should describe the necessary data and what would be required to obtain such data for offline use." The reference data provided will correspond to areas in the test dataset.

63. For Phase 1, the camera orientations are given. How are the camera orientations expressed, Cartesian, Euler Angles or Quaternion? For wall/roof mounted or mobile fixed cameras, is the north and roll-pitch-yaw given? If not, what is given? For overhead cameras mounted on the drones, is the north (or east) and roll-pitch-yaw given?

Answer: The format in which orientation will be provided will be specified at program kickoff.

64. For Phase1, page 10 of the BAA states that “Full sensor pose will be available to performer systems during test runs but pose measurements might not be synchronized with the video frames for cameras with variable pose (e.g. panning and moving cameras) and will contain uncertainty” Do cameras zoom change? Is the camera pose measurement/camera frame synch uncertainty range available?

Answer: Zoom changes for cameras are within scope. It is possible that some level of uncertainty range might be available at test time. Pose encompasses both location and orientation, so pose uncertainty includes location uncertainty.

65. For Phase 2, page 11 of the BAA states that “Sensor location will be available to performer systems during test runs, but orientation may not be present, and location measurements might not be synchronized with the video frames for moving cameras and will contain uncertainty” Do cameras zoom change? Is the camera location measurement/camera frame synch uncertainty range available?

Answer: Zoom changes for cameras are within scope. It is possible that some level of uncertainty range might be available at test time.

66. Per Section 1.D.1 Development Data, If part of the proposed solution is the inclusion of a pre-trained model (one that was trained in the past outside the VideoLINCS program). Is the source training data for that pre-trained model subject to the Development Data requirements? (e.g., IRB approval, sharing across T&E and performer teams)? Is the source training data itself considered an external data set?

Answer: Pre-trained models that were trained in the past outside of the Video LINCS program do not need to be shared with other performers and training data for those models do not need to be shared across T&E and performer teams. Any HSR performed on Video LINCS would require IRB approval. The source training data would be considered an external dataset if that data is used for Video LINCS funded development.

67. The submission deadline is stated as July 22 on grants.gov but July 15 in the provided BAA pdf document. Which one is the accurate submission deadline?

Answer: The Proposal Due Date for Initial Round of selections has been extended to August 1, 2024 @ 4:00 PM EST. The BAA Closing date is August 9, 2024 @4:00pm EST. Please reference Section 4.A which details the specific differences between the two dates. "General Information, item 7.3, Proposal Due Date for Initial Round of Selections, in order to be considered in the initial round. It is within IARPA’s sole discretion whether to evaluate any proposals received after this date but prior to the BAA Closing Date set forth in General Information, item 7.4." IARPA will accept any questions until July 17, 2024. The extended due date is reflected in the amended BAA.

68. Regarding who determines “what constitutes an object”: does the final user determine an object (e.g. by selecting a region in a video) and expects the system to return all places/times that this “user-determined” object appears? Or, is the system expected to output all things it believes to be objects (that could be potentially hundreds of objects in a video) and the user will search in this list of “model-determined” objects?

Answer: The system must re-identify all foreground objects as stated in the BAA in Section 1.A.2.B, "For generic objects, systems will need to autonomously detect and reID foreground objects throughout the collection, without any prior information regarding the types or characteristics of objects to be re-identified."

69. Will people be performing different activities or just walking?

Answer: There is no restriction on the types of activities of people in the videos.

70. How will the geo-localization be evaluated in the case of timestamp errors? Are performers expected to produce trajectories with corrected timestamps?

Answer: Geo-localization is not dependent on time. It is the geo-position of the person/vehicle/generic-object in a given video frame. There is no expectation or task for correcting incorrect timestamps.

71. What is the difference between confusers and persons/objects of interest? Is any person/object re-identified across multiple views considered of interest and anything else considered a confuser?

Answer: Confusers are those people/vehicles present to serve as potential false positives but will not be evaluated for true positive detections.

72. Objects of interest may not be moving in any view in phase 3. That means the system has to match all objects in all videos. Will there be a limit on the number of objects of interest to control the complexity of this operation? Alternatively, would there be some limitations/characteristics to be considered when selecting an entity as a candidate object (e.g., minimum size, belonging to classes of interest, or some other characteristic)?

Answer: All objects in the foreground would be in scope, as stated in the BAA in Section 1.A.2.B, "For generic objects, systems will need to autonomously detect and reID foreground

73. Can you elaborate more on the independent evaluation of TA2? TA2 needs estimation of camera parameters, for which access to the videos is needed. So, will the system be run once end-to-end and then run a second time for independent TA2 evaluation based only on camera parameters estimated in the first run, or, the videos will also be available in the second run?

Answer: As stated in Section 1.A.2, "Geo-localization will be tested both as an independent algorithm as well as for the end-to-end reID system. For independent algorithm testing, camera coordinates of subjects in the test video will be provided at test time and performer systems will remap to geo-coordinates which will be measured for accuracy. For end-to-end reID, geolocation accuracy of subjects identified by the system will be measured." Details regarding how the evaluation will be conducted will be finalized by the T&E team and shared with performers during program execution.

74. Does OpenStreetMap data satisfy your requirement for external data sources to be available in arbitrary locations?

Answer: Reference data, when available, will be made accessible to systems at test time, with the same data provided for all systems. Potential reference data sources are mentioned in the BAA in Section 1.A.2 "such reference data may include georeferenced orthographic satellite imagery such as that available from Google Earth or Maxar Vivid Basemaps". Proposers may also propose additional/alternate reference data as Section 1.C of the BAA states: "If a solution would require external data sources, proposals should describe the necessary data and what would be required to obtain such data for offline use."

75. How much overlap between the views can be assumed to exist and can be leveraged in the proposed solution?

Answer: While an overlap in views is possible, there should be no expectation that there will be any overlap in views between videos.

76. In the BAA, it is mentioned that "For outdoor scenes, such reference data may include georeferenced orthographic satellite imagery such as that available from Google Earth or Maxar Vivid Basemaps". At the same time, it is mentioned that: "Delivered solutions must include all resources and execute autonomously, not relying on internet access during execution. There may also be the ability to accumulate external data for system reference." - How do you envision access to Google Earth (for example) or other resources without access to the Internet? Would a solution require pre-downloaded data from such sources, assuming public availability, to be incorporated in the submitted container? If so, would such data need to become available to all performers at runtime?

Answer: As stated in Section 1.A.2, "Varying levels of reference data may be made available to systems at test time". Such data will be provided by T&E. As stated in Section 1.C, "There may also be the ability to accumulate external data for system reference. If a solution would require external data sources, proposals should describe the necessary data and what would be required to obtain such data for offline use." That data, if considered a feasible solution, would be collected by T&E and as stated in Section 1.A.2, "In the event that such reference data is made available at test time, it will be provided to all performer teams." This data would be provided to all performer team systems at test time.

77. In earlier phases of the program, when knowledge about the cameras is expected to be more accurate, would information about the intrinsic parameters and/or zoom level of a camera be available?

Answer: As stated in Section 1.A.2.a, "Full sensor pose will often be available to performer systems during test runs". There should be no expectation of availability of intrinsic parameters and/or zoom level information.

78. Can the Government please confirm that an offerors objection to contractor support in evaluation will not impact the evaluation and award-ability of our proposal.

Answer: Generally, an offeror’s objection to contractor support in evaluation would not impact the evaluation and award-ability of a proposal. Only Government personnel will make evaluation and award determinations under this BAA. An example of an instance where it could have an impact would be if the objection was to working with a particular T&E partner, see Section 4.A.

In that case, the government would need to consider if the program could be conducted while working around this objection. Please refer to the BAA, Section 4.A for specific details on the use of Contractor support.

79. Can the Government please provide additional detail regarding the information the contractor personnel shall have access to for evaluation in the bidding process. Is it limited to technical information or does it also include the cost proposal?

Answer: If necessary, non-Government technical experts with specialized expertise may advise government panel members and the PM during the technical proposal evaluation, (Please refer to BAA, Section 5.A). A limited number of contractor personnel will have access to both the technical and cost proposals to assist in the logistical aspects of the evaluation and the contract award process. Only Government personnel will make selection recommendations and decisions under this BAA. All contractor personnel shall have signed and be subject to the terms and conditions of non-disclosure agreements.

80. Can the government give us expected T&E data sizes?

Answer: The length of the test videos will be determined during the program. A rough order of magnitude estimate for the length of the self-test dataset is hundreds of hours.

81. When does the Government intend to publish the BAA for purposes of portal registration?

Answer: The Video LINCS BAA has been published to IDEAS as of 26 JUN.

82. In the event of an OTA award, please confirm the "Anticipated FAR General Clauses" will not apply. For example, clause 52.216-7 should not apply to an OTA. DCAA would not have audit rights to an OTA, yet this clause would require the contract to be reported on the Incurred Cost Submissions which is audited by DCAA.

Answer: Yes, in the event of an OTA, Anticipated FAR General clauses "may not apply. In general, with Research OTs, audits and access to financial records are subject to negotiation.

When audits may be necessary or prudent, the Government team has the flexibility to use internal Government auditors.

83. A definition is not provided for key personnel. Please define this term for industry so that offerors have a consistent approach to utilizing this terms.

Answer: Key Personnel are essential to carrying out the work of a project, typically those responsible for the design, conduct and reporting of the research. Key personnel examples include: PIs, Co-PIs and Sr. technical personnel.

84. Please define the expectation of availability for proposed key personnel and significant contributors during contract performance given the anticipated contract start date in May 2025.

Answer: Key personnel and significant contributors must be involved at a level that is appropriate for their role. Per section 4.B.1.c, item I, proposers must “Identify the expected levels of effort (percentage time, or fraction of an FTE) for all Key Personnel and significant contributors.” … “Participation by all Key Personnel and significant contributors is expected to exceed 25% of their time. A compelling explanation is required for any variation from this figure.”

Changes made to key personnel and significant contributors after the selection decision but before contract award, could impact whether the prime offeror will receive an award. The reason being that the proposed key personnel and significant contributors, may have influenced the government's final selection decision. Therefore, any changes after selection but before award, will need to be assessed by the contracting officer and program manager. After contract award, any addition of or revision to key personnel and significant contributors will be handled according to IARPA’s standard clause. See example below:

H- X KEY PERSONNEL

Participant Org role Unique, Relevant Capabilities

Role: Tasks Time

If one or more of the key personnel for whatever reason becomes, or is expected to become unavailable for work under this contract for a continuous period exceeding 30-calendar days, or is expected to devote substantially less effort to the work than indicated in the proposal as initially anticipated, the contractor shall within one week notify the Contracting Officer specified in Section G. Upon concurrence of the Contracting Officer or his authorized representative, the contractor shall promptly replace such personnel with personnel of at least substantially equal ability and qualifications.

All requests for approval of substitutions hereunder must be in writing and provide a detailed explanation of the circumstances necessitating the proposed substitution(s). They must contain a complete resume for the proposed substitute, and any other information requested by the Contracting Officer or needed by him to approve or disapprove the proposed substitution. The Contracting Officer or his authorized representative will evaluate such requests and promptly notify the contractor of the approval or disapproval thereof, in writing.

If the request for approval of substitutions is disapproved, the contract may be subject to termination.

85. What is the Governments distinction between Key Personnel and Significant Contributors?

Answer: Key Personnel are essential to carrying out the work of a project, typically those responsible for the design, conduct and reporting of the research. Significant contributors are individuals whose planned contribution will be impactful (i.e., technical personnel with unique or specialized experience). They are treated the same for the purposes of the contract.

86. Do we have a sense of what broad categories of objects will be involved?

Answer: All objects in the foreground would be in scope, as stated in the BAA in Section 1.A.2.B, "For generic objects, systems will need to autonomously detect and reID foreground

87. How will the re-ID task be specified? Are we given the detections / tracks of the boxes and asked to Re-ID, or do we also have to detect the object. If the latter then we also need to build a generic object detector and would need to know answers to Q1 above.

Answer: There will not be any manual annotation of the people/vehicles/objects to be re-identified. All foreground people/vehicles/objects would be in scope for re-identification. As stated in the BAA in Section 1.A.2.B, "For generic objects, systems will need to autonomously detect and reID foreground objects throughout the collection, without any prior information regarding the types or characteristics of objects to be re-identified."

88. Are there any constraints on the resolution of the objects to detect and re-ID?

Answer: As the BAA states in Section 1.A.1, "The objective is for Video LINCS to be effective in the open-world setting where there is no advance knowledge of the sensor types, sensor collection geometries, scenes, objects to be re-identified, etc." As a result, there are no constraints on resolution.

89. What time scales are we talking about for Phase 1/2/3?

Answer: There should be no expectation on the time scales of the video collection, but there are test conditions where there would be temporal proximity for people and vehicles that appear in the video collection. As stated in Section 1.A.2, during Phase 1, "each person’s clothing will remain the same (short-term / temporally proximate reID)" and during phase 2 "vehicle reID will be performed over short-term / temporally proximate intervals and provided metadata (e.g. time-stamps and camera pose, to the extent available) will be noise free".

90. Can we assume that the people are in an “upright” pose for re-ID, i.e. not sitting, lying down, etc.?

Answer: There is no restriction on the types of activities of people in the videos.

91. Would IARPA consider moving Volume 1, Section 2.F (BAA pg. 31) into Volume 1, Section 3 to better allow performers to keep within the Volume 1, Section 2 page limit of five pages?

Answer: IARPA will not be altering the current BAA.

92. Is the Research Data Management Plan required in both Volume 1, Section 3 and Volume 1, Section 4? (Per the BAA, pages 29 and 34, it appears to be required in both sections.) Would IARPA consider making it required only in Volume 1, Section 4?

Answer: The RDMP is required as Attachment 11 (see 4.B. Proposal Format) and is required in the Technical and Management Proposal. Section 4.B on page 29 identifies the RDMP as attachment 11 and its place within the proposal. Section 4.B1.c, Section 3L of the BAA (Pg.34) defines what is a Research Data Management Plan, and what should be addressed in your RDMP.

Appendix A.6: Research Data Management Plan (RDMP) BAA 24-04 listed in the BAA on page 60 is the template that should be submitted as an attachment within your proposal. There is only one requirement for the RDMP, and that is to add it in the Technical and Management volume per the BAA instructions.

93. Volume 1 shows as a 30-page total limit; however, there is a one-page limit for the transmittal letter (which is shown as not included in the page count) and 5-page limit for the Summary.

Could IARPA confirm whether the remaining sections should total 25 pages or are the 5 pages in addition to the 30 pages listed?

Answer: As stated in the BAA section 4.B. Proposal Format and Content and Section 4.b.1 Volume 1: Technical and Management Proposal: The proposal will not exceed 30 pages, and, yes, the Cover sheet and transmittal letter does count toward the 30-page limit, but the table of contents does not count towards the page limit. Please reference the BAA.

94. How does the Government intend videos to be loaded into the system? (e.g. system should offer an API that Government can call, Government exposes an API that system will call, cloud bucket, etc.)

Answer: An API will be provided by the T&E team after program kickoff. As stated in the BAA in Section 1.G.3.1, "The program will also require the use of a common processing framework, such as the Open Media Processing Framework [3] for development. The first version of the Video LINCS API and nominal framework will be provided to Performers at the beginning of the program and updated periodically thereafter. "

95. Government owned data could be very relevant for tasks such as relocating objects in a video.

Does the Government expect the system to be fully self-contained, or is the Government open to exposing other useful govt-owned data…