140G0326Q0165_Am1_QnA_0001.pdf

QUESTIONS AND ANSWERS

1. STAC Implementation & Providers.

a. Of the STAC providers listed in Table 1 of the SOW, does the Government have a preferred or prioritized provider (or providers) for this effort?

The Government does not have a preferred or prioritized provider, but we seek to develop STAC capabilities from a provider that includes low cost and computational efficiencies. The Contractor may include an assessment of the STAC in their workplan if that may help define a prioritized list of providers.

b. Should the Contractor focus primarily on optical imagery (Landsat and Sentinel-2), or is support for SAR (Sentinel-1) through STAC also expected?

Yes, the contractor should focus on the optical imagery data (Landsat and Sentinel-2). We do not require support for SAR (Sentinel-1) through STAC.

2. Validation & Alaska Test Sites.

What specific quantitative metrics and performance criteria does the Government expect for validating shoreline change results against the provided lidar, photogrammetry, and satellite-derived shoreline datasets?

Metrics for evaluating shoreline change results are suggested to follow existing scientific and statistical methods. For shoreline change data, state-of-the art accuracy assessments are based on simple data comparisons and traditional statistical techniques that include linear regression and error metrics, such as the root mean squared error (RMSE) See the following references as examples of existing methods and code availability for these comparisons:

Vos, K., Splinter, K.D., Palomar-Vázquez, J. et al. Benchmarking satellite-derived shoreline mapping algorithms. Commun Earth Environ 4, 345 (2023).

https://doi.org/10.1038/s43247-023-01001-2

Castelle B et al. Satellite-derived shoreline detection at a high-energy meso-macrotidal beach. Geomorphology 383, 107707 (2021).

https://doi.org/10.1016/j.geomorph.2021.107707

3. Are there any known data challenges or considerations for the 10 Alaska test sites (Table 2) that we should account for, such as seasonal ice, tidal effects, or persistent cloud cover?

DOI, USGS: Amendent 1/RFQ 140G0326Q0165 Page 4 of 10 https://doi.org/10.1038/s43247-023-01001-2 https://doi.org/10.1016/j.geomorph.2021.107707

Yes, the Alaska test sites have seasonal ice, snow, and persistent cloud cover.

Tidal effects exist worldwide, and the Alaska sites have relatively low tidal effects.

4. When will the Government provide the validation reference datasets (lidar, photogrammetry, and existing shoreline positions)?

The Government will provide all validation datasets during the first week of the contracted work.

6. Comparison with GEE Workflow.

What level of detail is expected in the comparison between the STAC-based workflow and the existing Google Earth Engine (GEE) CoastSeg workflow? Is a full quantitative side-by-side analysis required across all 10 sites?

The GEE and STAC workflows should be conducted fully for each of the test sites.

Quantitative side-by-side analyses will be needed with respect to run times, data download versus STAC access times, computing storage needed, and general workflow operations and problems.

7. Code Repository & Publication.

Which USGS-approved Git hosting platform should the Contractor use when publishing the updated CoastSeg code and repository in accordance with USGS Survey Manual IM OSQI 2019-01?

The preferred platform is GitLab at https://code.usgs.gov/.

8. Training & Documentation.

What format and duration are expected for the one virtual training session for

USGS/PCMSC staff?

Training session could be in person or virtual. Training would walk through overarching workflow, operational requirements and settings, and at least one complete step-by-step example of the workflow for the trainees. It is expected that this training would occur over a duration of 2-to-4 hours.

9. Compliance.

Beyond submission of an Accessibility Conformance Report (ACR) using the VPAT template, are there any specific deliverables or components that require additional Section 508 focus?

No.

10. SAR vs. optical scope. The SOW (Sections II–V) describes STAC-based ingestion of Landsat and Sentinel-2 optical imagery, but the Section 508 Accessibility Statement

DOI, USGS: Amendent 1/RFQ 140G0326Q0165 Page 5 of 10 https://code.usgs.gov/ and FISMA checklist are titled "Coastal Change Mapping with Synthetic Aperture Radar (SAR)," and SOW Table 2 is captioned "SAR-based coastal change assessments." Table 1 also lists Sentinel-1 GRD collections. Could you confirm whether Sentinel-1 SAR processing is within scope, or whether the effort is limited to optical imagery, with the SAR references being legacy document titles?

The effort should be limited to optical imagery, and the SAR references are legacy document titles as suggested.

11. Git hosting platform. The SOW requires development within a fork of CoastSeg on a "USGS-approved Git hosting platform for scientific software releases." Will the Government provision contractor access to the USGS platform (e.g., code.usgs.gov) and sponsor the IM OSQI 2019-01 review process, or should the contractor develop on a public platform such as GitHub with transfer to the USGS platform at publication?

The contractor will have full access to the USGS platform (code.usgs.gov), and all development and publication of code will occur on this platform.

12. Privacy Act / system access. FISMA checklist item 2 indicates contractor access to a Privacy Act System of Records. Given that performance is at the contractor's facility using publicly available satellite imagery and Government-furnished shoreline datasets, could you confirm whether any Privacy Act system access, USGS IT system accounts, DOI security training, or background investigations are actually required under this order?

Access to the USGS platform (code.usgs.gov) to share and publish code requires USGS IT systems accounts and associated training and background investigations.

13. Quote format. Are there page limits or format requirements for the technical quote?

No more than 10 pages.

14. May the technical quote include a link to a working code repository as evidence of technical capability?

Please include links to common repositories, such as GitHub. Links to websites that are not distinctly code repositories will not be opened.

15. Government-furnished data timing. Will the lidar, photogrammetry, and satellite-derived shoreline validation datasets for the ten Alaska sites be furnished at award, or staged during the period of performance?

DOI, USGS: Amendent 1/RFQ 140G0326Q0165 Page 6 of 10 http://code.usgs.gov/

Data will be furnished during the first week of the contracted work following award.

16. Publications qualification. For Technical Capability item (f), will peer-reviewed publications authored by proposed key personnel (including consultants or subcontractors) satisfy the requirement, and are published, DOI-issued open-source scientific software releases considered relevant evidence of scientific expertise?

Both peer-reviewed publications and software releases, depending on their content, will be considered to provide evidence of relevant scientific expertise.

17. For the technical qualifications and recent-project evidence in FAR 52.212-2, may an offeror rely on the qualifications and project experience of a named subcontractor or team member? If so, how should that experience be identified?

The Government will evaluate the qualifications, education, technical expertise, and relevant experience of the individual(s) proposed to perform the work. The Contractor/Offeror must provide a resume(s) of staff and project summaries with specific evidence for each of qualification elements as seen under provision 52.212-2.

18. Data rights and open-source licensing.

CoastSeg and its imagery-acquisition dependency (coastsat-package) are licensed under GPL-3.0. Since the extended workflow will be a derivative work of that code, the Government's rights in the derivative portions would be the (very broad) GPL-3.0 rights, while newly developed standalone components, documentation, data, and reports can carry full unlimited rights under FAR 52.227-14. Please confirm this satisfies the solicitation's Rights in Data requirement, consistent with USGS's existing publication of CoastSeg under GPL-3.0.

It is not clear what products would be required to be ‘standalone’ components to the extended workflow for CoastSeg. The existing CoastSeg package (doi:

10.5066/P13NYWEE) is completely contained under GPL-3.0 and includes documentation, data, and reports, all of which are available in code.usgs.gov. All new work under this contract would follow this model for publication and release.

19. Radiometric continuity (TOA vs. surface reflectance).

The current CoastSeg/GEE workflow ingests Landsat Collection 2 Tier 1 TOA and harmonized Sentinel-2 L1C, whereas the Table-1 STAC catalogs primarily serve Landsat C2 Level-2 (surface reflectance) and Sentinel-2 L2A. Does the Government have a preference between (a) using the STAC Level-2/SR products with explicit validation of segmentation performance on SR inputs, or (b) using TOA-equivalent

DOI, USGS: Amendent 1/RFQ 140G0326Q0165 Page 7 of 10

STAC collections where available to maximize radiometric continuity with the existing GEE workflow? We are prepared to implement and validate either.

The current CoastSeg workflows and ML training data (and the CoastSat workflows that form the basis of these) were developed using TOA data. We do not expect or require the contractor to revise these workflows and ML training for surface reflectance data. Please consider a TOA workflow for all proposed work.

20. Git hosting platform and review workflow.

For the “USGS-approved Git hosting platform for scientific software releases,” will the

Government provision contractor access to a USGS-managed instance (e.g., code.usgs.gov), or is a USGS-administered GitHub organization repository acceptable?

Additionally, will the IM OSQI 2019-01 code review be conducted by the USGS science lead, or should the contractor propose independent reviewers?

We have been given the guidance that CoastSeg and its derivatives need to be published and released on the USGS-managed GitLab (code.usgs.gov). The USGS will require contractor access to this GitLab repository after training is accomplished and background checks are complete. Regarding the code review, we will have internal USGS staff review the code, owing to specific requirements of the formal review process. The USGS science lead will facilitate the review process once the contractor has fully developed the code, but the contractor will be required to update the code and reporting in response to all review comments and document these changes in a review reconciliation document.

21. Government-furnished validation data.

Will the lidar, photogrammetry, and satellite-derived shoreline validation datasets referenced in Section II be available for all ten Table-2 sites at time of award, and in what formats/coordinate reference systems are they maintained?

The USGS will furnish the validation datasets during the first week of the award.

Validation data will include raster and vector geospatial data in standard coordinate systems for Alaska (NAD83 (2011) + NAVD88).

22. Imagery modality (SAR vs. optical).

The SOW specifies CoastSeg, which is an optical satellite-derived-shoreline tool

(Landsat 5–9, Sentinel-2, PlanetScope; its segmentation models operate on optical MNDWI/NDWI/RGB imagery). However, Table 2 is titled "SAR-based coastal change assessments" and the Section 508 statement is titled "Coastal Change Mapping with Synthetic Aperture Radar (SAR)," while Table 1 lists primarily optical STAC collections.

DOI, USGS: Amendent 1/RFQ 140G0326Q0165 Page 8 of 10 http://code.usgs.gov/

Can USGS confirm the intended scope is optical STAC imagery (Landsat / Sentinel- 2)consistent with CoastSeg, and that the SAR references are a labeling artifact? If Sentinel-1 SAR processing is genuinely intended, please clarify its role relative to the optical CoastSeg workflow, as these are distinct methods.

The effort should be limited to optical imagery, and the SAR references are legacy titles.

23. Testing Sites.

Ten Alaska sites are identified by bounding coordinates. Could the AOI boundaries be provided, or the approximate shoreline length and processing area for each site to support accurate estimation of the level of effort?

Each of the ten sites have bounding boxes (lat,lon for the southwestern and northeastern limits of the AOI) that provide all the data needed to address this question.

24. Meaning of “training data.”

Technical Requirement 1.a calls for "developing code, tools and training data to enhance the CoastSeg workflows... for use with STAC-derived imagery." CoastSeg's segmentation step uses the Doodleverse/Zoo Segformer models trained on the CoastTrain dataset. Does USGS intend for the contractor to retrain or fine-tune these segmentation models on STAC-sourced imagery (to account for differences between STAC products and the GEE-sourced imagery the current models were trained on)?

And may the contractor build on the existing "Alaska-only" Segformer model and CoastSeg.Arctic assets, or is a new model expected?

The contractor is not expected to retrain or enhance the training for the existing ML models used in CoastSeg. The work should focus on developing and verifying the STAC-based workflows using the existing CoastSeg models.

25. Qualifications equivalence.

Contractor Qualifications (f) and the Technical Capability factor call for evidence of expertise through peer-reviewed publications on coastal change and AI methods. Will USGS consider equivalent demonstrated experience (e.g., federally funded research such as NSF awards and federal contract work) where peer-reviewed publications are not available for a given qualification element?

DOI, USGS: Amendent 1/RFQ 140G0326Q0165 Page 9 of 10

All evidence for demonstrated expertise will be considered, although emphasis will be given to delivered and verifiable/publicly available products, code, and publications over existing contracts.

26. Section 508 / ACR applicability.

The Instructions to Offerors require an ACR VPAT 2.1+)for each commercially available

ICT item. This deliverable is a fork of an open-source scientific Python/Jupyter tool, not a commercial ICT product. How does USGS want 508 conformance documented for a command-line / notebook research tool of this type, given GS2125 treats 508 as a pass/fail gate rather than a scored factor?

It is within the Offeror’s discretion to address the applicable Section 508 requirements or identify those not applicable. Please note the guidance provided in the ‘Instructions to Offerors’ for any “Not Applicable” responses.

27. IM OSQI release ownership.

For deliverable 00030's code publication consistent with IM OSQI 2019-01, will USGS lead the software release review and approval workflow (Fundamental Science Practices review, IPDS record, official release page), or is the contractor expected to drive that process with USGS review?

The USGS will lead the formal review process workflow (FSP review, IPDS record), however the contractor will be required to document the responses to all review comments and revise code and other products as necessary to adhere to the review.

28. The Statement of Work describes extending the CoastSeg workflow to ingest optical satellite imagery (Landsat and Sentinel-2) from the STAC providers listed in Table 1.

However, the Section 508 ICT Accessibility Requirements Statement and the FISMA 18-Point Checklist are each titled "Coastal Change Mapping with Synthetic Aperture Radar (SAR)," and the caption of SOW Table 2 refers to "SAR-based coastal change assessments." Can you please confirm that these SAR references are administrative titles only, and that no SAR data processing is within the scope of this requirement?

The effort should be limited to optical imagery, and the SAR references are legacy document titles as suggested in this question.

DOI, USGS: Amendent 1/RFQ 140G0326Q0165 Page 10 of 10