AoU_CLAD_ROA_OTA-23-003 FAQ - Amendment 1.pdf

National Institutes of Health (NIH) Office of the Director (OD) All of Us Research Program

Frequently Asked Questions

All of Us Center for Linkage and Acquisition of Data No. OTA-23-003

Issue Date: March 15, 2023 Amendment 1

General Notes About the FAQ 3 The All of Us Research Program 3 Award and Proposal Considerations 6 Data Linkage Platform / Cloud 14 Data Linkage 18 FedRAMP 26 Health Information Network / Health Information Exchange Questions 26 Humans Subjects Research Protections 27 Security 28

General Notes About the FAQ This FAQ is provided to offerors to answer questions that have been received about the Center for Linkage and Acquisition of Data (CLAD) Research Opportunity Announcement (ROA). The FAQ includes questions that were asked during pre-solicitation meetings, in advance of the CLAD webinar, and subsequent to the webinar. As additional questions are asked by offerors, the FAQ will be updated and reposted as quickly as possible.

Offerors who wish to submit additional questions about this ROA may do so. The questions should be concise and direct. For each question submitted, the relevant section, page, and paragraph of the ROA must be referenced. Questions should be submitted to the email address listed in section 2.1 of the ROA. They must be submitted at least 14 calendar days prior to the full technical and cost proposal submission deadline. If questions are submitted in less than 14 calendar days from the deadline, they may not be answered.

The All of Us Research Program Question 1. What are the overall goals of the All of Us Research Program?

All of Us has set six goals for 2026:

○ Enroll 1 million participants who reflect the diversity of the U.S., cover the lifespan, and have shared all baseline elements. Of these participants, 500,000 are actively engaged in the program.

○ Expand data available for 1 million participants, to include surveys, health data streams, a whole genome sequence, environmental data, and physical measures.

○ Develop and launch a scalable ancillary studies program that expands the cohort and delivers additional phenotypic, lifestyle, environmental, and biologic data.

○ Establish a diverse global community of 10,000 researchers productively using All of Us data.

○ Incorporate participant return of value into data collections and assess its impact, including return of information to participants on genomics and Electronic Health Records.

○ Enable the full potential of the All of Us Research Program through organizational accountability, strategic and data-driven decision-making, effective core processes, and team capacity and development.

Question 2. What are the current data sources for the All of Us Research Program? What is the awardee’s expectation regarding continued support of these existing data sources?

All of Us collects:

○ EHR data from partners who are health care provider organizations; EHR data is also acquired from Apple Health records and from participant-mediated connections to provider portals.

https://allofus.nih.gov/about/program-goals

○ Survey responses directly from participants through the All of Us Research Program participant-facing platform.

○ Physical measurements data during on-site appointments and through height and weight measurements directly from participants through the All of Us Research Program participant-facing platform.

○ Mobile health data from Fitbit devices, and other physical activity devices that participants link to the program through Apple Health.

○ Biospecimens (blood, urine, saliva) for bioassays. DNA extracted from blood or saliva is sent to genome centers for genomic analysis, including whole genome sequencing, and genome-wide genotyping.

○ For more information, please refer to https://www.researchallofus.org/data-tools/data-sources/.

During the base period, the awardee will be responsible only for the linkages that are specified in the ROA. After a linkage has been successfully completed, the government will decide on the frequency of repeating a linkage. This frequency will be driven primarily by scientific utility.

Question 3. Does the All of Us Research Program have a Master Patient Index (MPI) infrastructure today?

The All of Us Research Program does not currently have a Master Patient Index (MPI) infrastructure.

Question 4. Does the All of Us Research Program have a Data Catalog for their existing data solution?

Please refer to the Controlled Tier Data Dictionary.

Question 5. What is the current technology stack utilized within the All of Us environment?

The All of Us environment is a mix of cloud-based technologies, including services from AWS, Google Cloud Services, and Azure. The data repository for researchers is managed through Google’s BigQuery, and the Researcher Workbench runs on a modification of the Broad Institute’s Terra platform, providing Jupyter Notebooks to researchers.

Question 6. How does All of Us retrieve EHR data? Is data retrieved via scheduled batch, streaming, or on-demand?

EHR data is retrieved from Health care Provider Organizations via secure protocols. Data is received by batch, mostly on a quarterly basis.

Question 7. How is EHR data received from existing data sources (HL7 messages, CCDA documents, API transactions, MLLP, flat files, etc.)?

EHR data is received primarily from All of Us Research Program Health care Provider Organizations (HPOs) as an OMOP extract using secure transfer mechanisms. Other EHR data, from participant mediated methods and Apple Health records, are received as FHIR-format JSON files.

https://www.researchallofus.org/data-tools/data-sources/ https://docs.google.com/spreadsheets/d/1ZTVpTJBCZgUaGVafmUBBgoQv_Q1x6zbzhuwmKom1d_8/edit#gid=183931508

Question 8. Does All of Us use IAL2 as a method for verification of participants upon recruitment?

All of Us does not require IAL2 for ID proofing. The current use case for ID proofing in the program is to meet FDA requirements when capturing electronic signatures for consent. All of Us must complete ID verification to be able to offer health-related return of results to participants.

All of Us sites use our policy as a minimum threshold, but sites may have specific requirements that go above and beyond this policy.

Question 9 (added 3/15/2023). Are any health insurers collaborating with the “All of Us Study and if so how are they involved?

Although the program has had a relationship with some insurers previously, this was for the sake of outreach and enrollment, and the offerors should plan on developing their own relationship with insurers as they feel is appropriate.

Question 10 (added 3/15/2023). Is the “All of Us Study “ Collecting , or planning to collect, participant metabolomic data?

Metabolomic data has been discussed, and it may be sought by way of an ancillary program in partnership with other parts of the NIH, but at this point the program does not have any commitment to the collection or production of metabolomic data.

Question 11 (added 3/15/2023). Are you thinking of VA location data as well as military deployment or temporary locations like college?

We would like to capture as much of the residential history as possible. It may not be possible to capture data regarding schooling and military service. Offerors should consider this as part of their proposal. Note, details of this will be considered during the "requirements gathering" objective in each of the data linkage milestones.

Question 12 (added 3/15/2023). Can a bidder submit proposals for both the CLAD opportunity and the ROA for the Participant and Partner Service Center?

Yes, these are independent ROAs.

Question 13 (added 3/15/2023). Is data from wearables only for those participants that have their own wearable, or is All of Us providing access to wearables? (section 4.2, pg 8, paragraph 1)

All of Us is providing devices to a sample of individuals, however, wearable data may also be shared by participants who have their own devices. Wearable data is not part of the CLAD baseline data linkages.

Question 14 (added 3/15/2023). Does the All of Us study currently have access to Fitbit API intraday data? (section 4.2, pg 8, paragraph 1)

This is not related to CLAD.

Question 15 (added 3/15/2023). How is the All of Us study currently getting wearable data?

(section 4.2, pg 8, paragraph 1)

Through Fitbit devices that the program provides to some participants. Other wearable data is shared by participants through their own devices.

Question 16 (added 3/15/2023). How is the study currently getting EHR data? (section 4.1, pg 7, paragraph 3)

Please see paragraph 3 of section 4.1, paragraph 3 in section 4.2, and paragraph 1 in section 5.3.6 of the ROA.

Question 17 (added 3/15/2023). We see a key objective is to acquire new and updated data sources without reconnecting with a participant. Does this include through opt-in surveys?

(section 1.1, pg 5, paragraph 1)

No, CLAD will not interact with All of Us program participants via opt-in surveys.

Award and Proposal Considerations Question 18. Are for-profit entities and small businesses eligible for the Center for Linkages and Acquisition of Data (CLAD) award?

The award is open to all entities listed in the Research Opportunity Announcement (ROA) Section 6: Eligibility Information, including for-profit organizations and small businesses. There is no specific set-aside for small businesses in this award. For-profit entities can propose profit in their cost proposal but their cost proposal must break out the profit for NIH review and negotiation.

Question 19. Will the program provide potential collaborating partners for offerors?

No. Offerors are encouraged to seek out collaborating partners as they see fit and that they are confident they can work with well. Collaboration is a key element of the All of Us Research Program. Consequently, all collaborating partners on the offeror’s team should anticipate working in a collaborative environment.

Question 20. Are third tier sub-contracts allowed as part of this mechanism?

Yes, but any Lower Tier Agreements with subcontractors must include the flow down terms required in the OTA Terms and Conditions for the CLAD award.

Question 21. Are lower tier agreements allowed?

During the performance of the awarded OT Agreement between the awardee and NIH, either party may recommend to the other the need for a Lower Tier Agreement. The authorization process for establishing Lower Tier Agreements will be documented in the terms and conditions of the awarded OT. A current draft of this process can be found in Article VI of the Terms and Conditions in Appendix A of the ROA.

Question 22. Does NIH have a defined list of expected roles or key team members the offeror will be expected to provide?

There is a list of anticipated key team personnel for CLAD. These key personnel are aligned with CLAD cores. Key personnel also have an associated minimum Level of Effort (LoE). The applicant may identify different or additional key members and their associated LoE to successfully execute this work. Details are provided in the full announcement.

Question 23. Will NIH issue the Other Transactions Award directly from the agency, or through an Other Transactions Consortium Manager? If the latter, what is the name of Consortium Manager that will be used?

The Other Transactions Award will be issued directly from NIH.

Question 24. Is a cost share expected under the Other Transactions Award?

Cost share is not a requirement under the Other Transactions Award.

Question 25. Is an estimate available for the number of Data Acquisition Partners and Health Information Networks that will be included in scope?

Offerors will find in the milestones the number of data linkages that are expected to be completed during the base period of the award. Offerors are encouraged to find partners, as needed, to fulfill the scope of work and milestones. The number of partners and the formulation of teams to complete milestones and acquire data is to be determined by each offeror.

Question 26. Are offerors expected or required to apply for all cores of the CLAD ROA?

Each proposal must include a team that can satisfy the functional capabilities for all four cores and the milestones / objectives. Offerors may choose how to staff cores. The submitting entity may submit alone or as a primary entity in a submission that includes one or more other collaborating entities. An entity is limited to a single submission as a prime, but may be included as a collaborating entity on more than one application.

Question 27 (added 3/15/2023). ROA Section 5.1, Page 10, Paragraph 1. This section states that Offerors are encouraged to propose partnerships to provide expertise across the four primary cores within their proposal. Are Collaborating Partners permitted to work with more than one Primary Offeror?

Yes, Collaborating Partners may work with more than one Primary Offeror.

Question 28. Are the All of Us Research Program Participant and Partner Services Center (PPSC) and the Center for Linkage and Acquisition of Data (CLAD) opportunities linked in any way?

The PPSC and CLAD are independent program opportunities. Please see OTA-23-002 for more details on the PPSC.

Question 29 (added 3/15/2023) Thinking about exfiltration of data costs and knowing that the DRC uses Google, are there infrastructure cost concerns with standing up CLAD that would help us structure our response?

The proposal will certainly be evaluated for cost and so cost control is helpful. CLAD is an independent proposal and will not be evaluated from a cost perspective with the DRC.

Question 30. Are any of the technical components of the solution open source?

The components of the CLAD solution are to be determined by the offeror.

Question 31. Is the CLAD awardee required to have all linkages or will you have a consortium each with one or more specialities?

CLAD will result in a single award. Offerors should decide on the mix of partners that are needed to fulfill all of the milestones associated with CLAD.

Question 32 (added 3/15/2023). Appendix F: Cost Proposal Breakdown,"It is likely that commercial entities responding to this ROA already have accepted commercial contract labor rates established and used in contract work with NIH today. Some of these commercial contract labor rates are defined by cost, such as those included on the NIH CIO-SP3 contract vehicle.

Will NIH permit the use of commercial labor rates to develop proposal pricing for this opportunity?"

Due to uncertainties and flexibility needed in the project, a fixed price agreement would not be appropriate, therefore a cost-reimbursable or cost-reimbursable plus fee payment agreement would be considered. The offeror can have sub awards with fixed priced labor rate arrangements for commercial services and fixed priced commercial products. Any fixed price cost shall breakdown labor categories, hours, and itemized list of materials/equipment and travel in proposal.

Question 33 (added 3/15/2023). ROA FAQ: Question 10,"In the response to FAQ Question 10, it is stated that ""for-profit entities can propose profit in their cost proposal but their cost proposal must break out the profit for NIH review and negotiation."" Can NIH please clarify that the type of contract to be awarded will be a Cost Plus Fixed Fee contract, or can Time and Materials plus Firm-Fixed Price contracts be awarded as well?"

Due to uncertainties and flexibility needed in the project, a fixed price agreement would not be appropriate, therefore a cost-reimbursable or cost-reimbursable plus fee payment agreement would be considered. The offeror can have sub awards with fixed priced labor rate arrangements for commercial services and fixed priced commercial products. Any fixed price cost shall breakdown labor categories, hours, and itemized list of materials/equipment and travel in proposal.

https://sam.gov/opp/f9fba45eefaf41a6915e3c7ef784ab37/view#attachments-links

Question 34 (added 3/15/2023). ROA Main Document: Cost Proposal,Personnel Costs: Can the NIH please clarify if all personnel in the Cost Proposal must be named individuals? Can unnamed TBD individuals be included in the cost proposal?

Only Key Personnel must be named. So, unnamed TBD individuals may be included in the cost proposal for non-Key Personnel.

Question 35 (added 3/15/2023). Appendix E: Proposal Intent Form,Can Letters of Support be included from subcontractors and teaming partners with the Proposal Intent Form?

The NIH only needs the information requested in the Intent to Submit Proposal Form (Appendix E) and letters of support are not required. The information submitted in the intent form is not binding and does not enter into the review of a subsequent proposal, the information that it contains allows the All of Us Research Program to estimate the potential review workload and plan the review.

Question 36 (added 3/15/2023). ROA Section 7.2, Page 29, Paragraph 3 - If my company is intending to be a collaborating partner, and not the Primary Offeror, are we required to complete the Intent to Submit Form? Our interpretation is that only the Primary Offeror is to complete this form but would like to confirm.

NIH is requesting that offerors who intend to submit a proposal in response to this ROA complete the Intent to Submit Proposal Form (Appendix E) and submit it to AllofUsDTPDAwardRequirements@nih.gov by March 15, 2023 at 3pm ET. Although a letter of intent is not binding and does not enter into the review of a subsequent proposal, the information that it contains allows the All of Us Research Program to estimate the potential review workload and plan the review. This should be submitted by the Primary Offeror rather than a Collaborating Partner.

Question 37 (added 3/15/2023). ROA Main Document: Phase II Demonstration, does NIH have estimated dates on when Phase II demonstrations may be held?

We anticipate that the demonstration will occur in mid to late July. Offerors who are selected for Phase II will be provided with a few weeks to prepare for the demonstration. Please refer to the Webinar slide deck for more details on the dates of each CLAD Phase.

Question 38 (added 3/15/2023). ROA Main Document: 8.1.3 Proposal Content/8. Milestone Plan, Under Section 8. Milestone Plan, it asks for a table with a column for Estimated Total Cost in Dollars. Is the NIH permitting offerors to include costs within their Technical Proposals?

The costs in the milestone table are high level cost estimates for each milestone and should match directly with the formal and full cost proposal.

Question 39 (added 3/15/2023). ROA Main Document: 8.1.3 Proposal Content/11. Cost Proposal. Can NIH please confirm whether Section 11. Cost Proposal should be submitted separately as a PDF along with Appendix F in Excel or included with the Technical Proposal as one submission document?

The cost proposal along with Appendix F should be submitted as a PDF and included with technical proposal submission into the NIH eRA ASSIST System.

Question 40 (added 3/15/2023). ROA Main Document: 8.1.2 Proposal Format/Section 8.1.3 Proposal Content/10. Key Personnel,"Under 8.1.2, Appendix A: Key Personnel NIH Biosketches and Resumes, it only lists Principal Investigator, Program Manager, and De-Identification/Disclosure Expert as Key Personnel. Under 8.1.3/10. Key Personnel, it recommends 7 other roles. Can NIH clarify Appendix A and confirm if there are any required Key Personnel?"

In section 8.1.2, the ROA specifies the following:

● Appendix A: Key Personnel NIH Biosketches and Resumes

○ Principal Investigator, Program Manager, and the De-identification / Disclosure Expert: for each, a maximum four-page NIH biosketch.

○ Other key personnel: for each, either a maximum four-page NIH biosketch or a two-page resume.

○ The biosketches and resumes in Appendix A do not count against the page limit.

Thus, for the Principal Investigator, Program Manager, and De-Identification / Disclosure Expert the offeror must provide a four-page NIH biosketch. For the other key personnel the offeror may provide either a maximum four-page NIH biosketch or a two-page resume.

Question 41 (added 3/15/2023). ROA Main Document: 8.1.2 Proposal Format,"The ROA has a font requirement of Times New Roman, 12 point. Will NIH permit offerors to use font in the Arial family at a minimum 9 point for tables and header/footer information, and 8 point for figures and illustrations with the option to be landscape? "

Header/footer information should be in Times New Roman, 12 point font. Smaller text for charts, tables, figures, and illustrations is acceptable, as long as it is legible when the page is viewed at 100%". Figures and illustrations may be portrait or landscape mode.

Question 42 (added 3/15/2023). The Appendix F only gives a breakdown through Milestone 12 and the first 18 months. Where should the bidders provide optional cost breakdowns?

Cost estimates for optional periods are not being requested in Phase I proposals but may be requested in Phase III for planning and budgeting purposes. The Phase III offeror may be requested to provide an estimate for the optional periods by assuming each of the linkages will be repeated during each optional period. This should also include the cost of documenting these linkages and the update to the tools that were created during the base period. It should also include the cost of operating and maintaining the data linkage platform.

Question 43 (added 3/15/2023). Can the bidders add rows and columns to Appendix F as needed to reflect their accounting system setup and the specific cost elements they're proposing as well as profit/fee?

Yes, the vendor can add as many rows and columns as needed to reflect their accounting system setup and the specific cost elements they're proposing as well as profit/fee.

Question 44 (added 3/15/2023). Data Rights, Sections A, B, C. Page 40-41. "Regarding the recipient's pre-existing software platform or other trade secrets and materials (part of the recipient's Background Data as defined in the agreement), the agreement contains contradictory terms on whether the NIH is expecting to have use of that software platform, trade secrets and materials only during the term of the agreement, or perpetually. (see Article VII: Data Rights, Sections A, B, C, pages 40-41 of PDF). What is NIH's expectation?

Per the ROA, the Terms and Conditions attached in Appendix A serve as a baseline for Other Transactions agreements awarded by the All of Us Research Program in Phase III. The NIH may modify these terms and conditions throughout the selection process. Requests by offeror(s) to modify the terms and conditions language will be considered by the NIH on a case-by-case basis and negotiated as deemed appropriate. Discussion/negotiation of agreement terms and conditions will take place with the invitees of Phase II and Phase III.

Question 45 (added 3/15/2023). Data Rights, Section D. Page 41-42. Regarding the recipient's Background Data - the proposed agreement contains contradictory terms on how this Background Data can be used. It first states: (1) the government has an “irrevocable right to reproduce, publish, or otherwise use the work for any Government purpose, and to authorize others to do so”, yet (2) the next paragraph contemplates the recipient being able to furnish the Background Data “with restrictions on use, release or disclosure”. The recipient's ability to designate restrictions on use, release or disclosure are not described anywhere else but this one phrase. a. How does the NIH reconcile these contradictory terms?

Per the ROA, the Terms and Conditions attached in Appendix A serve as a baseline for Other Transactions agreements awarded by the All of Us Research Program in Phase III. The NIH may modify these terms and conditions throughout the selection process. Requests by offeror(s) to modify the terms and conditions language will be considered by the NIH on a case-by-case basis and negotiated as deemed appropriate. Discussion/negotiation of agreement terms and conditions will take place with the invitees of Phase II and Phase III.

Question 46 (added 3/15/2023). Data Rights, Section D. Page 41-42. b. Would NIH consider modifying the proposed agreement terms by adding ""Notwithstanding the foregoing..."", [then continue with Recipient being able to define restrictions on use, release or disclosure]?"

Per the ROA, the Terms and Conditions attached in Appendix A serve as a baseline for Other Transactions agreements awarded by the All of Us Research Program in Phase III. The NIH may modify these terms and conditions throughout the selection process. Requests by offeror(s) to modify the terms and conditions language will be considered by the NIH on a case-by-case basis and negotiated as deemed appropriate. Discussion/negotiation of agreement terms and conditions will take place with the invitees of Phase II and Phase III.

Question 47 (added 3/15/2023). Data Rights, Section D. Page 41-42. c. Can the terms of the proposed agreement be modified so that the recipient will provide its Background Data for the term of the agreement only?

Per the ROA, the Terms and Conditions attached in Appendix A serve as a baseline for Other Transactions agreements awarded by the All of Us Research Program in Phase III. The NIH may modify these terms and conditions throughout the selection process. Requests by offeror(s) to modify the terms and conditions language will be considered by the NIH on a case-by-case basis and negotiated as deemed appropriate. Discussion/negotiation of agreement terms and conditions will take place with the invitees of Phase II and Phase III.

Question 48 (added 3/15/2023). Data Rights, Section D. Page 41-42. d. For some very useful data sources produced at private expense, the data vendor cannot agree to provide NIH with a perpetual, irrevocable, royalty-free license to reproduce, publish, or otherwise use the data for any Government purpose, nor to authorize others to do so. For such sources, would NIH be amenable to modifying the the terms of the agreement to accommodate these vendor requirements: custom data extracts or access to all data on an analytical platform may be provided: 1) on a "pay per use" (per research project) basis and/or for a pre-defined, limited time period 2) conditional upon satisfactory review by the vendor of each proposed data use/research project against data supplier agreements and vendor patient privacy standards; and 3) limited to authorized users.

Per the ROA, the Terms and Conditions attached in Appendix A serve as a baseline for Other Transactions agreements awarded by the All of Us Research Program in Phase III. The NIH may modify these terms and conditions throughout the selection process. Requests by offeror(s) to modify the terms and conditions language will be considered by the NIH on a case-by-case basis and negotiated as deemed appropriate. Discussion/negotiation of agreement terms and conditions will take place with the invitees of Phase II and Phase III.

Question 49 (added 3/15/2023). AoU_CLAD_ROA-OTA-23-003. 8.1.5 Cost Evaluation. Page

37. The government states that proposals will be evaluated by cost. Given that this program is being funded under an OTA and not an assistance agreement, will the government consider and allow for proposals to be evaluated by price (commercial pricing)? Can you also provide clarification on whether fixed price models based on commercial pricing are eligible for the program?

Due to uncertainties and flexibility needed in the project, a fixed price agreement would not be appropriate, therefore a cost-reimbursable or cost-reimbursable plus fee payment agreement would be considered. The offeror can have sub awards with fixed priced labor rate arrangements for commercial services and fixed priced commercial products. Any fixed price cost shall breakdown labor categories, hours, and itemized list of materials/equipment and travel in proposal.

Question 50 (added 3/15/2023). AoU_CLAD_ROA-OTA-23-003 Draft Terms and Conditions.

1. Cost Principles. Page 28. The government states that the resulting agreement is subject to reimbursement of actual costs subject to generally accepting and established federal cost principles for the entity receiving funding. Given that this opportunity is an OTA and not an assistance agreement or a FAR based contract, will the government consider proposals based on commercial pricing inclusive of fee/profit (as established in FAR 2.101 and other federal award and assistance guidance)? While the government includes a definition of fee/profit in the draft terms it does not make a statement as to whether fee/profit can be included in the proposal and on what basis.

For-profit entities can propose profit/fee in their cost proposal but the prime and any non-commercial sub awards must break out the profit/fee for NIH review and negotiation.

Question 51 (added 3/15/2023). Will the cover letter and/or letter of transmittal count against the page limit?

Neither count against the page limit. However, the cover letter should be no longer than 1 page.

Question 52 (added 3/15/2023). Is Appendix C required for the phase 1 submission?

Please refer to Milestone 3 Objective 3.7 on Page 20. This objective is to “Complete all requirements on the continuous monitoring (referred to as “ConMon”) checklist (see Appendix C).” We project that this should be completed between months 2-12 after award.

Question 53 (added 3/15/2023). Will it be considered as Conflict of Interest if the offeror has a Teaming Partner who has already had the required data sources as part of the business, and the offeror prices the cost to acquire that data from the Teaming Partner in the budget?

A teaming partner may already have the required data sources as part of their business. This cost of the partner to work as a partner should be separated out from the cost of the acquisition of the data from that partner.

Question 54 (added 3/15/2023). Phase II of the ROA evaluation process asks for a capabilities demonstration of the proposed DLP. Does that indicate a preference for existing systems that have at least the core capabilities and could be further customized?

Please review the description of Phase II in the ROA. We are open to a variety of ways to conduct the demonstration. Phase II does not require an existing system.

Question 55 (added 3/15/2023). How long after Phase I will Phase II of the competition process be scheduled?

Please refer to the Webinar slide deck for more details on the dates of each CLAD Phase.

Question 56 (added 3/15/2023). Since there is only one award, do respondents have to submit proposals for all cores?

Yes.

Question 57 (added 3/15/2023). Are you wanting to have the submitter have every solution you need (e.g. Network, De-identification, Data Normalization, etc.)? Can we submit for a portion of the contract?

No. There is one award for CLAD. Offerors need to supply all of the personnel, services, platforms, etc., to fulfill all of the requirements of CLAD. This includes all of the cores and milestones. We encourage you to partner with other organizations to fulfill CLAD.

Question 58 (added 3/15/2023). When evaluating the proposal, how much value will you place on past performance? (section 8.1.4, pg 36, paragraph 1)

Past Performance is not part of the evaluation criteria for CLAD but Relevant Past Experience is part of the evaluation of proposals. Please see ROA Section 8.1.3 Proposal Content and Section

8.1.4 Proposal Evaluation Criteria for more information.

Question 59 (added 3/15/2023). Does CLAD Base Period also need to account for Costs to run the platform in the cloud? Or does NIH anticipate picking up the Cloud bill?

Yes please account for costs to run the platform in the cloud.

Question 60 (added 3/15/2023). When submitting the final proposal, should offerors separate the proposal so that each numbered item in ROA section 8.1.3 is included as a separate attachment within the ASSIST system?

Unless required to be a separate attachment per eRA ASSIST instructions, the individual sections in ROA Section 8.1.3 are not required to be separate attachments but each section must be clearly labeled and contain the information and adhere to the page limitations stated in the ROA.

Question 61 (added 3/15/2023). Please confirm that offerors may submit two separate attachments for the cost proposal, one narrative document and one spreadsheet attachment.

The cost proposal must include a cost narrative and cost breakdown to complete each milestone in the milestone plan submitted. Examples are provided in Appendix F.

Question 62 (added 3/15/2023). Do engineers need to be sourced from someplace specific or are there any areas that are excluded?

The government does not provide recommendations on specific places where organizations should source their engineers or other staff from. Please review Section 6: Eligibility Information, and more specifically, 6.4 Non-Eligible Entities.

Data Linkage Platform / Cloud Question 63. Does All of Us have a preferred cloud provider?

No, but the program does prefer cloud solutions to on-premises solutions.

https://www.era.nih.gov/files/ASSIST-Instruction-Guide-for-NIH-Other-Transactions.docx

Question 64. Does the All of Us Research Program have a preferred cloud vendor; a preferred list of platforms (such as terra.bio); or preferred technologies that should be considered to better integrate with existing systems?

The existing participant and study staff applications and platforms supported under the Participant Technology Systems Center (PTSC) and the Participant Center (TPC) awards are hosted and managed using the Amazon Web Services platform. The program currently hosts all Data and Research Center (DRC) platforms, data, and analytic tools on Google Cloud Services, and is in the process of expanding that to Azure and Amazon Web Services. Awardees are not limited in their choice of cloud provider and are welcome to work with whichever vendor they prefer. Fluency in the different cloud providers is worth noting within your application.

The All of Us Research Program does not have a preferred list of platforms.

Question 65. Is the CLAD data linkage platform to be made available to external researchers?

No. The CLAD data linkage platform will only be available to the awardee, All of Us Research Program government staff, and specific partners. Examples of partners who may require access include the Data and Research Center (DRC) and contractors to the All of Us Research Program.

The offeror should anticipate that this would be about 50 individuals or less. These staff will need access to the dashboards that are created to monitor linkage progress and potentially some analytic capability to review data linkage datasets before they are delivered to the Data and Research Center. These staff will also need access to CLAD managed tools such as Jupyter Notebooks, R, and Python on the data linkage platform for these activities.

Question 66 (added 3/15/2023). SOW 5.2.4, pages 15. Privacy and Security: Assure and safeguard the All of Us Research Program participant privacy and the security of the DLP as this relates to CLAD. "Other than the contractor, who will be authorized access to the data from within the CLAD environment? All of Us employees, All of Us data administrators, Researchers, and/or Participants?

The CLAD data linkage platform will only be available to the awardee, All of Us Research Program government staff, and specific partners. Examples of partners who may require access include the Data and Research Center (DRC) and contractors to the All of Us Research Program.

The offeror should anticipate that this would be about 50 individuals or less. These staff will need access to the dashboards that are created to monitor linkage progress and potentially some analytic capability to review data linkage datasets before they are delivered to the Data and Research Center. These staff will also need access to CLAD managed tools such as Jupyter Notebooks, R, and Python on the data linkage platform for these activities. The Other Transactions Program Official (OTPO) will designate who may have access to the data linkage platform and the data.

Question 67. Does All of Us prefer a federated, centralized, or other approach to data connection and storage?

All of Us EHR data is currently aggregated at the Data and Research Center using a centralized approach for data collection and storage. Similarly for CLAD, a centralized cloud-based solution is required. Data from CLAD linkages will be transferred to the Data and Research Center.

https://allofus.nih.gov/funding-and-program-partners/participant-technology-systems-center https://allofus.nih.gov/funding-and-program-partners/participant-center https://allofus.nih.gov/funding-and-program-partners/data-and-research-center

Question 68. The notice of future ROA mentioned that “Data collected through passive data streams will be transmitted to the All of Us Research Program Data and Research Center (DRC) for inclusion in the Researcher Workbench.” Please clarify if CLAD’s cloud-based platform must be hosted by the same cloud provider as the Researcher Workbench. Is All of Us open to using another cloud-provider within the NIH STRIDES Initiative if the team determines that it is technically feasible?

The CLAD cloud-based platform does not need to be hosted by the same cloud provider as the Researcher Workbench. We are open to other providers that are technically feasible and meet the security requirements of NIH and the All of Us Research Program. The platform must meet FISMA Moderate compliance.

Question 69. Can the All of Us data stored on the CLAD platform exist in the same database or cloud environment as data used on other non-All of Us programs or effort?

No. All of Us data must not be commingled with data that is not part of CLAD and the All of Us Research Program.

Question 70. Will the CLAD data linkage platform be accessible to external researchers interested in analyzing this data?

No.

Question 71 (added 3/15/2023). ROA Main Document: Data Linkage Core,"Can the NIH please clarify if the DRC would prefer for the CLAD awardee to tokenize patient identifiers in the DRC cloud environment, or will the DRC share All of Us participant identifiable data to the CLAD cloud environment to tokenize in that environment?"

The CLAD awardee will not have access to the DRC environment to conduct tokenization, if the decision is made that tokenization is needed, in the DRC. The All of Us Research Program may request that the DRC produce tokens for CLAD. However, the awardee will also receive personally identifiable information on study participants from the DRC that the awardee will use to create tokens for matching between linkage data providers and CLAD. So, CLAD will need to produce tokens and work with providers of data to ensure that the tokens and tools that the providers use to create tokens are also used within CLAD. All of Us does not have a preferred provider of token creation tools and this may depend upon the source of the linkage data.

Question 72 (added 3/15/2023). How much data is currently in the system?

We are unsure of which system the question is referring to. If it is in reference to CLAD, CLAD is a new award and thus there is currently no data.

Question 73 (added 3/15/2023). Can you provide an architecture diagram of the current system?

We are unsure of which system the question is referring to. If it is in reference to CLAD, CLAD is a new award and thus there is no architecture diagram.

Question 74 (added 3/15/2023). The requirement states data will be exchanged with the DRC implying a bi-directional flow. What data will be coming from the DRC to the DPC?

The Data Linkage Platform will receive participant personally identifiable and perhaps other types of data from the DRC that will be needed to link participants to the acquired data. In addition, for each data linkage the CLAD awardee will be required to transmit the linked data, documentation, and tools to the DRC so that this data may be moved to the Researcher Workbench after it has been curated.

The frequency of data transmission to the DRC will depend on several factors. This will be optimally determined during the requirements gathering objective for each data linkage (e.g., Objective 6.1, Objective 7.1). We anticipate that at least one data acquisition for each of the base period data linkages will occur during the base period. Moreover, deliveries to the DRC will need to be synchronized with the All of Us Data Roadmap, which projects a schedule for yearly delivery of data to the Researcher Workbench. Currently, All of Us is planning on yearly releases of data to the Researcher Workbench for data that has been historically collected and is part of ongoing data collection operations (e.g., physical measures, existing questionnaires). New data streams, such as those for data linkages, may be released to the RW out of sync with the roadmap to ensure timely delivery of this data for research use. Lastly, a small percentage of All of Us participants do withdraw from the study. Thus, the CLAD awardee should be prepared to support removal of such participants data from the CLAD repository as directed by the OTPO (or designees).

Question 75 (added 3/15/2023). Is there an expectation that the offeror develop analytic tools to visualize CLAD data for internal operational purposes, or will that remain under the purview of the DRC as part of their tools from the Participant Data Repository?

Yes, we would like tools developed to visualize CLAD data for internal operational purposes, particularly as part of monitoring activities. These tools will be accessible to the Other Transactions Program Official (OTPO) and designees. This will not replace tools developed and used by the DRC.

Question 76 (added 3/15/2023). Do you plan on keeping Google's BigQuery or replacing it?

The work on CLAD will not replace the platforms or infrastructure used in the Data and Research Center.

Question 77 (added 3/15/2023). Can we propose to utilize the DRC cloud instead of creating a separate cloud infrastructure? For example if DRC is already using GCP can we use an account within that entity to decrease costs?

No. A separate cloud infrastructure and data linkage platform is required for CLAD.

Data Linkage Question 78. In cases where All of Us has identified new passive data streams for acquisition as part of the CLAD ROA, has All of Us identified the sources from where this data will be retrieved from? If so, will All of Us direct the awarded recipient of this ROA to the source of the passive data (i.e., a Data Warehouse, partner/provider organization, electronic health records), or will the awarded recipient be expected to locate and acquire this data independently?

The All of Us Research Program has requirements-gathering objectives for each linkage. This collaborative process will result in per-linkage recommendations for sources. Once the program has agreed to the source, the awardee will acquire the data. However, the ROA document has noted instances in which the All of Us Research Program has already identified a specific data source for a linkage.

Question 79. Which datasets should offerors use in their proposal for each of the base period milestones?

As part of the proposal, offerors will recommend a specific candidate dataset for most linkages, a rationale for that selection, and should account for this in the cost proposal. When offerors justify a candidate dataset, they should keep this in mind:

1. The candidate dataset should maximize scientific utility.

2. Consider the potential for return of value by returning data to All of Us participants.

3. All of Us has a special focus on collecting data from individuals that have been traditionally under-represented in biomedical research (UBR). Candidate datasets that result in linkage data for a high percentage of UBR individuals is important.

Specifically from the perspective of cost, offerors should include in their proposal: candidate datasets for mortality, residential history, geocoding, and claims. Furthermore, offerors should recognize that all of the addresses acquired for residential history will be geocoded as part of CLAD. Moreover, the CLAD environmental data linkage requires that offerors use data from the Centers for Disease Control and Prevention. This will require the residential history and or geocoded information. Finally, the HIN/HIE effort is a pilot and offerors have room to propose a design that aligns with the All of Us UBR targets. It is also an attempt to understand if existing All of Us Research Program EHR data gaps can successfully be filled with such an approach.

Nonetheless, acquiring data through an HIN/HIE may require address information. The residential history data may be useful for this purpose.

Question 80. Is data within the All of Us Research Program linked internally at this time?

Yes.

Question 81. How many data integrations does the All of Us Research Program hope to accomplish in the first period of the CLAD award?

Details are provided in the ROA.

Question 82. What types of environments should data linkage analytic tools be developed in?

Currently, the Researcher Workbench is built to support Jupyter Notebooks, R and Python. Thus, the offeror will develop data linkage tools for dissemination to the Researcher Workbench in Jupyter Notebooks, R, and Python. In the future, we anticipate that other environments such as SAS, Stata, and non-programming environments (e.g., Tableau) will be supported on the Researcher Workbench. At that time we anticipate asking the offeror to support analytic tools in these environments as well. Supporting these additional environments will not occur during the base period of CLAD.

Question 83. Within the existing data systems, will the selected offeror be required to establish methods to develop linkages inside the existing dataset or will new linkages be focused on the new sources brought in as part of the CLAD effort?

The selected offeror will be primarily required to establish methods to link new sources of data to the existing All of Us Research Program participant population. Some of this new data may require new methods of curation, harmonization, and standardization. The data will need to be processed in such a manner that it is de-identified and may be disseminated on the All of Us Researcher Workbench to increase research utility. There may be cases in which some of the data is to be returned to study participants. There may be some cases where the offeror is linking in new data for an existing source of data that is part of the All of Us Research Program. For example, EHR is currently collected on All of Us. The CLAD award has a specific milestone to pilot linking in EHR data from HIN/HIE.

Question 84. Will the CLAD data linkages be stored in conjunction with the existing data, or will it be connected at the Researcher Workbench?

CLAD acquired data will be transferred to the All of Us Data and Research Center (DRC). The DRC will take the final steps to move this data to the All of Us Researcher Workbench for dissemination to researchers.

Question 85. Will the Occupational History and Residential History data to acquire be specified, or is the role of the offeror to identify and the appropriate dataset(s) to integrate?

At this time All of Us is not requiring a linkage for occupational history during the base period.

For the base period linkages, All of Us will require the awardee to complete a requirements-gathering stage as part of each linkage milestone to determine the best source of data for the linkage. Where NIH has a preferred source for data, this will be specified in the ROA. Please refer to the ROA to see the list of specific data linkages that are to be completed during the base period. Note, for the proposal, awardees should specify the source of data that the offeror thinks is best to satisfy the data linkage requirements.

Question 86. How will the geocodes and residential history be used?

Geocodes may be used during the base period of the CLAD award to link in the environmental data from the CDC. It may also be needed for the HIN/HIE pilot.

The procedures for geocoding may encounter some challenges with accurately linking in specific levels of geography. For example, this may occur in rural areas, where there are irregular roads or under other circumstances. Consequently, the offeror should include in their response the dataset and methods that will be used to ensure accurate multi-level geocodes.

Question 87. Are these linkage data continually refreshed?

The data linkages will be refreshed based on scientific need, research utility, and available funding.

Question 88. Will the All of Us Research Program only do the linkages specified in the base period or will there be different linkages conducted during the optional period?

It is the intention of the All of Us Research Program to expand the availability of data on the Researcher Workbench with new data. During the base period several data linkages have been specified. It is anticipated that during the optional years of the award that additional linkages will be made. The specific linkages will depend on the scientific priorities of the program. Potential examples of additional data could include cancer registry data, immunization registry data, and additional environmental data.

Question 89. How should offerors provide costs for the optional periods if the linkages are not specified?

Cost estimates for optional periods are not being requested in Phase I proposals but may be requested in Phase III for planning and budgeting purposes. The Phase III offeror may be requested to provide an estimate for the optional periods by assuming each of the linkages will be repeated during each optional period. This should also include the cost of documenting these linkages and the update to the tools that were created during the base period. It should also include the cost of operating and maintaining the data linkage platform.

Question 90. What should an offeror use for data…