Mod_2_Translator_Mod_and_questions_corrected.pdf

Solicitation Title: Biomedical Data Translator User Interface Development

Amendment of Solicitation no. 75N95021R00018

Amendment no.: 02

Amendment date: August 6, 2020

Due date for proposals: August 16, 2021 at 11:00 a.m. Eastern time (UNCHANGED)

Issued by:

National Center for Advancing Translational Sciences, NIH NIDA Office of Acquisition, NCATS Section 3WFN Rm 03C18, MSC 6012 301 North Stonestreet Ave.

Bethesda, MD 20892

Primary Point of Contact:

Stuart Kern, Contract Specialist Email: stuart.kern@nih.gov

TO ALL OFFERORS:

The above-numbered solicitation is amended as set forth below. Offerors must acknowledge receipt of this amendment on each copy of the offer submitted. FAILURE OF YOUR ACKNOWLEDGMENT TO BE RECEIVED AT THE PLACE DESIGNATED FOR THE RECEIPT OF OFFERS PRIOR TO THE HOUR AND DATE SPECIFIED MAY RESULT IN REJECTION OF YOUR OFFER. If by virtue of this amendment you desire to change an offer already submitted, such change may be made by letter, provided each letter makes reference to the solicitation and this amendment, and is received prior to the time and date specified for receipt of proposals.

PURPOSE: to respond to formal questions submitted in response to the solicitation.

All other terms and conditions of the RFP remain unchanged.

RESPONSES TO QUESTIONS

The Government has provided answers below. Note that many questions were combined, paraphrased, and/or summarized for brevity and clarity.

IMPORTANT: Offerors are reminded to preregister for the NIH electronic Contract Proposal Submission (eCPS). Creating a new account to submit may take up to three business days. Please apply for a new account early to allow enough time for the registration process. Proposals must be submitted via eCPS, and eCPS does not accept proposals after the due date and time (See Attachment no. 1, Packaging and Delivery of Proposals for Use with the NIH electronic Contract Proposal Submission (eCPS) Website).

1. Question: The special contract requirements state human subjects research will not be conducted under this contract, however OMB clearance is required for any user surveys or mailto:stuart.kern@nih.gov

NCATS Cybersecurity Services Questions and Answers 75N95020R00009 interviews. Also, the statement of work requires user research. Shall we expect that human subjects research (to include IRB approvals, which is typical for UI development) is in fact needed?

Answer: PRA does require OMB clearance for surveys of more than 9 people, however, it is possible to obtain an NIH Research Exemption to these requirements for user-research. The Translator team will obtain the NIH research exemption for PRA on the contractor's behalf.

2. Question: The UI contract is to deliver “an intuitive user interface”. The solicitation states Translator “need not present these answers in classic knowledge graph form”. Though the Translator tool itself uses knowledge graphs, is it acceptable to design the UI so that end users may employ these or other representations as appropriate to their research questions?

Answer: From the background: The development of a user interface poses several distinct design challenges for the developer, including but not limited to:

• presentation of multiple complex knowledge graph “answers” in response to a single query;

• establishing confidence in the ability of Translator to address the user’s question by providing provenance of the knowledge returned to the user;

• capturing the collection and history of user decisions used to formulate, iterate, filter and merge results as they develop their query; and

• • gathering feedback from the user.

Our objective is to develop a user interface that facilitates biomedical researchers interactively exploring Translator resources and constructing new research hypotheses while ensuring that the emerging standards and development of the Translator component tools support user needs.

This may include knowledge graph representations, but should be utilized based on user research for this contract.

3. Question: A related question to the intuitive user interface is the requirement that the “visualizations presented will need to optimize cognitive fit and maximize users’ working memory”. Is there a formal definition for these terms, and/or a requirement to measure cognitive fit and working memory usage? There appears to be a need to utilize cognitive science to “facilitate meaningful and successful interpretation of” these constructs.

Answer: Please review the following articles:

The application of cognitive psychology to User-Interface Design

What cognitive science taught me about design

A review of visual memory capacity: Beyond individual items and towards structured representations https://uxdesign.cc/the-application-of-cogntive-psychology-to-user-interface-design-81599ad7fb55 https://www.invisionapp.com/inside-design/cognitive-science-design/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3405498/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3405498/

4. Question: One design requirement is to “capture the collection and history of user decisions” to tailor to that end user. What requirement is there to maintain state of a user’s queries--one session, over multiple sessions, etc.?

Answer: This requirement should be driven by the user research associated with the contract.

5. Question: Will NCATS or the contractor host the MVP for testing? If NCATS will host, are there technology platform compatibility requirements?

Answer: Yes, NCATS ITRB DevOps hosts Translator. The presentation “NCATS-Stack-for- Translator.pptx”, included on SAM.gov with this notice, lists the current technologies used.

6. Question: Is there a specified labor rate schedule for this opportunity?

Answer: No. Labor categories and fixed hourly rates will be agreed upon during discussions.

Fixed hourly rates include wages, overhead, general and administrative expenses, and profit.

See FAR 16.601.

7. Question: Can you share any examples of “templated” user questions?

Answer: Templated questions would be re-usable TRAPI queries with an intermediate node(s), such as:

Ex 1:

<Disease> <any relationship> <NamedThing> <any relationship> <Disease> <any relationship> <ChemicalSubstance> <any relationship> <Protein> <any relationship> <MolecularActivity> <any relationship> <MolecularActivity> <any relationship > <ChemicalSubstance>

Ex 2:

<Disease> <any relationship> <NamedThing> <any relationship> <Disease>

8. Question: Is there an existing user research panel? Will the contractor be responsible for recruiting user research participants? WIll NCATS facilitate identification of user research participants?

Answer: There is no existing user research panel. The contractor will be responsible for recruiting user research participants. NCATS will provide a list of characteristics of the ideal users. NCATS will evaluate appropriateness of candidate users and provide guidance as needed.

9. Question: Will the contractor be responsible for back-end tools to support the UI? e.g.

data/feedback storage server, proxy/buffer server, name resolution, ...

Answer: The contractor will not be responsible for back-end tools to support the UI. But will be responsible for defining the requirement of the interaction/integration with the ARS.

Deliverable from the SOW: Development of production quality open-source interface that directly interacts primarily with the Autonomous Relay System and other Translator software components.

10. Question: Will the contractor be responsible for developing and/or integrating with natural language processing (NLP) algorithms/tools to support the UI?

Answer: The requirement to Integrate with natural language processing (NLP) algorithms/tools to support the UI should be driven by the user research associated with this contract.

11. Question: Do we know whether the software will require user sessions/login/authentication?

The document “3.Translator and UI Background.pptx” indicates not, but might this be at odds with the goals of “collaboration with other users”, “iteration”, and/or “feedback”?

Answer: Initially, Translator will support anonymous sessions such that when the user first comes to the site, Translator will create a unique random number associated with the session/query, thereafter as long as that URL is being used, that URL will initiate the required session and track the query. A previous example is from the Feasibility Phase FOA Challenge, where teams were assigned random identifiers for re-use.

In the future, support for user accounts may indeed be necessary.

12. Question: Can the UI rely on third-party paid services? e.g. auth0, Plotly, etc.

Answer: All software and other tools used for this entirety of the Translator program must be open source and publicly available. See Article B.5. of the Schedule.

13. Question: What is the maximum page count of the Technical Proposal?

https://github.com/ncats/foa-challenge

Answer: Ten (10) pages.

14. Question: What is the maximum page count for each CV?

Answer: Five (5) pages. NIH Biosketch format is acceptable, but not required.

15. Question: Is subcontracting or consulting with a Canadian company permitted?

Answer: Use of a foreign company for consulting or subcontracting is not prohibited.

16. Question: There is a requirement for doing user studies to develop the interface. Interviews and surveys can fall into human subject research. What is the expectation for how to handle this since it says no human research can be funded?

Answer: PRA does require OMB clearance for surveys of more than 9 people, however, it is possible to obtain an NIH Research Exemption to these requirements for user-research. The Translator team will obtain the NIH research exemption for PRA on the contractor's behalf.

Nothing Follows.