Question Responses_1.docx

Commercial Data Sets Questions and Answers

1. In regards to the requirement below, is there a contract associated to the requirement? If yes, can you kindly provide the incumbent contractor and incumbent contract number?

ANSWER: The current contract number is N00014-19-C-4004. The incumbent contractor is Digital Science and Research Solutions Inc.

2. In section 4, are the types of access ordered in rank of priority? Or are the methods of access equal in priority?

a. For the types of data of interest, is this listed in priority order, i.e. S&T publications is of highest priority? (Under 4. Requirements, Types of Data of Interest)

ANSWER: At a minimum, a proposal must include access to S&T publications, domestic and international grants, patents and patent transactions, and business startup activities. These four are the priority.

3. In section 4, are all types of access a requirement? API, platform, and Bulk access.

ANSWER: We are looking for creative responses on how to provide this type of access and are open to suggestions on how to get access and stIll fit within budget.

4. For the three types of data access is this listed in priority order i.e. API access is of highest priority?

ANSWER: Yes

5.. What is the anticipated effective date of award?

ANSWER: Barring any unforeseen circumstances anticipated award will be NLT 30 September 2020.

6. In section 4, For the derived work that “must be shareable with users across the DoD S&T organizations…” please can you define the scope of this e.g. static reports, limited data extracts etc.?

ANSWER: This include all static reports created using the data along with limited access to the data behind the reports within reason. For example, if doing a study on a technology that is based on 20,000 records, we would want to be able to share the data behind the study. However, if doing a study involving 2,000,000 records, we recognize that sharing this amount of data may be problematic to vendors.

7. For the “vendor supported Boolean-enabled, search-based user interface” how many analysts will initially (Yr 1) require access?

ANSWER: We anticipate the majority of users will use the API access, it is anticipated that there will be cases where some require additional access. At this point, we anticipate that access in year 1 to be between 20-40 users but anticipate that future users would grow and would be interested in options that flexible to allow for growth.

8. For the bulk data download please indicate what volumes of data this must support?

ANSWER: At a minimum, publications, grants and patents. This is only for a very limited amount of users needed to perform our annual horizon scans.

FOLLOW UP ANSWER: One example of the type of volume for bulk data download would be annual horizon scan which downloaded all data for a 5 year time period (10M-20M records) to run against a machine learning algorithm. Another example is to show how material science evolved over a 20 year period which needed about 5 million records to analyze.

9. For the Analytical Support services is this in-addition to the requirement of provision of 2 FTEs or are the listed types of analytical support to be performed by the 2 FTEs?

ANSWER: The listed types of analytical support is to be performed by the 2 FTEs

10. If the Analytical Support services are in-addition the 2 FTEs, then please provide estimated annual man days and how this should be priced as CLIN 0002 is for firmed fixed pricing?

ANSWER: See above.

11. What is the range of Budget for the Types of data of interest?

ANSWER: The Government is looking for Best Value to obtain Best Data to fit our Requirement this solicitation is being amended to include a NTE monthly amount.

12. What is the range of Budget for the Analytical Services?

ANSWER: The Government is looking for Best Value to obtain Best Analytical Services to fit our Requirement. This solicitation is being amended to include a NTE monthly amount.

13. What is ONR’s priority for financial data? Can examples be provided of what ONR would like to pursue?

ANSWER: With the exception of VC data, the financial data is in the second tier (not required but encouraged) priority. Examples of this data would include but not limited to mergers and acquisition data, company subsidiaries, other names the company goes by, financial health of a company, and foreign ownership.

14. What does ONR view as the daily and weekly activities of the FTE’s?

ANSWER: Activities include supporting the integration of ONR’s system with the vendor’s API, providing analysis on data for specific studies that may not be included in the general award, serving as a subject matter expert in metrics derived from vendor data, and working with the team to help perform technology assessments and horizon scans on an as needed basis.

15. ONR has indicated API integration is the preferred method of integration and indicated “ONR is looking for solutions that allow for unlimited access from our internal MDM server with either flexible access fee”. Are there approximate volumes of searches by data type for the API Integration, ie: Company Details, Publications, Patents that ONR can provide to assist with providing relevant usage tiers?

ANSWER: It is difficult to provide an accurate answer to this question other than it is anticipated that this will grow over time which is why we are looking at a flexible pricing model based on usage. One of the main use cases is for a program officer to monitor the health of their portfolio which would vary between 5 to 50 performers and they would like to see all details around these performers on a dashboard integrated in with internal ONR data. This could be a daily or weekly view that may be used and is anticipated as the most common use case for the over 150 program officers at ONR but as this option is currently not provided to them nor at other federal agencies, we do not have a model that predicts the usage. However, as ONR becomes more sophisticated with their usage of the data, it is anticipated this will grow beyond that use case and they will utilize the data for a variety of other uses as they manage their portfolio.

16. Based on the existing usage description, we anticipate the ability to provide several options of content sets at several price points. Is that approach acceptable for ONR?

ANSWER: Yes and encouraged.

17. PWS (P3): Beyond analytic support addressed in P4 and P8 further in the document, can you elaborate on what “technical support for that data” would include?

ANSWER: This includes any support to further enhance, clean, interpret, aggregate or report the data beyond those offered from normal data access methodologies (for example via APIs or web-based)

18. PWS(P4): ONR indicated interest in eight (8) data categories. Can the government please address the following questions:

a. In P2, ONR has an ongoing pilot contract providing three of the eight data types (S&T Pub, Dom/Int’l Grants, and Patents Info). Is this solicitation (1) to provide the same sources to additional users or (2) provide additional sources in these categories to complement existing data access?

ANSWER: This solicitation is intended to provide the current three data types plus additional data types to a broader number of users.

b. Can the government provide the data sources for the three data categories from the 2019 ONR pilot?

0. ANSWER: The pilot was with Digital Science’s Dimension platform. The pilot will end on the date of award of this contract.

c. Is the government interested in other data categories not mentioned in the PWS related to S&T data? Some examples include: i. News articles: which address personnel, company, technology, and country related activities at a quicker refresh rate and often a leading indicator for data forthcoming in the other categories.

1 ii. Social media: for insights into personnel and companies 1 iii. Meta S&T publication research by topic or researcher networks (e.g., Scopus, WoS)

ANSWER: Meta S&T publication research is covered in the call for publication data as mentioned in the RFP. Any additional data sources proposed will be evaluated. This solicitation will be evaluated as Best Value to the Government.

19. PWS(P3): the government specified in its solicitation a non-user-based subscription. Can the government provide some insights into expected activity in the following examples:

a. For the 10-20 “power users”, what is the average query size (# of entities) performed daily?

0. ANSWER: Data is not typically done on a “daily” basis from power users. Usually this is accessed on an ad hoc basis to perform tech watch activities. Queries can range from 10s of thousands of records to hundreds of thousands of records.

b. The PWS stated “the majority of the analyses will require much smaller amounts of data such as looking at 10s to 1,000s of entities”. What is the anticipated # of entities queried when “large amounts of data” are required and how often do you expect this to occur?

0. ANSWER: An example of a large query would be to look at all nanotechnology research in the past 5 years. Such a query could produce a million records.

c. The PWS stated “modest data export capabilities (i.e., 10k-50k records via a browser-based portal)”. Is that expected daily? Weekly? Monthly?

ANSWER: It is expected for this to not be daily but to be on an ad hoc basis.

20. PWS(P4): With regards to historic data access, five or twenty years:

a. Is this a requirement for each data category?

ANSWER: The minimum of 5 years with a preference for greater historical data.

b. Our experience is some sources in these data categories may not support queries of information one to two decades in the past. Can you provide some specificity

ANSWER: We recognize that not all data sources are created equal. It is difficult to limit ahead of time what research areas. ONR covers a wide range of technologies as outlined in our S&T Framework. If a technology fits within that framework, then most likely we are interested in it.

21. PWS(P4): With regards to bulk download for ML or other NLP-based algorithms:

0. a. Is this capability expected for every data category?

0. b. The term “….updated quarterly or yearly” implies a fixed query refreshed at some time interval. Can the government provide an example of such a query? Was this capability exercised in the 2019 ONR pilot contract?

0.

0. ANSWER: (for a&b) An example of why this is needed is the annual horizon scan that ONR has been doing for the past 2 years. Previously it looked at everyone grant, publication and patent issued over the past 5 years and put this into a topic model that would be used to identify new and emerging S&T areas that may be of interest to ONR. Last years, the topic model included over 14 million records.

22. PWS(P8): The government stipulates “costs incurred to date” and “labor hour worked per labor category” as part of Month FSR (CDRL A002), yet Section L(1) of the Solicitation Instructions indicate this is a Firm Fixed Price (FFP) Task order? Those items seem to be in conflict – can the government please address?

ANSWER: The Government is amending the CDRLS to remove Monthly FSR CDRL A002.

23. Is this solicitation a net new capability/effort or the re-compete of existing work?

ANSWER: Expansion/New capability If the former, can you provide an IGCE (Independent Government Cost Estimate)?

ANSWER: The Government is not at liberty to provide an IGCE.

24. On page 5 of the solicitation, CLIN 0001 seems to indicate that the subscription service should be fully available 10 days after contract award. However, CDRL item A002 identifies an overview and implementation plan. Do you envision this as a ready-made subscription, or are you envisioning this as a development process with evolving capabilities?

ANSWER: It is anticipated that the subscriptions are available upon award completion but that levels of access and integration will evolve over time.

25. The solicitation identifies a number of categories of data that it would like to have access to as part of this project. Many of these subscriptions have fees. So that we may provide a proper price proposal, please identify if ONR CDA&AS currently has subscriptions to any services that we could leverage.

ANSWER: ONR has no subscriptions that can be leveraged for this.

26. Based on what is written regarding data access solutions required, specifically the requirement of unlimited access to the database from the internal MDM server - does this mean we would build an API for non ONR personnel to access the internal database? Would you envision a separate fee for them?

ANSWER: ONR is asking for API access. We are asking for proposers to come up with a fee structure that allows for maximum flexibility to allow for growth over time.

Follow up Question: ONR's answers to questions 26 and 30 indicate that the Navy is looking for a fee structure that provides "maximum flexibility” and "creative pricing”. In contrast, the RFQ indicates you are seeking a firm fixed price. Should we be expecting an update to the solicitation that allows for other pricing (other than FFP)? If this update to the solicitation is being considered or planned, will there be an extension to the deadline for submission?

FOLLOW UP ANSWER: This solicitation is looking for ways to ensure maximum flexibility and creative pricing on how vendors propose to allow for API access data rates balanced with the other ways of accessing the data as outlined in the RFQ. One example, but not limited to, is a similar pricing structure that cloud compute uses in which there is a firm fixed price structure on the price of compute with either a monthly or annual cap on how much money can be applied to this.

The Government has decided to amend the solicitation to add a NTE monthly amount for Data and extend the deadline date for Quotes due an additional 7 days.

27. Regarding the Boolean-enabled search, is this the only type of search you are interested in or is it possible to incorporate a different type of search which is instead AI powered?

ANSWER: We are interested in other proven approaches for search but require at the minimum Boolean capabilities.

28. Do you have any preferences or restrictions on cloud storage of the data collection?

ANSWER: It is not anticipated that the government will be hosting the data but only derived data via access either via the API and/or web interface. For the limited number of users who need full access to the data, we are flexible on how that is provided.

29. In the last paragraph of section 4, you cite the need for shared access across the DOD S&T organizations. Do you envision this being provided via API, sign-on, or another approach?

ANSWER: We envision that majority of usage being API and would want to limit as much as possible any accounts require sign-on.

30. Access and volume is a driver of the cost of data, particularly of historical data. Could you please expand on the need, frequency, and use cases for historical data?

ANSWER: We recognize these as the primary cost drivers and are looking for creative pricing that is flexible based on usage as we cannot accurately predict the usage ahead of time.

Follow up Question: ONR's answers to questions 26 and 30 indicate that the Navy is looking for a fee structure that provides "maximum flexibility” and "creative pricing”. In contrast, the RFQ indicates you are seeking a firm fixed price. Should we be expecting an update to the solicitation that allows for other pricing (other than FFP)? If this update to the solicitation is being considered or planned, will there be an extension to the deadline for submission?

FOLLOW UP ANSWER: This solicitation is looking for ways to ensure maximum flexibility and creative pricing on how vendors propose to allow for API access data rates balanced with the other ways of accessing the data as outlined in the RFQ. One example, but not limited to, is a similar pricing structure that cloud compute uses in which there is a firm fixed price structure on the price of compute with either a monthly or annual cap on how much money can be applied to this.

The Government has decided to amend the solicitation to add a NTE monthly amount for Data and extend the deadline date for Quotes due an additional 7 days.

31. Are visual graphics allowed in the body of the technical approach to assist in explaining the dashboard layout?

ANSWER: Visual graphics are encouraged but will be included towards the page limit.

32. Which databases are currently used to store files? What is the structure of the data?

ANSWER: It is anticipated that the proposer will be storing the data and making it available to the government.

33. Is the plan for weekly meetings changed with respect to Coronavirus health requirements? Is it possible to meet online instead?

ANSWER: ONR is encouraging maximum telework during our response to COVID and currently all interactions are virtual through DoD’s Microsoft Teams Environment.

34. Is an anticipated funding profile per year going to be published?

ANSWER: Solicitation is being amended to include a NTE monthly amount.

35. If not a full funding profile, is there perhaps an anticipated Rough Order of Magnitude (ROM)?

ANSWER: Solicitation is being amended to include a NTE monthly amount.

36. Other than the price quote, should all other information fit within the 12 page limit or is there any necessary information beside price that would be an exception to the 12 page limit?

ANSWER: If you chose to submit resumes they would not be counted within the 12 pages

37. Are resumes needed?

ANSWER: Resumes are not a requirement however welcome and could be submitted.

38. If resumes are needed, should they be a part of the Technical Volume?

ANSWER: Resumes are not a requirement however, if you chose to submit they should be added to the Technical Volume.

39. If resumes should be a part of the Technical Volume, would resumes count against the page limit?

ANSWER: Resumes will not be counted toward the page limit.

40. The solicitation document (Item 14) states the method of solicitation as RFP; however, amendment 0001 Section L refers to the method of solicitation as an RFQ. Can the government please clarify

ANSWER: This solicitation should be considered a RFQ. Solicitation is being amended to correct.

41. What is the ceiling for this contract?

ANSWER: Solicitation will be modified to include a NTE Amount for Data CLINS

42. Can the Government provide the following “To Be Determined” details on page 4 of the solicitation document: “The total amount of the base and exercised options is [To Be Determined]. The total potential amount for this action is [To Be Determined].”

ANSWER: The Government cannot provide details at this time. However the solicitation is being amended to include NTE Monthly amounts for Data.

43. Who are key personnel? Page 5 of the solicitation document says, “The following are identified as key personnel: [ ]” Would the government complete this sentence?

ANSWER: See section 5 in the PWS: “It is expected that at least 50% of this will include a dedicated Senior Data Scientist who will be considered key personnel”

44. The solicitation states that deliverables for CLINs 0001 and 0002 shall be delivered no later than 10 after award, yet the PWS stipulates that, “upon request by the government, the Contractor shall participate in a Contract Kick-Off Meeting with ONR personnel and other Government representatives within twenty (20) business days after award to review and discuss contract administrative matters, security requirements, the milestone schedule, invoicing, and other applicable matters.” Would the Government consider changing the deliverable schedule in the solicitation document from 10 days to 20 days after award to align with the PWS kickoff schedule?

ANSWER: The solicitation will be changed to match the kick-off meeting schedule

45. Does the ONR Enterprise-Wide Contractor Manpower Reporting Application (ECMRA) require a certified timekeeping system required to report labor hours?

ANSWER: No a certified time keeping system is not necessary

46. Will the Government allow application of Defense Federal Acquisition Regulation (DFAR) clause 227.7202?

ANSWER: This section of the DFAR’s only applies to this procurement in regards to the documentation of the API that will be provided to the government.

47. Would the Government like offerors to submit Volumes 1 and 2 under a cover sheet (not to be included in the page count) that includes Tax ID Number, CAGE Code, DUNS, and Business Size?

ANSWER: This would be helpful to the Government but has not been listed as a requirement.

48. Should offerors include the resume(s) of proposed Key Personnel? If so, will the Government exclude Key Personnel resumes from the 12-page limit in the technical volume?

ANSWER: Resumes are not required but would be welcome. They will not count toward the page limit.

49. Regarding the pricing model for API Access, the PWS states, “ONR is looking for solutions that allow for unlimited access from our internal MDM server with either flexible pricing based on data usage vs a flat access fee.” Can the Government clarify its preference? Are both flexible pricing and a flat access fee acceptable? Would the Government prefer flexible pricing rather than a flat access fee? Or is there a third pricing model that the Government prefers?

ANSWER: We recognize we are asking for a different way to access the data and are encouraging responders to respond with creative pricing options for accessing this data in the way outlined.

50. The PWS states that “a limited subset of analysts” will require access to a vendor-supported Boolean-enabled, search-based user interface with modest data export capabilities. Approximately how many analysts will require access in the base year?

ANSWER: We do not have a hard number of users but it is anticipated that the majority of users will access the data via the API.

51. The PWS states that ONR expects the number of analysts who require access to a vendor-supported Boolean-enabled, search-based user interface with modest data export capabilities to “grow over time.” What growth rate of analysts does the Government expect after the base year?

ANSWER: Growth will be dependent on how successful we are in providing this data in a way that is easy to use and fits within current business workflows. At this time, we do have a solid projection on how quickly it will be adopted.

52. The PWS states that ONR expects a “maximum of 10 named users in year 1” to require access to the bulk data download. Does ONR expect the number of named users requiring access to the bulk data download to increase in the option years? If so, by how many?

ANSWER: We do not anticipate this number growing.

53. The PWS states, “ONR also requires personnel to support analytics, not to exceed 2.0 FTEs. It is expected that at least 50% of this will include a dedicated Senior Data Scientist who will be considered key personnel.” Does this mean that the Government expects offerors to propose at least 1 FTE of a dedicated Senior Data Scientist to serve as Key Personnel?

ANSWER: We expect a minimum of 50% of the total personnel to be a Senior Data Scientist. If the proposer believes that only 1 FTE is required to support ONR, then a minimum of 0.5 FTE of that 1 FTE should be a Senior Data Scientist

54. To the extent that you can, would you make clear which capabilities you expect to be complete and available on day 10, versus those who are willing to see developed over the first year of the contract?

ANSWER: On Day 10, we expect the vendor to have the core datasets listed in the RFP to be available to ONR in some format. By year 1, we expect the data to be fully integrated into ONR’s data and analytic environment via the API. We encourage partnering with the data vendors that would not normally submit to this RFP because by themselves, they do not have all the data sets and do not have existing partnering relations with other data vendors who would give them a complete response to our RFP.