86614823R00007 HUD Exchange Management Platform Q and A.pdf

SOLICITATION - 86614823R00007

HUD EXCHANGE MANAGEMENT PLATFORM – QUESTIONS AND ANSWERS

Question Number

Section Question Response

1.

5.2.2 Cloud

Services

Can HUD please clarify where the existing Exchange platform is hosted? If it is cloud hosted can you provide the service provider and region?

Please refer to the system description attached to the PWS.

2.

5.3.1.2 Content

Migration

If HUD is satisfied with the current hosting environment, would it be acceptable for the awardee to assume ownership of the existing environment rather than introducing the risk (and additional cost) of migrating content to a new environment?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

3.

5.3.1.2 Content

Migration

If the incumbent environment is cloud hosted, would HUD consider migrating content to HUD's internal cloud environment rather than the new contractor's environment to increase governance controls?

The process for moving to a HUD environment is anticipated by this requirement. The transfer will not be complete prior to contract award.

4.

5.4 Website

improvements

With recent advances in Generative AI, would HUD be open to using Retrieval- Augmented Generation (RAG) to provide users with a conversational interface where answers could be generated from the existing library of HUDx documentation?

HUD expects the website improvement plan for HUD Exchange will be informed by user experience research.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the

5. Could you confirm if this is a new requirement or a follow-on project?

This is a New Requirement building from an existing project funded under a Cooperative Agreement.

6. If so, could you kindly provide details such as the incumbent's name, current contract number, period of performance, and the value of the existing contract?

The incumbent operating under a Cooperative Agreement is ICF International, Inc. The approximate period of performance and value associated with the scope of work relating to the tasks outlined in this PWS is 12 months and $6 million.

The scope of work established under the present Cooperative Agreement and the tasks outlined in this PWS measurably differ.

7. Can you please tell me if there is an incumbent? If so, contract information?

The incumbent operating under a Cooperative Agreement is ICF International, Inc. The approximate period of performance and value associated with the scope of work relating to the tasks outlined in this PWS is 12 months and $6 million.

The scope of work established under the present Cooperative Agreement and the tasks outlined in this PWS measurably differ.

8. Cloud hosting provider (AWS, Azure)

• Content Management System (CMS) in use. Drupal?

• Learning Management System in use

• Current contractor (incumbent)

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

9. Scope and Requirements for Website Hosting and Administration:

Please refer to the current system description attached to the PWS.

• Can you provide detailed specifications for the website hosting requirements, such as bandwidth, storage, and security needs?

• Are there specific platforms or technologies that HUD prefers for website hosting and administration?

• How does HUD envision the role of the service provider in ongoing website maintenance and administration?

The Government will assess the software, systems, and services proposed in the technical

10. Website Operations:

• What are the expected traffic volumes and user load for the website?

• Are there any specific uptime or reliability targets that the website must meet?

• How will the website's performance be monitored and reported?

In January of 2024, HUD Exchange users initiated 516,270 sessions, with an average of

3.33 page views per session

Please see 5.2 and 5.3 of the solicitation

Please see 5.1 of the solicitation.

11. Website Improvements:

• Are there any planned or anticipated upgrades or improvements that should be factored into the proposal?

All discussion of website improvements are detailed in 5.4 of the solicitation.

12. • How does HUD define and measure the success of website improvements?

All definitions and measures of success of website improvements are detailed in 5.4 of the solicitation.

13. • Will the contractor be expected to provide suggestions and strategies for continuous improvement?

Yes, this is detailed in 5.4 of the solicitation.

14. Marketing:

• What are the primary goals and target audiences for the website's marketing efforts?

The Marketing plan is detailed in 5.5 of the solicitation and provides this information.

15.

Are there existing marketing strategies or campaigns that need to be integrated or considered?

Per 5.5.1 of the solicitation: In developing the marketing plan, the Contractor shall consider how other organizations advertise technical assistance resources and input from TA providers, HUD staff involved in TA, and HUD’s Office of Public Affairs.

16. • How will the effectiveness of marketing initiatives be evaluated?

All performance standards are detailed in 5.5 of the solicitation for evaluating the marketing initiatives.

17.

Project Management (Reports and Support):

• What type of reporting is expected from the contractor regarding project management, and how frequently?

Section 5.1 of the solicitation details the reports required for project management.

18. • How does HUD want to handle communication and coordination between its team and the contractor's project management team?

Any communication and coordination with HUD shall be done via the COR for the project.

19. • Are there any specific project management tools or methodologies that HUD prefers?

No.

20. Is there any Enterprise API Gateway Platform used?

• Google APIGEE, MuleSoft, AWS API Gateway, API Umbrella etc?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical

21. • HUD Exchange Ecosystem?

o How many microservices are used? Is there any monolithic application onboarded using HUD Exchange

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

22. • We believe it’s hosted in Cloud. Is it AWS or Azure or GCP?

o What kind of "R" disposition

(Refactored, Rehost, Replatform etc)

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

23. What kind of Content Management Solution HUD Exchange is?

o EMC Documentum, Drupal, Acquia, Plone etc?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

24. Is Microservices are Single Page Application (SPA) if Yes, what is technology stack??

o Angular, React, Node/ExpressJs, jQuery etc

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the

25. Backend Microservices?

o Is it Spring boot, Serverless, Containers based ecosystem using ECS, EKS, Docker Containers?

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the

26. What kind of backend eco system?

o For SQL RDS (Aurora, PostgreSQL, MySQL, Oracle etc) o For NoSQL DynamoDB, MongoDB, Cosmos DB?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

27. List of Nonfunctional requirements o Observability, Logging, Auditing, Monitoring and Distributed Tracing (Splunk, ELK, Datadog RUM, Prometheus, Open Telemetry, Xray etc)

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical

28. How many COTS products, Low code/No code platform usage currently at HUD Exchange

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

29. Software Development and Maintenance Environment?

o Agile?

o SAFe?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical

30. CI/CD process?

o GitHub, GitLab, AWS Code Pipeline

(Code Commit, CodeBuild, CodeDeploy)

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

31. Deployment Cadence o Zero Down Time Deployment o Blue / Green Deployment o Canary o Toggle Features etc?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

32. Compliance / Security?

o NPI, PII, NIST, Fed Ramp etc?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

33. Incident Management o ServiceNow, Splunk, Big Panda, NewRelic?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

34. Documentation Tools o SharePoint, Confluence etc?

Please refer to the current system description attached to the PWS.

35. Architecture Governance and Framework o TOGAF/ADM, FEAF etc

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

36. Are you only looking for businesses that have prior experience with government contracts only? We have over 15 years’ experience in Software development, most recently in non-governmental contracts.

No.

37. I just wanted to confirm that the TA application is a separate application from the HUD Exchange application.

The TA application is a separate application from the HUD Exchange contract.

38. For the website operation, can you specify how many user requests/issues are received on a daily or weekly basis?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

39. Can I ask what is the Technology stack of the HUD Exchange Platform.

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

40. There is a reference to Parts I and II, with what appears to be 3 separate volumes

Technical Proposal is Part I. Please see the revised Solicitation.

requested in the solicitation, with only the technical being labeled as Part I. Could the government confirm that the format of our response is meant to consist of the three documents below?

Part I – Technical Quote (limited to 50 pages)

Part II – Past Performance (limited to 3 pages per project write-up)

Part III – Price Quote

41. Could offers use a font size smaller than 12-point for tables, charts, and graphics (e.g., 10-point font)?

Yes, 10-point for tables, charts and graphics.

42. Could the government confirm that the Proposal Matrix referenced on page 73 is not included in the 50-page limitation for the technical proposal response?

Yes, the Proposal Matrix is included in the 50-page limitation.

Please see revised Solicitation

43. contractor has performed within three years since only the first 5 past performances listed will be evaluated.

No response needed.

44.

Solicitation

On page 81, as one of the four components of the past performance evaluation, it states “HUD will then determine an overall level of sufficiency of the past performance information and assess the risk of nonperformance based on the degree of relevancy and level of quality of all recent past performance references.” Could the government clarify what it is seeking to evaluate under the sufficiency component?

The Government will determine whether the information submitted to demonstrate past performance is sufficient to be able to be assessed for relevance and quality.

45.

Can the government confirm that by using the tradeoff process for source selection, that when considering award in the best interests of the government and HUD customers, that technical quality will be scored by a holistic selection team that will include the program office officials to include experts with the knowledge and capacity to determine technical excellence, management skills and qualification of key personnel?

A qualified technical evaluation panel will rate the proposals.

46.

What criteria will the government use to assess the risk in a proposer’s technical approach?

The Government will use a best value trade-off selection, consistent with the terms of the solicitation. A primary measure of performance risk will be the relevance and quality of past performance.

47.

Using an Acceptable/Unacceptable evaluation of the Technical Approach and not including the Technical Approach in the best value trade-off analysis would appear to encourage vendors to propose the bare minimum acceptable quality in order to drive down price. Is this the government’s intent?

The Government will include an evaluation of the Technical Approach in the best value trade-off analysis.

48.

Will the quality of the proposer’s technical approach be evaluated in any way other than assessing bare minimum acceptability in meeting the requirements of the solicitation?

In other words, will the technical approach be considered as a part of the best value trade off analysis?

The Government will include an evaluation of the Technical Approach in the best value trade-off analysis.

49.

Understanding that the government can accept other than the lowest cost, has the

The Government will use a best value trade-off selection, consistent with the solicitation.

government predetermined what the non-cost factors will be in the Source Selection Plan in collaboration and/or agreement with the designated Source Selection Official?

50.

Solicitation

Proposals are to be evaluated on a best value past performance/price trade off basis. If several vendors have acceptable technical approaches and score equally on past performance, what criteria other than price will be used to assess best value?

The Government may consider the relative quality of the technical approach among respondents that achieve equal ratings for technical approach and may consider the relative relevance and quality of past performance among respondents which achieve equal ratings for past performance. This is the correct answer because of the updated eval criteria. In minimum eval criteria there isn’t a comparison it is just pass/fail.

51.

Since the government may award to other than the lowest priced offeror or other than the offeror with the highest rated past performance, what factors will the government consider in making that trade-off evaluation? For example, will the government consider factors such understanding of the work, capacity for delivery, innovation capabilities, key personnel, management capacity, or risk management expertise?

The Government will use a best value trade-off selection, consistent with the solicitation.

52. General Question

Is there an incumbent providing this level of effort (LOE)? If so, could the government please provide the recent contract award number?

The incumbent operating under a Cooperative Agreement is ICF International, Inc. The approximate period of performance and value associated with the scope of work relating to the tasks outlined in this PWS is 12 months and $6 million.

The scope of work established under the present Cooperative Agreement and the tasks outlined in this PWS measurably differ.

53. General Question

Could the government please provide the total value of the previous contract (base and options)?

Are you contractually satisfied with the incumbent?

The incumbent operating under a Cooperative Agreement is ICF International, Inc. The approximate period of performance and value associated with the scope of work relating to the tasks outlined in this PWS is 12 months and $6 million.

The scope of work established under the present Cooperative Agreement and the tasks outlined in this PWS measurably differ.

The incumbent is compliant with all terms of the Cooperative Agreement.

54. General Question

Could the government please provide an estimated Rough Order of Magnitude

(ROM)?

HUD’s ROM is $10M – $50M.

55. General

With respect to the background checks, are non-US citizens who are US permanent residents allowed to work in the HUD systems?

Yes, Permanent Residents can work on this Requirement as long as there are no adverse findings.

56. 1.3.1 Is it safe to assume that proposed methods which deliver usable functionality more quickly than 6 months will be acceptable? Is that type of acceleration preferred or are there business or mission reasons that the 6- 9 month timeframe is noted?

Yes. The delivery timeline for website improvements can be accelerated from the timeline established in the requirement. The timeline established in the requirement is intended to provide structure to the project management and to ease user adoption or adaptation to changes and improvements.

57.

4.4.1

The link referenced for Accessibility Standards does not resolve to a site. Please confirm that the intended content is at this link: https://www.ecfr.gov/current/title-

Confirmed.

36/chapter-XI/part-1194#Appendix-A-to- Part-1194

58. 4.5 Are the Technical/Technology Lead and alternate able to be employees of subcontractors?

Yes.

59. 5.1.1.4 Are the sources of data currently known and readily accessible via APIs?

No.

60. 5.1.1.4 Will the government consider measures recommended by the contractor beyond those specified?

Yes.

61. 5.1.1.4 Will the government provide resources to integrate YouTube data into the dashboard or expect the contractor to set up that integration?

The contractor is expected to set up this integration.

62. 5.1.1.4 Does HUD have an existing dashboard tool in mind? If no, will the contractor be responsible for soliciting tool selection for the dashboard?

No, HUD does not have an existing tool in mind. Yes, the contractor will be responsible for selecting and maintaining the dashboard.

63. 5.1.1.4 Does HUD already have data mapping complete or will the contractor be required to map, source, and clean the data for the dashboard?

No, HUD does not have data mapping complete. Yes, the contractor will be responsible for data aggregation and maintenance.

64. 5.1.1.4 Are KPIs limited to those indicated or can others be added?

No, KPIs are not limited to those indicated. Yes, others can be added.

65. 5.2.1 Confirm that the target availability 99.0% Yes, the target availability is 99%.

66. 5.2.2 Please clarify information around HUD

Security Policies and Standards. There are references in the RFP to "Section 5.1.5 - Security", this section does not exist in any of the RFP documents.

The references should be to Section 5.2.5.1.

67. 5.2.1 Is the current HUDx system designed for high availability or require refactoring to meet the SLA?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

68. 5.2.1 Does HUD have any existing monitoring tools that are required to be used/integrated?

Is cloud-native monitoring acceptable?

Please refer to the technical requirements in Section 5.2.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

69. 5.2.1 Is the current HUDx system to scale horizontally when adding additional servers or does it scale vertically by increasing the size of the servers?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

70. 5.2.1 Can HUD provide additional details regarding the current scale and footprint of HUD Exchange? (e.g. storage, servers, geographic footprint, etc.)

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

71. 5.2.2 How many HUD service-level managers require access to the real-time monitoring dashboard?

HUD anticipates four service-level managers will require access to the real-time monitoring dashboard.

72. 5.2.2 For Performance Standard c) What is the expected SLA or turnaround for the reporting analytics regarding technical and management performance?

73. 5.2.5.2 Does HUD envision documentation to be required to achieve an ATO beyond the list provided?

74. 5.2.9 "Confirm that the request is for the Contractor to secure and maintain the license and authority for control of www.hudexchange.gov (or alternative) AND www.hudexchange.info concurrently as noted in the solicitation document.

Yes, the Contractor shall secure and maintain both domains on behalf of the government.

75. 5.3.1.1 Is the current and target scope (number and estimated complexity) of templates known and sharable?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

76. 5.3.1.2 Is HUD able to share the scope of existing content?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

77. 5.3.1.2 Confirm that content will be first migrated to Contractor/Hosted environment and then a second migration to a HUD environment.

That is correct.

78. 5.3.1.2 Is there an estimate of the timing for the migration to the HUD environment during the contract duration (which year)?

HUD anticipates migration to the HUD environment will be available in year two or three of this contract.

79. 5.3.1.2 Is HUD looking to improve on it's existing CMS platform or implement a new platform?

HUD is seeking for HUD Exchange to use a CMS that meets or exceeds the requirement.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

80. 5.3.1.2 Is there an existing HUDx web design style guide that is intended to be adopted and modified or is the intent to create a new style guide?

As the HUD Exchange project is currently managed under a Cooperative Agreement, HUD did not prescribe and is not familiar with this specific element of the system design.

81. 5.3.1.6 Is there a different standard (greater than 1 day) for time-sensitive content updates which require creation of a new webpage or a new template?

Time-sensitive updates will not include new webpages or templates.

82. 5.3.1.8 Will contractors have access to historical data for utilization analytics comparisons (e.g. old website analytics vs. new)?

Yes.

83. 5.3.2.1 Can you provide more information regarding the current Adobe LMS licensing;

When does the current licensing period expire? What are the details regarding the current agreement (user counts, costs, etc.)?

As the HUD Exchange project is currently managed under a Cooperative Agreement, HUD is not familiar with the specifics of the license.

HUD’s understanding is that the 12-month cost is approximately $350,000 to $400,000. The license is set to be renewed in October 2024.

84. 5.3.2.1 Is HUD able to share the scope of existing LMS content and delivery? (e.g. number of users, number of courses, features currently in use, etc.)

85. 5.3.2.1.1 Is HUD looking to improve on it's existing LMS platform or implement a new platform?

HUD seeks for HUD Exchange to use an LMS that meets or exceeds the requirement.

approach to determine the best value to the Government.

86. 5.3.2.1.1 For Live Training Events does HUD have a preferred current and/or future videoconferencing platform? What is used today?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

87. 5.3.2.1.1 How satisfied are you with the current Adobe LMS? Is the expectation that the contractor will bring a new LMS to HUDx in the first 90 days?

No, that is not an expectation of this requirement.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

88. 5.3.2.4 Is it possible to quantify the average number and complexity of annual changes to the Income Calculator?

89. 5.4 Is HUD looking to improve on it's existing Website technology or implement on a new platform?

HUD seeks for HUD Exchange to meet or exceed the requirement.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

90. 5.4.2.2 Are there requirements to automate the deployment of the application and infrastructure?

No.

91. N/A Do you have dedicated resources to do change management and drive adoption?

Yes.

92. N/A Do you currently have a way to measure TA sentiment and satisfaction with changes made on HUDx?

HUD analyzes site traffic to estimate user sentiment and, via the incumbent, uses targeted user experience research in inform specific changes to the site.

HUD may employ other approaches through its Customer Experience office.

93. Multiple Sections

The government has listed in different locations within the solicitation, 3 separate NAICS codes (541419, 541511, and 518210). Which NAICS code is the correct one for the opportunity?

The Primary NAICS code is for this solicitation is 518210 Computing Infrastructure Providers, Data Processing, Web Hosting and Related Services.

94. 5.2 General

What is the Technology stack? Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

95. 5.2 General Question

"What is the current and growth plan for Network bandwidth"

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

96. 5.2 General Question

What is the current and growth plan for Compute?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical

97. 5.2 General

What is the current and growth plan for Storage?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

98. 5.2 General Question

What is the current average of Number of change tickets?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

99. 5.2 General Question

Does HUD have a formal release process? Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

100. 5.2 General Question

Does HUD have a CI/CD process? Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

101. 5.2 General Question

What is the size of the solution in terms of number of systems within the boundary?

Please refer to the current system description attached to the PWS.

102. 5.2 General Question

What is the current cloud service model? Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

103. 5.2 General Question

What web content management system is used?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the

104.

Solicitation – L8 Proposal

Content

There is a reference to Parts I and II, with what appears to be 3 separate volumes requested in the solicitation, with only the technical being labeled as Part I.

Could the government confirm that the format of our response is meant to consist of the three documents below?

Part I – Technical Quote (limited to 50 pages) Part II – Past Performance (limited to 3 pages per project write-up) Part III – Price Quote

Technical Proposal is Part I (limited to 50 pages).

Past Performance can be included in the Business Proposal, but separately identified which is Part II (limited to 3 pages).

Price Quote/Business Proposal is Part III (Amended Version 2, pricing worksheet to be filled in for each line item and each year total amounts.

105. Solicitation –

Content

Could offerors use a font size smaller than 12-point for tables, charts, and graphics (e.g., 10-point font)?

Yes, 10-point for tables, charts and graphics.

106. Solicitation –

Content

Could the government confirm that the Proposal Matrix referenced on page 73 is not included in the 50-page limitation for the technical proposal response?

The Proposal Matrix is included in the 50-page limit.

107. Solicitation – L8 Proposal

Content

Please confirm the Past Performance chart must include 5 relevant past performances within three years from the prime Contractor and 5 relevant past performances from any significant subcontractor. The chart does not need to list every relevant past performance the Contractor has performed within three years since only the first 5 past performances listed will be evaluated.

Correct, only 5 relevant past performances within the past three years.

108.

Section M

On page 81, as one of the four components of the past performance evaluation, it states, "HUD will then determine an overall level of sufficiency of the past performance information and assess the risk of nonperformance based on the degree of relevancy and level of quality of all recent past performance references." Could the government clarify what it is seeking to evaluate under the sufficiency component?

The Government will determine whether the information submitted to demonstrate past performance is sufficient to be able to be assessed for relevance and quality. Please see adjectival ratings in the solicitation

109.

Section M

Can the government confirm that by using the tradeoff process for source selection, that when considering awards in the best interests of the government and HUD customers, that technical quality will be scored by a holistic selection team that will include the program office officials to include experts with the knowledge and capacity to determine technical excellence, A qualified technical evaluation panel will rate the proposals.

management skills and qualification of key personnel?

110. Will the government consider the risks associated with industry reducing price and meeting only minimum requirements which may substantially impact delivery to HUD customers?

The Government will use a best value trade-off selection, consistent with the terms of the solicitation.

The Government may select other than the lowest price proposal.

111. What criteria will the government use to assess the risk in an offeror’s technical approach?

The Government will use a best value trade-off selection, consistent with the terms of the solicitation.

112. Using an Acceptable/Unacceptable evaluation of the Technical Approach and not including the Technical Approach in the best value trade-off analysis would appear to encourage offerors to propose the bare minimum acceptable quality in order to drive down price. Is this the government's intent?

The Government will include an evaluation of the Technical Approach in the best value trade-off analysis. The government has revised evaluation approach.

113. Will the quality of the offeror’s technical approach be evaluated in any way other than assessing the bare minimum acceptability in meeting the solicitation's requirements? In other words, will the technical approach be considered as a part of the best value trade-off analysis?

The Government will include an evaluation of the Technical Approach in the best value trade-off analysis. Please see revised solicitation

114. Understanding that the government can accept other than the lowest cost, has the government predetermined what the non-cost factors will be in the Source Selection Plan in collaboration and/or agreement with the designated Source Selection Official?

The Government will use a best value trade-off selection, consistent with the solicitation. Please see revised solicitation

115. Proposals are to be evaluated on a best-value past performance/price trade-off basis.

If several offerors have acceptable technical approaches and score equally on past performance, what criteria other than price will be used to assess best value?

The Government may consider the relative quality of the technical approach among respondents that achieve equal ratings for technical approach and may consider the relative relevance and quality of past performance among respondents which achieve equal ratings for past performance. The Government has changed its evaluation criteria, please see solicitation

116. Since the government may award to other than the lowest priced offeror or other than the offeror with the highest-rated past performance, what factors will the government consider in making that trade-off evaluation? For example, will the government consider factors such understanding of the work, capacity for delivery, innovation capabilities, key personnel, management capacity, or risk management expertise?

The Government will use a best value trade-off selection, consistent with the solicitation. Please see the revised evaluation criteria in solicitation.

117. Could the government confirm there is no task 5.4.6 in the PWS, and that 5.4.7 Mobile Compatibility should be task 5.4.6?

The PWS does not include a task 5.4.6, the numbering will change

118. Can the government please clarify what is meant by "not priced separately" in box 20 of the SF-1449?

Our contract system does not allow for SUB- CLIN pricing requirements however, for HUD to evaluate the pricing of a contract, we are requesting each contractor price out each task, deliverable as an example that makes up the overall total price of the contract for the base year and each option period.

119. Will the government require experience successfully supporting multiple ATOs in the past?

As achievement and maintenance of ATO is a term of this requirement, support for one or more ATOs in the past will enhance the relevance of the respondent’s past performance.

120. Should offerors add lines for ODCs and travel in Attachment 4?

ODC and Travel line items for each year has been added to Amendment 0002 and Attachment 4- Pricing Worksheet v.2.

121. In Attachment 4, what should offerors include in the "Notes" column?

If there are additional assumptions needed to address your pricings, please use this column

122. Do offerors need to complete Section B.4.1 and include it in their submission?

Yes, it is required.

123. In Section B.4.1, what unit of issue is "JZ"? HUD’s Contract System identifies “JZ” as JOB

124. In a typical best value criterion, the government rates from excellent to unsatisfactory the following factors:

technical approach, staffing/management, past performance, and price. Would the government consider using a typical best value criteria?

The Government will use a best value trade-off selection, consistent with the terms of the solicitation. Please see revised solicitation

125. PWS – 5.2.1

Web-Hosting

Given that the cloud costs of hosting third-party applications are unknown, highly variable, and outside the control of the offeror, would the government consider adding a cost reimbursable line to the pricing worksheet for cloud hosting costs specified in 5.2.1?

ODC line item for each year has been added to Amendment 0002 and Attachment 4- Pricing Worksheet v.2.

126. 5.2.1 Website hosting requirements

[OPTIONAL

TASK after base year of the contract]

To ensure consistency among offerors' proposals, would the government please provide a government-provided estimate of cloud hosting specified in 5.2.1?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical

127. 5.2.5.2 Authority to

Operate

With respect to Authority to Operate, will the government confirm that the risk level is moderate?

Yes.

128. 5.2.8 Technical support

How many technical support requests should the Contractor assume will be received per year?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

129. 5.3.1.1 Content management software

How many different content templates should the Contractor plan to make available in the CMS in each year?

Consistent with Section 5.4.4, no more than two in the initial 12-month contract period and no more than three in each successive 12-month contract period.

130. 5.3.1.2 Content migration

How many documents/assets/files does HUD anticipate will need to be migrated to the Contractor’s servers?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

131. 5.3.1.5 Content publication procedures

Can HUD provide the estimated number of content updates that the Contractor should anticipate making directly on a weekly basis?"

HUD anticipates between 1500 and 3000 updates annually, not to exceed 80 content updates in any one week. The Contractor should anticipate making all updates directly.

132. 5.3.1.5 Content publication procedures

Can HUD provide the estimated number of new web pages that the Contractor should anticipate creating directly on a weekly basis?

HUD anticipates between 150 and 300 new webpages annually, not to exceed 8 webpages in any one week.

133. 5.3.1.5 Content publication procedures

Can HUD provide the estimated number of HUD-initiated content updates that the Contractor should anticipate needing to review and publish on a weekly basis?

HUD anticipates between 1500 and 3000 updates annually, not to exceed 80 content updates in any one week. The Contractor should anticipate reviewing all updates consistent with the requirement.

134. 5.3.1.5 Content publication procedures

Can HUD provide the estimated number of HUD-initiated new web pages that the Contractor should anticipate needing to review and publish on a weekly basis?

HUD anticipates between 150 and 300 new webpages annually.

135. 5.3.1.2 Content migration

Given the absence of specific details provided in Task 5.3.1.2 regarding the requirements and constraints of a migration to HUD servers, it is challenging to accurately estimate the necessary level of effort under a firm-fixed price contract.

Would the government consider separating the migration to HUD servers into a separate task as a Time-and-Materials

(T&M) CLIN?

Please refer to the current system description attached to the PWS and Section 5.2 to best inform your proposal.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

The Government will not entertain a T&M CLINs

136. 5.3.1.2 Content migration

To ensure consistency among offerors' proposals, would the government please provide a government-provided not to exceed level of effort for migration to HUD servers (specified in Task 5.3.1.2)?

Please refer to the current system description attached to the PWS and Section 5.2 to best inform your proposal.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

The Government will not entertain a T&M CLINs

137. 5.3.2.1 Learning

Management System

Please specify the number of licenses required for Adobe Learning Management System so all offerors have the same basis of estimate.

Please refer to the current system description attached to the PWS.

138. 5.3.2.1 Learning

Management System

ALM licensing is highly variable based on the number of active users. Would the government consider adding a cost reimbursable ODC line to the pricing worksheet for the ALM license costs?

ODC line item for each year has been added to Amendment 0002 of this solicitation.

139. 5.3.2.1 Learning

Management System

To ensure consistency among offerors' proposals, would the government please provide a government-provided estimate of ALM licensing?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

140. 5.3.2.1 Learning

Management System

Should the Contractor assume a migration from the incumbent's LMS to the Contractor's instance of Adobe Learning Manager?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

141. 5.3.2.1 Learning

Management System

Can HUD provide details on the scope of LMS assets that will need to be migrated to the Contractor's instance of Adobe Learning Manager, including the number of users and the number of trainings?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

142. 5.3.2.1 Learning

Management System

Can HUD confirm that all user data must be migrated out of the incumbent's LMS to the Contractor's new LMS instance, including all archived trainings and user transcript data, so that users' historical transcript data remains intact?

143. 5.3.2.1 Learning

Management System

Should the Contractor assume that the LMS is integrated with the HUD Exchange using Single Sign-On so that users only need to log in to the HUD Exchange to access the

LMS?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

144. 5.3.2.1 Learning

Management System

Should the Contractor assume that registration buttons on HUD Exchange class pages will take logged-in users directly to that specific training (and not to a generic LMS page)?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

145. 5.3.2.1 Learning

Management System

Will the government please confirm that the proposed Adobe Learning Manager environment is required to be FedRAMP certified?

146. 5.3.2.1.1 LMS

training content and delivery

Several features described under section

5.3.2.1.1 will require a webinar platform in addition to the Adobe Learning Manager LMS; should the Contractor assume a webinar platform license is required as part of this scope of work?

Yes.

147. 5.3.2.1.1 LMS

training content and delivery

If a webinar platform license is required as part of this scope of work, what is the minimum participant capacity that the Contractor should assume for the license (e.g., 1,000)?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical

148. 5.3.2.1.1 LMS

training content and delivery

Aside from live chat and the ability to record, are there other webinar platform features that HUD requires (i.e., breakout rooms)?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

149. 5.3.2.1 Learning

Management System

Given the number of licenses, seats, and necessary features may vary and change over time based on government needs, would the government consider adding a cost reimbursable ODC line to the pricing worksheet for webinar platform license costs, as described in 5.3.2.1?

ODC line item for each year has been added to Amendment 0002 and Attachment 4- Pricing Worksheet v.2.

150. 5.3.2.1 Learning

Management System

To ensure consistency among offerors' proposals, would the government please provide a government-provided estimate of webinar platform costs?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

151. 5.3.2.1.2 Training Event Calendar and Registration

How many trainings should the Contractor assume will be added to the Training Calendar annually?

In Fiscal Year 2023, HUD Exchange posted 124 instructor led-courses, 11 in-person training events, 113 live webinar events, and 133 recorded trainings.

152. 5.3.2.2 Ask A Question

(AAQ)

Helpdesks and Frequently

Asked Questions

(FAQ) Database

How many total FAQ updates (additions, updates, deletions) should the Contractor assume must be made each year?

HUD anticipates between 75 and 150 FAQ updates annually and up to five new FAQ topics.

153. 5.3.2.1.4 Peer Cohorts

How many distinct peer cohorts should the Contractor assume will be created each year?

HUD declines to estimate the number of peer cohorts that will be established under Technical Assistance work plans, as the technical capability is new and the popularity of the modality is unknown.

154. 5.3.2.1.4 Peer Cohorts

How many distinct discussion boards should the Contractor assume will be created each year?

HUD declines to estimate the number of discussion boards that will be established under Technical Assistance work plans, as the technical capability is new and the popularity of the modality is unknown.

155. 5.3.2.4 Income Calculator

Will detailed specifications on income calculator updates be provided field-by-field, or should the Contractor assume that subject matter expertise will need to be applied to complete the necessary updates?

Please refer to the current system description attached to the PWS.

The Government will assess the software, systems, and services proposed in the technical approach to determine the best value to the Government.

156. 5.3.2.5 Third-party

Applications

How many third-party applications of each type should the Contractor assume will be added to the site each year?

Please see section 5.4.4.

157. 5.3.2.6 User accounts

If the Contractor is expected to maintain a database of organizations to support user account management, will HUD provide an updated list of organizations on an annual basis? If not, how should the Contractor plan to update the database?

Yes, HUD will routinely provide an updated list of organizations to support user account management.

158. 5.4 Website Improvements

The Website Improvement work volume is not specified in the RFP, allowing offerors to bid a wide range of labor profiles for that work activity. Would the government consider providing a volume of work or an estimated level of effort for this task to ensure consistency among offerors' proposals?

HUD anticipates website improvements will be driven by user experience research. The three areas of priority for improvement are information architecture, LMS functionality, and search.

159. 5.4 Website Improvements

Given that the scope and complexity of website improvements may vary over the life of the contract, would the government consider changing Task 5.4 to a Time and Materials CLIN?

HUD anticipates a level of effort between $200,000 and $300,000 on a quarterly basis for ongoing and continuous improvement of the website.

160. 5.4 Website Improvements

To ensure consistency among offerors' proposals, if the government decides to move 5.4 Website Improvements to a T&M CLIN, would the government please provide a not-to-exceed level of effort for work under that Task?

HUD anticipates a level of effort between $200,000 and $300,000 on a quarterly basis for

161. 5.4.2.2 Test environment & production release

The statement of work states that "the Contractor shall implement quarterly production releases according to the schedule in the website improvement plan."

Please clarify whether HUD intends for releases to be…