Attachment_2_Q_AND_A_86548B19R00002_Contract.docx

QUESTIONS AND ANSWERS TO SOLICITATION NUMBER 86548B19R00002

1. While questions on the solicitation are due to you April 19th per the instructions, the solicitation appears to have a few issues that make assessing it difficult. Sections 5.1 "TASK 1 - NASS BUSINESS REQUIREMENTS DOCUMENT (BRD), REQUIREMENTS MATRIX, AND UPDATES” and 5.2 "ASSESSMENT LABORATORY AND TESTING ANALYSIS” appear to be identical. In addition, there are two subsections in 5.1 that appear to be misplaced (5.2.1 and 5.2.2, both on page 17). Are these items correct as is, or should we anticipate a modification? PWS revised to add:

SPECIFIC TASKS AND DELIVERABLES

5.0 SCOPE OF WORK: The Contractor shall provide business support to the Integrated Assessment System (NASS) in the form of advisory and technical assistance, processes, analyses, system documentation, consultation, and system testing

5.1 TASK 1 – NASS BUSINESS REQUIREMENTS DOCUMENT (BRD), REQUIREMENTS MATRIX, AND UPDATES

5.1.1 The Business Requirements Document, known as the BRD, is a Word document communicating the business needs for the iNtegrated Assessment Subsystem (NASS) to all interested parties and is used to provide detailed requirements in the development and updating of the online NASS Secure System. Periodically, the online NASS Secure System needs to be updated based on regulatory changes, system requirement modifications or changes in PHAS regulations. System updates are accomplished via system releases and system testing. Production schedules are established by the Development Coordination Group (DCG) of the REAC. The DCG works with all REAC system development teams to provide support in the development, testing, production environments, and release management. During the period of performance of this contract, at least 4 to 6 releases are anticipated per year where the BRD would require updating based on system requirements matrix, development of user stories, facilitation of grooming sessions, system enhancements and integration with other PIH-REAC subsystems which is mandated from the Office of Chief Information Officer (OCIO) in compliance with the PHAS including notices, and PHAS rule changes. PHAS notices and regulations may be found at http://portal.hud.gov/hudportal/HUD?src=/programoffices/publicindianhousing/reac/products/prodphasintrule.

https://www.hud.gov/program_offices/public_indian_housing/reac/products/prodphas Business Requirements Document - The Contractor shall perform on-going development and updating of the BRD and processes for the business area being addressed including the business process flowchart for the quality assurance component of each business area, facilitate requirements and AGILE grooming sessions, develop User Stories, and where applicable, integration and participation in Agile related sessions and capturing meeting minutes. The Contractor shall provide new or revised BRD resulting from IT releases and policy changes, updated in accordance with the System Development Methodology (SDM) policy found at:

http://hudatwork.hud.gov/HUD/cio/po/i/it/security/cmb/qacmbtoc The Contractor shall deliver the NASS BRD, in accordance with the SDM, that includes the business process flowchart for the quality assurance component of each business area, and where applicable, integration with other PIH-REAC systems.

The Contractor shall maintain the BRD up-to-date by capturing any changes to the BRD as soon as a change is identified by either a HUD staff or recommended by the Contractor. Revised changes to the BRD shall be highlighted upon each update. Additionally, the BRD shall be updated after each system release.

5.1.2 Updates to the BRD Following System Release - The Contractor shall provide updates to the BRD to the GTM following each NASS Secure System release as well as quarterly updates to the BRD as the NASS system changes. These updates should reflect any business process changes, process flow and system requirement modifications.

5.1.3 Requirements Matrix Document - The Contractor shall develop and maintain the NASS Business Support Requirement Matrix that details the specific business requirements for each system release. This would include analysis of business requirements and User Stories to develop acceptance criteria needed to support the User Acceptance Testing (UAT) efforts. The document describes the business processes and rules for each functionality change, addition and/or modification and captures the requirements highlighting any changes for each planned system release/future release.

5.2 TASK 2 – ASSESSMENT LABORATORY AND TESTING ANALYSIS

The Contractor shall perform research, analysis, and testing of various approaches to be used for assessing the performance and compliance requirements of applicable HUD program constituency, i.e., public housing agencies. The Contractor shall also provide requirements analysis, design input and testing in support of the design and implementation of the NASS subsystem and operating processes. For the NASS business area, the contractor shall support continued development, operation, updating and/or enhancement of the existing subsystem and operating processes. For both new and existing projects, the Contractor shall assist the NASS Team staff in supporting internal and external outreach, customer service and training needs, provide workload analyses and evaluations, as directed by the GTM, to monitor, evaluate PHAS indicators and PHAS scores to ensure requirements are accurately implemented, to provide new approaches to assessment of PHA scores within the PHAS program, to record and provide input into the NASS Quality Assurance Program, and to record and provide system enhancements and to capture warranted/requested changes of the NASS team to REAC system functionality.

The Contractor shall conduct research, analysis and data modeling to develop business processes, performance indicators, and scoring/assessment methodologies that shall further identify program risks related to the integrated assessment system. The Contractor shall research existing PIH-REAC program requirements, assessment processes, and industry best practices, and recommend performance indicators for use in assigned PIH-REAC assessment systems, as well as to provide recommendations on the program improvement to support quality assurance, NASS Quality Assurance Program, and PHAS Indicator Improvement (NASS lab) for the purpose of releasing PHAS scores that are accurate, valid and reliable. The Contractor shall continuously review and refine the PHAS assessment system, specifically PHAS indicators, to ensure the proper measures are in use, are accurate, and working consistently. The results of these efforts will produce evaluations/reports/analyses with recommendation for improving PHAS program performance. The intended outcome is a report on indicator research, as well as recommendations. The NASS quality assurance team’s primary responsibility is the distribution of accurate, valid and reliable PHAS scores to the PHA’s, PIH and the public.

5.2.1 Based on the above, the Contractor shall develop an Indicator Research Report that provides: (1) analyses and evaluations to monitor/evaluate PHAS indicators and PHAS indicator scores to ensure requirements are accurately implemented; (2) new approaches to assessment of PHA scores within the PHAS program including basis for recommended PIH-REAC indicators and assessment processes; (3) input into the NASS Quality Assurance Program, to record the status of system enhancements and to capture warranted/requested changes by the NASS team to REAC system functionality utilizing the Master Enhancement List, and (4) recommendations for improving PHAS program performance including risks.

5.2.2 The Contractor shall provide support to NASS IT to enable NASS IT staff to conduct system compatibility and system enhancements testing, to include: (1) User Acceptance Testing (UAT) for upcoming test cycle(s) of major/emergency system releases, and (2) enhancements to HUD’s systems and the production of a Master Enhancement List. The test cycle(s) schedule(s) are established by DCG and will be provided to the Contractor by the GTM as soon as it becomes available. The Contractor shall write the test scripts for each UAT. The UAT scripts must include each step of each test input, specific action to be performed and expected results (with reference back to the BRD for each step) so that any designated government staff can perform the UAT by following the test scripts. The GTM will provide a sample of a UAT test script to the Contractor. At the conclusion of each test cycle, system compatibility or system enhancement testing, the Contractor shall provide a summary of the test result findings, system compatibility or system enhancement testing.

2. Could you please tell me if this is a new requirement, or if there is a current or previous contractor who has performed similar services? This is a recurring requirement. The current contract number is DU208WP-14-D-01.

3. The RFP states “To produce the requested report(s), the Contractor shall conduct data analysis utilizing data base tools, REACs secure system, a business intelligence tool, the Data Warehouse, and the Reporting Server.” Can the Government elaborate on what data base tools and business intelligence tools the Government would like used? (5.4 task 4). 5.4 TASK 4: data base tools and business intelligence tools are current HUD systems.

4. The RFP states “5.6.1.4 Participate in planning sessions and perform multiple dry runs for each new-year of scoring.” Can the Government elaborate on approximately how many how many planning sessions and dry runs there will be? 12; factor in to the relevant CLIN.

5. Are contractor employees covered by the Service Contract Act? If so, can the WD be identified and can the utilized labor categories be provided? Yes, the SCA does apply but the wage rates can vary by state. The RFP does not specify specific labor categories. Each vendor is requested to propose their own mix of labor categories with applicable wage rates.

6. The RFP states “Evaluated as part of the overall responsibility determination under LPTA, NOT as an individual evaluation factor. Small businesses determined non-responsible but otherwise eligible and in line for award must be referred to the Small Business Administration for a Certificate of Competency.” Does the Government require any past performance examples as part of our proposal to demonstrate responsibility? Past performance examples can be provided. The Government can research past performance using Past Performance Information Retrieval System (PPIRS).

7. The Government reserves the right to change hours of operation or restrict contractor access.

QUESTION: What scenario(s) would typically prompt a change of hours or operation or to restrict contractor access? A Government shutdown

8. Sufficient personnel transitioning-in. QUESTION: How may we gage sufficiency? Contractor discretion what personnel are needed for contract completion. Each vendor proposes its own solution.

9. Names of authorized personnel shall be provided to the Contractor by the Government, in writing and updated as necessary throughout the contract period. QUESTION: Will this be provided by the Contracting Officer or Contracting Officer Rep? Yes

10. What is the average number of reports over the past 12 months? NA

11. In Section 5.71, the contractor is to provide statistical and data analysis for PIH-related needs and specifically, a Statistical Analysis Report (1 per year); a rudimentary analysis (2 per year); a complex analysis (2 per year); and advanced statistical analysis (2 per year). Would HUD provide examples of each type of statistical deliverable that have been performed to date by the incumbent contractor? Sample small report provided

12. Page 65 of the PWS. The table makes reference to 5 types of transmittals (SFF, SAM, SFS, SMC, and SFT). What do these acronyms mean? Table updated.

13. Some deliverables and associated quantities differ between the information provided in Attachment A Price Sheet and the deliverables schedule provided on pages 23 – 26 of the PWS. Both have been revised.

14. In the past, HUD has requested other information to be provided in the pricing proposal, such as the Original SF-33, Certain Completed Clauses, etc. Can HUD confirm that HUD is requesting only a completed Attachment A - Price Sheet and a listing of personnel, labor categories, labor rates and hours supporting Attachment A in this pricing proposal. Clauses such as Key Personnel require vendor input; page 64 references the SF1449: Total annual price carried over from Attachment A. So, yes, the SF1449 should be submitted.

15. How many contractor full-time equivalents (FTEs) currently perform on the incumbent contract?

16. How will the Government manage and monitor the contract without specific management tasks and deliverables? The reports are the deliverables

17. Are these key personnel to be identified at the time of contract award? The instructions do not currently specify that key personnel are required for a response. Key personnel are to be identified under the Key Personnel clause (page 46), as well as identified in the Management Plan (page 63)

18. Should the QCP be considered a deliverable with the list in Section 6? No, this contract pays by reports as identified under the List of Deliverables.

19. Can an SBA-approved 8(a) joint venture be awarded this contract? Yes

20. Does the agency have a preference towards awarding this contract to an SBA approved 8(a) prime that is or is not an 8(a) Joint Venture? No preference

21. Can the government provide historical labor hours, FTE count and labor categories for this requirement? No – proprietary information

22. Is there a change (increase or decrease) in the expected labor hours for this contract versus the historical hours? This contract is based on delivery of reports, not labor hours

23. Does the successful offeror need to have experience and past performance supporting the NASS program and/or HUD operations? No

24. Re: Task 1: Does the successful offeror need to have experience conducting PIH-REAC assessments? No

25. Re Task 2: Does the successful offeror need to have experience and past performance supporting NASS subsystems and processes? No

26. Re Task 3: Does the successful offeror need to have experience working with HCVP? No

27. Re Task 3: What is the historical level of effort & labor categories for resources supporting the NASS Quality Assurance Program? Proprietary Information

28. Re Task 4, Can the government please provide specifics on the technologies supporting the Data Warehouse and Reporting Server? The Data Warehouse / Reporting Server is on an Oracle OLAP platform

29. Re Task 5: Does the successful offeror need to have experience using specifically MicroStrategy business intelligence technology? Yes

30. Re Task 5: Can the government use and accept other Business Intelligence technologies for report prototyping which are not dependent on MicroStrategy? REAC Business has standardized on MicroStrategy for reports creation and visualization. The MicroStrategy middleware piece is already in place on the network.

31. Re Task 5: What is the historical level of effort & labor categories for resources supporting Business Intelligence ? Proprietary information

32. Re Task 6: Does the successful offeror need to have experience in FASS/FDS, PASS, PMASS, Capital Fund Program and Rental Assistance Demonstration (RAD) program? No

33. Re Task 7: Does the successful offeror need to have experience working with the Public Housing Assessment System (PHAS)? No

34. Can the government update the RFP Response date to four weeks responses to the questions above are released? Extended

35. Can the Government provide the spend information on the incumbent contract for this effort? No

36. To allow the Government to review proposals with competitive pricing and staffing plans aligned with the program mission, will the Government clarify which LCATs are currently supporting which tasks? No

37. Does the past performance need to be within last certain number of years? NO- Any applicable.

38. Will the past performance of being a subcontractor on a contract qualify for the bid? Yes.

39. How many past performances need to be included in the proposal? Contractors discretion. Is there any limitation on number of pages for past performance? No.

How many resumes need to be included in the proposal? Contractors discretion. Is there any limitation on number of pages for resumes? No.

40. Does the solicitation require contractor to be on GSA IT-70 Schedule? No

41. Any Contractor employee, as far as they are legally authorized to work in United states and the Government is able to obtain background information, can be employed? All contract employees must pass a background checks. The work authorization requirements is not limited to Citizens and Permanent Residents, but open to anyone authorized to work in the United States? Same as answer above.

42. On Page 2 of the RFP, what is the difference between Unit and Quantity? For example, for item description – “Document No 0001AD-Updates to the Business Requirement Document – Quantity: 4-6 Annual / Unit Price $”, what does Quantity, Unit and Unit Price means? Assuming Amount will be based on Unit*Unit Price. As an example, if this update takes around 100 hours to fix and each hour is priced at $40, then Quantity is 100, Unit is Hours and Unit Price is $40 with the Amount being $4000. Unit price x quantity (number of reports) =Total price.

43. The work will need to be carried out from contractor’s facility. Will HUD provide a VPN software which can be used to connect directly to the HUD systems and servers to access the data? Yes Does the contractor facility need to have some sort of certifications? No Can contractor employee work remotely anywhere from within the United States, and use HUD VPN to connect to the systems? Yes Any system requirements for the laptop or desktop used by the contractor to carry out the work? No

44. Will the project follow Waterfall or Agile methodology? Agile

45. Can samples of contract deliverables under each task in this solicitation be provided? No

46. Will the preference be given to the contractor that has prior knowledge and experience of PHAS scoring methodology and approach? No

47. The solicitation refers to statistical analysis reports. Can details be provided as to which software is currently being used to deliver these reports? HUD Systems Can samples of such statistical reports be provided? One sample provided (small report)

48. The solicitation refers to Data Collection/Analytical Tool tasks. Can details be provided as to which software is currently being used to deliver these tasks? Can samples of such tasks be provided? Removed these CLINs

49. Any software being used to manage Requirements and Test Cases/Results? HUD Systems

50. The tasks will not require any work on Database or MicroStrategy back end/schema? It is assumed that IT Support will be able to make necessary updates as required. Yes

51. Page 2 shows the template to provide pricing per item for the Base Year. Once award happens, will the government procure all these line items from the contractor, or government may decide to procure some of the line items from the contractor and not all of them? The same question will be for the following Option Year 1-4. CLINs with variables have been removed other than 2 that may be needed in an emergency situation. Factor in pricing for the 2 emergency CLINs. They may or may not be utilized. All other variables are removed.

Recommendations The description of the task seems missing. Could the Government please review and clarify what is expected to be performed for this task? Covered under 5.5.1.7

Micro Strategy can be used to generate Reports from existing collected data but cannot be used for data collection. Could Government clarify the expectation for data collection or analytical tool? HUD has many systems to collect data. MicroStrategy is one that NASS uses to collect data.

A021 Delivery of Final Updates A022 Data Collection A023 Data Collection (Small) A024 Data Collection (Medium) A025 Data Collection (Large) Could the Government please provide the matching sub-task numbers against each of the 6 listed deliverables? Could the Government also provide sample report of this deliverable? All identified

A003 – Draft Business Requirements Document for Major System Release (Quarterly) A004 – Final Business Requirements Document for Major System Release (Quarterly)

PRICING Excel – Base Year Tab:

Major releases that will result in a Business Requirement Document-task 1 (Biannually) – QTY:2 Emergency releases that will result in a Business Requirement Document-task 1 (0 to 1) – QTY:1 The number of QTY and Frequency in Solicitation Document and PRICING EXCEL does not match for the required deliverable. Could the Government please review and confirm what is expected? 1 draft report quarterly and 1 final report quarterly. The Government pays for only the final report.

Also, in PRICING Excel, we see BRD Requirement for Emergency Release but we don’t see matching BRD deliverable in Solicitation Document. Could the Government please review and confirm the expectation? 5.2.2 The Contractor shall provide support to NASS IT to enable NASS IT staff to conduct system compatibility and system enhancements testing, to include: (1) User Acceptance Testing (UAT) for upcoming test cycle(s) of major/emergency system releases. Page 17.

A005 – Updates to the BRD (Quarterly)

PRICING Excel – Base Year Tab:

Updates to the Business Requirement Document QTY:4-6 The number of QTY and Frequency in Solicitation Document and PRICING EXCEL does not match for the required deliverable. Could the Government please review and confirm if this is expected to be QUARTERLY or every 2 months? 1 draft report quarterly and 1 final report quarterly. The Government pays for only the final report.

A006 – Annual Indicator Research Report

PRICING Excel – Base Year Tab:

Indicator Report-task 1 (Annually) The Solicitation Document states that the deliverable is for Task-2 while PRICING EXCEL indicates this deliverable for Task-2. Could the Government please review and confirm the expectation is for Task-1 or Task-2? Changed

A012 - Draft of NASS QA Plan Update (Semi-Annually) A013 - Final of NASS QA Plan Update (Semi-Annually)

PRICING Excel – Base Year Tab:

NASS and PIH-REAC Quality Assurance Plan Update-task 3 (Quarterly) The Solicitation Document states that the deliverable is Semi-Annually while PRICING EXCEL indicates this deliverable to be quarterly. Could the Government please review and clarify the expectation? Changed

A023 – Data Collection (Small) – 0-4 per year A024 – Data Collection (Medium) – 0-4 per year A025 – Data Collection (Large) – 0-4 per year

PRICING Excel – Base Year Tab:

Data Collection/Analytical Tool-small-task 5 (2 to 4) Data Collection/Analytical Tool-medium-task 5 (2 to 4) Data Collection/Analytical Tool-Large-task 5 (2 to 4) The QTY provided in the Solicitation Document and PRICING EXCEL does not match. Could the Government please review and clarify the expectation? Changed

A026 – PHAS Score Report Summary - Monthly

PRICING Excel – Base Year Tab:

No matching Deliverable The Solicitation provides deliverable name for Task-7 but we don’t see this deliverable name on PRICING Excel. Could the Government please review and confirm the expectation? Changed

A027 – Draft of Technical Standard Operating Procedure - Annually A028 – Final of Technical Standard Operating Procedure - Annually

PRICING Excel – Base Year Tab:

SOP/Technical Documentation, QA and Control, PHAS Score Database Users Guide-task 5 (Quarterly) The FREQUENCY provided in the Solicitation Document and PRICING EXCEL does not match. Could the Government please review and clarify the expectation? Changed

A029 – Draft of Statistical Analysis Report - Once A033 – Final of Statistical Analysis Report – None

PRICING Excel – Base Year Tab:

SOP/Technical Documentation, QA and Control, PHAS Score Database Users Guide-task 5 (Quarterly) The FREQUENCY provided in the Solicitation Document and PRICING EXCEL does not match. Could the Government please review and clarify the expectation? Changed

A030 – Rudimentary Analysis of date (Small) – 2 per year A031 – Involves more complex analysis of date (Medium) – 2 per year A032 – Involves more advanced statistical analysis of (Large) amounts of data – 2 per year

PRICING Excel – Base Year Tab:

SOP/Technical Documentation, QA and Control, PHAS Score Database Users Guide-task 5 (Quarterly) The FREQUENCY provided in the Solicitation Document and PRICING EXCEL does not match. Could the Government please review and clarify the expectation? Variables removed

One submittal Could the Government please provide the timeline for providing this deliverable for each of the option periods and number of days of submission after contract has been awarded for the base period to understand the expected timeline? Five days after award

similar format, which maps what page of the proposal addresses each Performance Work Statement requirement identified under Technical Approach, and which page of the proposal fully addresses each Evaluation Factor.

Could the Government please provide if this proposal matrix should be part of any proposal response volume or separate table? Page 63 states “Offerors are cautioned not to include pricing in the labor mix/level of effort matrix submitted with the technical proposal.”

Could the Government please provide direction if these sections needs to be in 1 document called Technical Volume or 3 separate documents? Could the Government specify if there is any specific instructions to be followed for font type, size, etc.? Page 63 states “Offerors are cautioned not to include pricing in the labor mix/level of effort matrix submitted with the technical proposal.”