Questions_and_Responses_Part_Two_4_April_2016.pdf

Questions and Responses Part Two (2) Solicitation No. PBGC01-RP-16-0011

Benefit Calculation & Valuation (BCV)

NOTE: This is Part Two of PBGC’S response to the Industry Questions. PBGC will post it’s response to the remaining questions on or before Thursday, 8 April 2016.

35. Question: Will your current operations and maintenance staffing model be provided to assist with the proposal process?

RESPONSE: Based on historical data, the BCV O&M staff consisted of a senior project manager, project manager, operations lead, several business analysts, two developers, and testers.

36. Question: Please provide clarification regarding the following paragraph:

1. Will this initial briefing be conducted within 10 days following the award of the IDIQ contract or award of the initial task order?

2. If the initial briefing will be conducted within 10 days following the award of the IDIQ contract, does the Government expect the meeting to occur in the event the initial task order is not yet awarded?

Response: It is expected that the initial briefing will be conducted 10 after Task Orders are issued.

37. Question: Are Technical Proposal Exceptions and Deviations included in any page limitations?

Response: No. Technical Proposal Exceptions and Deviations are not counted in the page limitations.

38. Question: Can the Government provide the following documents referenced in the footnote?

1. Executive Summary of the BCVS solution options, 3/4/2011

2. Decision: Reevaluate BCVS Investment Direction, 10/4/2012 Response: These documents were used as input for Attachment 10.Therefore, the Government will not be providing these documents.

39. Question: Can PBGC share with vendor’s legacy BCV O&M metrics such as the monthly average number of production support requests on the current system, the typical number of bug fixes or enhancements that go into a steady state release etc.?

Response: There are an average of 37 production support tickets per month. An average of 28 backlog items are included in an O&M release.

40. Question: Can PBGC clarify if the recommended phased plan discussion in the proposal is to be provided as part of the “planning phase - first task order” response or the current solicitation response?

Response: The Phased Plan discussion is required as part of the first DME task order and is not required as part of the Solicitation response.

41. Question: Are the software releases referenced in the statement “a task order will be issued and include at least one software release”, to be deployed in production?

Response: Yes, software release refers to a production deployment.

42. Question: Can PBGC clarify the reference to the corporate database? “Able to reconcile data in centralized database with corporate database.” Is it referenced in the architecture diagrams released as part of this solicitation?

Response: The term Corporate Database is synonymous with Enterprise Data. See Attachment 16.

43. Question: Can PBGC expand on what it means by the statement “host all legacy BSRS programs”? Is this an infrastructure migration of legacy BSRS programs or implementation of the functionality of the legacy BSRS programs?

Response: The BSRS programs need to be migrated and hosted in the new solution and allow for the BSRS programs to continue to function and produce the same results.

44. Question: States “a tolerance level for accuracy” as a challenge for the conversion process. Can PBGC share in real terms what the acceptable tolerance level as a percentage or some other quantifiable measure is?

Response: This can't be quantified at this time.

45. Question: states that the submission font shall be Times New Roman 12 pitch print or larger. Would the government allow for a smaller size and/or style of font for graphics and tables?

Response: The government will allow smaller and/or different font type for headers, footers, exhibits, diagrams, table, charts, inset text boxes, graphics, figures, illustrations, and those requested documents that cannot be modified to adjust the font to the requested size, but the type must be readable.

46. Question: states that "...the Compliance Traceability Matrix and Subcontracting Plan are not included in the page count limit and shall be included as a separate attachment.", however L.7.3 (Page 93) states that the "first page of Volume 1 shall be the Compliance Traceability Matrix". Would the government please confirm whether the Compliance Traceability Matrix should be submitted as a separate attachment?

Response: The Compliance Traceability Matrix shall be the first page of Volume 1 and shall not be included as a separate attachment.

47. Question: states that "... [the] Subcontracting Plan [is] not included in the page count limit and shall be included as a separate attachment.", however L.7.3.4 (Page 96) lists the Small Business Subcontracting Plan as part of Factor 4.0. Would the government please confirm that it is acceptable to submit as a separate attachment (or an appendix) but will still be considered as part of Factor 4.0?

Response: Offerors Subcontracting Plan shall be included as a separate attachment as part of Factor 3, Management Approach and is not counted towards page limitation.

48. Question: We assume that all the existing applications mentioned in the Application Landscape diagram are in scope of Operations, Maintenance? If there are certain application which needs to be excluded from Operations and Maintenance, please specify.

Response: Refer to Attachment 4 for list of applications considered Legacy BCV. These are the applications in scope for O&M.

49. Question: If available, please provide high level functionality and technical matrix for each application with below details:

a. Functionality - (example Benefit Calculations, Taxation, Vesting etc.)

b. complexity - (low, Medium, high)

c. Technology (Ex. Excel, MS Access, Mainframes, and Java etc.)

d. Number of Users of this application.

e. Peak Period when this application is mainly used (Ex. Once in year - for taxation, Daily or Monthly)

f. Software Development Methodology (Waterfall/Agile) Response: No matrix is available. Please refer to SOO attachments for more information.

50. Question: Are there applications where there are significant Defect leakages or very less test coverage? Please highlight the QA pain points also.

Response: Please refer to Attachment 10 for more information.

51. Question: As per our understanding, The scope of testing would include the following:

- Functional Testing

- Data Migration Testing

- ETL/Interface Testing

- System Integration Testing

- Regression Testing

- Automation Testing

- Performance Testing

- Mobility Testing (please provide the preferred list of devices)

- Security Testing

- User Acceptance Testing - Support (Defect management)

- End to End Testing

- IVR Testing Please provide details of any additional UAT support Please confirm if any other testing are in scope(Ex. Usability, Accessibility testing).

Response: Yes, Usability and Accessibility testing would be included in needed testing.

UAT support, aside from defect management, would include any support required by the user to accomplish testing.

52. Question: Is cross browser/ multi OS testing in scope? If yes, please provide the list of browsers (IE, Firefox, Safari etc.) and OS combinations. Please specify is iPad or Android Tablet testing is also needed.

Response: Yes, it is in scope. The following is a list of browsers and operating systems the system should work on: Browsers: Internet Explorer; MS Edge; Chrome; Firefox OS: Windows 7; Windows 10; Windows Server 2012 R2. Since this is an internal Application, there are no Apple products included within scope.

53. Question: As per our understanding, we have development and testing environments will be available to support development and Functional Testing & SIT, UAT and Performance Testing. Please confirm.

Response: The appropriate number of environments necessary will be determined after contract award.

54. Question: How is the test data generated currently? Is it generated manually or by using a tool? Please clarify. If it is a tool, please mention the tool name.

Response: Currently, test data is generated manually.

55. Question: Are there any existing Testing Artifacts from existing applications available which can be provided to the QA team? (ex. Existing Regression Test suite, Functional Test scenarios, cases, Test data for different applications).

Response: Yes, they are existing Testing Artifacts. They will be provided after contract award. The BCV team has multiple data sets that test different business rules that are used for the different types of testing.

56. Question: Are there any test automation activities currently being done for any of the applications / systems? If yes, please provide the following: - List the applications that are currently automated and the automation levels attained.

Response: Yes, some automated testing is being conducted on the following Legacy BCV applications: ACT-Runtime, Archive, PACS and ADT.

57. Question: Does PBGC have any list of application high on radar for automation?

Response: There are applications within Legacy BCV that are on the radar for automation. Refer to Attachment 10.

58. Question: What is the size of the current team & its split

- by group of applications, technology

- how may BAs, Actuaries, developers, PMs etc.

- by employee type - FTE, Contractor, and Vendor etc.

Response: Based on historical data, the BCV O&M staff consisted of a senior project manager, project manager, operations lead, several business analysts, two developers, and testers. For the TRMD Tools, it's mostly just one federal employee who acts as the business owner and developer of the tool.

59. Question: What types of testing are currently in place by applications, LOB

- Functional Testing (Manual & Automation)

- Non Functional Testing (Performance, Security etc.)

Response: Functional testing is completed manually on all BCV applications, and automated tools are used on several of the BCV applications. No specific nonfunctional test scripts exist.

60. Question: What are the testing tools currently in use?

- for Performance Test (e.g. HP Performance Center etc.)

- for Automation (UFT or QTP)

- for defect tracking (Is it PVCS entirely)

- Traceability (Requirement Traceability) Response: The following testing tools are used: Performance Test (e.g. HP Performance Center etc.) LoadRunner, Automation (UFT or QTP) – HP Unified Functional Testing 12.0, defect tracking, HP Application Lifecycle Management Quality Center, Traceability, HP Application Lifecycle Management Quality Center.

61. Question: Please provide release schedule for existing application in PBGC (monthly/quarterly etc.)?

Response: The government receives approximately two BCV O&M releases each year.

62. Question: What types of QA Metrics / KPI’s are followed for System, Performance (Load/Stress) and Test Automation?

Response: All Critical, Major, and Moderate severity defects must be addressed prior to production deployment. Any Minor severity defects must be addressed or accepted by the government and a tracker item opened so it can be addressed in future release.

63. Question: Does PBGC have any metrics available on UAT and production defect leakage from QA to measure existing QA effectiveness?

Response: Per release, there are approximately 8 QA issues, 13 UAT issues and 1 Prod issue.

64. Question: What are the expected number of concurrent users at average and peak times?

How much this data is expected to grow over time?

Response: The approximately 500 users of the system may all be accessing the system at the same time during peak and average times. Currently, PBGC trustees on average 100 plans per year.

65. Question: How is test data managed in the Test environment? Is production data copied into Test environment or data is mocked up in Test environment? If production data are copied into Test environment, does PBGC have requirements to mask sensitive data? If yes, is there any data masking or data obfuscation tool currently in use?

Response: Test data is comprised of both masked production data, as well as mock up data.

66. Question: What is the QA estimation process? (ex. Test case point driven etc.). Also specify Dev/QA ratio in PBGC for key applications.

Response: The QA estimation process used is Planning Poker and Story points per backlog item. There are 7 to 8 DEV/QA resources supporting the applications.

67. Question: What is the expected YOY percentage growth of plans & retirees?

Response: The projected growth of PBGC trusteeships is not available. However, over the last five years, PBGC has averaged 108 new plans trusteed per year and 43,000 new participants per year.

68. Question: For Database Building, Which ETL tool is preferred by PBGC? (ex.

Informatica, Datastage).

Response: Please refer to PBGC.gov and reference the Technical Reference Model for more information.

69. Question: How many business workflows are needed in the future state BCV application? (Workflow is a series of tasks performed through GUI screens to produce a desired outcome and it usually involves multiple stakeholders. Ex. File upload and approval workflow, online corrections and adjustments).

Response: The number of business workflows needed for New BCV is unknown at this time.

70. Question: What is the difference between Participant Management & CMS?

Is CMS centralized database for various other interfaces?

Response: Please refer to Attachment 10, page 19 for definition of CMS and its use for more information.

71. Question: Is file archival and retrieval process manual or automated? Is there any specific Application to retrieve the archived reports/Statements? What is the function of FBA in archiving the documents?

Response: There are currently no automated triggers for archiving and retrieving electronic files. With Legacy BCV, files are manually archived upon request by a centralized team using the Archive Utility application (discussed on page 409 of Attachment_3_-_ACT_Toolkit_Manual.pdf and on page 1 of Attachment_16_- _Existing_System_Information.pdf). Business users, including those at FBA offices, have access to view and open archived files through the Archive Access application (discussed starting on page 399 of Attachment_3_-_ACT_Toolkit_Manual.pdf and on page 1 of Attachment_16_-_Existing_System_Information.pdf). FBA employees do not directly archive electronic files but may request updates to existing archived files or that new files be archived.

72. Question: Will the existing benefit administration be implemented AS IS in new BCV application or will it be enhanced?

Response: The government anticipates implementing current or similar business process, with enhances, in New BCV.

73. Question: • During migration will the legacy system be available to validate the results

• is there any third party applications involved

• is there a separated QA environment? How the data is generated for QA?

• Are the old and new rules & regulations maintained in one application? Or these rules are referenced in excel wherever applied.

Response: Yes, Legacy BCV will be available to validate results. Yes, there are third party applications involved (Microsoft Access, PowerBuilder, etc.) Please, refer to RFP attachments for more information. Yes, there is a separate QA environment. Data is generated in QA through refreshes of production data, as well as manually inputted test data. The rules of the plan exist within the Excel spreadsheets.

74. Question: Applications mentioned in this RFP are the only applications under scope for

QA?

• ACT - All components of ACT are based on Excel?

• How frequently archiving is done?

• What is the current process of data reconciliation? What are the new features that BCV is looking for data reconciliation?

Response: Only applications mentioned in this RFP are those under scope for QA. The ACT suite consists of four components: ValTool, DOPT Report System (DRS), Design Tool, and Runtime Tool. Benefit calculation and valuation rules are programmed in Excel spreadsheets which are used by the ValTool and Runtime Tool components. Plan and participant data along with calculated values are stored in MS Access databases which are used by all ACT components. The layouts for individual participant benefit statements are stored in configuration text files which are used by the Design Tool and Runtime Tool components.

Over the past year, there have been approximately 900 requests to update or to add electronic files on the Archive Access application.

Participant data is periodically reconciled between the valuation database in Access and the PBGC enterprise payment database (Genesis) using Leo, the Valuation Database Construction Tool noted in Attachments 4, 11, and 16. Leo generates reconciliation reports which list data discrepancies that must be manually investigated and resolved.

We are looking for recommendations from the vendor for new features that will improve this process.

75. Question: • Are there any batch processes involved other than PACS? How frequently these batches run?

• How the batch is triggered? Is it automated?

Response: The BCV legacy system has no automated triggers for running batch processes. Any type of batch processing (which may include calculating valuation results for an entire plan, generating participants’ benefit statements, or running batch PACS calculations) must be implemented manually and does not involve scheduled events.

76. Question: What all LOBs' are currently supported in BCV? - Cash Balance, Trust Plans, pure DB plans.

Response: Any defined benefit plan trusteed by PBGC is supported in Legacy BCV.

77. Question: Please provide total number of active/inactive Plans, participants and assets that needs to be migrated to new BCV.

Response: Refer to the SOO for estimated number of plans. Participant numbers vary by plan and are not available at this time.

78. Question: What is the plan with respect to migrating inactive plans to new BCV?

Response: An analysis of plans will be used to determine what plans should be migrated to New BCV. It is anticipated that plans without any future retirees will not be migrated.

79. Question: Will the legacy BCV sunset and is the sunset of legacy BCV application in scope of this RFP?

Response: Yes, it is anticipated that Legacy BCV will be sunsetted and it is within the scope of this RFP.

80. Question: What are the processes involved in data migration of plan and participant when PBGC acquires any new plan? Is any specific data format needed from plan sponsor for data to be migrated?

Response: Plan sponsors provide plan and participant data to PBGC in any type of format (electronic files, mainframe data, standardized paper forms, hand-written paper records, etc.). There can be no expectation of a specific data format or quality level. One of the initial steps to the database building process is determining the most efficient way to convert the plan-provided data into the standard PBGC format.

81. Question: How the Participant Websites are getting the data from different source systems? Is the website directly call the source system or the data will be sent to one centralized location from there the website accesses?

Response: Participant data is gathered in a number of ways (e.g., hardcopy, data files, and manual entry). The VDCT in Legacy BCV does not directly call the source system.

82. Question: How are they storing the Statement(s)? How can they reprint the statement if participant requests it?

Response: Statements are stored in Image Viewer. FBA administrators have the ability to regenerate a statement with Legacy BCV.

83. Question: How are Tax Reporting happening? Do they generate Tax Forms and send them to participants? Do they report to IRS/States and other governments?

Response: Tax reporting is out of scope for this requirement.

84. Question: Which application is currently tracking the payment (Check/EFT) information?

Response: Spectrum tracks payment information. Tracking payment information (such as Check vs. EFT) is out of scope for this requirement.

85. Question: Which application is currently handling general ledger?

Response: There is no application handling the General Ledger. The General ledger is out of scope for this requirement.

86. Question: Out of the 3000+ plans to be migrated to new BCV, does PBGC have any preferred approach to migrate certain plans first followed by other plans?

Response: There is currently no established approach to migrate the plans. The offeror shall provide a proposed approach in response to the Solicitation.

87. Question: Are the existing NFRs (Nonfunctional requirements) clearly documented? If yes, please provide those NFRs.

Response: Yes, NFRs for New BCV will be defined during the New BCV Planning Phase.

88. Question: What's the plan with respect to migrating historical data to new BCV?

Response: There is currently no established approach to migrate the plans. The offeror shall provide a proposed approach in response to the Solicitation.

89. Question: What is the average transactions volume/day?

Response: Legacy BCV is not transaction-based. In respect to volume of use, the approximately 500 users can access pieces of the system at once.

90. Question: Are they primarily using MS access database for benefit calculation? IF yes, how many tables/Views we have in this?

Response: The participant and plan data in Legacy BCV is stored in MS Access databases that generally have the same structure (16 tables with permanent data 12 tables that hold data temporarily for recalculations). There is a separate Access database for each case (some cases have more than one Access database). Other than the legacy database building tool, Leo, MS Access is strictly used as the repository for data in the Legacy BCV with data being pulled into MS Excel spreadsheets for calculating results and there are no forms, queries, views, applications, etc. within MS Access. The legacy database building tool is a highly evolved complex MS Access application that also holds data in a similar database structure with one database per case.

91. Question: What kind of report(s) coming out from ADT? Does all report(s) run as part of batch or it can be generated on Adhoc basis?

Response: ADT generates summary reports which list participant counts and liability totals and list the counts and types of each validation error and warning. ADT also generates detailed reports listing the validation errors and warnings applicable to each participant record in the valuation database along with the relevant data causing the validations to fail. ADT reports are generated on an ad hoc basis for a specific plan database. The reports cannot be generated as part of a batch process.

92. Question: What is the statement(s) volume and frequency generated by Archive system?

Response: Tens of thousands of statements are generated annually. The frequency of statement generation is daily.

93. Question: There are quite a few XLS programs. So do all run as part of some batch or someone manually managing it?

Response: The Excel spreadsheets contain formulas programmed based on the benefit calculation rules for a specific plan. And, for a specific plan, there are system processes for calculating total valuation liabilities, validating calculated values, generating all benefit statements, and running automated tests of benefit recalculations. However, all plan-specific Excel spreadsheets are never used together as part of some batch process.

94. Question: How they are doing disaster recovery and how quickly they can restore the data from backup(s)?

Response: COOP plans require mission critical services to be restored within 24 hours while other services may be restored within 7 to 30 days.

95. Question: Do they maintain Audit/trail or log files?

Response: No audit logs are currently maintained on Legacy BCV.

96. Question: Do they transfer any file(s) to outside of company? If yes, what tool they use?

Response: No files are transferred out of PBGC from Legacy BCV.

97. Question: Is there any other database except MS Access for storing plan/participant data?

Response: The Payment Adjustment Calculation System (PACS) and Batch PACS (described on page 2 of Attachment_16_-_Existing_System_Information.pdf) use an Oracle enterprise database for plan and participant information. All other BCV components use MS Access databases.

98. Question: Is there a separate database for each plan in BCV currently?

Response: Yes, each plan has its own Access database.

99. Question: Can you please provide the volume of Prod support incidents by severity for legacy BCV.

Response: There are an average of 37 production support tickets per month (37 non urgent, 0 urgent).

100. Question: What tool/product is used to record and track production incident tickets?

Response: HP Service Manager is used to track incidents.

101. Question: Can you please provide the volume of planned/unplanned system outages (H/W and S/W)?

Response: Per year, there is an average of four planned outages for two O&M releases and two COOP testing. There has been on average, two unplanned outages in at least some of Legacy BCV.

102. Question: Can you please provide the current data archival/unarchival strategy of legacy BCV application?

Response: Participant and plan data is placed in a repository for access by users. This process is achieved through the Archive-Access and Archive Utility applications of Legacy BCV.

103. Question: To ensure the quality of current legacy BCV application code is good, we want to confirm if all the current production components follow change control process?

Response: Currently, the five applications of BCV follow the change control process. It is anticipated that, within the first year of the contract, the TRMD Tool will be maintained by the BCV vendor and will follow the same change control process.

104. Question: Are there any tool(s) currently used to perform data repairs? How frequently are these requests made? Please provide historical statistics?

Response: Legacy BCV has no tools which perform automated data repairs. Leo, the Valuation Database Construction Tool described in Attachment_16_- _Existing_System_Information.pdf, allows users to generate reconciliation reports against the PBGC enterprise payment database (Genesis). And, ADT (also described in Attachment_16_-_Existing_System_Information.pdf) allows users to inspect the valuation database for missing and inconsistent data. However, investigating and correcting data using the reports from Leo and ADT is largely a manual process.

105. Question: Does the application and database reside in same server?

Response: The application and databases do not reside on the same server. Legacy BCV is a client application. The databases are Access databases.

106. Question: Is Citrix or any other tool currently used for load balancing to servers?

Response: Brocade is used for load balancing on servers at PBGC.

107. Question: How many DEV environments will be available for development?

Response: The appropriate number of environments necessary will be determined after contract award.

108. Question: What is the current user logon authentication and authorization process (Active directory, Oracle single sign on, etc.?)

Response: Legacy BCV uses Active Directory and UPT on some of its applications.

109. Question: What is the difference between Valuation, Corporate, Enterprise and Centralized Database? Are all of them in MS Access and are they replicas (has the same set of tables)? Is the data in those databases being maintained at plan level?

Response: Please refer to Attachment 16 for more information on Corporate, Enterprise, and Valuation databases.

110. Question: Are there any performance monitoring tool available in PBGC?

Response: IBM EndPoint Protection is currently monitoring system performance at an operating system level.

111. Question: Is the current performance benchmarks available?

Response: No performance benchmarks on Legacy BCV are available.

112. Question: What are the different forms of statement output to other PBGC systems (e.g. letter generation, MyPBA, etc.) and the counts per form?

Response: The different forms of output are: statement, worksheet, and optional forms of annuity. Users can produce these outputs on demand.

113. Question: Count of reports currently in the system for each applicable formats, e.g. hard copy, PDF, XML, etc.

Response: Currently, Legacy BCV only has summary reports for ADT, available in PDF and HTML formats.

114. Question: Are the system documentation (manuals, handbooks, guides, reference materials, etc.) available for the current legacy suite of applications. If the system documentation of current BCV application is not available, then is it required to be created?

Response: Yes, Legacy BCV documentation exists and will not be made available upon Contract award.

115. Question: How many planned releases do you have in a year? Please provide historical release information with appropriate effort and counts.

Response: The government receives approximately two BCV O&M releases each year.

116. Question: Are the business rules for each functionality documented and available? If not, please provide the count the code components based on type (like VB, Powerbuilder, Excel macros, forms, scripts, etc.) with the LOC (lines of codes) of each component which will help us in estimating the effort needed to extract the business rules out of current BCV application.

Response: The Business Rules exist in each plan spreadsheet.

117. Question: Section L does not make reference to attachments, is there a limit to technical attachments for product information and other necessary data for evaluation?

Response: The page limits apply to the entire RFP submission, except as noted in the solicitation.

118. Question: The RFP Synopsis paragraph 1 says "This requirement is a 100% Full & Open Requirement. The appropriate NAICS code for these services is 541512 with a small business size standard of $27,500,000.00. “Does it mean that this RFP is for a small business set-aside?

Response: This requirement is a 100% Full & Open Requirement. This is not a Small Business Set-Aside.