Questions_and_Responses_regarding_OTE_RFP.pdf

Solicitation HSBP1013R0080 – Q &A

Questions and Responses regarding OT&E RFP

1. Is there a listing of HUBzone contractors that have been brought to you attention as interested and capable of servicing this requirement? We would like to use that list to contact these firms and make our services available.

Qualified HUBZone SBCs are identified by running a search on CCR/DSBS

(http://dsbs.sba.gov/dsbs/dsp_dsbs.cfm) and are listed on the HUBZone Web page at https://eweb1.sba.gov/hubzone/internet/general/approved-firms.cfm. In addition, requesters may obtain a copy of the List by writing to the D/HUB at U.S. Small Business Administration, 409

3rd Street, SW., Washington, DC 20416 or at hubzone@sba.gov.

2. I did not notice a Government estimate of the required labor to perform the requested services. If the Governmnet has an estimate for this requirement, please make it known to potential bidders.

The Independent Government Cost Estimate will not be provided to any potential bidders.

3. There are a variety of approaches for deploying test resources into sprint activities, which range from applying traditional manual testers who develop and execute manual test approaches after the completion of an increment to applying advanced testers with programming skills into the sprint working alongside the developers and crafting automated test scripts which are executed immediately after the increment is built. The RFP seemed to lean towards traditional manual testing but was not specific in this important sense. Please clarify if there is a preference for the test approach or if multiple test approaches are indicated. Also – the RFP does not indicate if

Performance Testing is a requirement of the program. I did not discern a preference in the

RFP In our experience, if the development team is using mature agile approaches, the latter approach –high-end automated testing –is ultimately more efficient and thorough from an overall cost and quality of release standpoint.

The Operational Test and Evaluation support described in this RFP validates the system under test can satisfactorily execute its mission in a realistic operational environment including typical operators. Automated test scripting and performance testing is outside of the scope of this work.

4. Is this also restricted to MOBIS contractors?

It is open to all Small Scale HUBZone Vendors.

5. Since the Proposal response is not due until September 4th, would DHS consider giving respondents until the due date of the proposal itself to provide PPQs. Generally we need a little more lead time to get our customers to respond.

Due date extended to Sep. 9 – see amendment mailto:hubzone@sba.gov

6. The proposal instructions require submittal of a SF1449 but there wasn’t one posted to the FBO notice. Please advise.

SF1449 provided in this amendment.

7. The solicitation states that at least 3 past performances are required. Will government consider past performance references of subcontractor? How many past performance should be from the prime contractor?

See Section 2.2 of solicitation

8. How long is it expected that each OT&E event would take? Or how much time is allocated/planned for them?

For cost estimation purposes, one-week of data collection at a port of entry can be assumed for each operational test event.

9. Assuming increments are staggered and that there is an OT&E event at the end of each increment, we would like more detail in terms of the schedule of OT&E events. In other words, will any of the planning/implementation of any of the OT&E events be expected to be run in parallel with other OT&E events to the point where parallel teams and/or environments are expected to be necessary?

Parallel teams or environments are not expected to be necessary.

10. Please describe the expected involvement of OT&E during the earlier engineering lifecycle. For example, do you expect that OT&E would have significant membership on the various ACE agile teams or do you expect a test approach where OT&E is largely independent from design, development, sprint test teams?

Engagement with the various ACE agile teams would primarily be one day every two weeks to observe demonstrations of development completed in the previous sprint and the development planning for the next sprint.

11. Is there an incumbent currently performing this work for CSPO. If yes, can the government provide the current FTE count? Can the government provide the name of the current incumbent?

There is no incumbent performing this work.

12. Can the offeror utilize sub contractor’s specialized past experience in Section 5?

See Section 2.2 of solicitation. If Past Performance Questionnaire (PPQ) of a sub-contractor is provided, the contract number of the prime contractor with the customer and the contribution of the sub-contractor on that contract needs to be identified – last entry on the PPQ.

13. Can the government clarify if all of the Specialized Experience in Section 5 requires an

Attachment C questionnaire to be completed by customers?

Past Performance Questionnaires completed by customers supplement specialized experience claimed by the offerer. Specialized experience identified by the offerer without an accompanying Past Performance Questionnaire may be considered in the government’s evaluation.

14. Scope of Work – In section 3, Scope of Work, the government states the scope of work includes

ACE, ACS, AES and interfacing systems. Can the government provide the number of two-week

Spints, number of releases and formal Operational Tests, the government expects over the period of performance? This will allow offerors to properly scope the level of effort.

The ACE Agile Quick Reference guide (identifying one one-week planning sprint and six two-week development sprints per increment) and the ACE Roadmap ACE DevDep Schedule

(identifying 13 increments and associated teams) have been provided to allow offerors to properly scope the effort. The government anticipates 7 Operational Test events.

15. Can the government provide ACE Capability Development Roadmap?

The ACE Capability Development Roadmap has been provided as Tab 1 in the .xls file titled SOW

- ACE Development/Deployment Schedule v3.8 7/24/13.xls in the updated solicitation

16. In section 9 of the SOW, it references the Development Contractor? Can the government provide the roles and responsibilities of the Development Contractor vs. the Test and Evaluation contractor described in the SOW?

Section 9 was included to provide offerors insight into the Agile practices used during development. The government expects the OT&E support contractor to work with development contractor to identify and resolve risks early. In section 9, the word “contractor” refers to the development contractor.

17. In section 9, there are requirements for the contractor to perform multiple tasks that should be performed by the Development Contractor? Can the government clarify which tasks in this section pertain to the Test and Evaluation Contractor?

Section 9 was included to provide offerors insight into the Agile practices used during development. Section 9 does identify artifacts and practices that should be considered by the

OT&E support contractor (e.g. story boards, product backlog, release planning). The OT&E support contractor is not responsible for any of the tasks identified in section 9. In section 9 the word “contractor” refers to the development contractor.

18. In Section 12.2.1, Key Personnel, are there any required key personnel for this solicitation?

At a minimum, the government considers the contractor’s designated lead a key position. Any additional resumes submitted by offerors will also be considerd key positions.

19. In Section 12.4, Security Clearances, is a CBP full BI required in order to start on the program? Can the contractor personnel start after completing the required CBP paper work?

CBP may approve interim security clearances based on an initial assessment of suitability

20. On page 1 of the solicitation document, an email address is provided for electronic submission of quotes. However, on page 11 of the same document, notation is made of the requirement for one original and three additional copies, plus a CD-ROM, of the submission. Please confirm that an electronic submission only is acceptable or whether hard copies will be required.

Electronic submission only is NOT acceptable – See “Proposal Submission Requirements” section in Solicitation.

21. Section 2.2, page 1 - What Agile Software Development Lifecycle phase is ACE currently in?

ACE is currently in the hardening/integration for Increment 2 capabilities.

22. Section 6.5.1 Test Requirements, page 7 - What type of technical reports are being referred to?

Technical reports refer to any additional reporting requested by the Operational Test Agent.

Technical reports are not considered a formal contract deliverable.

23. Does the ACE Operational Requirements Document (ORD) currently exist? If so, when was it promulgated/approved?

The ACE Operational Requirements Document was approved on 20 August, 2013.

24. Does the ACE Test and Evaluation Master Plan (TEMP) currently exist? If so, when was it promulgated/approved?

A mature draft of the ACE Test and Evaluation Master Plan currently exists. It has not yet been approved.

25. Deliverable Table ACE OT&E Plan Page 8 - There is a due date of 30 and 45 days from end of test.

Is there a period of time between the due dates where the government will review and comment? If so, how will that affect the second due date?

The time period between draft and final documents is intended allow collaboration between the

Operational Test Agent and the OT&E support contractor. Formal government review will start once the final deliverable has been submitted.

26. Deliverable Table Operational Test Report Page 8 - There is a due date of 20 and 30 days from end of test. Is there a period of time between the due dates where the government will review and comment? If so, how will that affect the second due date?

The time period between draft and final documents is intended allow collaboration between the

Operational Test Agent and the OT&E support contractor. Formal government review will start once the final deliverable has been submitted.

27. Deliverable Table Letter of Observation Page 8 - Due Date: As Required. Is there a maximum number of required Letters of Observation?

There is not a maximum number of required Letters of Observation.

28. Deliverable Table Operational Test Report Page 8 - There is a due date of 30 days from end of test. DHS OT&E instruction allows 90 days for report delivery. Is there an Instruction that supersedes the 90 day requirement?

The Statement of Work for this effort supersedes the 90 day requirement. The increased frequency of OT&E reporting is intended to allow shorter duration events.

29. Where should we put our BOEs and do they count against the page count? Would it be acceptable for the BOEs to go within the Tab B workbook, below the pricing tables for each worksheet? If so, do you want the BOEs contained within the Tab C product as well?

BOEs not included in page count and should be within Tab B workbook only.

30. Will DHS provide a completed 1449 (less company info, etc.) to submit with the offeror’s proposal or do you expect the offeror to develop and submit the 1449?

SF1449 provided in this amendment.

31. Based on the contents of the Tab B workbook are we to assume the two CLINs are “OT&E Planning” and “OT&E Execution and Reporting”?

Changed in “Pricing Summary” section of solicitation.

32. Are the SLINs, associated with the CLINs just the base and option years or were you looking for additional detail regarding each year’s SLINs? As an example, OT&E Planning would be CLIN 1 and Base Year 1 would be SLIN 1, Option 1 would be SLIN 2, and Option Year 2 would be SLIN 3.

Is this correct? If not, can you please provide your CLIN/SLIN structure?

The pricing template included in the solicitation provides a template for identifying costs.

33. Should we insert a cover page and table of contents for each of the required proposal Sections for each Volume? Do the cover page and TOC count against the page count?

A cover page and table of contents are not required for each of the required proposal Sections, but would make it easier to read. The Cover page and TOC of the proposal are not used for the page count.

34. Given that Volume 1, Sections 1-4 have a 15 page combined limit, is it acceptable to combine these four sections into one contiguous section, with headings covering the required section titles?

Accepted – see modified solicitation

35. What Volume/Section should we place the Proposal Transmittal Letter?

First page of the Technical proposal.

36. Can the government provide an estimate of the number of two-week sprints and the number of 13-week release cadences anticipated per contract year (Base and Options)?

The ACE Agile Quick Reference guide (identifying one one-week planning sprint and six two-week development sprints per increment) and the ACE Roadmap ACE DevDep Schedule properly scope the effort. The government anticipates 7 Operational Test events.

37. SOW 13 requests the Contractor to provide a reasonably detailed estimate of anticipated travel.

However, the pricing tables provide a travel amount with instructions to not change. If travel is to be estimated, can the government provide travel location to the two formal OT&E events each year, and travel locations for the OT associated with the two week sprints and 13-week release cadences?

Locations for the OT&E events have not been identified yet. The number identified in the pricing template should be used to estimate travel costs. Work associated with the two-week sprints should be assumed to take place at the primary location of work.

38. Are the tasks described in the SOW currently being performed, and if so by whom?

39. Is there a incumbent contractor providing Operational Test and Evaluation (OT&E) Support to

CSPO?

40. How many parallel development sprints does the government anticipate?

Currently there are 8 teams developing in parallel. The government anticipates a maximum of 15 teams developing in parallel.

41. Of the development efforts described in the executive summary, how many are currently underway or are slated to begin?

15 teams developing in parallel.

42. How many CBP OTEs are expected to be completed during the base year and two optional years for ACE, ACS and AES The ACE Agile Quick Reference guide (identifying one one-week planning sprint and six two-week development sprints per increment) and the ACE Roadmap ACE DevDep Schedule properly scope the effort. The government anticipates total of 7 Operational Test events.

43. You have stated that the “ACE Agile Framework” shall be provided to bidders. What procedure must we follow to obtain the document.

The “ACE Agile Framework” was included in a 21 Aug update to the solicitation.

44. The proposal delivery terms were a little unclear. It appears that it is acceptable to email the response, however, you are requesting that the Cost Volume be submitted as one original and one on CD-ROM. If we chose to electronically submit our proposal is it acceptable email a softcopy of Volume 2 See “Proposal Submission Requirements” section in Solicitation.

45. How many simultaneous Sprints, User Stories and OT&E events are anticipated at any given time?

15 teams developing in parallel. No OT&E events are planned to take place in parallel.

46. Will the Sprint Planning Meetings and the T&E WIPT meetings be at CBP, 1801 N. Beauregard Boulevard, Alexandria, VA 22311?

Yes.

47. Section 10 states “Some work may be performed at other DHS CBP office locations throughout the Washington DC Metro area.” Will the OT&E Contractor need to have full-time staff at these locations? If so, please specify the locations and which work is expected at those locations.

The OT&E contractor will not need full-time staff at DHS office locations throughout the Washington, DC Metro area.

48. Proposal Submission Requirements - Font TNR 12 point. Please allow 10 point font for charts and graphics to improve their present ability while retaining readability.

Accepted – see modified solicitation.

49. Any graphs and charts are to be submitted in Adobe Acrobat (.pdf) format.

This instruction would require graphs and charts to be separate documents rather than embedded in the respective volume. Will the Government please clarify that the submittal format can be either Microsoft Word 2010 or Adobe Acrobat (.pdf) rather than a combination of both formats?

Accepted – see modified solicitation.

50. I.23 Payment and Invoice Instructions, “For invoices with payment terms less than net 30, the subject line for all emailed invoices to the NFC must include the following text: “Per CBP, Net [state # days] Invoice”. Do offerors have the ability to receive less than net 30 terms for this contract?

Yes, as stated in the prescribed directions.

51. Section 10 Government Furnished Equipment and Section 11 Access to Government Property and Facilities/Place of Performance states that CBP will provide workstations, network, etc. for the OT&E staff. This implies that the OT&E contractor does not need to provide computers, software, and/or network connectivity for its staff as their work will be accomplished on DHS systems. Please confirm this assumption.

CBP will provide workstations, CBP standard software, and network connectivity. The OT&E contractor does not need to provide computers, software, and/or network connectivity for its staff.

52. Further, if this is a valid assumption, please identify which of the following SOW requirements do not then apply to this contract as they are written for systems being developed for, attached to, or storing data outside of the DHS environment. This is important as there are requirements for independent assessments that will need to be priced into offeror proposals if these are firm requirements for the OT&E contractor:

15. Interconnection Security Agreements (OT&E would not be implementing any interconnections)

16. Security Requirements for Unclassified Information Technology Resources – specifically 16.e which requires accreditation that includes security test and evaluation and disaster recovery plan/continuity of operations plan. (OT&E would be using DHS systems which would already have been tested and would have disaster recovery plans)

18. Security Certification/Accreditation (OT&E will not be a system owner)

19.2 Security Authorization (OT&E is not providing infrastructure in support of a DHS information system)

Sections 15, 16, 18, 19 are standard clauses and would only be applicable when appropriate. If no non-CBP systems are interconnected or established the requirements would not be applicable.

53. Section 9. Agile Practices. This section refers to the “Development Contractor” and then the “Contractor”. The activities listed herein primarily seem to be the responsibility of the Development Contractor and the OT&E Contractor would be supporting or participating in a subset of these activities. The requirements in Section 9 all state that “the Contractor shall use.”

It appears that the “Contractor” in this case primarily refers to the Development Contractor, not the OT&E Contractor. Will the Government please clarify which Section 9 activities specifically pertain to the OT&E Contractor in order to bound the scope of this effort?

Section 9 was included to provide offerors insight into the Agile practices used during development. Section 9 does identify artifacts and practices that should be considered by the OT&E support contractor (e.g. story boards, product backlog, release planning). The OT&E support contractor is not responsible for any of the tasks identified in section 9. In section 9 the word “contractor” refers to the development contractor.

54. When was the last OT&E event conducted? Was it in line with the plan to conduct OT&E events approximately every 26 weeks?

The last OT&E event was conducted in early 2012 prior to the program moving to the Agile development model. It was not in line with the plan to conduct OT&E events approximately every 26 weeks.

55. Part I (Section A-H) and Part II (Section I), Part III (Section J) and Part IV (Section K-M) although synopsized are not explicitly included in the solicitation package. Is this part of the FAR 12.6, simplified acquisition procedures?

56. Will CBP provide the SF 33/1449?

SF1449 to be provided in an amendment.

57. Pricing is specifically broken into two areas, OT&E Planning and OT&E Execution and Reporting.

Is it correct to divide the SOW Section 6 tasks as follows for pricing purposes?

OT&E Planning

6.1 Program Management,

6.2 Attendance at Project Meetings,

6.3 OT&E Planning,

6.4 OT&E Events

OT&E Execution and Reporting

6.5 OT&E Execution and Reporting

A pricing template has been provided with the solicitation to assist offerors.

58. With OT&E events described as occurring no more than once every 26 weeks is it correct to assume that CBP envisions 2 OT&E events per contract year for pricing purposes?

That is a valid assumption.

59. What is the historical duration of recent ACE OT&E events such as ACE M1 or SE?

The last ACE OT&E event for M1 was conducted at three ports of entry over the course of three weeks. It covered multiple years of software development and may not be representative of ACE OT&E events in the future.

60. Will Tasks 6.1-6.4 continue during the twice annual execution of Task 6.5?

Tasks 6.1-6.4 are expected to continue throughout the duration of the task.

61. What local travel costs are to be accounted for in the OT&E Planning ODC cost estimate?

Any local travel costs outside of travel associated with data collection during the OT&E event should be included.

62. When will ACE Program-specific documents (MNS, ORD, CONOPS, TEMP, Capability Roadmap, etc) be made available to bidders as these core documents drive the scope and extent of OT&E including the development of TEMP Sections D&E and the Operational Test and Evaluation Plans and so underpin the labor estimate for this effort?

The ACE ORD and TEMP are documents with distribution limitations and will be made available by e-mail to all offerors who request them with an e-mail to jagdish.narayan@cbp.dhs.gov. The ACE Capability Development Roadmap has been provided as Tab 1 in the .xls file titled SOW - ACE Development/Deployment Schedule v3.8 7/24/13.xls in the updated solicitation – see Amendment.

63. Does ACE or do the current ACE core functionalities to be delivered (Air Manifest, Cargo Release, Entry Summary, Exports and Finance) have a TEMP (Draft or Approved) which reflects the Agile OT&E strategy described here? When will they be made available to the bidders?

The ACE TEMP includes a description of the ACE OT&E strategy. The ACE TEMP has distribution limitations and will be made available by e-mail to all offerors who request it with an e-mail to jagdish.narayan@cbp.dhs.gov – see Amendment.

64. OT&E Strategy Comments –Is there a ACE OT&E strategy (Draft or Final) which reflects the Agile concept or will the OTA, with the contractor, be responsible for building that strategy as part of the larger ACE OT&E plan development effort?

The ACE TEMP includes a description of the ACE OT&E strategy. The ACE TEMP has distribution limitations and will be made available by e-mail to all offerors who request it with an e-mail to jagdish.narayan@cbp.dhs.gov – see Amendment. The OTA and the OT&E support contractor will continue to build and refine the strategy as part of the ACE OT&E plan development effort.

65. Does the CBP Test and Evaluation Gate Review (TEGR) process apply to this effort The CBP Test and Evaluation Gate Review process does not apply to this effort.

66. When will CSPO define the OT&E test environment/location? Will all OT&E be conducted in the WDC area?

The Operational Test Agent will approve the OT&E test locations as part of the OT&E test planning. It is unlikely all OT&E will be conducted in the WDC area.

67. Can work be performed at the contractor’s facility if approved in advance by the CO/COR?

mailto:jagdish.narayan@cbp.dhs.gov

It may be possible. However, the primary location for work is the government site in Alexandria, VA.

68. I just wanted to confirm that there is a 10 page proposal limit for this opportunity correct?

The pages limits for each section are as stated in the Solicitation.

69. You provide no indications of labor category structure you expect within the solicitation materials. How will the government fairly evaluate cost proposals across offers with potentially radically different labor structures and hours?

The government will evaluate proposals in accordance with the evaluation factors identified in the synopsis.

70. You provide little information on the extent of the past or future OT&E activities expected to be performed by the contractor regarding the varying workload of the CSPO. How are offerors expected to judge appropriate staff sizing to provide a realistic price proposal?

The government expects offerors to use the information provided in the Statement of Work and Attachments in addition to the offerors past experience with similar tasks to provide a realistic price proposal.

71. Are offerors expected to provide sample resumes for each possible labor category proposed?

Descriptions of labor categories would be sufficient. Resumes will be accepted as well.

72. Who is the incumbent contractor?

There is no incumbent.

73. What hours/costs have been spent on this type of activity in past years?

Declined.

74. Is there an incumbent for this opportunity? If so, is the incumbent eligible to compete on this opportunity in any fashion (i.e. prime, subcontractor)?

There is no incumbent.

75. What is the current/estimated level of effort for this opportunity (i.e. number of full-time and part-time employees and their corresponding labor categories, number of hours, etc.)?

The Independent Government Cost Estimate will not be provided to any potential bidders.

76. Would the Government consider extending the due date of the responses?

See Amendment.

77. For the Technical Proposal, does the page count also exclude the cover page and table of contents?