CRM_Pre-Proposal_Conference_Slides-23_May.ppt

This Briefing is:

UNCLASSIFIED

Headquarters Air Combat Command

SOURCE SELECTION SENSITIVE

CAF Cockpit/Crew Resource Management (CRM) Aircrew Training and Courseware Development Contract

Pre-Proposal Conference

SOURCE SELECTION SENSITIVE

ADMINISTRATIVE

To Familiarize Contractors With The Government’s Requirement

To Provide Clear Understanding Of The Evaluation Process

PURPOSE

AGENDA

0900 – Welcome And Introductions 0910 – Program Review / Source Selection and Contractual Issues 1000 – Break 1015 – Question review 1100 – Case Study / LMS-TMS Review 1145 – Break for lunch / Courseware Review 1330 – 1500 Courseware Review as required 1500 – Closing remarks / Submit Questions

INTRODUCTIONS

Mr. Kirt Cattron - ACC/TRSS Program Manager kirt.cattron@us.af.mil (757) 764-4432

Mr. Ray Churchill - ACC/TRSS CRM Program Manager raymond.churchill@us.af.mil (757) 764-4280

Capt Hannah Kosirog – ACC AMIC/PKB Contract Officer hannah.kosirog.1@us.sf.mil (757) 575-2153

Capt Cassie Hardcastle - ACC AMIC/PKB Contract Manager cassie.hardcastle@us.af.mil (757) 764-5259

Attendees present:

Crew Training International, Convergent Performance, AOCE, Spiral Solutions and Technologies, Inc.

RULES OF ENGAGEMENT

Contracting office (ACC AMIC/PKB) is the POC for information.

Any remarks, explanations, or answers to questions at this conference shall not qualify the terms of the solicitation.

Terms of the solicitation and PWS remain unchanged unless the solicitation is formally amended in writing and issued by the contracting officer.

All questions must be submitted in writing and will be posted on FEDBIZOPs (http://www.fbo.gov/).

PROGRAM OVERVIEW

CRM Program History Current CRM Program PWS Requirements Procurement Description Procurement Evaluation Program Funding / Schedule / Risk Submitted Questions

CAF CRM PROGRAM HISTORY

Fourth contract solicitation for this requirement:

Previous solicitations were: 1999, 2005 and 2011

Current CAF CRM customers:

Air Combat Command (ACC) Air Education And Training Command (AETC) Pacific Air Forces Command (PACAF) Air Force Global Strike Command (AFGSC) Air Force Reserve Command (AFRC – added May 2014)

Current acquisition:

Total small business set-aside Crew Training International (CTI) 1 Feb 2013 - 31 Jan 2018 No collective bargaining agreement

SOURCE SELECTION SENSITIVE – SEE FAR 2.101 AND 3.104

CRM PROGRAM

AFI 11-290 Mandates CRM Training Mandates training phases Mandates the 6 skills that must be taught Must be delivered by a trained facilitator

CRM Program Goals Maximize operational effectiveness and combat capability Preserve Air Force personnel and material resources

CRM Training Objectives Develop aircrew skills in recognizing and responding to the conditions that lead to aircrew error Develop aircrew proficiency in CRM skills

NEW REQUIREMENT

CAF Customers:

Air Combat Command (ACC) Air Education and Training Command (AETC) Pacific Air Forces (PACAF) Air Force Global Strike Command (AFGSC) Air Force Reserve Command (AFRC)

Could add new units or other MAJCOMs

6 year contract Base year with 5 options Feb 2018 thru Jan 2024

KEY PWS REQUIREMENTS

Contractor to provide all personnel, equipment, tools, materials, supervision, and services necessary to perform all aspects of the Performance Work Statement

Specific tasks for this contract are divided into three main categories:

Contract Aircrew Training (CAT) Courseware Development (CWD) General Tasks

KEY PWS REQUIREMENTS

Contract Aircrew Training IAW Appendix 1:

ACC: All MDS

AETC: Luke (F-16/F-35), Eglin (F-35) & Holloman (F-16) Randolph T-38 optional CLIN

PACAF: Fighters and AWAC units

AFGSC: B-1, B-2, B-52, E-4 and T-38

AFRC: Fighters, Bombers, C2ISR Aircraft and Aerospace Evacuation Aircrew Includes personnel who support the mission including flight surgeons and contract instructors

KEY PWS REQUIREMENTS

Contract Aircrew Training per Appendix 1 Trains approx. 9,000 personnel per year ~ 50 different bases world wide Unless otherwise noted, at least 2 site visits per year Class sizes vary by MDS and course Ideal class size 8 – 20 Smaller units may only have 4 Instructor classes are usually small No auditorium size CRM training Contractor is responsible for scheduling all CRM training Communication and flexibility are the keys to success

KEY PWS REQUIREMENTS

3 different MDS specific courses – Appendix 2

Formal Training (4 hours) Date is outlined in syllabi; priority for scheduling Different formats (4 hr; 2 x 2hrs; 2 hr, 1hr, 1hr) Continuation Training (2 hours) Can combine MDSs for some CT classes Instructor Upgrade Training (2 hours) Date also specified in instructor upgrade syllabi IUP course re-hacks currency for CT

KEY PWS REQUIREMENTS

Courseware Development (CWD):

Create, review and revise CWD to support all weapon systems FTU and Instructor upgrade courses revised as required Develop new CT courses every 2 years New case studies annually for AETC courses CWD Delivery Priorities (PWS 1.5.5) Develop all CWD IAW publications in Appendix 4 Develop new case studies in response to any mishap board recommendation Develop additional training materials IAW Appendix 3

KEY PWS REQUIREMENTS

Learning Management/Training Management System (LMS/TMS) Contractor maintained commercial off the shelf Access by a web site address Access for all USAF Active Duty, NGB, AFRC & Civilian Must contain the master schedule and the history of all classes that have been conducted Additional training materials as outlined in Appendix 3 Special access for attendance to track currency

FUNDING

Each MAJCOM will be responsible for their funding

CAT & CWD CLIN for each MAJCOM Optional CLIN for AETC T-38 at Randolph

Pricing CLINS will be set up as follows:

ACC 57% (Base year) 56% (Options years) AETC 18% (Base year) 19% (Options years) PACAF & AFGSC 9% each

AFRC 7%

PROCUREMENT / DESCRIPTION

General Tasks Maintain Master Files Maintain CRM training materials Plans Delivered with Proposal Phase-In / Phase-out Plan Training and Certification Plan Quality control Plan

Plans Delivered 30 days after award ISD Management Plan Courseware Development Schedule

PROGRAM RISKS

CONTRACT SCHEDULE

QUALITY ASSURANCE

Contracting Officer’s Representative (COR)

COR appointments will be provided to the awarded contractor in writing

Capt Cassie Hardcastle

ACC AMIC/PKB

SOURCE SELECTION AND CONTRACTUAL ISSUES

Evaluation Factors Initial Eval

Ratings

Initial Evaluation Debrief

EVALUATION PROCESS

Offeror Proposals

ENs

Close Discussions

Receive Final Proposals

Award w/o Discussion

Initial Evaluation Brief

Interim Evaluation

Competitive Range Determination Discussions ENs

Final Evaluation

Revise Ratings Award

Source Selection Decision Briefing Best Value Decision

ACQUISITION OVERVIEW

Authority

FAR Part 12: Commercial Contracts FAR Part 15.3: Negotiated Contracts AFFARS 5315.3: Mandatory Procedures DoD Source Selection Procedures effective 31 Mar 2016

Notice to Offerors

Sufficient number of Proposals/Competition is expected

Government intends to make award without discussion but reserves the right to conduct discussions if deemed necessary. Therefore, you are cautioned to submit your best proposal upfront.

ACQUISITION OVERVIEW

Lowest Price Technically Acceptable (LPTA) IAW FAR 15.101-2 and the DoD Source Selection Procedures effective 31 Mar 2016

Small Business (SB) set aside

NAICS: 611512

Base plus five, one-year Option Periods

Firm Fixed Price

PROGRAM REVIEW

Terms of the solicitation and PWS remain unchanged unless it is formally amended in writing and issued by the Contracting Officer

Questions during this conference must be in writing

RFP issued 05 May 17

Proposals due NLT 1500 hrs on 09 Jun 17

Conference questions and answers will be posted on FEDBIZOPs (http://www.fbo.gov/)

PROCUREMENT HISTORY/DATA

Follow-on acquisition; 100% small business set-aside

The incumbent is CTI

Contract Type

Non-personal services Firm-Fixed Price Off-Fiscal year contract -- Funding -- Performance Period

SOLICITATION REVIEW

SECTION 5 – List of Attachments

SOLICITATION REVIEW

NO PAST PERFORMANCE VOLUMES

For the purposes of contractor responsibility determination, past performance information may be obtained from sources such as the Government’s Past Performance Information Retrieval System (PPIRS), the Defense Contract Management Agency, interviews with Program Managers, COs, and Award Fee Determining Officials, and other sources known to the Government. (Pg 88, RFP)

CONTRACTING ASPECTS:

MILESTONE TIMELINE

EVALUATION AND AWARD CRITERIA

FACTOR 2 – Price

PROCUREMENT EVALUATION

Subfactor A: Manning Requirements and Analysis Aspect 1: Manning Requirements and Workload Analysis Subfactor B: Training Management and Approach Aspect 1: Training Management and Approach Aspect 2: Effective Management of Personnel Resources Aspect 3: LMS/TMS Subfactor C: All Other Plans / CFE Aspect 1: Phase-In/Phase-Out Plan Aspect 2: Training and Certification Plan Aspect 3: Quality Control Plan

TECHNICAL

Technical Rating

Proposals will be assessed a rating of "Acceptable" or "Unacceptable" at the subfactor level; an overall factor level rating will not be assigned To be eligible for award, offerors are required to receive an “Acceptable” rating in each technical subfactor; proposals that exceed the evaluation criteria will not receive higher ratings In assessing the acceptability of each subfactor, weaknesses or deficiencies will be noted; proposals that receive a deficiency or a weakness in a subfactor will receive an “Unacceptable” subfactor rating

PRICE EVALUATION

Price will be evaluated using techniques established in FAR 15.404-1 to ensure the Government receives a fair, reasonable and balanced price.

There is a great expectation of competition for this requirement; thus offerors are cautioned to present their best price proposal up-front.

The Government will evaluate offers for award purposes by adding the total price for all CLINS to include phase-in, basic year, option years, and fifty percent (50%) of the last option period price to cover the option to extend services IAW clause 52.217-8*.

*Offerors shall not include pricing in their proposal for this clause.

BREAK

ADVANCE QUESTIONS

QUESTIONS & ANSWERS

QUESTION 1: Will USG provide a count of all FTE and PTE personnel (and approximate PT hours per PTE) supporting the current effort?

ANSWER: No. The Government has provided all the workload that will be accomplished in this effort in the Appendixes 1, 2, and 3. It is up to each contractor to propose the number of FTE and/or PTE required to accomplish the work.

QUESTIONS & ANSWERS

QUESTION 2: With respect to the Draft PWS, Section 3 Government-Furnished Property, paragraphs 3.1 and 3.2 (Page 16), can the USG confirm that not a single student workstation setup, common classroom projector/smartboard, nor digital/electronic display of any type will be available in the classrooms where CRM training is to be provided?

ANSWER: Some of the materials referenced in the question may be provided at some locations. However, some units may only have the GFE as listed in the PWS para 3.1. The contractor should be prepared to bring all equipment required (and as outlined in their proposal) to each class that is being taught.

QUESTIONS & ANSWERS

QUESTION 2 Follow on: Government-Furnished Facilities, Government-Maintained (PWS para 3.1.

ANSWER: “…These rooms will contain only enough desks, tables and chairs to accommodate the students of the class-size identified in this PWS. The contractor should be prepared to provide any other equipment necessary to perform the contract as proposed (e.g., projectors, computers, dry-erase boards, etc.). These rooms will only be used by the contractor during the time that CRM training is scheduled and must be returned to their original state.”

QUESTIONS & ANSWERS

QUESTION 3: Will USG entertain, and Air Force facilities support, a virtual training solution (CRM contract instructor available virtually to provide all required classroom training) for all or part of the CRM requirement vice a contract instructor physically present at the site(s)? This approach to training would greatly reduce USG costs by significantly decreasing travel expenditures.

ANSWER: No. The Government expects a contract instructor to be physically present at each site listed in Appendix 1 of the PWS. The Government has tried CRM training virtually and found it to be less effective as a contract instructor being physically present in the classroom.

QUESTION 4: Reference sections 1.3 and 1.5 on page 29-30 of the Draft PWS. The tables do not show any workload for the T-38s at Beale. Should there be workload for the T-38s at Beale?

ANSWER: We eliminated the requirement for T-38 training at Beale because there are just a few T-38 only pilots there. Those T-38 only pilots receive their refresher (CT) CRM training with the U-2 pilots at Beale.

QUESTION 5: Reference page 89 of the RFP, what does “cross-matrixing” mean?

ANSWER: Refers to the utilization of a person across multiple labor categories.

QUESTION 6: Per the LMS/TMS, what format(s) are currently used to store the data/records? What is the size of the current archive? Is it COTS or proprietary?

ANSWER: The LMS/TMS is COTS to store all the data/records. All the information will be provided to the Gov’t “in a usable format at the completion of the contract”. This info will be passed to the winning contractor during phase-in.

QUESTION 7: Per downloading of training material on USAF desktop computers, what is the USAF baseline and does it need to be available for older window versions like XP and VISTA?

ANSWER: The current standard for USAF desktop configuration is Windows 7. The USAF is migrating to Windows 10 but will not be fully operational until after contract award.

QUESTION 8: Why does the Gov’t require a Secret facility clearance to develop an unclassified program?

ANSWER: Some CRM instruction may be conducted in secure locations where the contract instructor would be required to have a security clearance to access the area.

QUESTION 9: Asked 3 questions related to Appendix 1 Tables 5.2, 5.4 & 5.5 “These tables indicates that there are notes associated with the bases but notes are not visible”.

ANSWER: The notes listed below Table 5.1 “Fighter” relate to all tables under Section 5.0.

QUESTION 10: Ref App. 1, Section 3.1.3 “Training for T-38 pilots” does the gov’t wish to see a separate CLIN number, and price for every contract year?

ANSWER: Yes, this is an optional CLIN that needs to be priced separately from the other 8 CLINS and will be priced for every contract year.

QUESTION 11: Ref App 1, Section 3.2, the 2nd sentence says “Basic Qual courses will be split into 3 different phases: 3 hours TR, 1 hour A/A and 1 hour A/G? This equals 4 hours when para 1.5.1.1 states that the FTU course will be a 4 hour block of instruction. Which is correct?

ANSWER: The “3 hours TR” is a typo. It should read “2 hours TR”. The PWS will be revised to reflect this change.

QUESTION 12: On page 34 of the PWS (Appendix 2, Section 1, Bombers), the table does not show a need to develop an FTU TX course for these aircraft. However, page 32 of the PWS (Appendix 1, Section 4.2, Formal Training Unit (FTU) Personnel) seems to indicate the need for an FTU ReQual/SORC class for all types. Is this a full FTU (4 hour) class, or is this similar to the 2 hour FTU TX courses for other aircraft types?

ANSWER: The ReQual/SORC class for the bombers will be taught the current CT course.

QUESTION 13 & 14: Ref App 3, Section 5 “Additional CRM Training Materials” and “Briefing Guides for AFRC” represent totals for the contract per year, not totals per MDS per year?

ANSWER: Yes, it represents totals for the contract year, not per MDS.

QUESTION 15: For Course Ware Development purposes, can you please define what you mean and/or the difference between new courseware “development” vs “ completely revised”.

ANSWER: In this case they are the same. The wording in para 1.5.1.3 of the PWS has been re-worded.

QUESTION 16: To accurately price travel and labor hours, can you provide any potential options for the “TBD RPA Bases”?

ANSWER: The Government does not know where these RPA bases will be located. All we do know is that they will be located at a current USAF CONUS base.

QUESTION 17: Can we assume that AE units training will be accomplished on drill weekends or annual training event and that unit personnel will be made available during that time for CRM training?

ANSWER: Yes. Per the PWS, the contractor will make one trip annually to each of the AE units listed and train as many personnel that are available.

QUESTION 18: Class sizes: can we assume the customer will work to schedule and help us fill classes? Are there historical numbers available for average class sizes.?

ANSWER: Per 1st question, Yes. The contractor is responsible for working with each unit to schedule CRM training. Per 2nd question, also Yes but class size really depends by unit size. Some small FTU’s and small operational units have only 4 students per class. Larger units with hundreds of aircrew to train per year have larger class sizes.

QUESTION 19: Are there historical numbers on re-teach/accomplishment factors or non-availability of unit personnel for classes? If the vendor is scheduled to perform a class but less than 8 personnel are available to be trained, is there a remedy plan from the Government?

ANSWER: The contractor will provide training IAW its’ Training Workload Management Plan. Per PWS para 1.5.1, the contractor should be prepared to teach any CRM class that has 4 students up to 20 students.

QUESTION 20: Is there any specific requirements for the Learning Management System or Training Management System (LMS?TMS)? Do we need to integrate with individual units’ LMS/TMS?

ANSWER: All requirements of the LMS/TMS are listed in PWS para 1.7.2. This LMS/TMS is not required to integrate with any other USAF unit LMS/TMS.

QUESTION 21: Because the courseware will include FOUO data, should we assume that LMS/TMS access needs to be protected by measures such as Public Key Infrastructure (PKI) or other encryption methods?

ANSWER: The LMS/TMS does not need to be protected by measures such as PKI or other encryption methods. All training materials loaded on the website must be sanitized to meet all FOUO and USAF Mishap Prevention Program regulations.

QUESTION 22: On page 28 of the PWS (Appendix 1, table 1.3), there is a note (2) that states “Requires four trips annually to support SO FTU.” This SO FTU is not reflected in Paragraph 1.4 immediately below it. Should this be treated as a 2 hour course?

ANSWER: The SO FTU support is reflected in para 1.4 of App 1. It’s listed as “Grand Forks AFB, ND, RQ-4, Basic-Qual, and 60” to train per year.

QUESTION 23: On page 29 of the PWS, (Appendix 1, table 1.4), there is a UIP course listed for Tyndall AFB. Is this a different course than the INST course? If they are the same, is this number of students already accounted for in Table 1.1 of the PWS?

ANSWER: No, these are the same courses. The PWS has been revised the PWS by deleting “Tyndall F-22A UIP” row in Table 1.4 and added the 10 to the Tyndall CT requirement in Table 1.1.

QUESTION 24: Are pilots who meet the criteria of Section 4.1.3, but have no experience as a mission crew member in a particular MDS, acceptable as instructors for a class composed only of mission crew members?

ANSWER: Yes, if they were a pilot of that weapon system and are familiar with the mission crew and how they interact as a crew.

QUESTION 25: On page 5 of the PWS, Section 1.5 Overall Requirements, paragraph 1.5.1.4 Instructor CRM Training, states “The contractor shall train instructors to use CRM Performance Assessment Tools to…” Do these Performance Assessment Tools already exist, or will the contractor be required to create them?

ANSWER: Yes, they do exist and will be displayed at the Pre-proposal conference.

QUESTION 26: Based on the requirement found in PWS 4.1.3 and MDS count numbers by location found in Appendix 1 and to ensure the correct qualified Instructor/SME student Ratio, can the RQ-4, MQ-4, and RQ-170 be considered similar enough for CRM purposes to be taught by an instructor with any of those platform qualifications?

ANSWER: Yes.

QUESTION 27: Based on the requirement found in PWS 4.1.3 and MDS count numbers by location found in appendix 1 and to ensure the correct qualified Instructor/SME student Ratio, can all variants of the same basic platform (i.e., F-15C/E, F-16C/D be considered similar enough for CRM purposes to be taught by an instructor with any of those platform qualifications? Is the QF-16 to be trained under the F-16 syllabus or under the same syllabus as the RQ-4, MQ-4, and RQ-170?

ANSWER: Yes to the first question. Per the 2nd question, the QF-16 is listed as a separate CT course in Appendix 2, para 3.

QUESTION 28: IAW PWS 1.5.1.1 FTU BQ Training “course will be a 4 hour block of instruction” and IAW PWS 1.5.1.2 FTU Requal or TX training “This course shall be 2 hours in length”. In Appendix 1, 3.2.2 and 3.2.3, do the F-35 TX and BQ both require 4.0 hr training courses as they are FTU’s?

ANSWER: The BQ course is 4 hours and the TX course is 2 hours.

QUESTION 29: Are the Global Strike E-4 FTU count numbers listed Appendix 1, 4.2 at Offutt AFB in addition to the numbers listed in the Appendix 1, 1.4 CAF FTU Offutt or are they included in the Offutt numbers?

ANSWER: The numbers are separate. The reference to “E-4” was deleted in para 1.4 for ACC FTU training numbers at Offutt AFB.

QUESTION 30: Are there any new courses to be instructed that there is no existing courseware currently available at contract award?

ANSWER: No.

QUESTIONS & ANSWERS

QUESTION 31: Continued

ANSWER: Reference PWS para 1.5.1.3 Continuation Training (CT) CRM, 6th sentence states “AETC CT courses will have new case studies added on an annual basis (per the AETC requirement for annual CRM CT)”. Changing the case studies on an annual basis revises the courseware making it different than the previous course.

QUESTION 32: Is there an LMS/TMS currently in place and, if so, will it be made available to the awardee for review? If an LMS/TMS will not be made available to the awardee, will the files from the previous LMS/TMS be made available to the awardee?

ANSWER: Yes, the current LMS/TMS will be demonstrated during the pre-proposal conference. All files located on the LMS/TMS will be available to the awardee during phase-in.

QUESTION 33: IAW Ref 1.7.2, Pg. 12, “The contractor shall develop an automated CRM LMS/TMS.” What is the expected deliverable timeline on this system?

ANSWER: It should be operational at the start of the contract.

QUESTION 34: Is there a specific location where the Contract Manager and alternate as well as the Security Manager must reside or is it at the discretion of the contractor?

ANSWER: Discretion of the contractor.

QUESTION 35: IAW 4.1.3, pg. 18, “single seat fighter experience will be required for teaching single seat CRM and large aircraft experience will be required to teach large aircraft CRM”, what is required for the RQ-4, MQ-4, and RQ-170 i.e. drones?

ANSWER: Remotely Piloted Aircraft (RPA) experience is desired to teach all RQ-4, MQ-4 and RQ-170 CRM classes. Per PWS Para 4.1.3, “the Government will give some flexibility with this requirement in regards to the newer Weapon Systems”.

QUESTION 36: IAW 2.3, pg. 31, Note 2: “(2) The contractor will be prepared to teach any of the following CT courses during each site visit: Any bomber, any fighter and any C2ISR class. The unit will inform the contractor at least 30 days prior to the training the types of courses required during the annual visit.” Does this mean 30 days out the contractor will know exactly who needs to deploy to these locations or is the contractor still responsible for deploying a bomber, fighter, and C2ISR instructor on each of these deployments regardless?

QUESTION 36: Continued

ANSWER: The contractor will send one instructor to Guam annually to provide 2 days of CRM training. The contractor will decide which type of instructor to send 30 days prior based on the type of CRM classes to be taught.

COURSEWARE REVIEW

8 Binders containing different CRM training courses &materials

E-3 AWACS Flight Deck FTU Course (4 hours) A-10 FTU #1 (2 hours) A-10 FTU #3 (2 hours) C2ISR Instructor Upgrade (2 hours) F-35 FTU #1 & 2 (2 hours) F-35 FTU #3 (1 hours) F-35 FTU #4 (1 hours) RQ-4 Continuation Training (2 hours)

Performance Assessment Toolkits

AGENDA

0900 – Welcome And Introductions 0910 – Program Review / Source Selection and Contractual Issues 1000 – Break 1015 – Question review 1100 – Case Study / LMS-TMS Review 1145 – Break for lunch / Courseware Review 1330 – 1500 Courseware Review as required 1500 – Closing remarks / Submit Questions