Program_Management-Lessons_Learned.pdf

Experimental Spaceplane (XS-1)

Mr. Jess Sponable, TTO Program Manager

Management Lessons Learned

Flight Demonstration Programs

7 October 2013

Distribution Statement A – Approved for Public Release, Distribution Unlimited.

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 2

Program management: Long history

“It must be remembered that there is nothing more difficult to plan, more doubtful of success, nor more dangerous to manage than the creation of a new system. For the initiator has the enmity of all who would profit by the preservation of the old institutions, and merely lukewarm defenders in those who would gain by the new ones.”

The Prince, Machiavelli, 1513

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 3

Purpose

• Reviewed lessons learned from Government programs

• Early NAVSTAR GPS program

• Gen Abrahamson’s Seven Commandments of Management

• MSTI, Clementine, SSRT (DC-X) and other SDIO programs

• Reviewed industry lessons learned

• Lockheed Martin – Skunkworks Management

• McDonnell Douglas – LOSAT, Delta 180/181, Delta Star

• Rockwell International – X-29, etc.

• Consolidated ten lessons for future programs

• Brief originally compiled for DC-X program transition to NASA

• Updated 2005

Review Lessons from successful Space Demo Programs

Consolidate Lessons for Future Flight Demonstration Programs

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 4

Lessons learned: Early NAVSTAR GPS

• GPS background

• Program established in 1970’s

• Six RDT&E satellite constellation began launching in 1978

• Essentially X-Satellites

• Augmented later with additional RDT&E satellites

• Preceded by NRL’s Navigation Technology Satellite, NTS 1 & 2

• Early user support soft (except Navy)

• Lessons

• Program objectives clearly defined

• Navigation technology demonstrated on NRL NTS-1 & 2

• GPS RDT&E “X-Satellites” proved CONOPS & remaining technology

• Program “center of gravity” was satellites – user equipment followed

• “Streamlined” FFP contract with initial value of $42M

• Satellite office manning kept small

• Complexity of joint service program well managed

• Commercial potential grossly underestimated

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 5

Lessons learned: National Aero-Space Plane

• Technology objectives not realistic for that era

• Airbreathing doesn’t imply “Aircraft-Like” O&M

• Achieving aircraft like O&M is design process independent of propulsion mode or what vehicle looks like

• Program sold as all things to all people--don’t oversell

• Long/complex programs require interim successes to survive

• Complex DARPA/Navy/AF/NASA Joint Program Office/funding

• Some would say – never again a “National Program”

• Pre-Major Defense Acquisition Program status simplified execution

• Lack of space or launch vehicle experienced personnel

• It was more than an airplane, also a specialized launch vehicle

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 6

Lessons learned: X-33, X-34, X-37, SLI/NGLT

• Select programs

• Too much new technology + flight = Failure (X-33)

• Gov’t changed requirements: no single string avionics (X-34)

• Over-specified solution

• 9 month mission (X-37)

• Not allowed to use existing NK-39 rocket engine (X-34)

• Vehicle types, technologies, costs, schedules, etc. (SLI/NGLT)

• Overly conservative designs/margins (SLI/NGLT)

• All programs

• Cumbersome management – too many people

• Nationwide teams – not “Skunk Works” programs

• Complexity of NASA centers as subcontractors

• Very large Gov’t team/Lot’s of Gov’t Help

• Program needed a gov’t program manager, not a “partner”

• Demonstrator contracts awarded without adequate design

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 7

Lessons learned: BMDO programs

Delta 181

Delta 183/Delta Star Delta 180

LOSAT/LACE

ATP to Launch – 14 Months ATP to Launch – 13 Months

ATP to Launch – 18 Months ATP to Launch – 13 Months

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 8

Lessons learned: BMDO programs (cont.)

MSTI-I

DC-X

LOSAT-X

Clementine

ATP to Launch – 30 Months ATP to Launch – 24 Months

ATP to Launch – 12 Months ATP to Launch – 23 Months

© Ball Aerospace

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 9

Lessons learned: DC-X BMDO program

• Streamlined acquisition

• ~$67M through first flight

• 18 month to roll out/2 yrs to flight

• ~$80 total DOD investment

• Focus was “aircraft-like” ops

• Turnaround time: 26 hrs (8 hr goal)

• Maintenance hours/flight < 1,000

• Call up/alert: 2-3 hr response time

• Autonomous & small ops crew sizes

• Flight control: 2

• Ground control: 1

• On-aircraft labor: 6-12

• Total ops team: 25-35

• Min facilities required < $600K

• Streamlined ops control center < $6M

• Low marginal flight cost ~$200K

• “Aircraft-like” flight ops

• Flight abort/engine out landing

• Incremental flight testing

• All weather: sun, rain, sand,...

Contract Award Aug 91

First Flight 18 Aug 93

DC-X Demonstrated 2 yr, $67M Program

Key reasons for BMDO successes

• Single empowered program manager

• Small Gov’t and contractor teams

• Focused, limited demo(s) – didn’t set bar too high, focused on key issues

• Streamlined contracting and management processes

Key reason for failure

• No User to “catch” follow-on program

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 10

Industry perspective: Kelly Johnson’s Skunkworks rules

4. A very simple drawing and drawing release system with great flexibility for making changes must be provided in order to make schedule recovery in the face of failures. [Applies to computer codes as well.]

5. There must be a minimum number of reports required, but important work must be recorded thoroughly.

6. There must be a monthly cost review covering not only what has been spent and committed but also projected costs to the conclusion of the program. Don't have the books ninety days late and don't surprise the customer with sudden overruns.

7. The contractor must be delegated and must assume more than normal responsibility to get good vendor bids for subcontracts on the project.

8. Push basic inspection responsibility back to the subcontractors and vendors. Don't duplicate so much inspection.

9. The contractor must be delegated the authority to test his final product in flight. He can and must test it in the initial stages.

10. The specifications applying to the product must be agreed to in advance of contracting.

11. Funding a program must be timely so that the contractor doesn't have to keep running to the bank to manage cash flow.

12. There must be absolute trust between the customer and the contractor with very close cooperation and liaison on a day-to-day basis. This cuts down misunderstanding and correspondence to an absolute minimum.

13. Access by outsiders to the project and its personnel must be strictly controlled. Because only a few good people will be used in most areas, compensation must be based on performance and not the number of personnel supervised.

1. The Skunks Works program manager must be delegated practically complete control of his program in all aspects. He should have the authority to make quick decisions regarding technical, financial, or operational matters.

2. Strong but small project offices must be provided by the customer and the skunk works.

3. The number of people having any connection with the project must be restricted in an almost vicious manner. Use a small number of good people.

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 11

Gov’t management rules: Demonstration programs

I. Agree to clearly defined program objectives in advance

II. Single manager under one agency

III. Small government and contractor program offices

IV. Build competitive hardware, not paper

V. Focus on key demonstrations, not everything

VI. Streamlined documentation and reviews

VII. Contractor integrates and tests prototype

VIII. Develop minimum realistic funding profiles

IX. Track cost/schedule in near real time

X. Mutual trust essential

INCENTIVIZE…INCENTIVIZE…INCENTIVIZE

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 12

Agree to clearly defined program objectives in advance

• Establish program “vision” supported by relevant test objectives

• Push outside of envelope, create sense of urgency

• Don’t settle for incremental advances “just because”

• Pushing envelope creates controversy…

…ignore and press ahead

• Agree to cost, schedule and top level functional objectives vs. detailed requirements

• Include design reference missions for operational objectives

• Avoid over specification

S p e c P a g e s

U-2 SR-71 HAVE

BLUE

F-117 SSRT

DC-X

(SOW)

RULE ONE

• Two essential agreements needed

• Program Manager to Program Executive Officer (PEO) or equivalent

• Contractor to Government Program Manager

• Program Manager should zealously guard against changing requirements

Never Tell People How To Do Things. Tell Them What To Do and They Will Surprise You With Their Ingenuity.

Gen George S. Patton

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 13

Single manager under one agency

• Room for only one Gov’t Program Manager reporting through one Agency

• Empower Program Manager to make all decisions consistent with law

• Program Manager should report directly to source selection/decision making authority

• Joint agency funding must not necessitate joint agency management

• i.e., Joint funding works when one agency has clear lead

• Pick right Program Manager

• Should understand technology and system trades

• Should implement program objectives as honest broker

• Program success should be more important than next promotion

• Requires ability to make decisions – promptly and at right time

…Willing to ask forgiveness after, rather than prior permission

• Contractor Program Manager subject to same rules within corporation

• Corporate commitment to operate in fast track mode

Where “Unity of Command” of Command is Weak Problems Grow

RULE TWO

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 14

Small Government & contractor program offices

• Keep program offices tiny

• Small tooth to tail ratio

• Co-locate all team functions, technologies and disciplines

• Organize into concurrent engineering teams to develop a sense of “ownership”

• Design-in quality, O&S up-front…

…not after the fact

• Push responsibility to lowest level

“Business As Should Be” Requires Far Fewer People Than “Business As Usual” (BAU)

RULE THREE

M a n n in g L e v e l

F-117 X-31 DC-X

Govt Program Office

Contractor Program Office

Contractor Peak Manning

Program Office Manning

• Program Manager needs first-rate front line managers

• Chief Engineer, Finance manager, Contracting Officer

• All managers are working managers

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 15

Rockwell example: Small core team yields large cost savings

X-29 TYPICAL

BAU PROTOTYPE

X-31 BASB

(ESTIMATES)

ACHIEVED

FOR X-31

• No “Illities”

• Minimum Configuration Control

• Min OPS Analysis

• No Production Drawings

• No Special Systems Development

Min Tooling

Soft Tooling

Less Off-line Planning

Layout vs Detailed Drawings Machinists vs Machine Operators

Technicians vs Assemblers

No Quantity Buy Discount

Reduced QC

Reduced Planning

No Order Release

Use of M. E.

No Shop Coordinators

No Expeditors

• Overall Personnel Reduction (Mgmt, Engineering, Manufacturing)

• Experience Based Design

20%

90% - 100%

115%

20%

50%

60%

BASB APPROACH

ENGINEERING

TOOLING

FACTORY

LABOR

MATERIAL

SUPPORT

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 16

Build competitive hardware, not paper

• X = Demonstrator is Worth 1,000,000 Words

• Extensive prime and subcontractor competition reduces risk and cost

• But down-select quickly to flight hardware

• Don’t dilute program and resources by dragging out competition

• Compete initially, down-select quickly and build demo hardware

• Typical of DARPA and BMDO programs, aka DC-X, Falcon, etc.

• Don’t forget software – freeze and code core modules early in program

• Software rapid prototyping/auto coding capability must be stressed early-on

…Order of magnitude savings demonstrated on DC-X very complex NG&C

• Push, push, push demanding schedules

A High Priority Should be Given to Building and Testing Prototype Systems and Subsystems Before Proceeding With Full Scale

Development. This Early Phase of R&D Should Employ Extensive Informal Competition and Use Streamlined Procurement

Practices… Packard Commission

RULE FOUR

Words1000

Picture 1

Pictures 1000

Demo 1

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 17

• Differentiate between development of operational system and X-demo/Y-demo

• Operational system must support all requirements

• X- & Y-demos are steps to lower risk & cost

• If uncertainty exists, better to build X-demo, then Y-demo, then operational system

• Team takes credit for success, Govt/Program Manager takes blame for failure

• Build a little – test a little

• Focus on key demonstrations initially

• DC-X didn’t demonstrate everything

• Focused on key aerodynamics and ops concept demos

• Demonstrated most objectives for <<1% of follow-on system cost

….Increased Emphasis on Prototyping Should Allow Us to Fly and Know How Much it Will Cost Before We Buy.

PACKARD COMMISSION

RULE FIVE

R e la ti v e V a lu e

Operational

System EMD

Follow-on Y-

Demo

DC-X

Relative Cost of Demo

Relative Objectives Verified

Example:

DC-X & Follow On Systems

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 18

Lockheed example: Personnel Launch System (HL-20)

Flies in Half the Time at One Fourth the Cost

ATD Focused on Limited Key Objectives Can be Done Faster and Cheaper

FIRST

ORBITAL FLIGHT

(BASB)

FIRST

ORBITAL FLIGHT

(BAU)

1/4 th COST TO

FIRST FLIGHT

0 -1 1 2 3 4 5 6 6 8 9

YEAR

CUMMULATIVE

COST %

BAU

OPERATIONAL

SYSTEM

SKUNKWORKS

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 19

• Short, concise, simple SOW and contract

• Minimize, tailor and simplify required CDRL data

• Empower contracting officer to act autonomously without MAJCOM review

• Conform with Law, not necessarily Regulations

• Replace ICD’s with small working groups where feasible

• Have reviews/meetings only when decision makers are present

• Don’t micromanage, do insure decision logic is sound

• Institute incremental “rolling wave” drawing release and vendor procurement in lieu of formal PDR/CDR approvals

• Resolve open issues rapidly

• Minimize number of drawings and ICD’s

• Maintain simple and rapid drawing release system

• Use shop floor as part of concurrent engineering team

• Use best commercial practices when feasible

Strive for Business As Should Be Not Business As Usual

RULE SIX

R e la ti v e

C o s t

Space Qual Military Qual Commercial

Qual

F-86D

FJ-2

F-86H

F-86K

YF-100

F-100A

HL-20

X-31A

HL-20

D ra w in g s

Business As Usual (BAU)

Business As Should Be (BASB)

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 20

Contractor integrates and tests prototype

• Use prime contractor, not Government, as integrating and testing contractor

• Contractor has test expertise and systems knowledge

• Eliminates redundant government testing and costs

• Streamlined testing philosophy shrinks the test schedule which saves time and money

• Contractor retains “ownership”, liability and esprit de corps

• Define clear test objectives, accomplish them and move on

• Get data needed for next vehicle

• Retire high cost ATD as soon a objectives are achieved

• Don’t build test empire

Test Demonstrator to Build … Don’t Build to Test

RULE SEVEN

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 21

Contractor prototype testing successes

F-117A

YF-16

DC-X

YF-22

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 22

Develop minimum realistic funding profiles

• Develop realistic cost estimates consistent with program objectives and schedule

• Provide adequate funds, but maintain Gov’t control of funds distribution

• Don’t distribute too much, too soon

• When building hardware, up front loading essential!

• Gen Bennie Schriever heavily front loaded Thor IRBM funds and demanded rapid flight – launch occurred 13 months after ATP

• Relentlessly seek stable funding profile

• Advertise cost and schedule impact of funding cuts

• Document all cost and schedule impacts due to funding cuts

• Track Life Cycle Cost, not just development costs

Contractor Program Manager Should Personally Approve All Costs in Advance/Real Time

AND

Share Information with Government Program Manager

RULE EIGHT

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 23

Track cost/schedule in near real-time

• Use contractors in-house cost tracking system

• Keep cost accounts to a minimum and focus on maintaining project visibility and controlling risk

• Timely and accurate forecasting will minimize surprises

• Rapid cost/schedule problem reaction and avoidance essential

• Keep first rate business team on-board

• Encourage proactive early identification of problems

• Regular meetings between contractor and customer

• CFSR/CSCSC provides too much/too late

• Didn’t predict problems on DC-X

• Bottoms-up “Estimate at Completion” identified concerns

Where Uncertainty Exists, Push for Early and Thorough Bottoms-Up Estimate at Completion

RULE NINE

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 24

Mutual trust essential

• Government and contractor program managers should interactively share all program information, critical decisions and risks

• Government must trust contractor

• To do their job and test/fly hardware

• By not second guessing every contractor decision

• Contractor must trust Government

• To provide stable programmatic direction

• Not to drive cost/schedule with unreasonable demands

• Build team philosophy and esprit de corps

• Think team, team, team…anyone can, we do

• People need people…manager(s) must be there for team

• Minimize personnel turnover

Allows Empowered Government and Contractor Team to Push

Ahead Decisively and Succeed or Fail Together

RULE TEN

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 25

Incentivize…Incentivize…Incentivize

• Develop appropriate reward systems

• Establish win-win contract structure

• Ensure prime contractor is motivated

• Incentivize subcontractors, especially in critical areas where minimal control exists

• Hire people who are self motivating

• Don’t reward/do discourage “bureaucratic growth”

• Fundamental program management objective is to work yourself out of a job, not build-up staff…

…Reward those who do so

• Building/flying hardware is its own reward if pride of “ownership” exists

Facilitate, Reward and Strive for Excellence

RULE OF COMMON SENSE

Distribution Statement A – Approved for Public Release, Distribution Unlimited. 26

Conclusion

• Consistent lessons learned from throughout industry ... can do affordable space demonstrators

• Key is to build and operate demonstrators before committing to operational systems

• Use streamlined acquisition practices and common sense

• Don’t spend too much money

• Expect some failures and much success

• Requires fundamental culture changes within organizations to reap benefits

• Takes real government and corporate commitment

• Must empower program managers/contracting officers www.darpa.mil

Distribution Statement A – Approved for Public Release, Distribution Unlimited.