Lessons_Learned_and_New_Code_300_Positions.pdf

National Aeronautics and Space Administration www.nasa.gov

SAFETY and MISSION ASSURANCE

DIRECTORATE C o d e 3 0 0

Lessons Learned and the new Code

300 Positions

S AF E T Y a n d M I S S I O N AS S U R AN C E D I R E C T O R AT E C o d e 3 0 0

What problems are we solving?

• We unnecessarily repeat many things

• Lessons learned are not conveyed at all the right levels

• Lessons learned are not conveyed in an effective way

• Requirements do not appropriately account for our experiences

• We tend to do things because we’ve always done them

• Lessons learned are not considered in everyday practices

• Red herrings are running amok

Events to learn from

• Analyses performed

• Technical assessments

• Risk Assessments

• Failures

• Anomalies

• Mishaps

• Close calls

• Project conflicts

• Procurements

• Nonconformances and dispositions

• Cost overruns

• Schedule problems

Existing lessons learned artifacts

• SMA top ten

• Watchlist

• GIDEPs, NASA advisories, and MWARs

• SMA CE wiki

Handling Concepts, new and old

• Day-to-day responsibility within key positions

• Requirements evaluation board

– Testing for reqmts evaluation

– Requirements changes

• Close call monthly or quarterly briefing

• Wiki communication and discussion

• Code 300 risk board, 400 risk advisory board

• MSR briefings

• Alert mechanisms

– Watchlist

– GIDEP

– NASA advisory

• Entry into lessons learned system

People

• MPAEs

• PRAEs

• CRAEs

• QEs

• REs

• Auditors

Introduction to the new positions

• CRAE: Commodity Risk Assessment Engineer

• PRAE: Parts and Radiation Assurance Engineer

• MPAE: Materials and Processes Assurance Engineer

What are these positions about?

• Risk

– Ensure that proactive and reactive actions are informed by risk in proper context of the project

– Operating at the lowest risk posture supersedes simply meeting lower level requirements

• Learning

– Ensure that lessons at all levels are applied from project to project and that subsequent assessments continuously improve in efficiency and effectiveness.

– Lessons learned are among everyone’s job, but these positions are the leaders in applying the lessons learned in everyday activities.

– Lessons learned are implemented in daily practices for continuous improvement

PRAE (373)

(Assigned directly to multiple projects)

• Ensure EEE parts requirements and guidelines reflect experiences

• Ensure that risk is the primary driver for parts-related decisions

• Ensure that parts entering the parts control board are prioritized by risk

– Focus on high risk parts/high risk applications

– Minimize efforts on low risk parts/applications

• Establish cross-cutting dispositions and processes for EEE parts-related alerts and advisories

• Maintain database of parts experiences

• Establish acceptability/risk of vendor parts practices

MPAE (373)

(Assigned directly to multiple projects)

• Ensure materials and processes requirements and guidelines reflect experiences

• Ensure that risk is the primary driver for materials-related decisions and acceptance/denial of material usage

• Ensure that materials approvals are prioritized by risk

– Focus on high risk materials/high risk applications

– Minimize efforts on low risk materials/applications

• Establish cross-cutting dispositions and processes for materials-related alerts and advisories

• Maintain database of materials experiences, e.g., where process problems cause major project issues

• Establish acceptability/risk of vendor materials practices

Specifics

• Review all parts and materials lists

• Invited to all PCBs, MPCBs, etc. (not voting)

• Review or drive agendas for PCBs, MPCBs, MUA disposition

• Reach-out to vendors

• Review parts and materials related alerts for applicability and cross-cutting disposition

• Put parts and materials related decisions in project risk context

• Perform risk assessments when decisions cause problems in project or with vendors

• Document all issues encountered and risk assessments

• Ensure that vendor nonconformances and notable observations get to supply chain managers

• Act as a cross-cutting set of eyes

• Head off problems caused by requirements overreach and creep

• Focus overly broad prohibitions into proper context (e.g., press-fit connectors, RNC 90 resistors, table II and III materials, etc)

• Understand common vendor practices at all vendors

CRAE (371)

Senior Technical positions in 300

Commodity: Tangible or intangible entity that has a major impact on risk, cost or schedule for GSFC projects

• Expert in key discipline area with background and experience with reliability and risk

• Responsible and empowered to assign risks based on warnings, alerts, environments, and “what we are stuck with”

• Establishes testing programs and protocols to keep up with current design practices and common parts and components

• Sets the policies for the risk-based decisions on use of parts, components, and processes

• Establishes layers of risk reduction based on risk classification (ownership of GPR 8705.4)

• Determines the acceptability and risk of alternate standards or requirements, or deviations and non-conformances

• Answers, “are we ok?” “why are we ok?” “how ok are we?”

• Provides risk assessment to the project for the project to decide how they want to disposition

Commodity Areas

• Standard Spacecraft Components

• Printed Circuit Boards

• Electronic Packaging

• Digital Electronics (esp FPGAs and ASICs)

• Power Systems (MOSFETS, power converters, high voltage, batteries, …)

• Capacitors/inductors

• Transistors

• Resistors

• Hybrid microcircuits

• Optocouplers

• On-board processors

• Additive Manufacturing

• Software

• Materials

• Radiation

• Environmental testing

• Contamination

• Connectors

• ESD

Decision-making process

CRAE or commodity lead

CSO

MSE

PM/PI

Risk (LxC)

Recommendation

Project risk board

CRAE or commodity lead convenes relevant experts for top commodity issues, including reliability, PDL, materials, parts, etc., as needed. If there is no major commodity issue, project MRB will suffice

Decision to accept/reject

Communication Mechanisms

• SMA monthly

• MSR

• Lunch time seminars

• Systems engineering seminars

• Email distro to projects

• Code 300 all hands

• Safety awareness campaign

• Workshops

• 300/400/500 BOD

• CSO learning sessions

• OAGS (on-orbit anomalies of GSFC spacecraft) reports

• MARs, SMA plans, etc

300 day in the life example - yesterday project

Standard MAR (how GSFC has always done it) vendor

Delivered product

QA

inspectio n

Nonconforming

Conforming

Reject and try again, as many times as necessary

How many iterations will this take?

Does the nonconforming product entail unacceptable risk for the current project?

Are the requirements necessary and aligned with the project risk posture?

Did we learn anything from this process?

New 300 day in the life - generic product delivery example

Commodity Risk Assessment

• Derating and usage guidelines

• Risk layering requirements per risk class

• Nonconforming and out-of-family item risk assessment

• Learning through risk assessments, research, and testing

Quality Engineering

• Upfront involvement in design

• Design for manufacturability

• Inspection

• Nonconformance and problem identification in developed hardware/software

Management Systems

• ISO and AS9100 quality

• NCR follow-ups with vendors

• Audits and Assessments

• Lessons Learned capture project

Risk-based SMA requirements vendor

Delivered product

Quality Inspection

Nonconforming

Vendor follow-up, Root Cause Analysis

Risk assessment

Conforming

New 300 day in the life PCB example

Commodity Risk Assessment

• Derating and usage guidelines

• Risk layering requirements per risk class

• Nonconforming and out-of-family item risk assessment

• Learning through risk assessments, research, and testing

Quality Engineering

• Upfront involvement in design

• Design for manufacturability

• Inspection

• Nonconformance and problem identification in developed hardware/software

Management Systems

• ISO and AS9100 quality

• NCR follow-ups with vendors

• Audits and Assessments

• Lessons Learned capture project

Risk-based PCB requirements vendor

PCB production

PCB coupon

Coupon lab

Nonconforming

Vendor follow-up, Root Cause Analysis

Risk assessment

Conforming

Design & Implementation (yesterday)

MPE

Parts control board

Early Design

M&P Engineering (541, occasionally) -materials selection -process development -drawing development

Trade Studies

Initiate Implementation

Integration

Environmental test

Risk Mitigation (e.g., qualification)

Launch

Operations

Products Delivered

Inspections (373)

NonconformingConforming

Deviation from standard practices Failures/anomalie s

Failures/anomalie s alerts

300 risk board watchlistReject and try again

Ad hoc center approach

Design and Implementation

Commodity Risk Assessment (371)

• Derating and usage guidelines

• Risk layering requirements per risk class

• Nonconforming and out-of-family item risk assessment

• Learning through risk assessments, research, and testing

Quality Engineering (373)

• Upfront involvement in design

• Design for manufacturability

• Inspection

• Nonconformance and problem identification in developed hardware/software

Management Systems (383)

• ISO and AS9100 quality

• NCR follow-ups with vendors

• Audits and Assessments

• Lessons Learned capture

M&P control board

Risk-based Parts and materials requirements

Parts control board

Reliability (371 in-house) -fault tolerance -expected lifetime -qualification

Early Design

Quality (373 in-house)

- Design for manufacturability

- Sensitivity to workmanship issues

Parts Engineering (562 for in-house)) -parts selection

M&P Engineering (541 in-house) -materials selection -process development -drawing development

Trade Studies

Initiate Implementation

Application of alerts and lessons learned

MPAE (373) makes risk-based decisions to keep products flowing

PRAE (373) makes risk-based decisions to keep products flowing

Integration

Environmental test

Risk Mitigation (e.g., qualification)

Launch

Operations

Products Delivered

Inspections (373)

NonconformingConforming

Deviation from standard practices Failures/anomalie s

Failures/anomalie s

Problem with part or component?

alerts

300 risk board watchlist