Attachment_2_Production_Acceptance_Plan_(V.1.0).pdf

Attachment 1:

APPLICATION PROGRAM SET (APS)

PRODUCTION ACCEPTANCE TEST PLAN

Prepared for:

Marine Corps System Command

PROGRAM MANAGER

SUPPLY AND MAINTENANCE SYSTEMS (PM SMS)

MARINE CORPS SYSTEM COMMAND

QUANTICO, VA 22134

Prepared by:

TMDE Division, PPA, MDMC

Bldg. 2235

MCLC Albany, GA 31704

Distribution Statement C: Distribution authorized to U.S. Government Agencies and their contractors only. (Reason: Evaluation Use). (Date 01 Oct 2019). Other requests for this document shall be referred to Marine Corps System Command.

1.0 Purpose.

The purpose of this Production Acceptance Test Plan (PATP) is to demonstrate that the

Application Program Sets (APSs) are in compliance with the requirements identified in the contract.

2.0 Applicable Documents.

Statement of Work (SOW)

3.0 Test Requirements, Conditions and Ground Rules.

3.1 Acceptance Testing will be conducted at the Test and Integration Facility (TIF) located at

MCLC, Albany, Georgia. References to Automatic Test Equipment (ATE) within this ATP refer to the test equipment specified in the SOW. All Engineering Change Proposals (ECPs) and APS software affected by the ECPs shall have been reviewed and approved by the Government before testing begins. Acceptance testing will include the following:

a. Complete Test Program Hardware (TPH) End-To-End testing on all TPH from each

Application Program Set (APS).

b. Complete Units Under Test (UUTs) End-To-End testing on randomly selected APSs.

(1) For projects with ten or fewer APSs, run UUTs on three APSs selected at random.

(2) For projects with more than ten APSs, run UUTs on one in five APSs selected at random, running a minimum of three APSs.

c. Transportability Testing:

(1) APS testing for transportability demonstrates the APS can be used on different ATE stations. The APS must exhibit the same End-to-End test results when compared to the End-To-End test results when run on any other ATE stations.

(2) Each APS shall successfully pass all TPH End-To-End testing on a minimum of two

TIF ATE stations.

(3) The government shall retain signed copies of the TPH and UUT End-To-End Test results as they are completed; the test results shall be retained for the entire testing phase.

3.2 TIF and ATE stations shall be the only test equipment used for APS testing. If the existence or nature of a fault or the status of the UUT, TPH, or ATE station is in question, the uncertainty may be resolved using external equipment.

3.3 The ATE stations calibration will be verified as being current prior to the start of the APS testing. The ATE stations self-test procedure will be run daily prior to the start of APS testing.

3.4 The Government Test director shall maintain control of all testing data until testing is complete and the data is submitted to the Technical Point of Contact (TPOC) for final approval.

3.5 The UUTs required for acceptance shall be certified by the Government as being operational.

3.6 End-To-End testing will be performed with UUTs and TPH covers mounted.

3.7 Government testing personnel shall rely exclusively on the Test Program instructions for

Acceptance Testing.

3.8 If an apparent defect is detected and investigation proves it to be an ATE station or UUT defect, the defect shall be corrected and the demonstration resumed. The Government Test Team will determine if the test is resumed at the entry point just prior to appearance of the defect or at the beginning of the test.

4.0 Acceptance Test Team Composition.

4.1 The Acceptance Test Team acts as the officials responsible for acceptance testing of the

APS. The Government has final authority. Titles and responsibilities of the Acceptance Test

Team personnel are as follows:

a. Government Test Team.

(1) Government Representative

(2) Test Director (ATE station Operator)

(3) Other personnel assigned by the Government

b. Contractor Test Team.

(1) Program Manager

(2) Lead Project Engineer

(3) Lead Software Engineer, if applicable

4.2 Government Participation and Responsibilities.

a. Provide a Test Director to conduct acceptance testing and sign off on the test results.

b. Provide personnel to assist the Test Director.

c. Provide the Technical Point of Contact (TPOC) for acceptance of the DD-250 and QA acceptance of invoices, as required..

d. Provide support for Government assets required to conduct testing.

5.0 Failures.

5.1 Relevant Failures.

a. A test failure is relevant if the APS fails the TPH self–test, fails a UUT End-To-End test, is incomplete (e.g. missing components), or has a mechanical defect. All failures are initially considered to be relevant until determined to be non-relevant by the Government Test Team.

Relevant failures shall be corrected through an engineering analysis and/or the normal manufacturing repair process. Relevant failures may include but are not limited to the following:

(1) Mechanical defects or physical damage to hardware.

(2) Missing components or mislabeled parts.

(2) Measured values (current, voltage, resistance) deviate beyond acceptable limits established in the software code.

b. Relevant failures shall be analyzed by the Acceptance Test Team for resolution. Each finding and the associated corrective action taken will be documented in the applicable Test Data

Sheet and TDR.

5.2 Non-Relevant Failures.

a. An APS failure is non-relevant if it was caused by an external anomaly, is not repeatable, has not resulted in permanent degradation, or over stressed the ATE stations, APS or UUT.

Non-relevant failures may include but are not limited to the following:

(1) Incorrect installation of equipment due to operator error.

(2) Failure of the ATE Test Station or test unique stimulus/response equipment.

(3) Failure of the test facility such as a power failure.

(4) Single occurrence (non-recurring) failures that could not be duplicated in the same test environment or the remaining test environments prior to completing testing.

6.0 Re-test.

6.1 Non-relevant Failures. Non-relevant failures will be re-tested from the closest previous entry point to the failure. Cable connections and disconnects may be bypassed upon approval of the

Government Test Team so long as they do not jeopardize the test integrity or otherwise influence the re-testing of the failure.

6.2 Relevant Failures.

a. Relevant failures will be re-tested from the closest previous entry point to confirm the failure as relevant. After the corrective action is implemented, the APS will be re-tested from the beginning to ensure that the corrective action implemented was successful. The dates and results for re-testing shall be documented on the applicable Test Data Sheet. Re-run at an entry point or retest from the beginning will be at the discretion of the Government Test Team.

b. If the relevant failure occurs during Interface Device (ID) End-To-End testing, after successful re-testing, the failing ID will be required to successfully pass End-To-End testing on a third ATE station.

c. If the relevant failure occurs during complete End-To-End UUT testing, after successful re-testing, two additional APSs shall be added to the complete End-To-End UUT testing sequence.

d. If the initial number of APS End-To-End UUT testing doubles due to relevant failures, the Government Test Team will halt APS testing until the Government Test Team is confident that APS issues have been resolved and APS testing is ready to resume. Upon restart of APS testing, if the initial number of APS End-To-End testing is added on again due to relevant failures, the Government Test Team will again stop testing until APS issues have been resolved and APS testing is again ready to resume.

7.0 Documentation.

7.1 End-To-End Test Sheets will be completed, signed and submitted by the Test Director to the

TPOC for approval once acceptance testing is completed.

7.2 Test Reports.

a. Test Discrepancy Record (TDR). Relevant and non-relevant failures shall be documented on a TDR. An entry shall not be generated for multiple occurrences of the same failure once the original entry has been generated. Completed TDRs will be submitted to the Contractor as they are generated for analysis and corrective action.

b. Test Discrepancy Log. The Contractor shall generate a numbered Test Discrepancy Log to record TDR submissions. The completed Test Discrepancy Log will be submitted to the

Government for review at the end of Acceptance Testing. The Contractor will have 10 working days after completion of Acceptance Testing to correct all TDR findings.

c. System Trouble Report (STR). The Government shall generate an STR to document software or hardware issues beyond the scope of the SOW.

d. Data-Logging Files. The Test Director will generate Data-Logging Files for all test runs and include them in the Test Report.

8.0 Test Sequence.

Step 1. Verify ATE stations calibration is current.

Step 2. Run Self Test on ATE stations being used for APS Acceptance Testing.

Step 3. Set up all ATE stations for data retrieval.

Step 4. Tag all APSs with tracking tags.

Step 5. Visually inspect and inventory the APS.

Step 6. Run Self-Test on each ID in the APS, document results and update tracking tag.

a. Run two consecutive ID End-To-End tests on first APS.

b. If a Software change has been initiated due to an ECP, run all ID Test Modules individually after running two End-To-End tests on first APS.

c. Repeat step 6 on a third ATE station for any ID Self-Test failure.

d. Complete ID End-To-End Test Sign Off Sheets.

Step 7. Repeat step 4 and step 5 until all APSs have been inventoried.

Step 8. Run UUT End-To-End tests on all IDs in APS, document results and update tracking tag.

a. For projects with ten or fewer APSs, run UUTs on three APSs selected at random.

b. For projects with more than ten APSs, run UUTs on one in five APSs selected at random, running a minimum of three APSs.

c. Complete UUT End-To-End Test Sign off Sheets.

Step 9. Repeat step 8 until selected APSs have been tested.

Step 10. Remove all Acceptance Test files and restore ATE stations to pre-test condition.

START

- Verify calibration current on all

ATE used for testing

- Run Self Test on all ATE used for testing

Set up all ATE stations for data retrieval

Tag all APSs with tracking tags

Inventory APS

- Run ID Self-Test

- Document results

- Update tracking tag

Failure Detected ?

Failure Relevant ?

- Retest to confirm failure

- Rework and resubmit for complete re-test

- Run failed ID on three

ATE

- Correct failure

- Retest from closest entry point

YES

NO

NO

- Run an additional ID End-To-End Test

- Run all ID Test

Modules individually

First APS ?

NO

YES

A

C B

B

B A C

NO

YES

All IDs in APSs

Tested twice?

NO

YES

All failed APSs passed third ATE?

YES

NO

Test ID on second ATE

Test failed IDs on third ATE

E D

All IDs in APSs tested once?

- Run UUT End-To-End tests on all IDs in selected APSs

- Document results

- Update tracking tag

All selected

APSs tested?

- Remove all Acceptance Test files from ATE stations used

- Restore all ATE stations used to pre-test condition

END

NO

YES

D

Failure Detected?

Failure

Relevant?

- Retest to confirm failure

- Rework and resubmit for complete re-test

- Add two APSs to selected APS list

- Correct failure

- Retest from closest entry point

NO

NO

E

ID END-TO-END TEST SIGN OFF SHEET

For

(enter ID part number in box above)

S/N

(enter ID serial number in box above)

(enter ATE and serial number in box above)

The testing and/or results witnessed is true and correct to the best of my knowledge and meets the requirements of Acceptance Test procedures with the exceptions (if any) noted below:

Test Director: Date:

TPOC: Date:

EXCEPTIONS:

UUT END-TO-END TEST SIGN OFF SHEET

For ID

(enter ID part number and serial number in box above)

UUT

(enter UUT nomenclature, part number and serial number in box above)

(enter ATE and serial number in above box)

The testing and/or results witnessed is true and correct to the best of my knowledge and meets the requirements of Acceptance Test procedures with the exceptions (if any) noted below:

Test Director: Date:

TPOC.: Date:

EXCEPTIONS:

22 March 2011

Attachment 1

TEST DISCREPANCY LOG

TDR # Test Deficiency Change Documentation

Date Opened

Date Closed

TEST DISCREPANCY RECORD

PROGRAM: TDR NO.: 001

TEST ITEM: DATE:

TEST: TEST PROC. NO.:

POSITION IN TEST SEQUENCE:

A. DESCRIPTION OF SYMPTOMS

BY:

B. IDENTIFICATION OF PROBLEM

BY: DATE:

ANOMALY SOURCE

UNIT UNDER TEST

TEST EQUIP/SETUP TEST PROCEDURE OPERATOR ERROR

C. ACTION REQUIRED

BY: DATE:

CONTRACTOR REP: DATE:

CONTRACTOR QUALITY REP.: DATE:

TEST DIRECTOR: DATE: