Attachment 14 Life Cycle.pdf

Department of the Treasury

Department of the Treasury

INFORMATION SYSTEM LIFE CYCLE MANUAL

TD P 84-01

VERSION 3.0

March 2002

Preface

Information system life cycle management emphasizes decision processes that influence system cost and usefulness. These decisions must be based on full consideration of business functional requirements and economic and technical feasibility in order to produce an effective system.

The objectives of a life cycle management approach are to:

� deliver quality systems which meet or exceed customer expectations when promised and within cost estimates;

� deliver systems that work effectively and efficiently within the current and planned information technology infrastructure;

� deliver systems that are cost-effective to enhance and maintain;

� develop quality systems using an identifiable, measurable, and repeatable process;

� establish an organizational and project management structure with appropriate levels of authority to ensure that each information system is effectively managed throughout its life cycle;

� identify and assign the roles and responsibilities of all affected parties including functional and technical managers throughout the information system life cycle;

� ensure that information system requirements are well defined and subsequently satisfied;

� provide visibility and comprehensive information to functional and technical managers for all information system resource requirements and expenditures.

� establish appropriate levels of management authority to provide timely direction, coordination, control, review, and approval of the information system project;

� ensure project management accountability; and � identify project risks early and manage them before they become problems.

This Manual establishes the processes, procedures, roles, and responsibilities governing the planning, definition, design, development, deployment, operation, maintenance, management, and termination of information systems within the Department. This Manual applies to all Treasury employees and contractors responsible for information systems projects.

/s/ James J. Flyzik 3-11-02

Deputy Assistant Secretary (Information Date Systems) and Chief Information Officer

TABLE OF CONTENTS

I. INTRODUCTION

II. PURPOSE

III. INFORMATION SYSTEM LIFE CYCLE MODEL

1. Requirements Identification and Analysis

2. Planning

3. Development and Testing

4. Implementation

5. Operation

Periodic Information System Evaluation

6. Termination/Disposition

APPENDICES

Appendix A – Life Cycle Models

Commercial Off-the Shelf Software (COTS) Development

Select/Control/Evaluate Approach Appendix B

Functional Requirements Document Appendix C-1

Requirements Checklist Appendix C-2

Project Plan Appendix D-1

Planning Checklist Appendix D-2

Planning Review Template Appendix D-3

Methodologies and Tools Appendix E

Test Plan Appendix F-1

Quality Assurance Plan Appendix F-2

Design Checklist Appendix F-3

Development Checklist Appendix F-4

Test Checklist Appendix F-5

Final Design Review Checklist Appendix F-6

Test Readiness Review Checklist Appendix F-7

Test Analysis Review Checklist Appendix F-8

Implementation Plan Appendix G

Implementation Checklist Appendix H

Training Plan Appendix I

Configuration Management Plan Template Appendix J

Post Implementation Evaluation Appendix K

Disposition Plan Template Appendix L

Definitions Appendix M

References Appendix N

I. Introduction

The Treasury Department has an extensive variety of systems, all of which are governed by the rules and principles of the Information System Life Cycle (ISLC). This manual is provided to assist bureaus in the general standardization of life cycle management of their information systems. Standardization ensures that systems are developed, acquired, evaluated and operated in an efficient manner, within prescribed budgets and schedule constraints, and responsive to mission requirements.

II. Purpose

The ISLC manual provides Treasury and associated bureau project managers charged with developing systems with standardized modules, methodologies, and guidelines for implementing a structured and consistent approach to IT project development. Bureaus and offices that have already developed information life cycle management documents and implemented practices prescribed therein should use this manual as a means to generate thought for process improvement. Bureaus and offices without standardized methodologies should utilize this manual for the management of their systems life cycles.

The Department is in the process of identifying additional architecture-related documentation for each of the life cycle phases of major projects. Documentation requirements will be tied to the Treasury Enterprise Architecture Framework (TEAF). Upon finalization, additionally developed documentation will be added to the ISLC.

III. Information System Life Cycle Model

A lifecycle model may be tailored to the unique needs of an information system project. For example, the use of COTS products in an information system can result in the reduction or elimination of some phases or activities. The tailored process will be described in the project (management) plan. As an example, life cycle tasks and products may be reduced for an information system project to install a COTS software product on an existing bureau IT infrastructure. The project manager should consider the size, complexity, and scope of the information project when preparing the project plan. Some tasks and work products may be omitted as long as the resulting approach provides for the delivery of a quality system.

The model outlined in this document includes the following phases:

1. Requirements Identification and Analysis

2. Project Planning

3. Development and Testing

4. Implementation

5. Operation

6. Termination/Disposition

An illustration of the six-phase model described in this document is provided on the next page.

Other life cycle models are illustrated in Appendix A-1 through A-4. The models outlined in the appendices are listed as illustrations of system development methodologies that may be used.

There are other methodologies that can be used to support the effective development of systems.

Bureaus are not precluded from using other tools and techniques and are encouraged to share their best practice methodologies with other bureaus.

Phase 1 Requirements Identification & Analysis

Phase 2 Planning

Phase 3 Development and Testing

Phase 4 Implementation

Phase 5 Operation

• Needs analysis

• Investment management process

• Business case development

• Investment review board approval

• Quality Assurance

• Configuration management

• Project schedule and milestones

• Project work plan

• Project resource plan

• Project assumptions

• Quality Assurance

• Configuration Management

• Software items traceable to requirements

• Appropriateness of design methodologies

• Unit testing

• System qualification testing

• System acceptance testing

• Quality Assurance

• Accessibility testing

• Configuration Management

• Training

• Configuration management

• Quality Assurance

• Post implementation review

• Information system review

• Quality Assurance

• Configuration Management

• Project approach decision

• Project execution decision

• Project continuation decision

INFORMATION SYSTEM LIFE CYCLE MODEL

Phase 6 Termination/

Disposition

1. Requirements Identification and Analysis

The initial phase of an information system project is the identification and analysis of the requirements the project must meet. During the requirements identification process, the functional manager(s) should provide the first critical description of the information system requirements or opportunity for improvement and should document the need to secure resources to further examine the requirements, opportunity or potential solution.

Once it is determined that an IT investment is necessary to address a bureau's program goals and strategic objectives, an Integrated Project Team should be established to specify and define required functional technical requirements to address strategic objectives. Requirements should address the following:

� Mission function � Outsourcing � Use of COTS technology or supported reengineered work processes � Analysis of alternatives, cost-benefit analysis, estimated return on investment � Consistency with target architecture and sequencing plan � Risk-reduction criteria � Planned approach to development � Balanced acquisition strategy � Performance measures � Workload and related requirements � Records management requirements � Security requirements � Accessibility (Section 508) requirements for individuals with disabilities � Quality Assurance � Configuration Management � Test Management

There are some requirements that should be addressed throughout a system's life cycle, e.g., quality assurance and configuration management. For large or complex projects, a feasibility study and market survey should be conducted to provide a preliminary determination of needs and alternative strategies for meeting those needs.

The information derived from the identification and validation of requirements should be used as decision criteria with respect to the evaluation of proposed IT investments. This information becomes an integral part of an IT investment management process as mandated by the Clinger-Cohen Act. An IT investment management process is an integrated approach to managing IT investments that provides for the continuous identification, selection, control, life-cycle management, and evaluation of IT investments. The Information Technology Management Reform Act of 1996 (also known as the Clinger-Cohen Act) , as supplemented by the Capital Programming Guide Supplement to OMB Circular A-11, Part 3, "Planning, Budgeting, and Acquisition of

Capital Assets," defines the specific requirements for IT capital planning and investment control and mandates a select/control/evaluate approach.

As described in the November 8, 1999 memorandum from the Assistant Secretary for Management, I-TIPS (Information Technology Investment Portfolio System) is a mandatory-use capital planning and management tool for all Treasury capital IT investments for Fiscal Year 2002 and future budget cycles. The Department and bureau staff will use I-TIPS as part of a capital investment management process to select, control, and evaluate IT projects.

All proposed capital investments must be justified by supporting documentation. The Department has developed a standard business case template as guidance and encourages Bureaus to use it. The template is provided at http://intranet.cio.treas.gov/cirb/cioc2.nsf/SBCTemplate-OpenPage.htm

In the select phase, the costs and benefits of all available projects are assessed and the optimal portfolio of projects is selected. During the control phase, the portfolio is monitored and corrective action is applied where needed. In the evaluate phase, implemented projects are reviewed to assure that they are producing the benefits expected, and adjustments are made where appropriate. All phases may be underway at once as they are applied to projects at different stages of their lifecycle. The IT investment portfolio concept, as set forth in the Clinger-Cohen Act, emphasizes the need for Federal agencies to do a better job of prioritizing IT capital investments and being accountable for results. Appendix B illustrates the components of the select/control/evaluate approach as developed by GAO.

Policies, responsibilities and procedures relating to IT investment management processes are outlined in TD P 81-01, Department of the Treasury IT Manual. TD P 81-01 also provides detailed information on requirements identification and cost-benefit analyses.

TD P 81-01 is provided at http://intanet.cio.treas.gov/cio/ITManualV4.pdf

Section 508 Accessibility standards are provided at http://www.section508.gov

A description of functional requirements is outlined in Appendix C-1. A requirements checklist is outlined in Appendix C-2.

2. Planning

Planning begins after requirements have been sufficiently identified and validated, the project has been approved, and resources have been allocated, and continues throughout the life cycle of a project. Program developers and management should develop a detailed Project Plan, which indicates the project's work breakdown, schedule, associated resource requirements, and plans for managing quality and risk at the onset of the development phase. Once the Project Plan is approved, day-to-day project work can then be assigned and monitored, and overall performance of the project measured and assessed. During the planning phase, it should be emphasized that a "freeze" will be placed on any changes in requirements requested subsequent to the initiation of the development phase.

The Project Plan is a detailed implementation plan developed during the planning phase of the life cycle. It needs to be flexible enough to accommodate re-planning needs as the project progresses. The plan documents the baselines mutually agreed upon by program and project managers, and encompasses all project-planning documents needed to effectively perform and control project activity, including, but not limited to:

� Project Schedule and Milestones � Project Work Plan, including Work Packages � Project Resource Plan � Project Quality Action Plan � Team Organization Chart(s) � Project Risk Mitigation Plan � Project Assumptions

Program and project management must jointly resolve all baseline differences between the Project Plan and the initial estimates provided by program management.

The following information should also be provided as part of the Project Plan:

� Assumptions used in preparing the planning documents � Methods used in defining, estimating, and scheduling the work � Additional project risks identified and, if possible, their projected effect on the baselines � Alternative project approaches considered and rejected or warranting further discussion.

Illustrations of a project plan, a planning checklist and a planning review template are provided in Appendix D-1 through D-3.

3. Development and Testing

A. Development

Various methodologies exist today for application development and software engineering. These methods range from the widely known, such as Structured Design, to hundreds of lesser-known approaches, some supported by a single individual or organization. Combinations of methodologies and software tools play a key role in achieving a higher level of software quality and productivity. Software quality, from requirement analysis to final testing, integration and ultimately production maintenance, can be greatly improved by selecting an appropriate development methodology. In Appendix E, notable methodologies and tools are examined and compared.

The development phase of the system life cycle is the phase in which developers transform requirements into a design that is compliant with the applicable bureau and/or Treasury target architecture and sequencing plan. Guidance on information technology architecture is provided in the Treasury Enterprise Architecture Framework (TEAF).

Additional information on TEAF is located at http://www.treas.gov/teaf/. Developers are tasked with creating a detailed design for interfaces that are external and internal to the software item.

The following are requirements for the development phase:

� Ensure that the software is traceable to the requirements � Ensure external consistency of software and requirements � Ensure internal consistency among software components � Ensure appropriateness of design methodologies � Ensure viability of detailed design � Ensure viability of operation and maintenance

Testing and quality assurance activities are integral parts of the development phase.

B. Testing

Testing is the most labor-intensive activity performed during software development.

Testing often requires more effort than the combined total for requirements analysis and design by as much as 15%. Testing is also a significant source of risk that is often not recognized until cost and schedule overruns have occurred. Industry experts agree that there are two basic reasons why testing is risky. First, testing traditionally occurs so late in the life cycle model that defects become costly and time consuming to locate and correct. Secondly, the testing methodology employed is frequently inadequate and poorly defined. Many projects enter into the testing phase of the life cycle model without a clear idea of what to accomplish and how to accomplish it.

Software testing is a process that checks software execution against requirements. The goal of software testing, in the past, was to demonstrate correctness and quality. Today, this goal has been modified. Testing cannot produce quality software, or verify correctness; rather, testing can only confirm the presence of software defects. Software defects are usually symptoms of fundamental problems in the development process. As a result, software testing has evolved into an integrated set of quality assurance processes that cover the entire development life cycle. To engineer quality software, developers must inspect, test and remove errors from requirements, design, documentation, code, test plans, and tests.

Testing is usually divided into various levels:

i. Unit Testing

A unit is a component and a component is an aggregate of one or more components that can be tested as an aggregate, such as subroutines, functions, macros, the application and the subroutines it calls, communicating routines, or an entire software system. Unit testing is usually performed by the programmers who created the unit. Unit testing is usually conducted in an incremental design/code/test fashion, where more and more of the completed system is progressively tested during each increment. There are two basic types of testing that are performed at the unit and system level: structural testing and behavioral testing.

Structural testing ideally involves exhaustive execution of all paths of control flow in a module or system. Exhaustive tests are usually impossible because the number of potential paths can be infinite. Also, path testing cannot detect missing paths or data sensitivity defects. Therefore, structural test case design should be based on random and/or selective testing of control flow. Some structural testing techniques include:

� Statement coverage � Decision coverage � Condition coverage � Decision/condition coverage � Multiple decision/condition coverage � Independent path coverage

Behavioral testing focuses on requirements. Testing should consist of ensuring that the application meets all features mentioned in the specifications and requirements. Behavioral testing is performed without knowledge of how the object tested is constructed and focuses only ensuring the program or module behaves as indicated in the specifications and requirements. In contrast with exhaustive path testing, behavioral testing focuses on exhaustive input testing, which is also usually impossible due to the potentially infinite amount of inputs.

Thus, behavioral test case design should be based on random and/or selective testing of inputs. Behavioral testing techniques include:

� Equivalence partitioning � Boundary analysis � Cause effect graphing � Error guessing

During the unit test, it is important to note that neither test approach is sufficient alone. Behavioral testing should be used throughout the development process, while structured methods are best used later in the process. Both methods are complimentary, however, some redundancy does exist. To this effect, automated tools that build test cases, which maximize yield and minimize redundancy, may be a sound investment.

ii. Subsystem Integration Testing

As basic units are integrated into subsystems, even those units that have received adequate unit-level testing may produce failures when used in connection with the other units that comprise the subsystem. Specifications written by component developers do not always describe behavior in sufficient detail to define how the component will interact with every other component.

The directed tests are constructed from the interactions between units as specified in the architecture. These representative tests search for those failures that users of the system are mostly likely to encounter. The representative tests are constructed from the use cases used to represent the business process requirements. The integration test plan should describe tests that cover each protocol between pairs of components. A protocol is a complete sequence to messages going back and forth between two components. Each protocol must be consistent with standard input-process-output. Test cases should include instances where the protocol is violated with the expected result that one of the two components will recognize the violation and raise an exception.

iii. System Qualification Testing

System qualification testing is performed to demonstrate to the customer that system requirements have been met. It covers the system requirements in the system/subsystem specifications and in any associated interface requirements specifications. If a system is developed in multiple modules, qualification testing of the completed system will not occur until the final module is developed. System qualification testing in each module is interpreted to mean planning and performing tests of the current module of the system to ensure that the system requirements implemented in that module have been met.

iv. System Acceptance Testing

The acceptance test is a formal test that demonstrates to the user that the system meets all specified needs. The criteria for an acceptance test will be used as part of the system test. If a project has passed the system test, then fails an acceptance test, the system testing was seriously inadequate.

v. System Security Testing

The security test focuses on verification of the protection mechanisms built into the system. It is designed to see how secure the system is from improper penetration and malicious use.

vi. Accessibility Testing

Accessibility testing provides assurance that design specifications for hardware and software meet accessibility (Section 508) requirements. Section 508 requirements (standards) are provided at: http://www.section508.gov

vii. Quality Assurance

Quality Assurance (QA) refers to a group of related activities employed throughout the life cycle of a system to quantify and improve the quality of software. It is important to note that the QA process spans the entire life cycle of the system from requirements definition to disposal. Standard QA methodologies and assessment guidelines include ISO 9000 and Capability Maturity Model (CMM)-Based Appraisal for Internal Process Improvement (CBA IPI). The quality of software is assured in the QA process through defect detection practices such as formal inspections, reviews, and testing.

QA procedures and policies should be embedded within the software development life cycle. The purpose of these embedded QA procedures is to reduce error rates in system development, which ultimately results in less re-work and increased cost savings. QA methods focus on detection and elimination of defects as early in the life cycle as possible.

Independent oversight functions can also be a part of the QA process. An independent test function, or testing which is observed by an independent organization, such as a contractor, is another method of assuring quality products. Other options include tests witnessed by users, expert review of test results, and audits of the system being developed.

Many methods are used to perform the QA process. Several methods employed by effective organizations today follow:

� Independent Verification and Validation (IV&V) - Independent verification and validation is a term used to describe a process where an independent group (e.g., contractors) verify and validate the project's adherence to established standards and procedures such as ISO 9000 and CMM.

� Formal Inspection - Formal inspections are an examination of the completed product of a particular stage in the development process (such as requirements definition or code generation). Formal inspections typically employ checklists and expert inspection teams to determine the quality of the completed product.

� Reviews - Reviews are applied as an alternative to formal inspections.

Informal design and code review methods are difficult to quantify since they are generally completed at the discretion of the project manager. Informal review is vital if formal inspections are not utilized. Reviews also refer to project meetings (e.g., product design reviews) which have the end effect of evaluating the value of the project.

� Walkthroughs - Walkthroughs are meetings in which the developer acts as a presenter to step through the development process in a structured manner.

The objective of a walkthrough is often to raise and/or resolve design or implementation issues.

� Testing - Testing is a technique that has the primary goal of error detection.

Testing is conducted first on individual components during intermediate stages of development, subsystems after integration, and the entire software system once systems are deployed

Analysis entails addressing process weaknesses that allowed product defects to be manifested. In analysis, processes are analyzed to prevent defects from occurring during development of the system. Common approaches utilized in analysis include root cause analysis and process brainstorming.

A team of individuals, which should be comprised of both developers and analysts, determines the root cause of the defect insertion through a comprehensive review of processes utilized in development. If the cause is systemic and/or may be repeated, brainstorming for a remedy is performed to reduce the likelihood of similar defects which could manifest themselves later in the development process. Ideas for process improvement are generated and passed to the project management. Based on the information derived in the process brainstorming session(s), analyses can be performed at various stages throughout the system life cycle. Project management should strive to minimize the elapsed time between defect discovery and analysis.

Statistical methods can also be used to assess both process and product quality.

These methods include Pareto analysis, Shewart control charts, histograms, and scatter diagrams. These methods will not be discussed in this document, but are named to bring awareness about other statistical models which exist to assist in quality control.

Bureaus and offices should also note that if CMM is utilized for project development, quality assurance tools and methodologies are already built into the CMM model. As an agency progresses from CMM level 1 to CMM level 2 and beyond, the ability to generate quality software automatically becomes a part of the development process.

The following appendices provide information in the form of plans and checklists that should be considered during the development and testing phase:

Appendix F-1: Test Plan

Appendix F-2: Quality Assurance Plan Appendix F-3: Design Checklist Appendix F-4: Development Checklist Appendix F-5: Test Checklist Appendix F-6: Final Design Review Checklist Appendix F-7: Test Readiness Review Checklist Appendix F-8: Test Analysis Review Checklist

4. Implementation

The implementation phase focuses on transitioning the assembled from the development phase into a production system for real users. Typically, software-engineering activities in the development may fail to address certain software design elements and may not address some requirements. Because of this, configuration management plans and training plans should be adhered to and included as integral parts of the overall implementation plan. Quality must be a central focus in this phase of system development.

An illustration of an implementation plan is provided in Appendix G. An implementation checklist is provided in Appendix H.

A. Training

Training encompasses all activities related to ensuring that the users of the system are aware of the system functionality and how to use the system. Training should focus around discussing the transition from the “as-is” to the “to-be” system and the benefits of making the transition. The user manual should be used as the primary mechanism for setting up actual system training.

All training activities should be listed in the training plan. Specific details of information to include in a training plan are provided in Appendix I.

B. Configuration Management/Change Control

Configuration management provides a set of tools and procedures to control changes to a system after life cycle activities have begun and also controls the establishment and maintenance of production, testing, training, and development libraries. Additionally, configuration management provides the functional and physical characteristics of hardware and software as specified in the technical documentation and achieved in the end product.

The objectives of configuration management are:

� To provide a method to ensure the functional and physical characteristics of the hardware and software are identical as specified in the technical documentation and archived in the end product.

� To identify and control changes to the functional and physical characteristics of the hardware and software and to ensure the proper documentation and maintenance of approved changes in system and application manuals.

� To establish policy that addresses the development of specifications, test plans, test results and a formal review of test results in order to ensure consistency/uniformity in IT practices and procedures.

� To ensure that the structured methodology for establishing changes provides adequate controls; e.g., implementing procedures to address the appropriate segregation of duties of requesting, authorizing, programming and testing officials.

� To establish policy and procedures that ensure the proper retention of specifications, test results, and quality assurance documentation.

� To provide a structured methodology for monitoring, reviewing and verifying the implementation of configuration changes needed to hardware, firmware, software (operating systems, COTS products, and applications) and telecommunications supporting information technology functions.

� To provide a method for formal evaluation of the impact of proposed changes to the functional and physical characteristics of the hardware and software.

� To provide reports on the status of the functional and physical characteristics of the hardware and software.

� To establish formal oversight of system changes to maintain an open path of communication for all impacted parties, as well as, to provide routine notification/status of any system changes.

� To identify, establish, and control the configuration baselines.

Configuration management includes four functions:

� Identification: selecting and labeling all functional and physical characteristics of the hardware and software.

� Control: the evaluation, coordination, approval and implementation of all approved changes to the contents of an established configuration baseline, i.e., the initial functional and physical characteristics of the hardware and software.

� Accounting: provides the administrative support required for maintaining system baselines and monitoring the status of the system throughout the life cycle as well as recording and reporting the information that is needed to effectively manage the functional and physical characteristics of the hardware and software.

� Audit: a form of examining the configuration records to verify the success of the change control and identification processes.

A configuration management plan template is provided in Appendix J. The template addresses minimum standards for configuration management and should be modified by bureaus as needed.

5. Operation

The operation phase focuses on operating and maintaining the newly implemented system that was created in the development phase. The system must meet the formal service targets and metrics established in earlier phases. The team conducting the operations phase must also provide feedback for improvements based on measurements of actual performance against those targets.

The goal of the operation phase is the continuous provision of service to achieve and sustain the benefits of the new system. The operations phase is comprised of a service operations phase and an overarching application management practice. The service operations stage focuses on operating the new system and the application management practice entails the management of the underlying application (i.e.

technical support).

The principles that differentiate the operation phase from the other phases are that:

� This phase represents the operation of the new system that was developed and deployed during the development phase, including all related business processes, human performance initiatives, and technology innovations.

� The focus for all activities is long-term (i.e. several years, instead of weeks or months).

� Operation does not consist of a series of projects that have a finite life span;

operation activities are started once a new system is deployed, and are ongoing.

� Continuous improvement is used to sustain value. The application management team's standards are raised periodically in order to improve results. This process can achieve a reduction in cost and an increase in value.

The operation phase focuses on outcome, not deliverables. Although there are several deliverables, they are used to support the work in the phase itself (e.g. trouble logs, etc.). The success of the operation phase depends on consistent and improved outcomes.

Periodic Information System Evaluation

This phase of the life cycle begins when the system is fully operational, and ends when the system is retired from use or terminated. This phase is intended to:

� Ensure that user needs are met.

� Ensure that the system continues to perform as specified in the operational environment.

� Ensure that all other measures are on target.

� Ensure technical and strategic compliance with the applicable bureau and/or Treasure enterprise architecture.

� Identify lessons learned from the system development activities as well as operations that may be applied to other systems.

� Identify improvements in the investment management process.

A number of project approach, project execution, project continuation, and program management decisions are considered and made during the evaluation of an information system.

Project approach considerations include:

� What evaluations of the system/data should be conducted?

� What new or additional user support activities are needed?

� What improvements in system/data functionality, quality, and/or performance are needed?

� What improvements or adjustments to the business process are needed?

� What adjustments to the system/data management approach are needed?

Project execution considerations include:

� What changes/enhancements to the system/database(s) are needed?

� Should a particular enhancement be implemented during this phase or given its own life cycle?

Project continuation considerations include:

� Does the information system requirement continue to exist?

� Does the production system address the requirement sufficiently to be continued in operation?

� Are sufficient funding and other resources available for the remainder of the life cycle?

Program management considerations include:

� Are any lessons learned applicable to other current or planned systems?

� Are any improvements required to the investment management process?

Ultimately, to support these decisions, review activities may occur several times throughout the operating phase. Each time the system is reviewed, one of two decisions is made: either the system is continued in operation (with or without modifications) or the system is terminated, and its functions and data transferred to other systems.

A. Post-Implementation Reviews

The post-implementation review is used to evaluate the effectiveness of the system development after the system has been in production for a period of time. The objectives are to:

� Assess the system’s effectiveness in meeting the original objectives.

� Identify the achieved benefits, assess whether they match projected benefits, and determine the reasons for any discrepancies.

� Evaluate whether original business assumptions used to justify the system were valid, and are still valid.

� Compare actual costs incurred against projected costs.

� Determine how well the project met time schedules and implementation dates.

� Evaluate issues that still require management attention.

Post-implementation reviews are conducted between three and twelve months after implementation of the system and ensure that the system functions as planned, that benefits are derived, and that the system is within the estimated costs. The bureau may also conduct a user satisfaction review, if deemed appropriate. Users may be defined as the internal users, other external public organizations, or private citizens. Surveys of private citizens must ultimately be approved by the Office of Management and Budget, in concert with requirements specified in the Paperwork Reduction Act, as amended, and OMB implementing regulations.

A representative from the functional development group or other member of the major user organization participates in the review. The project sponsor ensures that all documentation needed for the review is preserved and available, and that all personnel needed to participate in the review are accessible.

The reviewer and an assigned team collect the information needed for the post-implementation review by interviewing end users and their managers, system administrators, and computer operations personnel.

The post-implementation review may be a free-form report and not all sections will be relevant or necessary to the final product. It is the bureau’s responsibility to develop, document and initiate the processes and procedures for post implementation reviews. The bureau must also determine the appropriate report format and questions. Descriptive information of post-implementation reviews is provided in Appendix K.

B. Periodic System Reviews

A Periodic System Review is a continuous system review. It is performed to evaluate system performance, user satisfaction with the system, adaptability to changing business needs, and new technologies that might improve the system. A periodic system review is diagnostic in nature and can lead to development or maintenance activities. Any major system modifications needed after the system has been implemented will follow the life cycle process from planning through implementation. A project plan, including feasibility study, will identify modification to existing system documentation (change pages) rather than new system documentation (e.g., functional requirements document, internal design document, etc.). The appropriate reviews and testing will be conducted, based on the scope of the modification. Post Implementation Reviews and Periodic System Reviews are very similar in nature to the reviews that must be conducted during the system development phases, particularly to support decision milestones.

Periodic System Reviews are conducted every three years, and as long as the system is in operation (continuously) to ensure that system performance is optimized and that users are satisfied with the system.

Regardless of specific evaluations, all systems require normal maintenance activities to ensure that any previously undetected errors are fixed. Maintenance may take several forms:

� Taking advantage of hardware upgrades or new releases of system software and application software packages used to operate the system (e.g., upgrades and releases).

� Identifying potential modifications needed to ensure that the system continues to operate as intended and produces quality data.

� Identifying modifications to the system and database(s) are needed to resolve errors or performance problems, or to provide new capabilities. New capabilities may take the form of routine maintenance, or may constitute enhancements to the system or database(s) that respond to user request for new/improved capabilities. The maintenance manual is used and updated accordingly.

C. Investment Management Process

Project reviews are pivotal to an organization’s investment management process.

The review provides an assessment of the system to the Investment Review Board or designated oversight group. Deliverables may also be produced for the benefit of other management officials. Besides being a feedback mechanism about a specific system, these reviews are a critical feedback mechanism on the effectiveness of an organization’s investment management process. Ultimately, for these reviews to be effective, the organization must ensure that the necessary methodologies, processes and mechanisms are not only functional but also effectively employed. The following issues and questions should be addressed when developing required process, mechanisms and methodologies:

� Reviews and resulting conclusions/recommendations are to be communicated to, and reviewed by, senior management.

� Review reports and results should be tracked and centrally maintained.

Aggregate results are of particular interest to senior management.

� Trend analysis of the review reports and results can be informative.

� The organization can ensure compliance with recommendations and decisions.

� Specific roles and responsibilities are delineated.

� The purpose of the reviews are clearly communicated, since problems may be identified.

6. Termination/Disposition

Disposition of an information system occurs at the end of the system life cycle. Disposal procedures ensure the orderly termination of a system and preserve vital information about the system in the event the system, or portions of the system (including deliverables) would need to be re-instituted. A disposition plan is prepared to address all facets of archiving, transferring and disposing of the system and the data. It is important to ensure proper preservation of the data processed by the system so that the data is effectively migrated into another system (conversion) or archived in accordance with applicable records management regulations and policies for potential future access. Disposal procedures preserve information not only about the current production system, but also about the evolution of the system through its life cycle.

A number of project approach, project execution and project continuation decisions are made during the disposal of a system.

Project approach decisions include: what evaluations of the system/data should be conducted to determine the need for a new system to replace the functionality of the old system; what new or additional user support activities are needed (if any); what improvements in system/data functionality quality, and/or performance are needed (if the old system is being replaced by a new system and not just being phased out); and what adjustments to the system/data management approach are needed (i.e. application of lessons learned from implementation of the old system).

Project execution decisions include: what changes/enhancements to the system/database(s) are needed, should a particular enhancement be implemented or an existing system completely replaced?

Project continuation decisions include: does the information system requirement continue to exist, does the production system address the requirement sufficiently to be continued in operation, and are sufficient funding and other resources available for the remainder of the life cycle?

Additional decisions to consider at the end of the system life include: what will be the system termination date, which software components should be preserved, which data should be preserved, what should be done with remaining equipment, and how should life cycle products be archived?

A. Disposition Plan

The disposition plan is critical to ensure proper disposition of the information system.

The plan will vary according to system and bureau requirements. The objectives of the plan are to end the operation or the system in a planned, orderly manner and to ensure that system components and data are properly archived or incorporated into other systems. At the end of this task, the system will no longer exist as an independent entity. The completion of the system life cycle is carefully planned and documented to avoid disruption of the organizations using the system, or of other systems that will use the data and/or software of the present system.

The software, hardware, and data of the current system are disposed of in accordance with organizational needs and pertinent laws and regulations. Software or data of the system may be transferred to other existing systems, migrated to an entirely new system, or saved and stored for future use. Hardware is made available for future use, added to surplus, or discarded.

In conducting the disposition task, several items are to be considered:

� All known users should be informed of the decision to terminate operation of the system before the actual termination date.

� Although the current system may be terminated, in many cases the data will continue to be used through other systems. The specific processing logic used to transfer the data to another system is developed as part of the data conversion planning for that system.

� In some instances, software may be transferred to a replacement system. For example, a component of the current system may become a component of the replacement system without significant rewriting of programs.

� Effective reactivation of the system in the future will depend heavily on having complete documentation. It is generally advisable to save and store all documentation, including the life cycle products generated during the earliest tasks of the life cycle as well as the documentation for users and for operation and maintenance personnel.

The disposition plan addresses how the various components of the system are handled at the completion of operations, including software, data, hardware, communications, and documentation. The plan also notes any future access to the system. The plan is lead/performed by the project manager, supported by the Records Officer, the project team and the functional staff, and reviewed by the quality assurance manager.

Other tasks include the following:

� Notify users of termination date. Notify all known users of the system date of the planned date after which the system will no longer be available. Work with the FOIA/PA representative to process any Federal Register regarding system of records notification.

� Store or transfer data. Copy data to be saved onto permanent storage media, and store media in location designated by the disposition plan. Work with the project management team for other systems to effect a smooth transfer of data from current system to these systems.

� Store or transfer software components. Copy software onto permanent storage media, and store media in location designated in disposition plan. (Software to be stored may include communications and systems software as well as application software.) Work with the project team for other systems to ensure effective migration of current system software to be used by these systems.

� Save life cycle products. Store other life cycle products, including system documentation, in archive locations designated by the disposition plan.

� Dispose of remaining equipment. Dispose of equipment used exclusively by this system in accordance with the disposition plan (refer to excess procedures).

� Complete Disposition Plan. Update the disposition plan to reflect actual disposition of data, software and hardware.

A disposition plan template is provided in Appendix L.

APPENDICES

Appendix A – Life Cycle Models

Waterfall Model Appendix A-1

The Waterfall Model can be thought of as the traditional development model. In the earliest days of software development, code was written and then debugged, repeatedly. As complex systems began to evolve, the code and debug approach became less than optimal. To deal with the problems introduced by development of large, complex systems, the Waterfall model was created.

The Waterfall model is an approach to development that emphasizes completion of a given phase of software development before proceeding to the next phase. Transition from phase to phase is accomplished by holding a formal review that is attended by the developers, including contractors (if utilized) and appropriate government representatives. These reviews provide insight into the progress of the software development.

Deliverable documents are reviewed at the end of each phase and serve as a baseline for the subsequent phases. If the need for change is identified, a formal change process is followed to implement these changes. In the Waterfall model, phases do not overlap, which makes the model straightforward and relatively simple to understand.

When to use the Waterfall model:

The Waterfall model, though easy to understand, fails to recognize the complexities and constantly changing requirements in software development. The Waterfall model may be appropriate for use on small system development projects and on some legacy system maintenance.

Strengths of the Waterfall model include:

� A formal method, built on a firm foundation of rigor, planning, extensive and standardized documentation, analysis, review and change controls � A top-down model that encourages analysis and documentation before coding begins � It is composed of independent phases to be completed sequentially and has well defined entry and exit conditions for each phase � It has well defined major milestones and review requirements � It specifies requirements (both content and format) for all significant planning documents and document products encourage consistency

Weaknesses of the Waterfall Model include:

� A complete set of requirements is required at the onset � Enforcement of…