Annex_D_Data_Collection_Instructions.docx

Software Maintenance Data Collection Questionnaire Detailed Data Definitions

Introduction The purpose of this data call is twofold:

1. Document what data exists/does not exist on software maintenance systems, with respect to the data elements requested in this questionnaire.

2. Where data does exist, begin to develop enterprise information about software maintenance.

The following terms are defined here and used in the questionnaire:

Software maintenance is defined as “the totality of activities required to provide cost-effective support to a software system”[footnoteRef:1]. In this questionnaire, the words software maintenance and software sustainment are synonymous. For purposes of this questionnaire, software maintenance begins once EMD has completed and a software release has been provided to an external party, whether a formal release or an engineering release. [1: ISO 14764:2006, "Software Engineering -- Software Life Cycle Processes – Maintenance." This document describes software maintenance activities.]

Software Release refers to a set of changes to the existing baseline software that has gone through Formal Qualification Test. Releases may be delivered to end users or may be engineering releases that are provided to external organizations for test or demos. Include any releases that require formal testing, i.e., a new software configuration has been created, including patches and emergency releases. This questionnaire uses the term “release” to refer to commonly used terms such as release, build, increment and drop.

This questionnaire consists of seven parts:

1. System-Level Context data.

2. System-Level Annual cost and effort data.

3. Release-Level data including schedule, cost, and size data.

4. Details about Software Licenses.

5. Information about the Update History.

6. Definitions of Application Domains from the Release-Level data.

7. Tables with definitions for the software change severities and the IAVA categories from the Release-Level data.

8.

For releases, actual execution data is collected. If a release has been completed prior to the submission of this questionnaire, actual execution data for the entire release is collected (Release A in the figure below). If a release is in the maintenance process at the time of data collection, and close to completion, then actual execution data to-date plus planning data for the remainder of the release is collected (Release B). If the release has not started, data for this release is not collected (Release C). Release A Release B Release C Submission Date Actual data values Planned data values for remainder of the release Do Not Report

Data Values All data fields in this questionnaire require a response. Valid responses are listed below. If the information or value requested (response #1 below) is not provided, use the comment field in each section to provide an explanation.

The following are valid responses: Please enter a response in every field.

1. The information or value requested.

2. Estimates of the “actual” value requested. Explain in the comment field that the value is an estimate and explain how it was derived.

3. N/A, not applicable, with an explanation of why it is not applicable. If an activity is not done, i.e. No C&A is required, then enter a value of N/A and explain why it is not done.

4. Zero (0) meaning the value is zero, i.e. the value or cost is known to be zero. For example, if C&A is done every 3 years, then put a zero as the entry for the years when C&A is not performed, and the actual value in the 3rd year when it is performed.

5. Unavailable: Use this response if the data is not separately tracked, is part of something else, an activity is done but it is funded or performed by a different organization, or the submitter does not have access to the data. Provide an explanation of why it is not available in the comments section.

Effort Data This questionnaire asks for effort hourly data: If effort hour data are not available, provide the equivalent hours by multiplying Full Time Equivalent (FTE) counts by the standard hours per year for your organization. This should be based on engineering judgment with appropriate notes in the comment field in each section.

If you don’t have actual data or FTE data, please provide planning hours data and make a note in the comment field in each section.

System-Level Data - Context Provide information about the system.

System Name. Enter the name given to the acquisition system, information system, or effort/initiative. If the system is part of a larger Program-of-Record (POR), enter the system name here, and provide an explanation of how it relates to the POR in the system description.

System Description. In a few words, provide a top-level system description of the final overall product maintained or to be maintained. For example: two engine attack helicopter, landing craft used to transport heavy equipment from ship to shore.

Service(s). Army / Air Force / Navy / Marine Corps. If this system is provided to multiple services or countries, list all, with any necessary notes in the comment field. For multiple services, indicate which organization is the lead.

Acquisition Category Level. Enter the appropriate system ACAT level. If no ACAT level is defined, use N/A.

Phase/Milestone.

Enter the current Phase/Milestone for the system from the selection below. If a system is in multiple phases concurrently, select the highest applicable phase. For example, if a system is in O&S, but also has some new development work ongoing, select O&S, and put a note into the comment field in section 1.6.15.

Pre-A: Material Solution Analysis Phase A: Technology Maturation and Risk Reduction Phase B: Engineering and Manufacturing Development Phase C-LRIP: Production and Deployment Phase - Low-Rate Initial Production C-FRP: Production and Deployment Phase - Full-Rate Production O&S: Operations and Support Phase Enter the actual date the system entered that phase. The date format is month, day, and year MM/DD/YYYY.

Context Information.

Number of Unique Software Baselines Maintained. Provide a count of the number of unique baselines of the software that are concurrently maintained. Multiple baselines may exist to support different platform configurations, because transition to the latest version is not complete, or because different configurations require different software. As an example, if there are four active software maintenance releases associated with each of three different configurations of software, enter 12.

Number of Total Hardware Platforms Fielded. Provide the total number of fielded hardware platforms that are supported by this software. If this number is not known, especially for IT systems, then mark it UNAVAILABLE and put a note into the comment field (section 1.6.15).

Number of Unique Hardware Variants. Provide the number of unique hardware variants that are supported by this software.

Number of Users. Provide the number of users. For weapons systems who provide a count of hardware platforms, this field can be marked as N/A.

Maintenance Allocation. Provide a description of how the maintenance work is allocated between the Government and Contractors. Is it government only, primarily contractor (government managed), or an integrated team made up of government and contractor personnel?

Maintenance Profile. Describe whether the system is doing mainly capability enhancements, maintenance, cyber-security, or other changes. If other, please explain.

Maintenance Process. List the steps/phases/stages of the Maintenance Process (CCB, Development Complete, Dev. Testing Complete, Acceptance Testing, Deployment) and the activities performed (requirements, design, code, testing, etc.).

Operational Tempo. Select the highest operational tempo for the field units using this software. Entries may be:

Extensive: Full-time use of the system; system is central to mission accomplishment.

Regular: Routine, but less than full-time, use of the system.

Event-driven: Occasional use of the system.

Other: Please describe.

Life Cycle Sustainment Plan. Is there a life cycle sustainment plan: Yes / No.

Life Cycle Sustainment Plan Description. When was the plan approved or last updated? Does it reflect the current maintenance strategy? Who is the Point of Contact for the plan?

Software Process Maturity. Report the characterization of the maintainer’s software process maturity using a methodology such Carnegie Mellon’s Capability Maturity Model Integration (CMMI), or an alternative equivalent rating. The reported software process maturity should reflect the rating that the primary maintenance organization has formally certified as of the date of the reporting event. The date format is month, day, and year MM/DD/YYYY.

If no formal certification has been conducted, enter as “none”. If a single submission is used to represent the work of multiple organizations, enter the level of the organization that will be expending the most effort on the maintenance project.

If the Government has accepted an alternative assessment mechanism, such as the Air Force’s Software Development Capability Evaluation (SDCE) or ISO/IEC 15504, enter those results and explain the meaning of the assessment.

Data Rights.

Data Rights Type. What type of rights does the system have? Select one of the following:

· Unlimited Rights. Developed exclusively at Government expense, and certain types of data (e.g., Form, Fit, and Function data [FFF]; Operation, Maintenance, Installation, and Training [OMIT]). These rights involve the right to use, modify, reproduce, display, release, or disclose technical data in whole or in part, in any manner, and for any purpose whatsoever, and to have or authorize others to do so.

· Government Purpose License Rights. This right involves the right to use, duplicate, or disclose technical data for Government purposes only, and to have or permit others to do so for Government purposes only. Government purposes include competitive procurement, but do not include the right to permit others to use the data for commercial purposes.

· Limited Rights. A limited rights agreement permits the Government to use proprietary technical data in whole or in part. It also means that the Government has to obtain the expressed permission of the party providing the technical data to release it, or disclose it, outside the Government.

· Restricted Rights. Developed exclusively at private expense.

· Specifically Negotiated License Rights. This right pertains whenever the standard license arrangements are modified to the mutual agreement of the contractor and the Government. In this case, the exact terms are spelled out in a specific license agreement unique to each application.

· Multiple Data Rights. Use this category if different software elements (components) have different data rights, and then explain which data rights are applicable to which software elements in explanation field in 1.6.12.3.

Data Rights Cost. How much do the data rights cost?

Data Rights Ownership. Explain to what extent the government or contractor owns the data rights and technical data packages for the system? Describe any limitations or issues.

Analogous Systems. If applicable, list at least three analogous/similar systems. Analogous systems are defined as those similar in software size, scope, and complexity. List those systems developed or maintained by the same software organization or development team that are similar in software size, scope, and/or complexity, and provide a description as to how each analogous system is similar.

Appropriation(s). Indicate all type(s) of appropriation used to fund the contract or the particular contract line item being reported.

Research, Development, Test and Evaluation (RDT&E) Production Operations and Maintenance (O&M) Overseas Contingency Operations (OCO) Other Program Allocation (OPA) Foreign Military Sales (FMS)* Other (please describe) *For FMS cases, please indicate if FMS dollars are part of project. If FMS dollars are "fair-share" of system dollars (contribute a portion of project costs), then include them in the dollars reported in the rest of the form. If FMS dollars are used for separate hardware or distinct releases, then include that in a note in the comment section and do not include the dollars in the remaining sections.

System-Level Context Comments. Provide any additional information that would permit a cost analyst to correctly interpret the above data.

Maintenance Organization.

Data Collection Date. Enter the appropriate numeric data for the year, month, and day in the format MM/DD/YYYY.

Point of Contact Info: Name, Reporting Organization and Division, Telephone Number, and E-Mail Address.

Program Executive Office. Provide the current or past name of the Program Executive Office for this system, e.g. PEO Aviation, PEO C3T. If the system is not associated from a specific PEO, then use “N/A” and explain in 1.7.9.

Software Engineering Center (SEC). The name of the applicable SEC, e.g., CECOM, AMRDEC, TARDEC, ARDEC. If the system will not transition to an SEC at any point, then use “N/A” and explain in 1.7.9.

Transition to SEC Date. When did or will the transition take place? Is there an MOA or transition plan?

Managing Organization. What organization is responsible for managing the technical decisions on the system at this time, e.g. PEO or SEC?

Developer(s). What Government or contractor (prime or subcontractor) organization(s) developed the original software for the version that is currently being maintained? Identify the major development organizations (prime developer and major subs) and the role of each organization as prime or subs.

Maintainer(s). What Government or contractor (prime or subcontractor) organization(s) are doing most of the primary software maintenance activities discussed in this questionnaire (the major players)? Identify the role of each organization as prime or sub.

Maintenance Organization Comments. Provide any additional information that would better explain the above data.

System-Level Data – Annual

For effort and cost data, actual data is preferred. If effort hour data is not available, provide hours based on Full Time Equivalent counts per year, using engineering judgment. Multiple the FTE count by the average hours per year to provide hours. Cost can then be calculated by multiplying hours by the average labor rate per person. In this case, add an appropriate note in the comment field in each section.

If you don’t have actual data or FTE data, please provide planning data and make a note in the comment field in each section. If data is not available, use the responses defined on page 2 of this questionnaire (N/A, Zero, Unavailable).

Report all effort data in hours (not FTE counts - translate those into hours). Report all cost data in dollars (not thousands of dollars). If any other unit of measure is used, please record that in the comment field.

In the comment fields, for each WBS elements, please provide the appropriate data source used, information on how the data is collected and the level at which it is collected, as well as any additional information that would better explain the above data. For example, describe whether the system tracks release data separately, and if the contractor collects effort/cost data at a detailed WBS level. Some examples of data sources are, but not limited to: Cost Reports, GFEBS, engineering judgment, etc.

Please enter a response in every field.

Fiscal Year. Provide the fiscal year for which the cost data below applies. Use a separate column for each year.

Total Effort and Cost. All effort and cost associated with the software system for the year. This should be the sum of all other data, including the portion of the release data spent in the specified year (total of the WBS elements in sections 2.3 to 2.10 plus 3.4).

Total Effort Hours Expended - Government. The number of government hours expended to perform all software maintenance activities.

Total Effort Hours Expended - Contractor. The number of contractor hours expended to perform all software maintenance activities.

Total Cost Expended - Government. Funding expended performing all government software maintenance activities.

Total Cost Expended - Contractor. Funding expended performing all contractor software maintenance activities.

Total ODC Cost Expended. All ODC costs expended, including facility and equipment ODC, travel, and license costs.

Total Effort/Cost Comments. Provide any additional information that would better explain the above data.

Project Management Effort and Cost. Effort and cost associated with system specific software maintenance project and technical management. These activities include planning, execution management, configuration management, resource management, contracting, and measurement. For program offices, include only the personnel who are doing software project management.

Number of Project Mgt. Hours Expended - Government. Provide the number of government labor hours that were expended in project management.

Number of Project Mgt. Hours Expended - Contractor. Provide the number of contractor labor hours that were expended in project management.

Project Mgt. Cost Expended - Government. Provide the government cost expended in project management.

Project Mgt. Cost Expended - Contractor. Provide the contractor cost expended in project management.

Project Mgt. Effort/Cost Comments. Provide any additional information that would better explain the above data.

Software License Management Effort and Cost. Cost and effort associated with the procurement and renewal of software licenses for NDI, COTS, GOTS, or open-source software. This data is for managing licenses for the maintenance facility as well as deployed systems. The effort and cost for configuring and integrating the licensed software is reported separately in the release data.

Number of License Management Hours Expended - Government. The number of hours expended to manage software licenses, for government personnel.

Number of License Management Hours Expended - Contractor. The number of contractor hours expended to manage software licenses.

Cost of License Management Expended - Government. Funding expended for the management of software licenses, for government-specific costs.

Cost of License Management Expended - Contractor. Funding expended for contractors, for the management of software licenses.

Software License Costs. Provide the total amount spent in each fiscal year for software licenses, including both Right-to-Use and Maintenance agreements.

License Mgt. Effort/Cost Comments. Provide any additional information that would better explain the above data.

Software License Details (separate tab).

Provide cost information on each software license used by the system. Provide the details of the current set of licenses, on the tab entitled “Software Licenses”. These include licenses for the maintenance facility as well as deployed systems. List all applicable licenses, and the cost the last time it was purchased. There can be two rows for each license: one for the right-to-use license, and one for the maintenance agreement. For any enterprise licenses, the cost data can be marked “unknown”.

Software License Name. List the names of the applications or products that constitute part of the final delivered product or part of the system being supported, whether they are NDI, COTS, GOTS, or open-source products. If a proprietary application or product that is not generally commercially available will be included, identify it here and include any necessary explanation in the license comments (section 2.5.9).

Software Company Name. List the name of the company that created the software product that has the license.

Quantity. The number of licenses for each license name. “Unlimited” can be used for enterprise licenses or those with unrestricted usage.

Entitlement. Describe how the software is licensed by indicating the number of processors, number of processor cores, one “seat” or computer, one site, enterprise-wide, or unlimited.

Total Cost. The total cost for all licenses under this license name. Specify costs using a dollar format: $N,NNN. For enterprise licenses, if the cost is unknown, enter “unavailable”.

Type. Indicate whether it is a Right-to-Use or Maintenance license. Right-to-Use provides licensing to operate the NDI, COTS, GOTS, or open-source products software. A maintenance license provides additional assistance on using or modifying the software.

Duration. The duration of the license as either annual renewal (renewed every year) or indefinite (free or one-time cost for unlimited license time).

License Award Date. The license start date.

Software License Comments. Provide any additional information that would better explain the above data.

Certification & Accreditation (C&A) Effort and Cost. Effort and cost associated with verifying a software system against externally defined domain performance criteria for this release. These activities include preparing for and managing certification and accreditations. Activities include Cyber-Security (including Risk Management Framework, DIACAP, IAVA), Interoperability testing, Common Operating Environment testing, Safety, Airworthiness, and Networthiness.

Number of C&A Hours Expended - Government. The number of hours expended to manage or conduct C&A activities, for government personnel.

Number of C&A Hours Expended - Contractor. The number of hours expended to manage or conduct C&A activities, for contractor personnel.

Cost of a C&As Expended - Government. Provide the government-specific costs for conducting C&As on a release.

Cost of a C&As Expended - Contractor. Provide the contractor costs for conducting C&As on a release.

C&A Types and Frequency. Describe the types of C&As performed, and the frequency of conducting each (e.g. quarterly, annually, every 3 years).

C&A Effort/Cost Comments. Provide any additional information that would better explain the above data. Indicate if the data includes the internal preparation for C&A activities, or only the formal C&A costs.

System Facilities (SF) Effort and Cost. Effort and cost associated with establishing, operating and sustaining software development facilities. Facilities include government and contractor acquisition, development, maintenance, test, and external test facilities. Test facilities may include test facilities at the PEO or SEC, the contractor, the C&A organization, interoperability test facilities, IOT&C facilities, and operational test facilities. In the comments section, provide a list of all facilities used, even those which are paid for separately and not included in this system’s data (note that they are paid for separately).

Number of SF Hours Expended - Government. The number of hours expended to manage system facilities, for government personnel.

Number of SF Hours Expended - Contractor. The number of contractor hours expended to manage system facilities.

Cost of SF Expended - Government. Funding expended for government costs associated with system facilities.

Cost of SF Expended - Contractor. Funding expended for contractor costs associated with system facilities.

Cost of SF ODC Expended. Funding expended for the purchase of any system facility assets.

System Facilities Effort/Cost Comments. Provide any additional information that would better explain the above data.

Sustaining Engineering (SE) Effort and Cost. Effort and cost associated with engineering (e.g. system specific test support, investigations, PM support) and user support (e.g. user training, help desk, software delivery). SE does not include any effort or cost for either maintenance (corrections) or capability enhancements: these are included in the release data. User support should not include FSEs, nor data in other sub-categories.

Number of SE Hours Expended - Government. The number of government hours expended to perform SE activities.

Number of SE Hours Expended - Contractor. The number of contractor hours expended to perform SE activities.

Cost of SE Expended - Government. Funding expended for government costs in performing SE activities.

Cost of SE Expended - Contractor. Funding expended for contractor costs in performing SE activities.

Sustaining Engineering Effort/Cost Comments. Explain the SE activities included in the data provided. Provide any additional information that would better explain the above data.

Field Software Engineering (FSE) Effort and Cost. Effort and cost associated with the on-site support of a deployed software product or system in its operational environment. FSE duties may include on-site technical assistance, problem troubleshooting, software installation, operational assistance, or on-site training. If FSE personnel are used, but paid for by another organization, put “unknown” into the hours/cost and a description of who pays for that personnel into the comment field (if the FTE equivalent is known, please provide that data in the hours field).

Number of FSE Hours Expended - Government. The number of government hours expended to perform FSE activities.

Number of FSE Hours Expended - Contractor. The number of contractor hours expended to perform FSE activities.

Cost of FSE Expended - Government. Funding expended performing government FSE activities.

Cost of FSE Expended - Contractor. Funding expended performing contractor FSE activities.

Field Software Engineering Effort/Cost Comments. Provide any additional information that would better explain the above data. If FSE hours are paid for from another source, or provided as part of a larger effort (e.g. FSEs cover all PEO systems or the FSE does both hardware and software), then use “unknown” in the data fields and provide an appropriate comment.

Operational Management (OM) Effort and Cost. System-level effort and cost allocated to pay for OM resources. This may be done as a tax, within overhead, or through inclusion of certain OM resources. OM resources are those associated with establishing and operating the organizational infrastructure required to implement common software maintenance business and technical processes across multiple software systems. These activities include operations, organization management, personnel management, financial management, information management, process management and change management.

Number of OM Hours Expended - Government. The number of government hours expended to perform OM activities.

Number of OM Hours Expended - Contractor. The number of contractor hours expended to perform OM activities.

Cost of OM Expended - Government. Funding expended performing government OM activities.

Cost of OM Expended - Contractor. Funding expended performing contractor OM activities.

Operational Management Effort/Cost Comments. Explain the OM activities included in the data provided. Provide any additional information that would better explain the above data.

Labor Effort and Cost Data.

Labor Hours per Person Year - Government. Specify the standard number of labor hours per person year for the government organization, e.g. 1720 hours per work year.

Labor Hours per Person Year - Contractor. Specify the standard number of labor hours per person year for the contractor organization, e.g. 2000 hours per work year.

Average Annual Labor Rate - Government. Provide an average yearly government labor rate relating to the total cost, e.g. $200,000.00 per year. This should be the fully burdened cost including any overhead. This allows translation between hours and cost, if both are not available.

Average Annual Labor Rate - Contractor. Provide an average contractor yearly labor rate relating to the total cost, e.g. $300,000.00 per hour. This should be the fully burdened cost including any overhead. This allows translation between hours and cost, if both are not available.

Labor Effort and Cost Data Comments. Provide any additional information that would better explain the above data.

Release Schedule / Cost / Size Data.

For a delivered release or an engineering release that has gone through Final Qualification Test, provide for the cumulative actual values. For a partial release, provide the planning data including the estimate to complete for the remaining work, as well as the cumulative-to-date actual values.

In the comment fields, please provide the appropriate data source used, where applicable, along with any additional information that would better explain the above data. Some examples of data sources are, but not limited to: Cost Reports, GFEBS, USC Code Counting Tool, CM tools such as DOORS, etc.

Report all effort data in hours (not FTE counts - translate those into hours). Report all cost data in dollars (not thousands of dollars). If any other unit of measure is used, please record that in the comment field.

Each column in the data collection form is used to record data for one release regardless of release type (explained in section 3.1.3).

If a release contains only IAVA fixes, fill out rows 3.1 to 3.4 (excluding 3.2.5) and 3.9 only. If your release has updates in additional to IAVAs, then fill out the full worksheet. Short IAVA releases that are done monthly may be aggregated into quarterly releases, for the purposes of this data collection.

Provide the data for any release that were completed.

Release Context.

Release Name. Provide the name/number of the release.

Release Description. Provide a brief functional description of what is contained in this release.

Release Characterization. Identify the percentages of changes to the existing software baseline in terms of the percentage of enhancements, maintenance, cyber-security upgrades, or other. Percentages should add to 100%.

Release Characterization Comments. Provide any additional information that would better explain the above data. If other, please explain.

Release Status. Enter the state of the release for which the data is being collected:

Partial: Any release close to being delivered and contains actual data to-date along with planning data for the entire release.

Final: A delivered release and contain all planned and actual data for that release.

Product and Maintenance Description.

Software Release Anomalies. Include a description of any unusual circumstances that impact the size, effort or duration of the performed maintenance, e.g. maintenance system funding, staffing, major changes, unplanned technical refresh.

Operating Environment. Identify the operating environment in which the maintained software system operates:

Surface Fixed: Software is embedded in a system at a fixed site.

Surface Mobile: Software is embedded in a system that is moved somewhere and setup.

Surface Portable: Software is embedded in a handheld or portable device.

Surface Vehicle: Software is embedded as part of a moving vehicle.

Air Vehicle: Software is embedded as part of an aircraft.

Sea Systems: Software is embedded as part of a surface or underwater boat/ship.

Ordnance Systems: Software is embedded as part of a missile or rocket.

Space Systems: Software is embedded as part of a spacecraft.

Multiple: System operates in more than one of the above environments. Identify all the relevant environments in the comments in 3.2.6.

Manned vs. Unmanned. For the operating environment above, indicate if it is a “Manned” or “Unmanned” environment. In general, AIS systems are “manned”.

Application Super-Domain. For the work being done on this release, identify which super-domain characterizes the overall release. Select the primary super-domain for this release. Definitions are provided in Annex A. Indicate if the system is:

Real Time Engineering Mission Support Automated Information System

Application Domains for Release. For the work being done on this release, identify all relevant application domains that apply by entering the approximate percentage of effort relative to total release effort expended (or planned) for this release. Definitions are provided in Annex A. The total percentage should add to 100%. Indicate if the system is:

Release Context Comments. Provide any additional information that would better explain the above data.

Release Schedule. Provide the schedule for the release.

Actual Start Date. The release start date is the review and approval of requirements (or equivalent) that will be implemented in the release. Provide the actual date using the format MM/DD/YYYY.

Actual End Date. The release end date should be Final Qualification Test (or equivalent). Provide the actual date using the format MM/DD/YYYY.

Schedule Comments. Provide any additional information that would better explain the above data, e.g. alternative criteria for start / end dates.

Total Release Effort and Cost. Report the total expended effort and/or dollar cost for the release to implement software changes. This includes all hours and cost associated with changing software to maintain the operational capability of the system. It includes all hours associated with addressing/correcting errors, addressing cyber-security issues and changing software, including requirements analysis, design, implementation, and software testing (including IV&V). Provide this data for each organization working on this system.

If you don’t have actual expended data, please provide data based on equivalent FTE data multiplied by the average labor hours per person year. If you do not have FTE data, please provide planning data. In either case, make a note in the comment field in each section.

Number of Total Release Hours Expended - Government. Provide the number of government labor hours that were expended in this release.

Number of Total Release Hours Expended - Government. Provide the number of contractor labor hours expended in this release.

Total Release Cost Expended - Government. Provide the total government cost expended in this release.

Total Release Cost Expended - Contractor. Provide the total contractor cost expended in this release.

Total Effort/Cost Comments. Provide any additional information that would better explain the above data.

In the following sections, there are multiple size measures. Provide the data that is relevant to your system. For example if your system tracks requirements and tracks Change Requests, you would fill out sections 3.5 and 3.8. If your system tracks SLOC, fill out section 3.6. Use N/A for any measures that are not applicable to your system. In general, every system should be able to provide a count of requirements, either a SLOC or non-SLOC software size, software changes, and IAVAs (if applicable).

Requirements / Interface Size Reporting.

Software Requirements Description. Describe the maintainer’s specific rules and/or tools used to count requirements. Identify the source for requirements data in the comments field. Primarily, the software requirements form the Software Requirements Specification should be provided. If the SRS requirements are not available, then an alternate requirements count may be provided, with a definition of the type used.

Number of Software Requirements at Release Start. Provide the number of software requirements for this system, at the start of this release. This should be the total number of software requirements for this system.

Number of Software Requirements Implemented. Provide the number of software requirements that were implemented in this release (e.g. new, modified, or deleted).

External Interface Count Description. Describe the maintainer’s specific rules and/or tools used to count external interfaces. The definition must also identify the source used for interface data (i.e., SV-6, ICD).

Number of External Interfaces at Release Start. Provide the total number of software external interfaces at the start of this release. This should be the total number of software external interfaces for this system.

Number of External Interfaces Changed. Provide the total number of software external interfaces that were changed in this release (e.g. new, modified, or deleted).

Requirements / Interface Size Comments. Provide any additional information that would better explain the above data.

SLOC-Based Software Size. Report the planned or actual delivered size of the product maintained by software language, in Source Lines of Code (SLOC). Do not include any code that might be needed to assist maintenance and that will not be delivered (such as temporary stubs, test scaffoldings, or debug statements). Additionally, the code size shall be reported by source code categories: Baseline, New, Modified, Reused, Auto-Generated, and Deleted. Report SLOC counts using single units (as opposed to thousands of units). Equivalent SLOC counts are not acceptable.

When reporting planning data, provide planning data as estimated at the start of the release and not at the time of this data submission.

SLOC Counting Rules. In general, logical lines of code (e.g. non-embedded semicolons) should be used. Describe the SLOC counting rules including:[footnoteRef:2] [2: The Software Engineering Institute (SEI) technical report, “Software Size Measurement – A Framework for Counting Source Statements,” has an example of a checklist for describing what is included/excluded in a code count.]

What is counted (i.e., terminal semi-colons) What is included in the size count (i.e., job control language, included files, comments) Tool used to count SLOC Source of the data Software Language. Indicate the programing language(s) of the code that is being sized.

Baseline Code Actual Count. Source code size at the start of the release (sometimes called carry-over code). Provide the count of actual baseline code size for this release.

New Code Planned Count. Source code that is programmer-generated. Provide the count of planned new code size for this release.

New Code Actual Count. Provide the count of actual new code size in this release.

Modified Code Planned Count. Source code that is reused with changes requiring some degree of redesign, recode and/or reintegration effort. For example, if 5,000 SLOC are modified in a 30,000 SLOC size component, report 30,000. Provide the count of planned modified code size for this release. Provide the count of planned modified code size in this release.

Modified Code Actual Count. Provide the count of actual modified code size in this release.

Reuse Code Planned Count. Code reused without modification is code that has no design or code modifications. Provide the count of external reuse code planned for this Release.

Reuse Code Actual Count. Provide the count of external reuse code implemented in this Release.

Auto-Generated Code Count. Provide the count of code generated from a software application in this release based on higher-level abstractions such as algorithms or design patterns.

Deleted Code Count. Provide the count of code deleted from the baseline code in this release.

Comment Line Count. Provide the count of the number of comment lines in the code.

Delivered Code Planned Count. Provide the count of Delivered Code Planned to be delivered in this Release.

Delivered Code Actual Count. Provide the count of Delivered Code in this Release.

SLOC Software Size Comments. Provide any additional information that would better explain the above data.

Non-SLOC-Based Software Size. Report the use of non-SLOC sizing methods here. Examples of other sizing measures are Function Points, RICE-FW Objects, Use Case Points, Story Points, Feature Points, or Predictive Object Points. Provide the planned or implemented measures for each release. Provide the detailed counts for the method used, on a separate tab. For example, if using RICE-FW, provide the planned and implemented counts for reports, interfaces, conversions, extensions, forms, and workflows, on a separate tab.

Sizing Method Name. Provide the name of the sizing method.

Sizing Method Description. Describe the sizing method here. If the measure has been defined by a standard, explicitly reference the standard or fully explain the method.

Sizing Method Planned Counts. Report the planned amount of size to be changed in a Release.

Sizing Method Implemented Counts. Report the amount of size implemented in a Release.

Sizing Method Comments. Provide any additional information that would better explain the above data.

Software Change Counts. Report the number of software changes that are applicable to this release. Changes may be called Change Requests, Problem Reports, Issues, Defects, or other organizational / system specific names.

Software Change Name. Provide the name used on this system for software changes.

Software Change Description. Describe what is considered a software change. Provide the source and internal definitions for change count measures being reported and specific rules used to count the measures. If software changes are tracked, describe how they are tracked, what tools are used to do the tracking, and the source of the data.

Total Number of Software Changes. Provide a count of the total number of software changes that were addressed during this release.

Number of Planned Priority 1 Software Changes. Provide a count of the number of software changes by severity as described in Table 1 that were planned and authorized at the beginning of this release.

Number of Actual Priority 1 Software Changes. Provide a count of the number of actual software changes implemented by severity as described in Table 1.

Number of Backlog Priority 1 Software Changes. Provide a count of the number of software changes that were not implemented by severity as described in Table 1, i.e., backlogged.

thru 3.8.18 - repeat planned, actual, and backlog information for priority 2 through 5 software changes.

Table 1. Software Change Severity

Jeopardize safety, security, or other requirement designated “critical”.

Adversely affect technical, cost, or schedule risks to the project or to life cycle support of the system, and no work-around solution is known.

Adversely affect technical, cost, or schedule risks to the project or to life cycle support of the system, but a work-around solution is known.

Result in inconvenience or annoyance for development or support personnel, but does not prevent the accomplishment of those responsibilities.

Software Change Comments. Provide any additional information that would better explain the above data.

IAVAs. Report the number of Information Assurance Vulnerability Alerts (IAVAs) that are applicable to each release. IAVAs are typically categorized as:

Table 2. IAVA Categories

Number of IAVAs Planned for the Release. Provide the number of software IAVAs planned to be addressed in this release.

Number of IAVAs Addressed. Provide the number of software IAVAs that were addressed in this release.

Number of Actual Cat1 IAVAs Addressed.

Number of Backlog Cat1 IAVAs. Provide the number of CAT1 software IAVAs that were identified during this release, but deferred to a future release for implementation.

Number of Actual Cat2 IAVAs Addressed.

Number of Backlog Cat2 IAVAs.

Number of Actual Cat3 IAVAs Addressed.

Number of Backlog Cat3 IAVAs.

Number of IAVAs Comments. Provide any additional information that would better explain the above data

Page 23 of 23 Pages

Annex A Application Domain Definitions

Super Domains

Super Domain

Real-Time (RT)

Real-Time is the most constrained type of software. These are specific solutions limited by system characteristics such as memory size, performance, or battery life. These projects take the most time and effort due to constraints, e.g.,

- May have guaranteed task execution requirements, i.e., missed deadline means catastrophic result

- May have to be compact and efficient due to limited storage capacity and high throughput requirements

- Could have very high reliability requirements (life critical, manned mission)

- Might have tightly coupled interfaces

- Program code may be imprinted on hardware devices

- May process sensor inputs and directs actuator outputs.

- Sometimes executed on special-purpose processors

Microcode & Firmware; Signal Processing; Vehicle Control; Vehicle Payload; Other Real-Time Embedded; Command & Control; Communication

- May have a fast response time requirement

- May have more storage capacity

- Might need to be highly reliable but not life critical

- May have multiple interfaces with other systems

- May implement complex algorithms, models or protocols

- Program code can be modified and uploaded

- Executes on general purpose processors that may be embedded in special purpose hardware

System; Process Control; Scientific and Simulation; Test, Measurement, Diagnostic and Evaluation

- Has to have an acceptable response time most of the time

- Less limited by storage or throughput

- Less stringent reliability requirement

- Software restarts are acceptable

- Fewer interfaces

- Relatively low complexity algorithms, models or protocols

- Program code can be modified and uploaded

- Executes on general purpose processors on general purpose computer boards Training; Software Tools

- Must have acceptable response time

- Fewer storage or throughput constraints

- Must be reliable enough to prevent data loss

- May consist of a single COTS / GOTS solution or multiple products coordinated with customer software

- Algorithms, models and protocols are well understood

- Code may not be available for modification

- Software restarts are acceptable

- Executes on commercial processing hardware Mission Planning; Custom AIS Software; Enterprise Service Systems; Enterprise Information Systems

Application Domains

Super Domain