21R0202 - Attachment A13 Test Type Table.pdf

Attachment 13: Test Type Table

Test Type Definition

Access Control Testing

A type of testing that attests that the target-of-test data (or systems) are accessible only to those actors for which they are intended, as defined by use cases. Access Control Testing verifies that access to the system is controlled and that unwanted or unauthorized access is prohibited. This test is implemented and executed on various targets-of-test.

Benchmark Testing:

A type of performance testing that compares the performance of new or unknown functionality to a known reference standard (e.g., existing software or measurements). For example, benchmark testing may compare the performance of current systems with the performance of the

Linux/Oracle system.

Build Verification

Testing

(Prerequisite: Smoke

Test)

A type of testing performed for each new build, comparing the baseline with the actual object properties in the current build. The output from this test indicates what object properties have changed or don't meet the requirements. Together with the Smoke test, the Build Verification test may be utilized by projects to determine if additional functional testing is appropriate for a given build or if a build is ready for production.

Business Cycle Testing

A type of testing that focuses upon activities and transactions performed end to end over time. This test type executes the functionality associated with a period (e.g., one-week, month, or year). These tests include all daily, weekly, and monthly cycles, and events that are date-sensitive (e.g., end of the month management reports, monthly reports, quarterly reports, and year-end reports).

Compliance Testing A type of testing that verifies that a collection of software and hardware fulfills given specifications.

Component

Integration Testing

Testing performed to expose defects in the interfaces and interaction between integrated components as well as verifying installation instructions.

Configuration Testing

A type of testing concerned with checking the programs compatibility with as many possible configurations of hardware and system software. In most production environments, the particular hardware specifications for the client workstations, network connections, and database servers vary.

Client workstations may have different software loaded, for example, applications, drivers, and so on hand, at any one time: many different combinations may be active using different resources. The goal of the configuration test is finding a hardware combination that should be, but is not, compatible with the program.

Contention Testing A type of performance testing that executes tests that cause the application to fail with regard to actual or simulated concurrency.

Data and Database

Integrity Testing

A type of testing that verifies that data is being stored by the system in a manner where the data is not compromised by the initial storage, updating, restoration, or retrieval processing. This type of testing is intended to uncover design flaws that may result in data corruption, unauthorized data access, lack of data integrity across multiple tables, and lack of adequate transaction performance. The databases, data files, and the database or data file processes should be tested as a subsystem within the application.

Documentation Testing

Documentation testing is a type of testing that should validate the information contained within the software documentation set for the following qualities: compliance to accepted standards and conventions, accuracy, completeness, and usability. The documentation testing should verify that the required information is provided in order for the appropriate user to be able to properly install, implement, operate, and maintain the software application.

Error Analysis Testing

This type of testing verifies that the application checks for input, detects invalid data, and prevents invalid data from being entered into the application. This type of testing also includes the verification of error logs and error messages that are displayed to the user.

Exploratory Testing

A technique for testing computer software that requires minimal planning and tolerates limited documentation for the target-of-test in advance of test execution, relying on the skill and knowledge of the tester and feedback from test results to guide the ongoing test effort. Exploratory testing is often conducted in short sessions in which feedback gained from one session is used to dynamically plan subsequent sessions.

Failover Testing

A type of testing test that ensures an alternate or backup system properly

"takes over" (e.g., a backup system functions when the primary system fails). Failover Testing also tests that a system continually runs when the failover occurs, and that the failover happens without any loss of data or transactions. Failover Testing should be combined with Recovery Testing.

Installation Testing

A type of testing that verifies that the application or system installs as intended on different hardware and software configurations, and under different conditions (e.g., a new installation, an upgrade, and a complete or custom installation). Installation testing may also measure the ease with which an application or system can be successfully installed, typically measured in terms of the average amount of person-hours required for a trained operator or hardware engineer to perform the installation. Part of this installation test is to uninstall. As a result of this uninstall, the system, application and database should return to the state prior to the install.

Integration Testing An incremental series of tests of combinations or sub-assemblies of selected components in an overall system. Integration testing is incremental in a successively larger and more complex combinations of components tested in sequence, proceeding from the unit level (0% integration) to eventually the full system test (100% integration).

Load Testing

A performance test that subjects the system to varying workloads in order to measure and evaluate the performance behaviors and abilities of the system to continue to function properly under these different workloads.

Load testing determines and ensures that the system functions properly beyond the expected maximum workload. Additionally, load testing evaluates the performance characteristics (e.g., response times, transaction rates, and other time-sensitive issues).

Multi-Divisional Testing A type of testing that ensures that all applications will operate in a multidivisional or multi-site environment recognizing an enterprise perspective.

Parallel Testing

The same internal processes are run on the existing system and the new system. The existing system is considered the “gold standard”, unless proven otherwise. The feedback (e.g. expected results, defined time limits, data extracts) from processes from the new system is compared to the existing system. Parallel testing is performed before the new system is put into a production environment.

Performance

Monitoring Testing

Performance profiling assesses how a system is spending its time and consuming resources. This type of performance testing optimizes the performance of a system by measuring how much time and resources the system is spending in each function. These tests identify performance limitations in the code and specify which sections of the code would benefit most from optimization work. The goal of performance profiling is to optimize the feature and application performance.

Performance Testing

Performance Testing assesses how a system is spending its time and consuming resources. Performance testing optimizes a system by measuring how much time and resources the system is spending in each function. These tests identify performance limitations in the code and specify which sections of the code would benefit most from optimization work. Performance testing may be further refined by the use of specific types of performance tests, such as, benchmark test, load test, stress test, performance monitoring test, and contention test.

Privacy Testing

A type of testing that ensures that (1) employee data are adequately protected and (2) systems and applications comply with the Privacy and

Security Rule provisions.

Product Component

Testing

Product Component Testing (aka Unit Testing) is the internal technical and functional testing of a module/component of code. Product Component

Testing verifies that the requirements defined in the detail design specification have been successfully applied to the module/component under test.

Recovery Testing

A type of testing that causes an application or system to fail in a controlled environment. Recovery processes are invoked while an application or system is monitored. Recovery testing verifies that application or system, and data recovery is achieved. Recovery Testing should be combined with

Failover Testing.

Test Type Definition

Regression Test A type of testing that validates existing functionality still performs as expected when new functionality is introduced into the system under test.

Risk Based Testing

A type of testing based on a defined list of project risks. It is designed to explore and/or uncover potential system failures by using the list of risks to select and prioritize testing.

Security Testing

A type of test that validates the security requirements and ensures readiness for the independent testing performed by the Security

Assessment Team as required by the Test and Certification Process.

Smoke Test

A type of testing that ensures that an application or system is stable enough to enter testing in the currently active test phase. It is usually a subset of the overall set of tests, preferably automated, that touches parts of the system in at least a cursory way.

Stress Testing

A performance test implemented and executed to understand how a system fails due to conditions at the boundary, or outside of, the expected tolerances. This failure typically involves low resources or competition for resources. Low resource conditions reveal how the target-of-test fails that is not apparent under normal conditions. Other defects might result from competition for shared resources (e.g., database locks or network bandwidth), although some of these tests are usually addressed under functional and load testing. Stress Testing verifies the acceptability of the systems performance behavior when abnormal or extreme conditions are encountered (e.g., diminished resources or extremely high number of users).

System Testing

System testing is the testing of all parts of an integrated system, including interfaces to external systems. Both functional and structural types of testing are performed to verify that the system performance, operation and functionality are sound. End to end testing with all interfacing systems is the ultimate version.

Usability Testing

Usability testing identifies problems in the ease-of-use and ease-of-learning of a product. Usability tests may focus upon, and are not limited to: human factors, aesthetics, consistency in the user interface, online and context-sensitive help, wizards and agents, user documentation.

UAT

UAT is a type of Acceptance Test that involves end-users testing the functionality of the application using test data in a controlled test environment.

User Interface Testing

UI testing exercises the user interfaces to ensure that the interfaces follow accepted standards and meet requirements. User-interface testing is often referred to as GUI testing. UI testing provides tools and services for driving the user interface of an application from a test.