The setup is the basis for the System Test with Test Management and other constructs.

If you have a new system to test, one has to start with the setup. It builds and is the basis for the test activities. Test Management is responsible for the setup of System Test — not only done initially, but improved and adjusted constantly.

Test Management#

Test Management handles the strategic planning, organization, monitoring and execution of all test related activities.

Test Administration

The starting point from the idea to test that leads to all that follows. It is the agency of the individual responsible for System Test.

Test Structure

For effective Test Management the structure of the test organization must be clear, including test concepts, test plans, organizing the Test Domains and leading quality management efforts.

  • Test Preparation: Test Specification, Test Infrastructure, Test Planning
  • Test Execution: actual testing
  • Test Processing: Problem Management, Reporting

Test Team

A group of individuals working interrelated on the test activities. Can be embedded in a project team or larger organizational structure.

Test Workflow

The work executed by the test team follows a defined workflow through test management and test domains to produce test related output.

Test Facilities

Physical and digital equipment to enable test work — test benches, test tools, etc.

Test Objects

The subject that is tested. In System Test: components that interact together and features provided by the system. It is important to exactly specify what is the Test Object.

Test Concept#

The Test Concept outlines the approach of testing the system. It gives guidance on the landscape of test possibilities and how testing for a specific system is done.

Quality attributes to consider: functionality, usability, reliability, performance, safety, security, privacy, compatibility, portability, maintenance.

Test Levels

  • Component level / unit test: one specific component
  • Functional level / integration test: looking at specific functions
  • Product level / system test: finished, end-to-end product
  • Customer level / acceptance test: product from a customer point of view

Test Methods

Black-Box: Equivalence Partitioning, Boundary Value Analysis, Decision Table Testing, State Transition Testing, Use Case Testing

White-Box: Statement Coverage, Branch Coverage, Path Coverage, Condition Coverage

Experience-Based: Error Guessing, Exploratory Testing, Checklist-Based Testing

Test Strategy

Defines the conditions of test execution: analytical, methodical, expert-based, reactive, model-based, process-compliant, change-based, regression-averse strategies.

Test Types

  • Dynamic tests: use the product in action and observe behavior
  • Static tests: based on documents — walkthroughs, reviews, inspections, analysis tools

Test Plan#

The Test Plan documents the general information about the system in the test context and the specifics about testing. It is the architecture of system testing for a specific test object.

Strategic Test Plans

Master Test Plan: high-level concept, generic test plan for a project, created during early phase, basis for other plans.

Operational Test Plans

  • Test Plan Schedule: focus on timeline, schedules test activities
  • Sub Test Plan: based on the Master Test Plan, for a specific trigger or component
  • Test Execution Plan: specifies which test is to be executed when, where and in which order

Test Artifacts#

Project-Based Artifacts

  • Project plan: official project document
  • Requirements: information that describes the product

Test-Based Artifacts

  • Test plan: main document of testing
  • Test case: documented test specification
  • Issues: documentation of tasks and open topics

Issue Types

  • Error: confirmed wrong doing, can lead to defect
  • Defect: discovered in static tests, can lead to failure
  • Failure: discovered in dynamic test, leads to a bug/failure mode
  • Bug / Failure Mode: state of failure with notable impact

Traceability

Ensures every document from specification to test case and every test activity is linked. The main path is from requirement to test case — bidirectional linkage allows forward and backward traceability.

Organization Work#

Test Metrics

  • Amount of tests, remaining risk, achieved quality, economic factors
  • Test related 'Definition-of-Done': based on test cases, defects, requirements, cost/time

Risk Management

Complex systems bring inherent risks: interface errors, dependency failures, resource constraints, variant configuration mismatches, environmental disturbances.

Key methods:

  • Fault Tree Analysis (FTA): top-down analysis to find root causes of failure
  • FMEA: systematic risk analysis evaluating Severity × Occurrence × Detection = Risk Priority Number

Single Point of Entry

Only one direct responsible individual that has influence to the workings of the System Test and the Test Team.