Quality is the foundation of System Test and the link to development.

Quality is something everyone naturally strives for. The term 'quality' originates from the Latin word 'qualitas', coined by the Roman philosopher Cicero in the 1st century BCE. He derived it from the Latin adjective 'qualis', meaning 'of what kind'.

The origin of systematic quality efforts lies in guilds, which are professional associations. Craftsmen were responsible for producing quality themselves. If they did not meet the quality requirements of customers, they were excluded from the guild.

Nowadays quality is a competitive advantage. Excellent organizations deliver excellent products with highest quality.

Quality Definitions

  • ISO: 'The degree to which a set of inherent characteristics fulfills requirements.'
  • ISO: 'The totality of features and characteristics of a product or service that bears on its ability to satisfy stated or implied needs.'
  • Conformity to requirements, freedom from deficiencies and fit for purpose.
  • Quality (Management) makes sure the product satisfies customer and user expectations.
  • Quality is a state of valuable, reliable and excellent products.

Quality is the cross-cutting principle that serves as a foundation for System Test. It is an essential and integral part of development.

Quality Management#

Quality Management (QM) is the overarching framework for managing all quality related activities.

  • Quality Management (QM): framework for all quality related matters
  • Quality Assurance (QA): prevents defects through processes
  • Quality Control (QC): finding and fixing issues

Quality Management Systems (QMS) are systems to manage quality. They typically include quality policies, objectives, planning, control, assurance, and improvement. QMS may fulfill standards like ISO 9001.

The basis of quality management is the PDCA cycle:

  • Plan: planning resources and processes
  • Do: realizing processes defined in the plan-phase
  • Check: control what was executed in the do-phase
  • Act: fix deviations found in the check-phase

Quality Characteristics#

Products have both inherent and assigned characteristics:

  • Inherent characteristics: properties that are fundamental and immutable (e.g. color)
  • Assigned characteristics: explicitly assigned and can be altered (e.g. price)

Only inherent characteristics are quality indicators.

Quality Levels

  • Product quality: product quality characteristics and customer demands
  • Process quality: includes process, where the product is part of
  • System quality: sees product and the process as whole system

Quality Perspectives

  • Product-based quality: measurable and classifiable product characteristics
  • Customer-based quality: places emphasis on perceived customer demands
  • Manufacturing-based quality: secondary characteristics required for production
  • Value-based quality: refers to the cost or price of the benefit

Quality Dimensions

  • Performance: objective, measurable characteristics
  • Features: secondary characteristics
  • Reliability: probability that the product will work over time
  • Conformance: in line with pre-established standards
  • Durability: economic and technical life span
  • Serviceability: speed, courtesy and competence of repair
  • Aesthetics: subjective measure related to the look and feel
  • Perceived quality: subjective measure of overall quality

Quality Standards#

Standards and norms stimulate global trade, guarantee a specific quality level and facilitate clear communication between partners.

  • ISO 9000: quality management systems
  • ISO 25010: software product quality requirements and evaluation (SQuaRE)
  • ISO 12207: software development life cycle processes
  • ISO 27001: information security management systems
  • IATF 16949: quality management standard for the automotive industry
  • AS 9100: quality management standard for aerospace, aviation, and defense

ISO 9000

The ISO 9000 is a bundle of international quality management standards with several principles:

  • Customer focus
  • Leadership
  • Engagement of people
  • Process approach
  • Improvement
  • Evidence-based decision making
  • Relationship management

Quality Responsibility#

Quality can be seen as a mindset or attitude. Every individual has an understanding of quality and within an organization a certain responsibility for it. System Test applies quality for a specific project or function.

Quality Costs (PAF Framework)#

  • Prevention costs: setting up QMS, preventive maintenance, quality training, robust design
  • Appraisal costs: inspections, quality audits, performance tests, monitoring
  • Failure costs: internal (before delivery) and external (after delivery) costs related to defects

Effort-Outcome Balance

It is important to find the right degree of quality efforts. Too high quality requirements can slow the design phase. Too many resources in quality effort can make the test phase slow and expensive. Any imbalance also leads to conflicts with development.

Test Pyramid#

The Test Pyramid distributes test activities in three layers:

  • Unit tests: software code testing, fast execution, easy to automate
  • Functional tests: integration of components, moderate number, may include simulations
  • End-to-end tests: full system, customer-like testing, difficult to automate

From bottom to top: number of tests decrease, cost increases, feedback speed decreases, effort increases, automation difficulty increases.

V-Model#

The V-Model (Verification and Validation Model) is one of the most popular models for System Test of embedded complex systems. It has three layers:

  • System layer: system requirements ↔ end-to-end tests
  • Functional layer: system design ↔ functional tests
  • Unit layer: specification design ↔ unit tests

Quality Constructs#

Basic Approaches

  • Kaizen: continuous improvement with small, incremental changes
  • Lean Manufacturing: eliminating waste through continuous improvement
  • Poka-Yoke: method to prevent mistakes ('fail-safing')
  • QFD: translating customer requirements into technical requirements

Fundamental Methods

PDCA Cycle: Plan-Do-Check-Act (Deming)

DMAIC Method: Define-Measure-Analyze-Improve-Control (foundation for Six Sigma)

Advanced Methods

  • Six Sigma: data-driven method to reduce process variation, aiming for zero defects
  • Eight Disciplines (8D): method to resolve recurring problems through systematic analysis
  • Deming's 14 Points: systematic ways for quality management emphasizing continuous improvement

Customer Satisfaction#

Kano Model

  • Excitement attributes: not expected but proportionally lead to satisfaction
  • Performance attributes: higher performance = greater satisfaction
  • Basic needs: must-have attributes
  • Indifferent attributes: lead to neither satisfaction nor dissatisfaction
  • Reverse features: absence of a feature leads to satisfaction