File preview
Systems and Software Engineering Standards for the Medical Domain
Vera Pantelic
McMaster Centre for Software Certification Department of Computing and Software McMaster University
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
1 / 24
Outline
1 2 3
Standards and Concepts of Interest Safety and Risk Safety Integrity Levels (SILs) IEC 61508 IEC 62304 Process vs. Product Requirements Assurance Cases Conclusions
4 5 6
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
2 / 24
Standards and Concepts of Interest
Introduction
We are mainly interested in the standards important for certification of systems containing software in the medical domain. Questions:
How are Safety Integrity Levels (SILs) defined, derived, and applied? Are process and/or product requirements defined? Is there an assurance/safety case behind a standard?
If yes, how? Is the rationale explicit?
Do standards address the concept of assurance cases?
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
3 / 24
Standards and Concepts of Interest
Standards in Medical Domain (IEC 62304, Annex C.1)
Medical device management standards ISO 14971, ISO 13485 requires
Lays out a foundation to develop a medical device
affects
affects
Medical device product standards IEC 60601-1, IEC 61010-1
Gives specific direction for creation of a safe medical device
Medical device process standards IEC 62304
Gives detailed direction how to develop and maintain safe software system
affects
Implementation of medical device software
Other sources of information ISO/IEC 12207, IEC 61508-3, ISO/IEC 90003…
Gives additional guidelines, techniques, etc that may be used
inspires
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
4 / 24
Safety and Risk
Vocabulary from ISO/IEC Guide 51:1999
harm physical injury, damage, or both to the health of people or damage to property or the environment hazard potential source of harm hazardous situation circumstance in which people, property or the environment are exposed to one or more hazards risk combination of the probability of occurrence of harm and the severity of that harm safety freedom from unacceptable risk
Vera Pantelic (McSCert) Standards: Medical Domain May 7th, 2012 5 / 24
Safety and Risk
Estimating Risk
Sequence of Events
Hazard
Legend Safety-related concepts Output occurs only if both inputs occur Causality
Hazardous Situation
Harm
Probability of occurrence of harm is P1 × P2 (often hard to quantify), where P1 is the probability of a hazardous situation occurring, and P2 is the probability of a hazardous situation leading to a harm.
Vera Pantelic (McSCert) Standards: Medical Domain May 7th, 2012 6 / 24
Safety and Risk
We focus on:
IEC 61508 series (with emphasis on IEC 61508-3:2010, Functional safety of electrical/electronic/programmable electronic safety-related systems – Part 3: Software requirements), IEC 62304:2006 Medical device software – Software life cycle processes.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
7 / 24
Safety Integrity Levels (SILs)
IEC 61508
IEC 61508: Basics
It assumes the existence of equipment under control (EUC) and its control system. The risks posed by the EUC and its control system should be mitigated until they reach tolerable targets.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
8 / 24
Safety Integrity Levels (SILs)
IEC 61508
Safety Requirements and SIL
Safety requirements are specified as
1
2
safety function: what should be done by an E/E/PE safety related system (or other risk reduction measures) to keep risks at tolerable level, and safety integrity: probability of an E/E/PE safety-related system satisfactorily performing the specified safety functions
Safety integrity can be specified using a safety integrity level (SIL):
one of four possible levels 1 corresponds to the lowest level of integrity, 4 to the highest.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
9 / 24
Safety Integrity Levels (SILs)
IEC 61508
Application of SILs
SILs drive the lifecycle processes: higher integrity demands higher rigour. For specified lifecycle phases, techniques and measures dependent on a specified SIL are recommended and their effectiveness in satisfying certain properties of artifacts defined as outputs of the lifecycle phases is graded. Furthermore, independence of functional safety assessment personnel depends in part on SIL, etc.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
10 / 24
Safety Integrity Levels (SILs)
IEC 61508
SIL as a Reliability Measure
Safety integrity levels relate to (dangerous) failure rates:
Low demand mode of operation
High demand or continuous mode of operation
For novel systems and software, the target failure rates are not useful and, even worse, potentially misleading.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
11 / 24
Safety Integrity Levels (SILs)
IEC 61508
SIL and Confidence
Systematic capability: measure (expressed on a scale of SC 1 to SC 4) of the confidence that the systematic safety integrity of an element meets the requirements of the specified SIL, in respect of the specified element safety function, when the element is applied in accordance with the instructions specified in the compliant item safety manual for the element [IEC 61508-4, Definition 3.5.9]
... NOTE 3 A Systematic capability of SC N for an element, in respect of the specified element safety function, means that the systematic safety integrity of SIL N has been met when the element is applied in accordance with the instructions specified in the compliant item safety manual for the element.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
12 / 24
Safety Integrity Levels (SILs)
IEC 61508
Software Safety Integrity Levels and Confidence
Software safety integrity level: systematic capability of a software element. “SIL N software” means “software in which confidence is justified (expressed on a scale of 1 to 4) that the (software) element safety function will not fail due to relevant systematic failures.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
13 / 24
Safety Integrity Levels (SILs)
IEC 62304
Safety Integrity Levels: IEC 62304
Only severity of harm is used for the estimation of risks. There are 3 safety classes:
class A (no injury or damage to health is possible), class B (non-serious injury is possible), class C (death or serious injury is possible).
For each requirement of the standard, it is indicated what classes it applies to.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
14 / 24
Process vs. Product Requirements
Process vs. Product Requirements
IEC 62304: Only tasks (processes) are required to be documented. IEC 61508-3:
1 2
3 4
Software artifacts are defined as the outputs of lifecycle phases. Then, techniques and measures are suggested for each artifact dependent on SIL. Target attributes are defined for each artifact. Then, for each technique/measure, the standard estimates the effectiveness with which an attribute is achieved.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
15 / 24
Process vs. Product Requirements
Towards Product Requirements: IEC 61508-3
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
16 / 24
Process vs. Product Requirements
Techniques/Measures for Software Safety Requirements Specification (SSRS): IEC 61508-3
Table: Software safety requirements specification, Table A.1 from IEC 61508-3 (normative)
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
17 / 24
Process vs. Product Requirements
Semi-formal methods for SSRS: IEC 61508-3
Table: Semi-formal methods for SSRS, Table B.7 from IEC 61508-3 (informative)
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
18 / 24
Process vs. Product Requirements
Techniques/Measures and Properties of SSRS: IEC 61508-7
Table: Properties for systematic safety integrity - Software safety requirements specification, Table C.1 in IEC 61508-3 (informative)
Vera Pantelic (McSCert) Standards: Medical Domain May 7th, 2012 19 / 24
Process vs. Product Requirements
Properties of SSRS: IEC 61508-7 (informative)
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
20 / 24
Assurance Cases
Implicit Assurance Cases Behind Standards
IEC 61508-3:
The top claim is the software is acceptably safe. The rationale can be found in the use of well-defined and carefully planned safety lifecycle activities, management of functional safety, independent functional assessment, use of competent staff. Outputs of each phase together with their attributes define required evidence.
IEC 62304:
The top claim is that the medical device software is acceptably safe and effective. The rationale is that a set of well-established processes has been followed. Evidence is found in process documentation.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
21 / 24
Assurance Cases
Assurance Cases in Standards
Safety cases in ISO-TR 80002:
“One could view a safety case as a risk management or residual risk summary with references to more detailed documentation for supporting information and the evidence in the risk management file. It could also include cross references to demonstrate specification and test coverage for all risk control measures.”
ISO 15026:
1 2 3 4
ISO 15026-1:2010 Concepts and vocabulary ISO 15026-2:2011 Assurance case ISO 15026-3:2011 System integrity levels ISO 15026-4 Assurance in the life cycle (Under development)
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
22 / 24
Conclusions
Conclusions
Important concepts are still being vaguely and inadequately defined/used in the standards (SILs, product requirements). The use of assurance cases has not yet been standardized in the medical domain. New standards emerge addressing the increasing complexity and connectivity of devices (e.g., IEC 80001-1:2010).
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
23 / 24
Conclusions
Acknowledgments
Marc Bender, Chris George, Paul Joannou, Zarrin Langari, Mark Lawford, Tom Maibaum, Jackie Wang, Alan Wassyng
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
24 / 24
Vera Pantelic
McMaster Centre for Software Certification Department of Computing and Software McMaster University
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
1 / 24
Outline
1 2 3
Standards and Concepts of Interest Safety and Risk Safety Integrity Levels (SILs) IEC 61508 IEC 62304 Process vs. Product Requirements Assurance Cases Conclusions
4 5 6
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
2 / 24
Standards and Concepts of Interest
Introduction
We are mainly interested in the standards important for certification of systems containing software in the medical domain. Questions:
How are Safety Integrity Levels (SILs) defined, derived, and applied? Are process and/or product requirements defined? Is there an assurance/safety case behind a standard?
If yes, how? Is the rationale explicit?
Do standards address the concept of assurance cases?
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
3 / 24
Standards and Concepts of Interest
Standards in Medical Domain (IEC 62304, Annex C.1)
Medical device management standards ISO 14971, ISO 13485 requires
Lays out a foundation to develop a medical device
affects
affects
Medical device product standards IEC 60601-1, IEC 61010-1
Gives specific direction for creation of a safe medical device
Medical device process standards IEC 62304
Gives detailed direction how to develop and maintain safe software system
affects
Implementation of medical device software
Other sources of information ISO/IEC 12207, IEC 61508-3, ISO/IEC 90003…
Gives additional guidelines, techniques, etc that may be used
inspires
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
4 / 24
Safety and Risk
Vocabulary from ISO/IEC Guide 51:1999
harm physical injury, damage, or both to the health of people or damage to property or the environment hazard potential source of harm hazardous situation circumstance in which people, property or the environment are exposed to one or more hazards risk combination of the probability of occurrence of harm and the severity of that harm safety freedom from unacceptable risk
Vera Pantelic (McSCert) Standards: Medical Domain May 7th, 2012 5 / 24
Safety and Risk
Estimating Risk
Sequence of Events
Hazard
Legend Safety-related concepts Output occurs only if both inputs occur Causality
Hazardous Situation
Harm
Probability of occurrence of harm is P1 × P2 (often hard to quantify), where P1 is the probability of a hazardous situation occurring, and P2 is the probability of a hazardous situation leading to a harm.
Vera Pantelic (McSCert) Standards: Medical Domain May 7th, 2012 6 / 24
Safety and Risk
We focus on:
IEC 61508 series (with emphasis on IEC 61508-3:2010, Functional safety of electrical/electronic/programmable electronic safety-related systems – Part 3: Software requirements), IEC 62304:2006 Medical device software – Software life cycle processes.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
7 / 24
Safety Integrity Levels (SILs)
IEC 61508
IEC 61508: Basics
It assumes the existence of equipment under control (EUC) and its control system. The risks posed by the EUC and its control system should be mitigated until they reach tolerable targets.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
8 / 24
Safety Integrity Levels (SILs)
IEC 61508
Safety Requirements and SIL
Safety requirements are specified as
1
2
safety function: what should be done by an E/E/PE safety related system (or other risk reduction measures) to keep risks at tolerable level, and safety integrity: probability of an E/E/PE safety-related system satisfactorily performing the specified safety functions
Safety integrity can be specified using a safety integrity level (SIL):
one of four possible levels 1 corresponds to the lowest level of integrity, 4 to the highest.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
9 / 24
Safety Integrity Levels (SILs)
IEC 61508
Application of SILs
SILs drive the lifecycle processes: higher integrity demands higher rigour. For specified lifecycle phases, techniques and measures dependent on a specified SIL are recommended and their effectiveness in satisfying certain properties of artifacts defined as outputs of the lifecycle phases is graded. Furthermore, independence of functional safety assessment personnel depends in part on SIL, etc.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
10 / 24
Safety Integrity Levels (SILs)
IEC 61508
SIL as a Reliability Measure
Safety integrity levels relate to (dangerous) failure rates:
Low demand mode of operation
High demand or continuous mode of operation
For novel systems and software, the target failure rates are not useful and, even worse, potentially misleading.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
11 / 24
Safety Integrity Levels (SILs)
IEC 61508
SIL and Confidence
Systematic capability: measure (expressed on a scale of SC 1 to SC 4) of the confidence that the systematic safety integrity of an element meets the requirements of the specified SIL, in respect of the specified element safety function, when the element is applied in accordance with the instructions specified in the compliant item safety manual for the element [IEC 61508-4, Definition 3.5.9]
... NOTE 3 A Systematic capability of SC N for an element, in respect of the specified element safety function, means that the systematic safety integrity of SIL N has been met when the element is applied in accordance with the instructions specified in the compliant item safety manual for the element.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
12 / 24
Safety Integrity Levels (SILs)
IEC 61508
Software Safety Integrity Levels and Confidence
Software safety integrity level: systematic capability of a software element. “SIL N software” means “software in which confidence is justified (expressed on a scale of 1 to 4) that the (software) element safety function will not fail due to relevant systematic failures.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
13 / 24
Safety Integrity Levels (SILs)
IEC 62304
Safety Integrity Levels: IEC 62304
Only severity of harm is used for the estimation of risks. There are 3 safety classes:
class A (no injury or damage to health is possible), class B (non-serious injury is possible), class C (death or serious injury is possible).
For each requirement of the standard, it is indicated what classes it applies to.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
14 / 24
Process vs. Product Requirements
Process vs. Product Requirements
IEC 62304: Only tasks (processes) are required to be documented. IEC 61508-3:
1 2
3 4
Software artifacts are defined as the outputs of lifecycle phases. Then, techniques and measures are suggested for each artifact dependent on SIL. Target attributes are defined for each artifact. Then, for each technique/measure, the standard estimates the effectiveness with which an attribute is achieved.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
15 / 24
Process vs. Product Requirements
Towards Product Requirements: IEC 61508-3
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
16 / 24
Process vs. Product Requirements
Techniques/Measures for Software Safety Requirements Specification (SSRS): IEC 61508-3
Table: Software safety requirements specification, Table A.1 from IEC 61508-3 (normative)
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
17 / 24
Process vs. Product Requirements
Semi-formal methods for SSRS: IEC 61508-3
Table: Semi-formal methods for SSRS, Table B.7 from IEC 61508-3 (informative)
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
18 / 24
Process vs. Product Requirements
Techniques/Measures and Properties of SSRS: IEC 61508-7
Table: Properties for systematic safety integrity - Software safety requirements specification, Table C.1 in IEC 61508-3 (informative)
Vera Pantelic (McSCert) Standards: Medical Domain May 7th, 2012 19 / 24
Process vs. Product Requirements
Properties of SSRS: IEC 61508-7 (informative)
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
20 / 24
Assurance Cases
Implicit Assurance Cases Behind Standards
IEC 61508-3:
The top claim is the software is acceptably safe. The rationale can be found in the use of well-defined and carefully planned safety lifecycle activities, management of functional safety, independent functional assessment, use of competent staff. Outputs of each phase together with their attributes define required evidence.
IEC 62304:
The top claim is that the medical device software is acceptably safe and effective. The rationale is that a set of well-established processes has been followed. Evidence is found in process documentation.
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
21 / 24
Assurance Cases
Assurance Cases in Standards
Safety cases in ISO-TR 80002:
“One could view a safety case as a risk management or residual risk summary with references to more detailed documentation for supporting information and the evidence in the risk management file. It could also include cross references to demonstrate specification and test coverage for all risk control measures.”
ISO 15026:
1 2 3 4
ISO 15026-1:2010 Concepts and vocabulary ISO 15026-2:2011 Assurance case ISO 15026-3:2011 System integrity levels ISO 15026-4 Assurance in the life cycle (Under development)
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
22 / 24
Conclusions
Conclusions
Important concepts are still being vaguely and inadequately defined/used in the standards (SILs, product requirements). The use of assurance cases has not yet been standardized in the medical domain. New standards emerge addressing the increasing complexity and connectivity of devices (e.g., IEC 80001-1:2010).
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
23 / 24
Conclusions
Acknowledgments
Marc Bender, Chris George, Paul Joannou, Zarrin Langari, Mark Lawford, Tom Maibaum, Jackie Wang, Alan Wassyng
Vera Pantelic (McSCert)
Standards: Medical Domain
May 7th, 2012
24 / 24