The Operational Domain Model (ODM) defines the scope of the operation of an AS within which the AS can be demonstrated to be acceptably safe when carrying out its autonomous capability. When AS are developed and put into operation it is based on assumptions made about when, where, and under what conditions that AS will need to operate. The operational scope defined by those assumptions determines the scenarios that may be encountered by the AS as it interacts with its environment. It is illustrated in Figure 6 below how the ODM effectively reduces the number of possible scenarios that the AS may need to deal with when performing its autonomous capability.

medium_defining an operational domain model 2.png

small_defining an operational domain model 2.png

thumbnail_defining an operational domain model 2.png

defining an operational domain model 2.png

The assumptions must be made explicit in the form of a defined ODM that characterises the operating domain. If the ODM is insufficiently defined then the AS may encounter scenarios during its operation that were not considered during the development of the system, and which could therefore be unsafe and for which no assurance is provided. It is crucial therefore that all relevant elements of the operating domain, including those with with the AS may interact are included within the ODM. In identifying relevant elements of the ODM it is important that “non‐mission interactions” are also considered [18].

For autonomous cars, the term Operational Design Domain (ODD) [23] is often used to refer to the ODM. A number of approaches to defining the ODD for an autonomous car have been proposed. NHTSA [45] defined and categorised an ODD taxonomy based on a review of over 50 sources of literature in automotive and other domains. The taxonomy is intended to be descriptive, recognising that other organisations of the elements are possible. The report provides a set of sample baseline ODDs for different automated driving features or capabilities, such as those shown in Figure 7 below for an Automated Highway Drive (HWD) function. 

Note that the examples in Figure 7 are used to define what is asserted as being within the ODM for that particular capability (function). If the operation of the vehicle is outside of this definition then the safety of that capability is no longer assured. We return to the issue of operation outside of the defined operating context at Stage 2.

Autonomous cars

medium_table1.png

small_table1.png

thumbnail_table1.png

table1.png

One of the major challenges when defining an ODM can be identifying an appropriate level of detail at which to specify elements of the ODM. If elements are not specified with sufficient granularity then important differences in the features that may impact safety may be missed. On the other hand, too much granularity can make any analysis based on the ODM intractable due to the high number of element combinations.

It may be necessary to revisit the granularity of the ODM as a result of information learned from other activities; for example, it may be informed by the nature of the relevant operating scenarios identified at Activity 3.

Challenges when defining an ODM

In the ODM for an autonomous robot operating in an office environment, the ODM may include ‘walls’ or ‘doors’ as part of the model. However, relying only on ‘wall’ or ‘door’ as an element of an ODM will miss crucial attributes of walls or doors that will affect the detection capability of an AS. A wall or door made of translucent material would not be capable of detection by some technologies (e.g. LiDAR). In the ODM, the material of the walls and doors should also be specified.

Office environment

In the ODM for an autonomous shuttle bus operating in pedestrianised areas, the ODM must include people as part of the model. However, defining “people” as a feature of the ODM may miss crucial differences between children and adults that affect the safety of the AS operation. “Children” and “adults” should therefore be defined as separate elements of that particular ODM. The appropriate elements to choose and their granularity would be expected to differ for each ODM depending upon the particular operational context of the system.

Defining people

Once created the ODM shall be validated to check that both the scope of the defined model and its level of detail are appropriate. The results of the validation activity shall be explicitly documented ([C]).

The process of defining and validating the ODM is an iterative process, and any initial ODM specification may need to be expanded or reduced based on the outputs of later process activities. This may include new information that is obtained about the design, operation, capabilities, limitations and/or potential failure conditions of the AS.

Define and validate the Operational Domain Model (ODM)

This activity requires as input the SOC assurance argument pattern ([N]), as well as the artefacts from Activities 10 and 11 ([L]and [M]). The activity uses these artefacts to create an instantiated AS operating context assurance argument ([O]) which demonstrates that the defined SOC sufficiently mitigates the hazardous scenarios identified for the AS operation.

SOC argument pattern

Instantiate SOC assurance argument pattern

Justification shall be provided for how each of the key design decisions that have been made at tier n ([V]) help to ensure that the safety requirements can be met by the AS. In particular this should consider how the design decisions relate to the robustness and fault tolerance of the AS, and the way in which this supports the safety of the system. It is also important that the design decisions are reviewed to check that no inappropriate decisions are taken that mean that the safety requirements cannot be satisfied by the proposed design. The justification for the design at each tier shall be documented in the AS design justification report ([Y]).

Even if appropriate design decisions have been taken, the design should also be reviewed to check that errors have not been made in the design. It is particularly important for AS, where substantial use is made of software components, that this includes review of the proposed software design. In particular, the review should check for potentially hazardous errors. The review of the design shall 
 be undertaken by a suitable person who is independent from the design activity itself. The results of the design review at each tier shall be documented in the AS design review report ([Z]).

The sufficiency of the design process used (as documented in [X]) shall also be reviewed to check it is sufficiently rigorous. It should also be checked that the defined process has been correctly followed when developing the design.

The level of independence that is required of the people responsible for the review may vary depending upon the level of risk associated with the AS. For high risk systems it may be required for the review to be undertaken by personnel from a different organisation to the design organisation. For lower risk systems it may be sufficient for the review to be undertaken by personnel from the same organisation who are capable of providing an independent view.

Independent reviewers

Review and justify design at tier n

This activity requires as input the AS operating context assurance argument pattern ([G]), as well as the artefacts from Activity 1, Activity 2 and Activity 3 ([B], [C], [D], [E]and [F]). The activity uses these artefacts to create an instantiated AS operating context assurance argument ([H]) which demonstrates that the defined ODM is sufficient to support the safe operation of the AS.

AS operating context assurance argument pattern

Instantiate AS operating context assurance argument pattern

This activity shall consider the transitions of the AS from autonomous to non‐autonomous operation when it crosses the ODM boundary. This seeks to identify ways in which those transitions could be unsafe such that suitable mitigations can be provided. This requires as input an ODM transition model ([JJ]) for the AS. Based on the transition model, the ways in which the possible AS transitions may be unsafe shall be determined. This will allow measures to be defined to minimise the risk of the unsafe transitions for the AS operation. The table in Figure 27 in the ODM transition model ([JJ]) provides examples of the nature of typical unsafe transitions. The transitions in the table refer to those from the ODM transition model ([JJ]) shown in Figure 28 in ([JJ]).

An autonomous ship can self‐navigate without intervention as long as it has sufficient channel width margins. If it progresses down a seaway of varying width, the autonomous navigation
function may need to engage and disengage, repeatedly handing control to the ship’s crew. In this scenario there are a number of different unsafe transitions that could occur. This includes
remaining in autonomous control when the channel width becomes too narrow (which may be beyond the safe capability of the autonomy), as well as relinquishing control when in a wide
channel (when the crew may not be in a position to safely take control). In certain situations the ship may attempt to mitigate such deviations by getting back inside the defined ODM, for example by taking a route that involves wider channels.

The results of assessment of the transitions shall be documented along with any assumptions made ([LL]).

ODM transition model

Assure transitions in and out of ODM

In defining the context of operation of an AS it is important to first define the scope of the system’s autonomous capabilities. It should be noted that this will in many cases be a sub‐set of the full capability of a system where only some of the functionality is provided autonomously, or the system is only capable of performing tasks autonomously some of the time.

Autonomous capabilities of an AS may be provided entirely by the system itself or with shared responsibility between a human, or humans, and the AS. This includes systems where a human is required to monitor the autonomous operation of the AS and intervene if required. Again the limitations on such capabilities should be clearly defined along with the separation of responsibilities.

Defining capabilities

In defining the autonomous capability of an autonomous robot deployed for delivering small packages around an office environment it is specified that the robot will not be capable of autonomously loading/unloading the packages. A human operator is relied on for the safe loading of packages.

Delivery robot

The autonomous capabilities of a self‐driving car vary for different levels of driving automation. For example an SAE level 3 vehicle [[40]]() may provide autonomous traffic jam pilot capabilities under defined conditions, whereas a level 4 vehicle may provide fully autonomous driving capability under defined conditions.

Self‐driving car

The definition of autonomous capabilities of the AS will often be an iterative process with activity 2 since the capabilities must be chosen in order to match the requirements of a given ODM.

Iterative process

AS concept definition

Define autonomous capabilities of AS

An operating scenario of an AS represents a particular set of actions and events that the AS may undertake in a defined environment situation. We adopt the characterisation used in [10] as a description of the AS, its activities and/or goals, its static environment, and its dynamic environment.

The following distinction is often made between “scenes” and “scenarios” [46]:
- A scene describes a snapshot of the environment including the scenery and dynamic elements, as well as all actors’ and observers’ self‐representations, and the relationships among those entities. Scene descriptions will inevitably be incomplete and vary from one or several observers’ points of view.
- A scenario describes the temporal development between several scenes in a sequence. Every scenario starts with an initial scene, and the temporal development is characterised by a set of actions and events.

Scenes and scenarios

An example of a scene could be a typical office environment, with office furniture, artificial lighting, and an autonomous robot in a corridor that is also used by human staff members. A scenario could develop where an autonomous robot and a human are approaching a right‐angle in a corridor from opposite directions.

Figures 8 and 9, taken from [10] provide an example of a scenario description for an autonomous vehicle.

Autonomous vehicle scenario

thumbnail_scehmatic overview of the scenario.png

scehmatic overview of the scenario.png

medium_a qualitative description of the same scenario.png

small_a qualitative description of the same scenario.png

thumbnail_a qualitative description of the same scenario.png

a qualitative description of the same scenario.png

It is crucial for safety assurance of the AS that the operating scenarios that are relevant to the AS within its defined ODM are identified as completely and correctly as possible. If we refer to [Figure 6](/sace/activities/define-and-validate-the-operational-domain-model-odm#page), we must identify the operating scenarios present in the area inside the orange boundary defining the scope of the ODM. The green area in the diagram represents all of the identified scenarios within this area. It is not possible for a complex, open environment, such as an urban street, to provide an exhaustive detailed specification of the operating scenarios, since the set of scenarios is simply too large.

Where operating scenarios exist within the scope of the ODM, but are not correctly identified, these could pose a safety risk to the AS since the scenario could be hazardous, but would not be assessed as part of the safety assurance process and therefore no mitigations put in place. This is represented by the red area in Figure 6. It is very important therefore for safety assurance to have confidence that this area is as small as possible by ensuring the operating scenarios are sufficiently well defined.

It is partly as a result of the possible existence of the red region in Figure 6 that resilience is such an important requirement when designing ASs; these unanticipated operating scenarios require that the system is able to adapt in a safe manner to unplanned events. This is discussed in more detail at Stage 6.

Resilience

The operating scenario descriptions shall be validated to provide evidence that they are sufficiently complete and correct. Evidence can be provided based upon review of the scenario specification. A rigorous specification to a defined format makes the scenarios more amenable to review and reduces ambiguity. It is important that review is carried out by a range of experienced stakeholders. Providing simulations of the defined scenarios may help to ensure the reviewers have a correct understanding of the scenarios. The defined scenarios can also be checked against collected field data from AS in operation to ensure that all encountered scenarios are captured in the operating scenario specification.
The results of the data validation activity shall be explicitly documented ([F]).

The definition of scenarios should be an iterative process, where the scenarios are refined based on increased understanding of the relevant and important aspects of the scenario space. This refinement may be informed by the outcome of the validation activities described above, particularly the results of simulation and field‐based validation tests.

Define and validate the operating scenarios

Having modelled the steps and interactions of the operating scenario, each of the identified decision points can then be analysed in order to determine the nature of any hazardous scenarios that may arise.

Each of the decision points identified in Activity 3 should be selected. For each, the possible environmental states that may arise within or emanate from the operating environment are identified before enumerating the decisions that the AS could select at that decision point.

A decision point for an autonomous robot moving in a building may correspond to the detection of an object in the robot’s path. The options available to the AS at this point are:
1. Continue on the current path at the current speed, OR
2. Continue on the current path at a reduced speed, OR
3. Change path to avoid the object, OR
4. Stop and wait.

Decision point for an autonomous robot 

The different possible scenarios and environmental states that relate to the decision can then be identified by considering the real world state and the belief state of the AS at the point at which the decision is made, along with each of the possible options. The real world state represents the actual state of the operating environment, whilst the belief state represents the understanding that the AS has of the state of the operating environment.

An example of some enumerated situations for an autonomous robot encountering an object in its path is shown in Table 1. The real world and belief states are represented as Boolean states where ‘True’ represents the presence of the object and ‘False’ represents that the object is not present. So situation 5 in Table 1 represents the situation where there is an object in the path of the robot, but the robot is unaware of the object and continues on its current path at its current speed.

Enumerated situations for an autonomous robot

For each of the identified scenarios, the outcome can be defined in order to identify those that may be potentially hazardous.

small_table2.png

thumbnail_table2.png

table2.png

For an autonomous car turning right at a roundabout, one decision point is the car entering the roundabout. The options for the car at this decision point are:
1. Enter the roundabout
2. Stop and wait.

A possible interaction at this point is with another road user, in this example a cyclist. Relevant scenarios can be identified by considering the real-world presence of the cyclist in combination with the car’s belief that a cyclist is present, and each of the options identified above. For example, one situation is that a cyclist is not present, but the car believes that there is a cyclist and decides to stop and wait. This scenario could lead to a hazardous outcome as an unnecessary and unexpected stop could potentially cause a rear‐end collision.

Autonomous car turning at a roundabout

For an autonomous insulin pump that is monitoring a patient’s blood sugar level, one decision point is whether to alter the infusion rate to the patient. The options for the pump at this decision point are:

- Increase the insulin infusion
- Decrease the insulin infusion
- Maintain the current rate of insulin infusion
- Stop the insulin pump.

Relevant scenarios can be identified by considering the real-world change in the patient’s blood sugar level in combination with the pump’s belief in the current sugar level, and each of the options identified above. For example, one scenario is that the patient’s blood sugar level rises, the pump also has a belief state that the blood sugar level has risen and yet decides to maintain the current rate of insulin infusion. This scenario could lead to a hazardous outcome as the patient's blood sugar level could increase to unsafe levels.

Autonomous insulin pump

Considering the possibly hazardous scenarios for an autonomous robot encountering an object in its path in Table 1, it is possible to also assign a severity of outcome, and consider whether minor or serious injuries (or even fatalities) are likely.

It is also possible to further consider how this severity could be impacted by factors pertaining to the
operating enviornment.

An autonomous robot moving in a building that fails to detect an object in its path, and therefore continues on the current path at the current speed, could impact a static object. If the static object is an adult human, the severity of the impact could be minor (bruise or laceration). Should the static object be a small child, then the severity of the impact could be major (broken bones). The robot may have detected the static object, but decided to maintain its current path, but reduce speed. Wet floor surfaces may reduce the effectiveness in braking (through wheel slippage), and this higher then expected collision speed could impact the severity of the collision.

Autonomous robot 

For an autonomous car turning right at a roundabout, the autonomous car may fail to detect a dynamic object that would intersect it should it enter the roundabout. If this dynamic object is a car, and was collided into by the autonomous vehicle, the occupants may suffer only minor injuries (if any at all). Should the dynamic object be a cyclist, then the collision may result in a fatality. Should the road surface be icy, the force of impact may displace an impacted car onto a fixed object in the environment, thereby increasing the severity of the collision for the car’s occupants.

Autonomous car 

For an autonomous insulin pump that is monitoring a patient’s blood sugar level, the autonomous infusion pump could decrease the insulin infusion without medical need, should the patient be experiencing a spike in blood sugar levels at the time the infusion was reduced, then the patient may become hyperglycaemic. Should the patient have additional health complications, or the reduction in insulin infusion go unobserved by a clinician, then the severity could be impacted and result in a fatality.

The analysis undertaken in this activity shall be documented ([WW]).

Analyse AS decisions

The proposed design of the AS at each tier shall be analysed to identify the potential hazardous failures that could arise as a result of that design. Although an AS has been developed to satisfy the identified safety requirements, it still may be the case that the AS may be capable of doing something else, under certain conditions, that may be hazardous. Such hazardous failures can often not be identified until detail of the design solution, and in particular the characteristics of the system components is understood.

Potentially hazardous failures are identified by considering possible deviations from intended behaviour that may arise for the AS. Analysis shall be undertaken based upon the information 
regarding the design at the current tier under consideration ([VW]). The analysis shall consider information about the particular components proposed as part of that design. In particular the known limitations of the components, their known failure modes, and the conditions under which the components may fail, or under which their performance may deteriorate, shall be considered in the analysis.

The cameras chosen to be used on an autonomous passenger shuttle to determine its distance from the footpaths adjacent to the road are found to not function well in conditions that present sunlight that is both of high intensity, and at an acute angle to the road. This is a potentially hazardous failure since it may mean that the shuttle drives too close to the footpath.

Autonomous passenger shuttle 

Analysing the AS design for possible deviations must be undertaken predictively at higher levels of abstraction where specific component solutions have not been chosen. For this analysis, a technique such as HAZOP [17] that uses a set of defined guidewords applied to elements of the design in order to prompt the identification of possible deviations, can be applied. For more detailed levels of design, once the properties of the actual components used are known a more specific deviation analysis techique such as failure modes and effects analysis (FMEA) [6] can be used, perhaps in combination with Fault Tree Analysis [26] in order to understand the causal path. Safety analysis such as this forms part of traditional safety engineering processes. For AS it is important that deviations are specifically identified relating to the autonomy of the system (particularly understanding and decision making deviations).

Although a fairly traditional HAZOP analysis may be applicable to AS, particular attention must be paid to subtle deviations when interpreting the guidewords. For example, when analysing a proposed perception component, careful consideration must be given to how the following guidewords may be defined and interpreted:

- More (more than one object detected when only one is present)
- Less (less objects are detected than are actually present)
- As well as (an extra area is classified as navigable as well as the intended route)
- Part of (only part of an object is detected)
- Other than (an object is classified other than what it is)

It may also be the case that additional guidewords are required when considering AS such as:
- Intermittent (considers intermittent detection and/or classification)
- Erroneous but credible (elicits failure conditions relating to information that is incorrect
but ‘believable’)

HAZOP analysis

Having identified the possible deviations, the hazardous failures are determined by considering which of the deviations, if they occurred in the AS, could result in a hazardous outcome.

The results of the analyses, including the identified hazardous failures shall be documented in the safety analysis report ([BB]). The safety analysis report shall also provide a justification for the sufficiency of the analysis approach used, including the suitability of the approach for the particular tier under consideration and the appropriateness of any modifications made to existing 
techniques to consider autonomy related deviations.

Identify potential AS hazardous failures at tier n

This activity requires as input the AS hazardous scenarios assurance argument pattern ([I]), as well as the artefacts from Activities 5 to 8 ([WW], [XX], [YY]). The activity uses these artefacts to create an instantiated AS hazardous scenarios assurance argument ([J]) which demonstrates that the hazardous scenarios have been correctly identified.

AS hazardous scenarios assurance case pattern

Instantiate AS hazardous scenarios assurance case pattern

Based upon the analysis performed in Activity 6 it is possible to define the hazardous scenarios associated with the operation of the AS in the defined operating context. These may be specified using the general form presented earlier:

<AS operating scenario> <relevant environment state(s)> AND <decision>

- An AS hazardous scenario for an autonomous robot is: <The autonomous robot is following a planned path>< with a static object present in the path>  **AND**  < the robot maintains its current speed and direction>

- An AS hazardous scenario for an autonomous car is: <the car is approaching a roundabout><with a cyclist on the roundabout to the vehicle’s right>  **AND**  <the car enters the roundabout>

- An AS hazardous scenario for an autonomous insulin infusion pump is: <the insulin pump is monitoring the patient’s blood sugar level >< when the sugar level rises> **AND** < the pump maintains the current insulin infusion rate>

Hazardous scenarios

The system environmental belief state is NOT an element of what constitutes a hazardous scenario, as belief states are merely hypothesised causes of a failure.

System environmental belief state 

These AS hazardous scenarios shall be explicitly documented ([XX])

Identify AS hazardous scenarios

Safe operation of an AS requires that the ODM boundary is correctly recognised. If the AS is unaware that its operation has moved outside of the ODM as defined in Activity 2 then its safety may not be assured.

A car is only capable of operating autonomously on a motorway, but during operation there are lane closures and move to a minor road running due to an accident. In this case the car must recognise that this situation represents an ODM boundary, as minor roads are not included in the ODM.

Autonomous car leaving a motorway

The approach that the AS will use during operation to determine and interpret the ODM boundary shall be determined based upon a consideration of the capability of the AS to sense and understand the ODM. Since perfect recognition of the ODM boundary will not be possible, it will always be necessary to make approximations and assumptions to reflect the AS sensing capabilities. In some cases it may not be possible to directly detect the ODM features using the sensors available to the AS. In such cases it may be that “proxy” measurements are required to be used to recognise the ODM boundary.

The ODM for an autonomous car specifies a maximum permitted intensity of rainfall. The car is fitted with a rainfall sensor as part of the automatic windscreen wiper system. The vehicle makes use of this sensor for recognising the ODM boundary. As such the rapid wiper threshold is used as a proxy for the maximum rainfall intensity permitted by the ODM. Since the rapid wiper threshold is less than the intensity defined for the ODM this is determined to be acceptable to use for ODM recognition.

Car's rainfall sensor

Recognition of the ODM boundary is often challenging for a number of reasons including:

- Many parameters may interact to form the ODM boundary, such as weather conditions, speed etc. For instance, an autonomous vehicle may only be able to progress through fog when visibility is at least 10 metres, during daytime, when speed is less than 60mph.
- There may be a complex boundary shape/envelope/volume with ‘holes’ or difficult geometry. For example a medical image recognition system which can be used for detection of tumours in radiological images only for patient age ranges 40‐45 and 65‐85 (due to the extent of the available training data).
- The AS itself may have to use an interpretation of the boundary which may be a different, or simplified approximation of the actual ODM boundary, to give appropriate margins. For example instead of a more complex boundary condition where a drone is able to fly in wind speeds less than 15 km/h and gusts of up to 20 km/h, the interpretation of the boundary the drone uses is wind speeds less than 12 km/h.
- It may take some time for the processing and analysis of the sensor data to establish whether the AS is near or has crossed the ODM boundary.
- The sensor data used to determine the ODM boundary may be approximate, noisy or subject to infrequent updates. All sensors have an accuracy, resolution and a reading lag time; some also have a polling interval. Sensors may age and deteriorate and readings drift or become subject to bias or noise over time. Individual sensor data may be subject to errors or variations and may have to be averaged with others or over time. This is particularly a problem in harsh environments such as marine, automotive or aviation. In this case the sensing of the ODM may be delayed, or incorrect for some time. Margins therefore have to be implemented for operation, i.e. working to a smaller ODM, so that in all reasonable scenarios the actual ODM boundary can be sensed.
- ‘Flip‐flopping’ between boundary states may occur, i.e rapid switching between in and out detections. The maximum rate at which changes with respect to the ODM boundary are detected by the AS must be considered. It may be better to indicate the AS as outside of ODM for a minimum period of time if there are likely to be many boundary detection events in a short period of time. Hysteresis biased towards early recognition, i.e. conservative detection, may be needed.

Challenges to recognising the ODM boundary

The approach for determining the ODM boundary during operation shall be documented ([HH]) together with any assumptions and approximations made. It shall be demonstrated that the AS is able to recognise the ODM boundary as interpreted during operation. This will involve consideration of at least four recognition cases:
1. AS is approaching the ODM boundary from within the ODM.
2. AS is crossing the ODM boundary.
3. AS is approaching the ODM boundary from outside the ODM.
4. AS is re‐entering the ODM from outside.

For each of the four cases above, it shall be determined how these may be unsafe through consideration of:
1. Timeliness ‐ ODM boundary recognised too early or too late.
2. Accuracy ‐ false positive recognition (ODM boundary recognised mistakenly) or false negative recognition (ODM boundary not recognised when it should be).
3. Hysteresis ‐ AS holds on to a ODM boundary recognition state for too short or too long a period.

For any of the cases that are determined to be potentially hazardous it shall be demonstrated that those cases are sufficiently mitigated by the AS. There are a number of approaches for this including testing of scenarios relating to each of these cases. Testing alone however may not be able to provide sufficient evidence where the ODM boundary is complex (as testing can only sample the boundary space). Simulations and analysis of the ODM boundary recognition may therefore also 
be needed (see Stage 8 for further guidance on this).

The results of the ODM boundary recognition assessment shall be documented ([II]).

Assure the Recognition of the ODM Boundary

This activity requires as input the design assurance argument pattern ([U]), as well as the artefacts from the previous activities of this stage ([V], [W], [X], [Y]and [Z]). The activity uses these artefacts to create an instantiated design assurance argument for the AS ([AA]) which demonstrates that all tiers of the AS design are sufficient to ensure that the defined safety requirements can be met.

AS design assurance argument pattern

Instantiate design assurance argument pattern

Correctly understanding the interactions that the AS has with elements of the operating environment is a crucial part of identifying the hazardous scenarios. This activity focuses on defining those interactions.

A complex operating environment can have a large impact on any hazardous behaviour of an AS, since it increases the probability of the system encountering unusual and unanticipated scenarios. Although complex environments are not unique to AS, traditional systems often rely on the involvement of a human as an operator of the system to deal with any resulting unanticipated or unusual scenarios. For autonomous operation, a human operator is not available to deal with these scenarios, so the AS itself must be able to deal safely with such situations so that they do not become hazardous (through inappropriate decision making). This requires therefore that analysis of hazardous scenarios for an AS incorporates consideration of the operating environment in a more systematic and explicit manner than is currently the case for human‐controlled systems.



medium_SUDA agent model of an AS.png

small_SUDA agent model of an AS.png

thumbnail_SUDA agent model of an AS.png

SUDA agent model of an AS.png

The ODM ([B]) is a key input used to identify the interactions that the AS may encounter during operation.

The consideration of interactions is not limited solely to the interactions between an AS and other agents explicitly required for the purpose of carrying out a task (sometimes referred to as ‘mission interactions’). Consideration must also be given to unexpected interactions with features such as the type of terrain encountered, as well as with unexpected agents that are not ordinarily involved in the task (‘non‐mission interactions’) [18].

Mission and non-mission interactions

- The ODM for an autonomous robot operating in an office environment may identify possible interactions with office workers, visitors, other robots etc.
- The ODM for an autonomous car may identify possible interactions with different types of road surfaces and weather conditions etc.
- The ODM for an autonomous insulin infusion pump may identify possible interactions with different healthcare professionals (e.g. doctors, nurses), as well as the patient, other medical devices etc.

ODM and identified interactions

Having identified the elements of the ODM with which the AS may potentially interact, the defined operating scenarios ([E]) can then be used to identify the particular interactions that may occur with those elements. This requires each scenario to be broken down into a set of steps that are undertaken by the AS as described below:

1. **Define start and end points of the scenario**: identify the logical start/end points (the former may often be the event that triggers the scenario, and the latter asserted as the point at which the scenario is successfully completed).
2. **Define the steps**: establish the steps involved in undertaking the scenario. Some of these steps may represent an ‘understanding point’ in the scenario where the AS requires information about a particular feature of the operating environment. These understanding points should be captured as questions with a binary (yes/no) output, such as “is an obstacle present in the planned path of the robot”. Other steps may represent a ‘decision point’ where the AS must make a choice, such as to reduce speed.
3. **Identify the interactions**: Determine which of the defined steps may involve interaction with elements of the operating environment defined in the ODM.

Define interactions between the AS and the environment

The safe operating concept (SOC) specifies sufficiently safe operation of the AS within the operating context defined by [B], [D] and [E] taking account of the identified hazardous scenarios from [XX] at Stage 2. The SOC shall include a definition of system-level safety requirements that specify how the AS must behave in order to provide sufficient mitigation for the hazardous events. The system safety requirements may be specified for the AS using either natural language or more formalised representations. Whatever representation is chosen, of most importance is that the requirements are clear and unambiguous.

There is existing good practice for requirements specification that can be used to guide the specification of system safety requirements as part of the SOC, such as [32] which provides a generic requirements syntax:

**\<optional preconditions> \<optional trigger> the \<system name> shall \<system response>**

In addition to five templates for different requirement types:

**Ubiquitous requirement**
These are requirements that always hold and therefore have no preconditions or trigger (sometimes referred to as invariants). These requirements should have the form:
**The \<system name> shall \<system response>**

**Event‐driven requirement**
These are requirements that are initiated when, and only when, a triggering event (such as a change in the operating environment) is detected at the system boundary. These requirements should have the form:
**When \<optional preconditions> \<trigger> the \<system name> shall \<system response>**

**Requirements to handle unwanted behaviour**
These requirements are a variant of the event‐driven requirements that specifically deal with unwanted events (such as failures). These requirements should have the form:
**If \<optional preconditions> \<trigger>, then the \<system name> shall \<system response>**

**State‐driven requirement**
These are requirements that are valid while the system is in a defined state. These requirements should have the form:
**While \<in a specific state> the \<system name> shall \<system response>**

**Optional feature requirement** These are requirements that are valid only in systems that include a given feature or capability. These requirements should have the form:
**Where \<feature is included> the \<system name> shall \<system response>**

**Requirements with complex conditions**
Combinations of keywords When, While and Where can be use to build complex expressions to specify more complex As behaviours. For example:
**While the robot is moving, when a person is present, the robot shall issue an audible warning**

A structured syntax approach such as that described above provides a method for mitigating against syntactic ambiguity.

Mitigating against semantic ambiguity requires ways to ensure that the meaning of words and phrases is clearly defined within the relevant context and that only that meaning is used. This can be achieved through the use of domain or project specific dictionaries or through developing context‐specific ontologies. These should include definitions of concepts and elements included in the ODM.

Requirements syntax

The SOC for an autonomous car includes the following safety requirement: “The AV shall maintain sufficient distance between itself and any vehicle in front in order to provide enough time to react if the car in front suddenly brakes.” This requirement has been specified in order to mitigate the hazardous event of the autonomous car colliding into the rear of another vehicle. For such a requirement, the distance can be calculated for different driving speeds based upon a number of assumed properties such as the minimum and maximum braking performances of the cars and the frictional coefficient of the road surface (see [42] for details of such calculations). These assumptions must be validated and explicitly documented as part of the assurance case.

Autonomous car braking requirement

The SOC for an autonomous excavator operating on a construction site includes the following safety requirement: “The excavator shall ensure maximum tilting angle is never exceeded.” This requirement has been specified in order to mitigate the hazardous event of the excavator toppling whilst digging [41]. This could occur due to an incorrect configuration of the excavator whilst digging on uneven or unstable ground or as a result of resistance to the bucket.

SOC for autonomous car on construction site

The SOC for an autonomous insulin infusion pump used in a busy intensive care unit treating multiple patients concurrently may contain the following safety requirement: “The alarms and warnings function of the autonomous insulin infusion pump shall not unnecessarily distract or disturb ICU nurses from their other tasks”.

Autonomous insulin infusion pump

The safety requirements defined as part of the SOC should be defined with reference to the system‐level behaviour of the AS, that is behaviour that is visible on the boundary of the AS, rather than referring to any particular sub‐system or component. For instance, the safety requirement relating to rear collision in Example 17 should not be written as a requirement on the braking system, but, as is shown, on required behaviour of the vehicle as a whole. This requirement and its assumptions may then lead to safety requirements being derived upon the braking system through a consideration of the system design later in the lifecycle (see Stage 5).

System-level safety requirements

In addition to specifying system safety requirements, as part of the SOC it may also be necessary to define a reduced operating domain (ROD). As shown in Figure 15, a ROD is a definition of additional constraints on the ODM in order to reduce the operational scope of an autonomous capability under certain conditions. By operating in a ROD, the intention is that the AS is not exposed to scenarios that may lead to hazardous events under those defined conditions.

There are a number of reasons why a ROD may need to be defined. Often a ROD will be defined with respect to a particular system state where it would be unsafe to operate within the entire scope of the ODM. This may often be in response to a particular system or component failure mode.

small_defining a reduced operating domain 2.png

thumbnail_defining a reduced operating domain 2.png

defining a reduced operating domain 2.png

If there is a failure of the long range object detection sensors on an autonomous passenger shuttle, the vehicle enters a reduced operating domain that requires a low maximum operating speed for the vehicle. This enables the vehicle to transport the passengers to a safe location, but to do so without increasing the risk of collision whilst relying only on short range sensing capability.

Autonomous passenger shuttle

If there is a data communication failure between an autonomous insulin infusion pump, and the centralised electronic patient healthcare records system, the autonomous pump may be prevented from increasing or decreasing the rates of insulin infusion until either communications are restored, or action is taken by a healthcare professional.

Although RODs may be identified early in the lifecycle, it is likely that RODs will often be formulated or revised later in the lifecycle in response to understanding gained through later activities such as hazardous failures identified in [BB]at Stage 6.

In order to define an SOC that provides sufficient mitigation for the hazardous scenarios under the specified conditions (including failures modes), there is generally a choice as to the safety strategy that is adopted:

1. The AS continues to provide all of the Autonomous Capability (AC) under those specified conditions but a ROD is defined that limits the scope of operation under which that AC is provided.
2. The AS continues to operate under the same ODM but the AC is limited.
3. A combination of 1) and 2) is adopted.

Sufficient mitigation for hazardous scenarios

An autonomous car driving on a highway enters a zone with emergency roadworks in place. There are different safety strategies that could be adopted under these conditions. The car could continue to operate fully autonomously, but the maximum allowable speed could be reduced (Strategy 1). The car could continue to operate up to the normal speed but only provide lane-keeping capability, handing other control to the driver (Strategy 2). Or the car could have both its speed constrained as well as restricting its capability to lane keeping only (Strategy 3).

Autonomous car in emergency roadworks

Definition of sufficiently safe

Define safe operating concept of AS

Mitigations shall be defined for all the identified potential hazardous AS failures ([BB]). The mitigations could take various forms such as:

- defining required design changes
- limitations to the operating concept
- deriving additional safety requirements

Where changes are made to the AS design in order to mitigate identified hazardous failures then Activity 19 shall be repeated to ensure additional hazardous failures have not been introduced by those changes. The suitability of the design changes shall be justified as part of the design justification report ([Y]).

Limitations on the operating concept may include changes to the reduced operating domain (ROD) for the AS to provide additional constraints. The changes to the ROD shall be reflected in the safe operating concept (SOC) definition ([L]).

Any additional safety requirements that are derived shall be added to the existing safety requirements definition ([Q]) for implementation. For some of the identified potential hazardous failures it may be determined that the existing design is already sufficient to mitigate those failures (such as through redundancy in the architecture). Where this is the case, this justification shall be 
documented as part of the design justification report ([Y]).

For an autonomous robot operating in an office building, a potential hazardous failure identified from analysis of an object detection component is it may under certain conditions fail to detect walls made of translucent material. In mitigation to this, a design change is proposed to add an additional sensor of a different type.

Sensors for autonomous robot

Define mitigations for identified hazardous failures

An appropriate verification strategy shall be adopted. At the highest level this will require a decision on whether verification shall be undertaken using testing, formal verification or a combination of both approaches. The advantages and disadvantages of using either approach for the verification of AS are discussed below.

**Testing**
Testing involves exposing the system to a defined set of inputs and checking that the outputs or behaviour of the system is as specified and safe. The main advantage of using testing is that evidence may be generated against the real system operating in the target environment. This means that demonstrating the relevance of the verification evidence to the AS operation is comparatively straightforward.

Some testing of AS may however be performed in a controlled environment, such as a test site or a laboratory setting, requiring that the similarity to the actual operating environment is demonstrated. The main disadvantage is that testing cannot be carried out exhaustively, so achieving an acceptable level of coverage of the operating domain can be very challenging, particularly for AS in complex operating environments that represent a very large state space. Providing justification for the acceptability of the level of coverage that is achieved is also a major challenge.

**Simulation**
Testing does not always need to be performed using the target AS in its intended operating environment. Particularly for AS, simulation may also be used for the purposes of testing. Simulation may be used for a number of reasons:
- it can allow verification activities to begin earlier in the development lifecycle, since a fully developed system is not required
- it can also enable tests to be carried out that would otherwise be difficult to create (due to their rarity in the real world) or dangerous to perform (since they would require exposing people to unnecessary risk)
- it can also be used as a way of increasing the coverage that can be achieved through testing, by enabling test cases to be created quickly and cheaply.

The disadvantage of using simulation is that all simulations are models of the real world and justifying the representativeness of the simulation models can be challenging. This is discussed further in Activity 29.

“In‐the‐loop” simulation is an approach that can be used as part of the verification of an AS in which real system components are used to provide inputs to the simulation. Using an in‐the‐loop approach can help to ensure that the data used by the simulation during verification is representative of the data to which the AS will be exposed during operation.

There are many examples that illustrate the use of in‐the‐loop simulations for AS in many different applications, such as autonomous driving ([11]) and unmanned aerial vehicles ([27] and [34]).

In‐the‐loop simulation

**Formal verification**
Formal verification involves using mathematical techniques to prove that a formally specified model of (some aspect of) the AS will always satisfy a set of formally specified properties. The main advantage of using formal verification techniques is that the properties are proved to be true for the entire scope of the model of the AS, meaning that the challenge of demonstrating sufficient coverage that existed for testing is avoided. The disadvantage of using formal analysis is being able to demonstrate that the analysis results are a true reflection of the real system in operation. Firstly, in order to support verification, the formally specified properties that are defined must sufficiently capture the intent of the safety requirements being verified. Secondly, the formal model, upon which the properties are proven to hold, must be sufficiently representative of the system itself, and any elements of the operating environment included in the model. All models require abstractions and assumptions to be made. For complicated AS operating in complex environments demonstrating the acceptability of the abstractions and assumptions made is challenging. Due to these challenges, formal methods should be used with caution for AS, and should be done so in combination with testing. An overview providing further details of different applications of formal methods to AS is available at [30].

The chosen verification strategy shall be documented ([RR]).

Determine verification strategy

The safety requirements documented for each tier in [Q] shall be validated in order to check that they adequately capture the intent of the more abstract safety requirements defined at the previous tier.

small_the decomposition of AS safety requirements.png

thumbnail_the decomposition of AS safety requirements.png

the decomposition of AS safety requirements.png

This will require that it is checked that each of the higher level requirements can be satisfied if the safety requirements for this tier are correctly implemented. It is therefore important that the validation activities focus on the semantic equivalence of the safety requirements at different tiers.

Demonstrating that the intent of the safety requirements is captured requires more than simply stating that a relationship exists between safety requirements at different tiers. Some explanation and justification for the sufficiency of that relationship must be provided. Concepts such as “Rich Traceability” [12] could help in this regard.

Ensuring the intent of the safety requirements is maintained throughout decomposition may often be more challenging for AS than it is for traditional systems due to the sometimes large ”semantic gaps“ that can exist. The nature of these gaps and the challenges of addressing them for AS is discussed in more detail in [7]. 

Relationships between safety requirements

The output from the safety requirements validation activity shall be documented in the safety requirement justification report ([R]).

Validate safety requirements

Given a set of safety requirements defined at a tier of decomposition ([Q]) of the AS design, a design for that tier shall be produced that ensures those safety requirements can be met. This will involve making design decisions that are appropriate given the overall context of the safety requirements, the operating context and the known failure modes. In particular decisions relating to the system architecture are of particular importance when considering the satisfaction of safety requirements for an AS.

In the context of AS, we use the following definition of system architecture, adapted from [1]: “An architecture comprises the rules and constraints for the design of a system, including its structure and behavior, that would meet all of its established requirements and restrictions under the anticipated environmental conditions and component failure scenarios.”

Definition of system architecture

A system architecture may capture either the logical or physical structure of the system. Decomposition of an AS design may often involve a transition from logical architecture to a physical one. During such a transition there is often an increased likelihood that errors may be introduced to the design.

Logical and physical structure of the system

The design of the system architecture at each tier shall consider the need for robustness, fault tolerance and runtime monitoring in order to satisfy the safety requirements defined for that tier.

***Robustness*** can be defined as the delivery of a correct service in implicitly‐defined adverse situations arising due to an uncertain system environment [31]. Robustness is therefore a mechanism that is particularly important for AS since it enables the AS to mitigate hazardous system failures associated with hard to predict, and thus unexpected, changes in a complex operational 
environment. This may for example include objects in the environment that were not included as part of the ODM which the system fails to detect, or unexpected effects of particular lighting conditions that lead to “phantom objects” being detected. Since these events have not been anticipated, specific hazardous failures will not have been identified as part of the assurance process. However, it is expected that hazardous failures of this type would be identified, leading to a requirement for the AS to be designed such that it is robust to those types of failures.

Robustness in AS systems is typically achieved through redundancy in the architecture. The purpose of the redundancy is to provide compensation in the design of the system for potential limitations in system components that could result in unexpected system failures. Such redundancy may be required for both hardware and software elements of the architecture, and may be used at multiple architectural levels.

Redundancy in the architecture alone will not necessarily provide the robustness for the AS that is required. Identical components will contain the same limitations and failure modes. This would mean that the components, when exposed to the same set of inputs, would suffer the same failures. To be effective, there must therefore be diversity in the redundant architectural elements. This requires some level of independence between the components, for example by using components developed by different teams of people in an attempt to ensure the same mistakes are not replicated in each component, or by providing conceptual diversity through providing different specifications for each of the redundant components.

Ensuring diversity for software components particularly challenging [29], [4] and therefore requires particular attention for AS (with their high reliance on software). The use of artificial intelligence techniques such as machine learning (ML) for developing components also provides unique considerations for diversity.

Diversity in architecture redundancy 

***Fault tolerance*** can be defined as the delivery of a correct service despite faults arising from the AS itself. The focus of fault tolerance is therefore on the detection and recovery from anticipated failures, which should be identified as part of the hazard assessment of the AS (see Stage 6). Since the  hazardous failures are known during AS development, checks can be included in the system design in order to detect the faults. This could include, for example detecting inconsistencies between the system state as characterised by the sensors and the system state predicted by a model [31]. Once a fault is detected, fault recovery strategies enable the continued provision of the service. Standard software fault tolerance strategies can be applied effectively to AS [39].

There are three basic fault tolerance approaches that can be used: recovery blocks, N‐version programming, and N‐self‐checking software [25].

A recovery block approach provides fault tolerance for a component by using alternate variants of the component plus an adjudicator. The adjudicator tests the outputs from the components. If a component fails then the next alternate component is used and so on. This mechanism can be used to ensure that the behaviour of the component continues to be provided to the AS once a failure occurs in the component. The tests performed by the adjudicator must be sufficient to identify the hazardous failure based on consideration of the safety analysis results [BB] obtained from Stage 6.

In contrast, with an N‐version programming approach, all variants provide outputs to the adjudicator which then decides by considering all the outputs together which result to use. All variants of the component must be functionally‐equivalent, but diversely‐designed. An advantage of this approach is that it does not require tests to be defined for the adjudicator, so can be used for situations where it may be challenging to specify specific test criteria for a hazardous failure.

N‐self‐checking software is a hybrid approach that involves the use of self‐checking software components (SCSC). Each SCSC could use a recovery block or an N‐version program approach. At least two SCSC are required, whose outputs are then checked. Such an approach provides fault tolerance for components or subsystems which are themselves fault tolerant.

Fault tolerance approaches

***Runtime monitoring***

The provision of a runtime monitor as part of the AS design enables the behaviour of the AS during operation to be checked against defined constraints or behavioural predictions. The runtime monitor should be independent from the components it is monitoring, but may take the same inputs. One of the biggest challenges with using runtime monitors is being able to correctly define the constraints or bounds on behaviour that the monitor will check. This definition should make reference to the SOC ([L]) and the hazardous scenarios ([G]) identified from previous stages.

Runtime monitoring may, in addition to monitoring for unsafe behaviour of the AS, also monitor behavioural trends of the AS over periods of time. This would enable the identification of, for example, the deterioration in performance of the AS which may act as a leading indicator of future unsafe behaviour. In this case it is important to determine which information to monitor and how that can be interpreted as representing a threat to the safety of the AS.

The key design decisions that are taken at each tier shall be documented in the AS development log ([V]).

Design process for tier n

Create design at tier n

This activity requires as input the AS verification argument pattern ([UU]), as well as the artefacts from Activities 28, 29 and 30 ([RR],[SS]and [TT]). The activity uses these artefacts to create an instantiated AS verification assurance argument ([VV]) which demonstrates how the verification activities show that the defined safety requirements are sufficiently satisfied by the AS.

AS verification argument pattern

Instantiate AS verification argument pattern

It shall be demonstrated that the minimum risk strategy defined for outside the ODM ([MM]) will be satisfied by the AS during operation. This requires that evidence is generated which shows that the requirements of the strategy are implemented by the AS. This shall be achieved by performing verification activities, which could take a number of forms. The verification process for AS is discussed in detail in Stage 8. The results of verification activities shall be documented in the outside ODM verification report ([OO]).

Demonstrate risk strategy is satisfied outside ODM

Having decided upon a verification strategy, the activities required to be undertaken to carry out that strategy shall be defined. The sufficiency of those planned verification activities shall be justified to explain how they provide sufficient evidence with respect to the safety requirements. The verification activities and their justification shall be documented as part of a verification log ([SS]).

AS verification Log

Prepare and justify verification activities

The key output of the SACE process is a safety case for the AS. We adopt a commonly‐used definition of a safety case as a “structured argument, supported by a body of evidence that provides a compelling, comprehensible and valid case that a system is safe for a given application in a given operating environment.” [37]. As for the safety assurance process, SACE does not provide general guidance on how to create a compelling safety case for a system. Instead, SACE focuses on what modifications, enhancements or additions are required with respect to a safety case for a more conventional system in order to address the challenges of autonomy. As the baseline for this, we consider a very simple argument structure as shown in the safety case patterns in Figures 2 and 3 below. A safety case pattern documents a reusable argument structure and types of evidence that can be instantiated to create a specific safety case instance [22].

In these figures, and throughout SACE, safety case patterns are represented using the Goal Structuring Notation (GSN) [16]. GSN is a graphical notation for explicitly capturing safety arguments that is widely used in many industries for documenting safety cases. For a detailed description of the notation, the reader is advised to consult the publicly available GSN standard [16].

Figure 2 shows what the top level of a simple safety argument for an AS may look like, documented in the form of a pattern. The intention here is to provide the general form that such an argument should take, rather than to mandate a particular structure. In practice, it is expected that a more detailed safety argument would need to be provided, but that it will retain the characteristics defined by the pattern as discussed below. The safety case for the AS also requires further argument relating to the decomposition of the design and requirements throughout the AS development process as discussed earlier. The safety argument pattern for this system decomposition is shown in Figure 3 and discussed further later in this section.

Figure 2 identifies in white, the elements of the argument structure that are required in a safety argument for an AS, but that would also need to be addressed in a safety case for any system. These are the elements that are affected much less by the autonomous nature of the system. In contrast, those elements of the argument structure that are coloured in Figure 2 represent aspects of the safety argument that are either novel to AS or which are most affected by autonomy. These correspond to the parts of the safety assurance process that are challenged by autonomy as discussed earlier. In this guidance, we provide argument patterns relating to these autonomy‐related aspects identified in Figure 2. The details of the patterns are provided as part of the relevant process activity along with descriptions of the relevant artefacts.

In the argument patterns, we make use of assurance claim points (ACPs) [16], indicated by black squares in the argument pattern, to represent points in the argument at which further argument and evidence demonstrating the confidence in particular elements is required. It can be noted in Figure 2 that many of the argument patterns we provide for AS relate to the confidence arguments for ACPs relating to the key artefacts arising from the AS safety process. This reflects the importance that must be placed on managing uncertainty when considering AS. For example in Figure 2, there can be seen to be an ACP relating to the hazards identified for the AS. This shows it is necessary to provide a confidence argument relating to the sufficiency of the identified hazards. The AS Hazard Identification Argument Pattern provides guidance on the structure of that confidence argument.

Below we describe the elements of the argument in Figure 2.




Out of Context Operation Assurance Argument Pattern

The top-level safety claim that we consider is that the AS is sufficiently safe throughout its operational life. The definition of what is considered to be sufficiently safe for an AS can be extremely challenging since it must take account of complex factors such as:

- Risk perception ‐ The tolerability of risk for an AS may be different from an equivalent non-autonomous system meaning that a direct comparison may not be an effective way to judge sufficiency.
- Risk trade‐off ‐ Autonomous operation will introduce new risks that must be traded‐off against the potential safety benefits that they may bring.
- Ethical considerations ‐ Any definition of sufficiently safe must take account of the complex ethical issues that are raised by autonomous operations (see [8], and [47] for example).

In addition to this there are many domain‐specific and legal factors that inform this definition.

These issues are the subject of extensive on‐going research that is outside of the scope of this document. For the purposes of SACE we assume that there exists an acceptable definition of "sufficiently safe”. Additional guidance in this area can be found at [3].


The strategy that is adopted is to split the safety argument into a claim regarding the safe operation of the AS when it is operating within the defined autonomous operating context (G1), and also a claim that it remains sufficiently safe when outside of that operating context (G7). This requires that the operating context has been explicitly defined (Stage 1 describes how this is done). The operating context description is provided at C1. However, it is also crucial for assurance of AS that an argument is provided, supported by evidence, to demonstrate that the operating context that is defined is sufficient to support the safe operation of the AS.

The Operating Context Assurance Argument Pattern ([G]) is provided in order to guide the development of this argument, and is discussed in detail in Stage 1. The link to the assurance argument for the operating context is established in Figure 2 using an Assurance Claim Point (ACP) [16] (indicated by the black square). ACPs represent points in the argument at which further assurance is provided.


This safety claim relates to the safe operation of the AS when it is operating within the defined autonomous operating context.


The safe operation of the AS in the defined autonomous operating context is demonstrated through an argument that all of the hazardous scenarios associated with the operation of the AS have been sufficiently mitigated. The hazardous scenarios that are identified are provided at C2. The completeness and correctness of the identified hazardous scenarios for the AS must be demonstrated. The AS Hazardous Scenarios Identification Argument Pattern ([I]) is provided in order to guide the development of this argument, and the activities and artefacts are discussed in detail in Stage 2.

The identified hazards for the AS are mitigated through ensuring that the AS operates in such a manner that the defined Safe Operating Concept (SOC) is satisfied. The SOC provides a specification for safe operation of the AS taking account of the operating context and the identified hazards. The SOC is discussed in detail in Stage 3. It must be demonstrated that the operation specified by the SOC sufficiently mitigates the identified hazards. The SOC Assurance Argument Pattern ([N]) is provided in order to guide the development of this argument.

This safety claim is supported by an argument that considers the control and mitigation of hazards and the associated safety assurance of the AS throughout the decomposition of the development lifecycle (as presented in Figure 4). This AS Decomposition Argument Pattern is presented in Figure 3 and discussed below.

This safety claim relates to the safety of the AS when operating outside of the defined operating context. Although the AS is only expected to operate autonomously within the defined operating context, there may be occasions when it is operating outside of that. This may be planned non‐autonomous operation where a human operator takes control of the system, or it may be unplanned excursions from the defined operating context. In all such cases it is important to be able to demonstrate that the system remains sufficiently safe. The Out of Context Operation Argument Pattern ([PP]) is provided in order to guide the development of this argument, and the activities and artefacts are discussed in detail in Stage 7.

Figure 3 shows the further development of the high‐level safety argument in Figure 2. This provides a pattern of argument that captures the general form that the safety argument should take for each level of decomposition in the system development process (described as a “tier” in the argument pattern). As discussed earlier, in this guidance we make no assumptions about the number of levels of decomposition in the AS development process, instead we require that a safety argument is developed that considers safety at each of those levels, irrespective of how many levels there may be. The argument pattern shown in Figure 3 is based upon patterns originally developed for the software aspects of systems [19], but which also apply more generally. The characteristics of this pattern are discussed below.

How to use

How to use the SACE guidance