AS hazardous scenarios identification
Objectives
- Identify and define potentially hazardous scenarios for the AS.
- Validate the AS hazardous scenarios.
- Create the AS hazardous scenarios assurance argument.
Inputs to the stage
- [B] : Operational domain model
- [E] : Operating scenarios definition
- [I] : AS hazardous scenarios assurance argument pattern
Outputs of the stage
- [WW] : AS decision analysis report
- [XX] : AS hazardous scenarios definition
- [YY] : AS hazardous scenarios validation report
- [J] : AS hazardous scenarios assurance argument
Description of the stage
Hazardous scenarios are those scenarios that the AS may encounter during its operation that could, under certain conditions, lead to an unsafe outcome. For AS we focus in particular on the interactions between the AS and elements of the operating environment, and on the decisions that are made by the AS as part of its autonomous capability. The hazardous scenario for the AS should therefore be described using the general form:
<relevant environment state(s)> AND , where:
- An AS operating scenario describes what the AS is undertaking (identified from [E])
- A relevant environment state is one or more states occurring within or emanating from the operating environment. Environment states may be identified by considering elements of the ODM ([B]).
- The decision is the selected course of action as a result of the operating scenario and relevant environment state(s).
As shown in Figure 11 above, this stage consists of activities that are performed to identify and validate the potentially hazardous scenarios associated with the operation of the AS. The artefacts generated from this stage are used to instantiate the AS hazardous scenarios assurance argument pattern as part of Activity 9.
Note 6 - Deployment of autonomous technology
This guidance focuses specifically on hazardous scenarios related to the deployment of autonomous technology. It is assumed that consideration of the hazardous scenarios associated with the more conventional (non‐autonomous) aspects of a system are considered concurrently and additionally to this.
Note 7 - Differentiating between understanding and deciding
It is clear that understanding the decisions that may need to be taken by an AS during its operation is crucial to identifying the potential hazardous scenarios. It is important therefore to understand what is meant by ‘decision‐making’ for an AS. The decision determines which action the AS should take in any given situation, and an incorrect decision can lead directly to an unsafe action. In order to make a decision, an AS must understand the state of the environment and the system, as shown in figure 12.
It can sometimes be difficult to differentiate between ‘Understanding’ and ‘Deciding’. For example, in the case of a mobile robot, the act of detecting a static object represents ‘understanding’, whereas the decision is on whether moderation of speed or course alteration is required (we would not therefore characterise the identification of the object as a ‘decision’ in this case). In the case of an autonomous medical device the act of classifying patient vital signs represents ‘understanding’, the decision is whether to increase medication or not (again, the classification of vital signs would not be characterised as a ‘decision’ in this case). Note here that a decision NOT to increase medication could in itself lead to harm due to delayed treatment.
In both of these cases, although incorrect understanding may be a causal factor, it is the decision that is made that ultimately determines if the outcome of a scenario is safe or not.
