Part III — Evaluating Risk, Intelligent Technologies, Digital Systems, Biotechnology and Human-Centred Technologies

Chapter 9. Risk, Security, Governance and Regulation — Part 1 of 3

9.1 Introduction

Every technology creates some form of risk. Risk may arise from technical failure, human error, malicious attack, inappropriate use, organizational weakness, regulatory uncertainty, environmental consequences or interactions between multiple systems. The challenge is therefore not to identify technologies as either "safe" or "unsafe" in absolute terms but to understand the conditions under which particular risks arise and how those risks can be reduced, transferred, monitored or accepted.

Risk assessment has traditionally involved the identification of hazards, estimation of likelihood and assessment of consequences (Kaplan & Garrick, 1981). Contemporary technology evaluation requires a broader approach because emerging technologies often operate under conditions of uncertainty, incomplete evidence and changing operating environments.

The International Organization for Standardization defines risk in relation to the effect of uncertainty on objectives, while ISO 31000 emphasizes that risk management should be integrated with organizational decision-making rather than treated as an isolated technical activity (International Organization for Standardization [ISO], 2018).

For emerging technologies, this distinction is particularly important. The absence of evidence of harm should not automatically be interpreted as evidence that no harm exists. New systems may not have accumulated sufficient operational experience to reveal rare failures, secondary effects or long-term consequences.

9.2 Understanding Technology Risk

Technology risk may be conceptualized as the possibility that a technological system, component, process or application will produce an outcome that negatively affects organizational, human, economic, environmental or societal objectives.

Risk can arise from at least four interacting components:

the technology itself;

the context in which it is used;

the people and organizations operating it; and

the external environment surrounding it.

This means that risk is not always an intrinsic property of the technology.

For example, an automated diagnostic system may have strong technical performance in a controlled clinical trial but perform less effectively when deployed in a hospital with different patient demographics, equipment, workflows or data quality. Similarly, a cloud platform may provide high availability but create organizational vulnerability if a company becomes dependent upon a single provider.

Risk evaluation should therefore examine both technological characteristics and deployment conditions (Aven, 2016; Leveson, 2011).

9.3 Categories of Technology Risk

A comprehensive technology evaluation should consider multiple categories of risk.

9.3.1 Technical Risk

Technical risk arises from defects, performance limitations, incompatibility, inadequate testing, component failure or unexpected system behaviour.

Examples include:

software defects;

hardware failure;

model instability;

inadequate computational capacity;

interoperability failure;

poor data quality;

insufficient testing;

system degradation;

inaccurate outputs; and

failure under unusual operating conditions.

Technical risk is especially significant during early technology development because evidence is often limited.

9.3.2 Operational Risk

Operational risk concerns failures arising from processes, people, procedures and organizational systems.

A technically reliable technology can therefore become operationally unreliable if:

employees are insufficiently trained;

procedures are poorly designed;

responsibilities are unclear;

maintenance is neglected;

escalation procedures are absent; or

management assumes that automation eliminates human responsibility.

Reason (1997) demonstrated that organizational accidents frequently arise from combinations of latent organizational conditions and active failures. This principle remains relevant when evaluating emerging technologies because technological systems operate within organizational environments.

9.3.3 Cybersecurity Risk

Digital technologies create risks associated with unauthorized access, malicious manipulation, data theft, ransomware, denial-of-service attacks, software vulnerabilities and supply-chain compromise.

The NIST Cybersecurity Framework 2.0 provides a general framework for organizations to understand, assess, prioritize and communicate cybersecurity outcomes (Pascoe et al., 2024). Its applicability is deliberately broad, extending beyond traditional critical-infrastructure environments.

Cybersecurity should therefore be evaluated as a continuing organizational capability rather than a one-time technical certification.

9.4 Threat, Vulnerability and Consequence

A useful risk model distinguishes among threat, vulnerability and consequence.

A threat is a potential source of harm.

A CONCEPTUAL FRAMEWORK

Threat, Vulnerability and Consequence

A consequence is the resulting impact that occurs when a threat exploits a vulnerability.

Figure: Relationship between Threat, Vulnerability and Consequence

Interpretation: A threat represents a potential source of harm. A vulnerability is a weakness that may be exploited by the threat. The consequence is the resulting impact when the vulnerability is exploited.

Example mapping

vulnerability is a weakness that may allow the threat to produce an adverse outcome.

However, modern technology systems may involve multiple interacting vulnerabilities. A single weakness can therefore propagate across an ecosystem.

This is particularly important in cloud computing, Internet of Things environments, connected vehicles, smart cities and AI-enabled infrastructures.

9.5 Privacy Risk

Privacy risk concerns the possibility that technological systems may collect, infer, retain, process or disclose personal information in ways that create harm or violate legitimate expectations.

Data-intensive technologies create particular challenges because personal information may be combined with apparently unrelated datasets. Machine learning can also infer sensitive characteristics without directly collecting them.

Privacy evaluation should therefore ask:

What data are collected?

Why are they collected?

How long are they retained?

Who can access them?

Where are they stored?

Can individuals correct them?

Can data be deleted?

Can information be combined with other datasets?

Can sensitive characteristics be inferred?

What happens when data are transferred between organizations or jurisdictions?

Privacy must consequently be considered throughout the technology lifecycle.

9.6 Financial Risk

Financial risk may involve:

cost overruns;

uncertain returns;

implementation failure;

vendor dependency;

infrastructure costs;

maintenance costs;

cybersecurity expenditure;

regulatory compliance costs;

workforce transition costs; and

reputational losses.

Financial evaluation should therefore go beyond initial purchase price. Total cost of ownership is particularly important for technologies requiring continuous software updates, cybersecurity investment, specialized personnel or proprietary infrastructure.

9.7 Legal and Regulatory Risk

Emerging technologies frequently develop faster than regulatory systems.

Legal and regulatory risk may therefore arise because:

laws are unclear;

responsibilities are disputed;

existing legislation does not address the technology;

different jurisdictions apply different requirements;

liability is uncertain; or

regulatory requirements change during implementation.

Technology evaluation should include a legal mapping exercise identifying relevant legislation, standards, professional requirements, contractual obligations and regulatory expectations.

Legal and regulatory risk is particularly important when organizations adopt technologies whose functions do not fit neatly within existing legal categories. Traditional laws are often designed around established products, professional roles and organizational responsibilities. Emerging technologies can challenge these assumptions by introducing new forms of automation, data processing, decision-making and interaction between humans and machines.

A technology may therefore be technically feasible while its legal status remains uncertain. The absence of a specific prohibition should not automatically be interpreted as evidence that an application is legally acceptable. Organizations should examine the broader legal environment, including existing obligations concerning privacy, intellectual property, consumer protection, employment, discrimination, safety, cybersecurity, contractual responsibility, professional standards and sector-specific regulation.

Data-intensive technologies provide an important example. A system may collect, process, store or transfer large quantities of personal or organizational information. The legal assessment should consider what information is collected, why it is collected, where it is stored, who can access it, how long it is retained and under what circumstances it may be transferred to third parties.

Intellectual property can also create uncertainty. Emerging technologies may process copyrighted materials, proprietary information, software, images, research outputs and other protected content. Technology evaluation should therefore examine ownership, licensing, permitted use, attribution and contractual restrictions.

Liability is another important issue. When an automated system produces an incorrect recommendation or decision, responsibility may be distributed among the technology provider, system developer, organization, operator and end user. The allocation of responsibility should be understood before the technology is deployed in consequential settings.

Contractual obligations may also create risk. Technology providers may impose terms concerning data use, service availability, warranties, liability limitations, audit rights, intellectual property, termination and data portability. Organizations should understand these conditions before becoming dependent on a particular technology provider.

Cross-border technology creates additional complexity. Cloud services, software platforms, data-processing systems and supply chains may operate across multiple jurisdictions. The legal requirements of the country where the organization is located may differ from those of the country where data are processed or where the service provider is established.

Regulatory change is another important consideration. A technology may comply with applicable requirements at the time of implementation but face new obligations as regulators respond to emerging risks. Technology evaluation should therefore consider not only current compliance but also the organization's capacity to respond to future regulatory developments.

Compliance should consequently be treated as an ongoing process rather than a one-time approval. Organizations should establish mechanisms for monitoring regulatory developments, reviewing policies, updating contracts, training employees and reassessing technology use when legal requirements change.

Standards can also support legal and regulatory evaluation. International standards, professional standards and sector-specific frameworks may provide useful guidance even when they are not themselves legally binding. However, standards should not automatically be treated as substitutes for applicable legislation.

Legal evaluation should also distinguish between compliance and responsible practice. A technology may satisfy the minimum legal requirement while creating broader ethical or social concerns. Technology governance should therefore consider legal compliance as a baseline rather than necessarily the complete measure of responsible technology use.

The legal evaluation process can therefore follow a structured sequence:

Technology function → applicable law → regulatory requirements → contractual obligations → professional standards → liability allocation → compliance requirements → monitoring and review.

This approach helps organizations identify legal risks before they become operational problems.

For emerging technologies, legal uncertainty should also be documented explicitly. Where the law is unsettled, organizations should identify assumptions, seek appropriate professional advice and establish conservative controls for high-consequence applications.

Legal and regulatory risk is therefore closely connected to governance risk. A technology cannot be evaluated solely according to whether it functions technically. The organization must also determine whether it can operate the technology within the applicable legal and regulatory environment and whether it can continue to do so as that environment develops.

9.8 Environmental Risk

Technology risk also includes environmental consequences.

Potential impacts include:

energy consumption;

greenhouse gas emissions;

resource extraction;

water consumption;

hazardous materials;

electronic waste;

biodiversity effects; and

environmental contamination.

Life-cycle assessment is useful because environmental impacts may occur at different stages of a technology's life rather than solely during operational use (ISO, 2006a, 2006b; Moni et al., 2020).

Environmental risk should be understood as the possibility that technological activities may create environmental harm, increase exposure to environmental liabilities or reduce the organization's ability to operate sustainably over time.

The environmental dimension of technology risk has become increasingly important because modern technological systems depend on extensive physical infrastructure. Digital technologies, for example, may appear intangible to users but depend on data centres, telecommunications networks, semiconductor manufacturing, electricity systems, cooling infrastructure, buildings and extensive supply chains.

Energy consumption is one important source of environmental risk. Technologies may require electricity during operation, while energy is also consumed during manufacturing, transportation, maintenance and disposal. The environmental significance of energy consumption depends partly on the source of electricity. Electricity generated from different energy systems can have different lifecycle emissions and resource requirements.

Greenhouse gas emissions should therefore be examined across the technology lifecycle rather than only during operation. Embodied emissions associated with manufacturing can be significant, particularly for equipment containing energy-intensive materials and components.

Resource extraction represents another dimension. Technologies may require metals, minerals, rare materials and other natural resources. Increased technological adoption can increase demand for these resources and may create environmental pressures associated with mining, processing and transportation.

Water consumption is increasingly relevant to technology risk. Manufacturing processes, electricity generation, cooling systems and industrial operations may require significant quantities of water. Technologies deployed in water-stressed regions may therefore face both environmental and operational risks.

Hazardous materials can create risks during manufacturing, operation and disposal. Organizations should understand what materials are present in technological equipment, how they are handled and what procedures apply when equipment reaches the end of its useful life.

Electronic waste represents another major concern. Rapid technological change can shorten the economic life of equipment even when components remain technically functional. Poor end-of-life management can result in material loss, environmental contamination and increased waste-management costs.

Biodiversity effects should also be considered. Mining, infrastructure construction, energy generation and transportation can alter habitats and ecosystems. The environmental evaluation should therefore examine whether technology deployment contributes to land-use change, habitat fragmentation, pollution or other ecological pressures.

Environmental risk can also be indirect. A technology may reduce one environmental impact while increasing another. For example, an energy-efficient system may reduce electricity consumption while requiring additional materials during manufacturing. An electric technology may reduce direct emissions while creating upstream environmental burdens associated with electricity generation and battery materials.

This creates the possibility of environmental burden shifting. Environmental evaluation should therefore consider multiple environmental indicators rather than relying on a single measure.

Environmental risk can also develop over time. A technology may initially have manageable environmental impacts but become problematic when deployed at very large scale. Conversely, technological improvements may reduce environmental impacts as manufacturing processes, energy systems and recycling capabilities develop.

Scale is therefore important. The environmental risk associated with one device may be relatively small, while millions of identical devices can create substantial aggregate resource requirements and waste streams.

Geographical context is equally important. Environmental impacts may be concentrated in particular locations even when technological benefits are distributed globally. Resource extraction, manufacturing, data-centre operations, waste processing and infrastructure development may affect specific communities and ecosystems.

Environmental risk should therefore be assessed through lifecycle thinking.

Raw materials → manufacturing → transportation → deployment → operation → maintenance → reuse → recycling/disposal.

At each stage, the organization should identify resource requirements, environmental impacts, potential mitigation measures and residual risks.

Environmental risk management can include energy efficiency, renewable electricity, responsible sourcing, longer equipment lifetimes, repair, refurbishment, reuse, recycling, water conservation, pollution prevention and environmental monitoring.

However, mitigation should itself be evaluated. A mitigation measure may reduce one environmental risk while creating another. Environmental management should therefore remain consistent with the broader multidimensional evaluation framework.

Environmental risk is also connected to climate resilience. Technologies may contribute to environmental change while simultaneously being exposed to environmental change. Higher temperatures, flooding, drought, water scarcity and extreme weather can affect technological infrastructure and supply chains.

The organization should consequently consider two directions of environmental risk:

Technology → environment

Environment → technology.

The first concerns the environmental effects created by technology. The second concerns how environmental change affects technological performance and organizational continuity.

This two-directional perspective strengthens technology-risk assessment by recognizing that environmental sustainability and operational resilience are interconnected.

9.9 Reputational Risk

Technology failure can affect organizational reputation.

A company may lose public trust following:

a major data breach;

discriminatory algorithmic decisions;

unsafe autonomous operation;

misleading AI outputs;

medical technology failure;

environmental damage; or

unethical surveillance.

Reputation is difficult to quantify, but it can influence customer confidence, employee retention, regulatory attention and investor behaviour.

Reputational risk arises when technology-related events affect how stakeholders perceive an organization, its products, its leadership or its commitment to responsible practice.

The significance of reputational risk has increased because technological incidents can become widely visible very quickly. Social media, online news, professional networks and digital communication platforms can distribute information about an incident to large audiences within a short period.

A relatively localized technical problem can therefore develop into a broader organizational issue if stakeholders perceive that the organization failed to manage the technology responsibly.

Reputational damage may result from actual technological failure, but it can also result from poor communication concerning a failure. An organization that does not communicate clearly about an incident may create uncertainty among customers, employees, regulators and other stakeholders.

Transparency is therefore an important component of reputational risk management. Organizations should have clear procedures for communicating significant technology incidents, acknowledging known facts, correcting inaccurate information and explaining corrective measures.

The credibility of organizational claims is also important. Organizations increasingly communicate about artificial intelligence, sustainability, cybersecurity, innovation and digital transformation. If claims concerning technological capability or environmental performance are not supported by evidence, the organization may face reputational consequences.

Technology evaluation should therefore distinguish between actual performance and promotional claims.

Reputational risk can also emerge from the behaviour of technology suppliers. An organization may experience reputational consequences arising from the conduct of a third-party provider even when the organization did not directly create the underlying technology.

Third-party risk assessment is therefore important. Organizations should consider the reputation, governance practices, security controls, environmental practices and compliance record of major technology suppliers.

Employee perceptions can also influence reputational outcomes. Employees may respond negatively if technology is introduced without adequate consultation, training or explanation. Conversely, appropriate engagement can help organizations understand concerns before they become broader organizational problems.

Customers and users may also evaluate whether technology reflects the values of the organization. Issues concerning privacy, accessibility, fairness, transparency, safety and sustainability can influence stakeholder trust.

Reputational risk should therefore not be reduced to public relations. It reflects the relationship between organizational behaviour and stakeholder expectations.

Trust can be difficult to establish and relatively difficult to restore after a major incident. Technology governance should consequently focus on prevention as well as crisis response.

Reputational risk can be assessed through several forms of evidence, including stakeholder feedback, customer complaints, employee surveys, media analysis, incident records, regulatory communications and changes in customer behaviour. None of these indicators provides a complete measure of reputation, but together they can provide useful evidence of changing stakeholder perceptions.

The organization should also distinguish between temporary dissatisfaction and sustained reputational deterioration. A single technical problem may be resolved quickly if the organization responds effectively, while repeated incidents may indicate deeper governance problems.

Reputation is therefore partly an outcome of technology governance. Strong technical performance alone does not guarantee trust. Stakeholders may also evaluate whether the organization uses technology transparently, responsibly and consistently with its stated commitments.

9.10 Systemic Risk

Systemic risk occurs when failures propagate across interconnected systems.

Examples include:

financial technology interconnectedness;

national digital infrastructure;

cloud dependency;

telecommunications;

electricity systems;

healthcare information systems; and

supply-chain technologies.

The greater the interconnectedness, the greater the possibility that local failures will create wider consequences.

Systemic risk differs from conventional technology risk because the consequences may extend beyond the organization that originally experiences the failure. A local technological disruption can spread through networks, suppliers, platforms, financial relationships, communications systems or shared infrastructure.

Modern organizations are increasingly dependent on interconnected digital ecosystems. A university may depend on cloud services, internet connectivity, payment systems, learning platforms, identity-management systems, telecommunications providers and external software applications. A failure in one component may therefore affect multiple organizational functions simultaneously.

Cloud dependency provides an important example. Organizations may outsource computing and storage to a small number of large service providers. This can improve scalability and reduce the need for local infrastructure, but it can also create concentration risk. A disruption affecting a major provider may affect many organizations simultaneously.

Telecommunications systems create another form of systemic dependency. If communications infrastructure is disrupted, organizations may lose access to authentication, cloud services, remote work systems, emergency communications and other digital functions.

Electricity systems are similarly interconnected with digital technologies. Data centres, telecommunications networks, hospitals, financial institutions and industrial systems depend on reliable electricity. Electricity disruption can therefore create cascading technological consequences.

Healthcare information systems demonstrate the potential seriousness of systemic risk. If multiple healthcare organizations depend on interconnected information systems, a failure or cyber incident can affect patient services across institutions rather than remaining isolated within one organization.

Supply chains can also create systemic risk. A shortage of semiconductors, specialized components or critical materials can affect multiple industries simultaneously. Technology evaluation should therefore examine dependencies beyond the immediate organization.

Network structure is an important consideration. Highly interconnected systems may produce efficiency and flexibility while also creating pathways through which failures can spread.

This creates an important technology-governance principle:

Interconnectedness can increase both value and vulnerability.

The objective is therefore not necessarily to eliminate interconnection. Instead, organizations should understand critical dependencies and develop appropriate resilience mechanisms.

Systemic risk assessment should identify:

critical dependencies;

single points of failure;

concentration risks;

shared infrastructure;

common suppliers;

interoperability dependencies;

cascading failure pathways;

recovery requirements; and

alternative arrangements.

The analysis should consider what happens if one component fails and then examine the consequences for other components.

For example:

Cloud failure → application unavailability → loss of institutional services → operational disruption → financial and reputational consequences.

Another pathway might be:

Cybersecurity incident → compromised information system → disruption of connected services → interruption of organizational operations → stakeholder impact.

Systemic risk can also involve feedback loops. A disruption may create conditions that increase the likelihood or severity of further disruptions. For example, a shortage of one component may increase demand for alternative suppliers, which may create additional pressure elsewhere in the supply chain.

Scenario analysis and stress testing can help organizations examine these relationships. Rather than asking only whether a system is likely to fail, the organization can ask what would happen if a critical component became unavailable for a specified period.

Recovery time is particularly important. A system that can tolerate a brief interruption may be unable to tolerate a prolonged outage. Critical systems should therefore have recovery objectives appropriate to the consequences of failure.

Systemic risk also reinforces the importance of diversification. Dependence on a single provider, technology, geographic location or infrastructure system can increase concentration risk. However, diversification can also increase complexity and cost. The organization should therefore evaluate the trade-off between efficiency and resilience.

The systemic dimension also extends beyond organizational boundaries. A technology may be embedded in national, regional or global systems. Financial platforms, telecommunications, cloud computing, energy infrastructure, transportation networks and supply chains are examples where organizational decisions can interact with wider systems.

Consequently, technology evaluation should consider not only:

What happens if our technology fails?

but also:

What happens if our technology fails while other interconnected systems are operating under stress?

This is particularly important for critical infrastructure and high-dependency environments.

Systemic risk therefore requires a broader view of technology evaluation. The evaluator must consider relationships, dependencies, concentration, cascading consequences and recovery capacity rather than evaluating each technological component as an isolated object.

9.11 Risk Appetite and Risk Tolerance

Organizations should define their risk appetite before making major technology decisions.

Risk appetite refers broadly to the amount and type of risk an organization is willing to accept in pursuit of its objectives. Risk tolerance refers to acceptable variation around specified objectives.

Different technologies may require different thresholds.

For example, an organization may tolerate minor inconvenience from an office productivity application but have a much lower tolerance for error in:

medical diagnosis;

aircraft control;

nuclear systems;

financial settlement;

emergency communications; or

autonomous transportation.

Risk tolerance should therefore be linked to consequence severity.

Risk appetite and risk tolerance provide an important bridge between risk identification and technology decision-making. Identifying risks is not sufficient. The organization must also establish how much uncertainty, disruption, loss or potential harm can reasonably be accepted.

Risk appetite is generally broader and strategic. It concerns the organization's willingness to take different categories of risk in pursuit of objectives. Risk tolerance is more specific and concerns the limits within which particular activities or systems are expected to operate.

These concepts should be established before major technology decisions are made because the same level of risk may be acceptable in one context and unacceptable in another.

A small error in an internal administrative application may have limited consequences. The same error rate in a medical diagnostic system could have substantially different consequences. The appropriate risk threshold must therefore reflect the consequences associated with failure.

Consequence severity is one of the most important factors in establishing risk tolerance. Other relevant factors include probability, exposure, reversibility, detectability, duration and the availability of alternative controls.

A technology risk assessment can therefore consider:

likelihood of failure;

severity of consequence;

duration of impact;

number of people affected;

ability to detect the problem;

ability to recover;

availability of alternatives;

regulatory requirements; and

ethical significance.

Risk appetite should also be aligned with organizational objectives. An organization may accept greater technological uncertainty when pursuing innovation in a controlled experimental environment than when operating a critical production system.

This creates a distinction between experimentation and operational deployment. Emerging technologies may be tested under controlled conditions with limited exposure. Once the technology becomes part of a critical organizational process, substantially stronger controls may be required.

Risk appetite can therefore change according to the stage of technology adoption.

Research and experimentation → pilot implementation → limited deployment → institutional deployment → critical dependence.

As dependence increases, the consequences of failure may also increase. Risk management should therefore evolve throughout the technology lifecycle.

Risk tolerance should also be measurable where possible. Instead of stating that an organization has a "low tolerance" for a particular risk, management may define measurable thresholds concerning downtime, error rates, financial losses, data incidents, response times or service availability.

For example, an organization may define an acceptable maximum interruption period for a critical information system. It may establish an error threshold for a particular automated process. It may also define circumstances under which human review becomes mandatory.

These thresholds should be documented and communicated to those responsible for operating the technology.

Risk appetite should not be confused with willingness to accept any risk in pursuit of innovation. Responsible innovation requires the organization to understand the consequences of risk and establish appropriate safeguards.

High-risk technologies may require stronger controls, independent review, human oversight, redundancy, testing, monitoring and contingency planning.

Low-risk technologies may require simpler controls.

The principle is proportionality. The intensity of risk management should reflect the potential consequences of the technology.

Risk tolerance should also account for vulnerable stakeholders. A technology may technically meet an organizational performance threshold while creating disproportionate consequences for particular groups. Risk assessment should therefore incorporate social, ethical and environmental considerations rather than focusing solely on financial or operational loss.

Risk appetite can also change over time. An organization may initially accept a higher degree of uncertainty when a technology is experimental. As evidence accumulates, management may revise the acceptable risk level. Similarly, a new regulatory requirement, major incident or change in organizational strategy may require risk appetite to be reconsidered.

This makes risk governance a dynamic process.

Risk identification → risk analysis → risk appetite → risk tolerance → controls → monitoring → review → adjustment.

Risk tolerance should also be connected to escalation mechanisms. When a technology approaches or exceeds a defined threshold, the organization should know who must be informed, what actions should be taken and under what circumstances the system should be restricted, suspended or reassessed.

Human oversight is particularly important for high-consequence technologies. Where automated systems can produce decisions with significant effects on people, risk tolerance should include requirements concerning human review, explainability, auditability and the ability to challenge or reverse decisions.

The concept can therefore be integrated into the broader multidimensional technology evaluation framework:

Technology capability → potential benefit → identified risk → consequence assessment → risk appetite → risk tolerance → control measures → monitoring → decision.

This approach prevents risk assessment from becoming a purely descriptive exercise. It connects risk evidence directly with organizational governance and decision-making.

Ultimately, risk appetite and risk tolerance should be understood as contextual rather than universal. There is no single acceptable level of technology risk for all organizations or all applications. The appropriate threshold depends on the purpose of the technology, the consequences of failure, the people affected, the availability of alternatives, the organization's capabilities, applicable legal requirements and the broader social and environmental context.

The central principle is therefore that technology risk should be proportional to consequence.

The greater the potential consequence of technological failure, the stronger the evidence, controls, oversight, resilience and accountability that should normally be required before and during deployment.

9.12 Risk Registers

A risk register provides a structured mechanism for documenting risks.

The register should be dynamic rather than static.

9.13 Failure Mode and Effects Analysis

Failure Mode and Effects Analysis (FMEA) identifies potential failure modes, their causes, their effects and possible controls. It is a structured and preventive approach to risk assessment that seeks to identify how a technology, process, component, or system could fail before the failure occurs in practice. Rather than beginning with the consequences of an incident that has already happened, FMEA begins by asking how failure could occur, what might cause it, what could happen if it occurred, how likely it would be to occur, whether it could be detected before causing harm, and what controls could be introduced to reduce the risk. This makes FMEA particularly relevant to technology evaluation because emerging technologies frequently involve combinations of hardware, software, data, people, infrastructure, suppliers, communication networks, and organizational processes.

For technology evaluation, FMEA can examine:

component failure;

software failure;

user error;

data failure;

communication failure;

security failure; and

maintenance failure.

These failure categories should not be treated as isolated technical events. In a modern technology system, one failure mode can initiate or contribute to another failure mode. A hardware component failure may cause software instability; software instability may produce incorrect information; incorrect information may lead to an inappropriate human decision; the resulting decision may create operational, financial, safety, legal, or reputational consequences. Similarly, a communication failure may prevent a monitoring system from detecting a component problem, while inadequate maintenance may allow a small technical defect to develop into a major system failure. Consequently, FMEA can be used not only to identify individual failures but also to examine relationships between failures and the conditions under which they may propagate through a wider socio-technical system.

The central logic of FMEA can be represented as:

Technology or process → potential failure mode → cause → local effect → system effect → consequence → existing control → detection capability → additional control → monitoring and review.

A failure mode refers to the specific way in which a component, process, function, or system may fail to perform its intended function. The failure mode should be described sufficiently precisely to allow its causes and consequences to be examined. For example, “AI system failure” is generally too broad to provide a useful basis for analysis. More specific failure modes might include incorrect classification of input data, failure to process a request, generation of an unsupported recommendation, loss of connectivity, unauthorized access, corrupted data, failure to update a model, or incorrect transmission of an output to another system. The greater the specificity of the failure mode, the easier it becomes to identify appropriate preventive and detective controls.

The cause of failure represents the condition or event that could produce the failure mode. Causes may arise from technical defects, design limitations, programming errors, inadequate testing, poor-quality data, human error, environmental conditions, organizational processes, supplier dependencies, inadequate training, cybersecurity incidents, maintenance deficiencies, or unexpected interactions between system components. In emerging technologies, causes may also arise because the technology is being used in circumstances that were not anticipated during its original development. A system that performs adequately under laboratory conditions, for example, may behave differently when exposed to larger datasets, different user populations, changing environmental conditions, or integration with legacy systems.

The effects of failure describe what happens when the failure mode occurs. Effects should be considered at several levels rather than only at the immediate technical level. A component may fail locally without immediately affecting the wider system. Alternatively, a local failure may interrupt a critical service, produce inaccurate information, expose confidential data, delay a decision, create a safety hazard, increase operating costs, or cause a cascading failure. The evaluation should therefore distinguish between the immediate effect, the operational effect, the organizational effect, and the broader stakeholder or societal effect where relevant.

For example, if a digital payment platform experiences a database synchronization failure, the immediate effect may be that transaction records are not updated correctly. The operational effect may be delayed or duplicated transactions. The organizational effect may involve financial reconciliation problems and customer complaints. The wider effect could include loss of confidence in the service or disruption to organizations that depend upon the platform. This illustrates why technology risk assessment should extend beyond the technical boundary of the technology itself.

FMEA can also distinguish between prevention and detection. Preventive controls are designed to reduce the probability that a failure will occur. Examples include robust design, quality assurance, redundancy, validation, testing, staff training, secure coding practices, data-quality controls, preventive maintenance, access controls, and supplier qualification. Detective controls are designed to identify a failure or abnormal condition before it produces serious consequences. Examples include monitoring systems, automated alerts, audit trails, anomaly detection, diagnostic tools, logging, human review, exception reporting, and performance dashboards.

The distinction is important because a system may contain a failure mode that cannot realistically be eliminated. In such circumstances, risk management may focus on reducing the likelihood of failure, detecting the failure early, limiting its consequences, and ensuring rapid recovery. A technology evaluation should therefore avoid assuming that every risk must be completely eliminated. In many complex systems, the more realistic objective is to reduce risk to an acceptable level while maintaining the intended functionality and ensuring appropriate safeguards.

FMEA is particularly useful during the design and development stage of technology evaluation. At this stage, design modifications may be relatively easier and less costly than after the technology has been deployed extensively. Identifying that a particular sensor can fail under certain environmental conditions, for example, may lead to the introduction of redundancy, improved environmental protection, additional calibration requirements, or a different sensor design. Identifying that users may misunderstand an automated recommendation may lead to interface changes, explanatory information, additional training, or mandatory human review.

FMEA can also be applied after deployment. Operational experience may reveal failure modes that were not identified during initial design. User behaviour may differ from the assumptions made by developers. The technology may be integrated with other systems in unexpected ways. New cybersecurity threats may emerge. Suppliers may change components or services. Software updates may introduce new interactions. Environmental conditions may change. Therefore, FMEA should not necessarily be treated as a one-time exercise. It can form part of a continuing technology risk-management process in which failure information is periodically reviewed and the analysis is updated.

The analysis can be especially valuable for emerging technologies because historical failure data may be limited. Traditional statistical approaches often benefit from large amounts of historical information. An emerging technology may not have sufficient operational history to establish reliable failure frequencies. FMEA provides a structured qualitative or semi-quantitative method for examining potential failures even when historical evidence is incomplete. However, where assumptions or expert judgments are used, they should be clearly documented. The distinction between observed evidence, estimated frequency, expert judgment, and assumption is important because it prevents a preliminary risk assessment from being mistaken for empirically established evidence.

A conventional FMEA may assign ratings for the severity of a failure, the occurrence or likelihood of the failure, and the detectability of the failure. These ratings can then be combined to support prioritisation of risk-management attention. However, numerical ratings should be interpreted carefully. A numerical score does not automatically represent an objective measurement of risk. Different evaluators may assign different values to severity, occurrence, or detection depending on their assumptions, available evidence, organizational context, and understanding of consequences. Therefore, the rationale for each rating is often more informative than the number itself.

Severity should reflect the consequences associated with the failure if it occurs. In a low-consequence administrative application, a temporary system interruption may be relatively manageable. In a medical, transportation, industrial safety, emergency communication, financial settlement, or critical infrastructure environment, a similar technical failure could have substantially greater consequences. Severity should therefore be assessed in relation to the function of the technology, the stakeholders affected, the duration of disruption, the possibility of harm, and the ability of the organization to recover.

Occurrence concerns how frequently or plausibly a failure may arise. Historical incident data can provide useful evidence where such information exists. However, for new technologies, occurrence may need to be estimated using testing results, supplier information, engineering analysis, expert judgment, analogous technologies, pilot deployments, or scenario analysis. Such estimates should be treated as uncertain rather than presented as precise probabilities when the evidence does not support that level of precision.

Detection concerns the ability of existing controls to identify a failure before it produces significant consequences. A failure that is almost certain to be detected immediately may have a different risk profile from an equally severe failure that can remain undetected for an extended period. Detection is therefore particularly important in autonomous, automated, or highly interconnected systems where human operators may not directly observe every process.

The FMEA process can be integrated into the broader technology evaluation framework by considering the relationship between failure, consequence, control, and organizational response. A simplified conceptual sequence is:

Technology function → possible failure → failure cause → immediate effect → wider consequence → existing control → detection → corrective action → residual risk → monitoring.

This sequence reinforces the distinction between inherent risk and residual risk. Inherent risk refers to the level of risk associated with a technology or process before additional controls are considered. Residual risk refers to the risk that remains after controls have been implemented. A technology may therefore be associated with significant inherent risks but have a substantially different residual-risk profile if effective safeguards, redundancy, monitoring, human oversight, and recovery mechanisms are established.

FMEA can also contribute to resilience analysis. A resilient technology is not necessarily one that never fails. Complex systems may experience failures despite careful design. Resilience concerns the ability to anticipate disruption, absorb or withstand failure, continue essential functions, recover, and adapt. FMEA can therefore identify not only how a technology may fail but also whether the system has sufficient capacity to tolerate the failure. For example, backup systems, redundant communication channels, alternative suppliers, manual operating procedures, independent power supplies, data recovery mechanisms, and emergency response procedures can reduce the consequences of individual failure modes.

This becomes particularly important where technologies contain single points of failure. A single point of failure is a component, service, supplier, network connection, database, or process whose failure could significantly disrupt the wider system. FMEA can help reveal such dependencies. If several critical functions depend upon one external cloud provider, for example, the technology may contain a concentration risk that is not visible when each application is evaluated independently. Similarly, if multiple systems rely on the same communication network or common database, a localized failure could have much broader consequences than initially anticipated.

Human interaction should also be incorporated into FMEA. Users may introduce failure through misunderstanding, incorrect configuration, inappropriate reliance on automated outputs, inadequate training, fatigue, distraction, or deliberate misuse. At the same time, humans can also serve as an important control mechanism. Human review can detect abnormal outputs, override automated decisions, initiate emergency procedures, and provide contextual judgment that may not be available to an automated system. FMEA should therefore examine both human error and human resilience.

The treatment of human error should be particularly careful. It is often insufficient to describe a failure simply as “user error.” Such a description may conceal deeper design or organizational causes. If users repeatedly enter incorrect information because the interface is confusing, the underlying cause may be poor interface design rather than individual negligence. If employees bypass security procedures because the procedures are excessively difficult, the organization may need to examine the design of the control environment. If staff misunderstand an automated recommendation because the system does not communicate uncertainty clearly, the technology itself may contribute to the failure. FMEA can therefore support a more systemic understanding of human-technology interaction.

Data should likewise be treated as a potential source of failure. Modern technologies increasingly depend upon data for operation, decision-making, prediction, monitoring, and automation. Data failure may arise from inaccurate data, incomplete data, outdated data, duplicated records, inconsistent definitions, inappropriate training data, biased datasets, corrupted data, unauthorized modification, or loss of data. In artificial intelligence systems, poor-quality or inappropriate data may result in outputs that appear technically plausible while being unsuitable for the intended context. FMEA can therefore extend beyond software functionality to examine the entire data lifecycle.

Cybersecurity represents another important category. A cybersecurity failure may involve unauthorized access, malware, credential compromise, denial of service, data manipulation, data exfiltration, or exploitation of software vulnerabilities. Cybersecurity failure can also interact with other failure modes. An attacker may compromise data integrity, causing a decision-support system to generate incorrect outputs. A denial-of-service attack may create communication failure. Compromised maintenance credentials may allow an attacker to alter system configurations. FMEA can help identify these relationships and determine where preventive and detective controls should be strengthened.

Maintenance failure should similarly be evaluated across the technology lifecycle. Technologies may deteriorate, become obsolete, require software updates, depend upon replacement components, or become incompatible with surrounding systems. Failure to maintain a system can gradually increase risk even when the technology initially operated effectively. Preventive maintenance schedules, software patching, calibration, component replacement, configuration management, inspection, staff competence, and supplier support can therefore form part of the control environment.

Supply-chain dependencies can also be incorporated into FMEA. A technology may depend on specialized components, semiconductor manufacturers, cloud providers, software libraries, data suppliers, logistics providers, or maintenance contractors. Failure at one supplier may consequently affect the technology even though the organization itself has not experienced an internal technical failure. FMEA can identify these external dependencies and examine whether alternative suppliers, inventory buffers, contractual safeguards, interoperability arrangements, or substitution strategies are available.

FMEA is also useful for comparing alternative technology designs. If two technologies provide similar functionality but have different failure characteristics, the FMEA results can provide evidence for understanding those differences. One design may have fewer potential failure modes but more severe consequences if a failure occurs. Another may have more frequent minor failures but strong detection and recovery mechanisms. The purpose is not simply to count failure modes but to understand their characteristics, controls, consequences, and residual risks in relation to the intended use.

The analysis should also consider the context in which the technology is deployed. A failure that is acceptable in one environment may be unacceptable in another. For example, temporary loss of an entertainment application may cause inconvenience, whereas loss of a communication system during an emergency may interfere with critical response activities. Geographic conditions can also influence failure. Extreme temperatures, flooding, humidity, power instability, connectivity limitations, or other environmental conditions may alter the likelihood and consequences of failure. FMEA should therefore be sensitive to the actual operating environment rather than assuming that laboratory or controlled testing conditions represent all deployment contexts.

Another important consideration is the difference between isolated and cascading failure. An isolated failure affects a limited component or function and does not substantially affect the wider system. A cascading failure occurs when one failure triggers additional failures. For example, a power interruption may disable a communication system, which prevents an automated monitoring system from transmitting an alert, which delays human intervention, which then allows another system failure to develop. FMEA can contribute to identifying these chains, although more complex dependencies may require additional methods such as fault-tree analysis, event-tree analysis, system dynamics, network analysis, or scenario-based stress testing.

FMEA can therefore operate as one component of a broader risk-analysis toolkit. It is particularly useful for structured failure identification and preventive thinking, but it should not be assumed to capture every dimension of complex technology risk on its own. Strategic uncertainty, systemic risk, social consequences, ethical concerns, environmental impacts, organizational transformation, and long-term technological change may require complementary evaluation methods. The strength of FMEA lies in its ability to provide a systematic examination of how specific failures may occur and how those failures can be prevented, detected, contained, and managed.

A further advantage of FMEA is that it encourages organizations to move from reactive to preventive risk management. Reactive risk management often becomes active only after an incident has occurred. By contrast, FMEA asks decision-makers to anticipate possible failure before deployment or before the failure becomes consequential. This can reduce the likelihood of avoidable incidents and can improve preparedness for failures that cannot reasonably be eliminated.

A technology evaluation using FMEA should therefore consider the following sequence:

Technology function → intended performance → potential failure modes → causes → immediate effects → operational consequences → stakeholder consequences → severity → occurrence or likelihood → detectability → existing controls → additional controls → residual risk → monitoring → review.

The results of FMEA should also be connected to organizational decision-making. A failure mode with potentially severe consequences may require redesign, additional testing, stronger controls, greater human oversight, restricted deployment, redundancy, or a different implementation strategy. A lower-consequence failure may be managed through routine monitoring and maintenance. In this way, FMEA does not merely create a technical risk register; it provides evidence that can support decisions about whether, where, and under what conditions a technology should be deployed.

For emerging technologies, the FMEA should also remain adaptable. As evidence accumulates through pilots, user experience, incident reporting, maintenance records, cybersecurity monitoring, customer feedback, and operational performance, previous assumptions can be reassessed. New failure modes may emerge, while some anticipated failure modes may prove less significant than expected. The analysis should therefore be periodically reviewed rather than treated as permanently complete.

The major advantage of FMEA is that it encourages preventive thinking. Instead of asking only why an incident happened after the fact, FMEA asks what could fail, why it could fail, what would happen if it failed, whether the failure could be detected, what controls already exist, and what additional measures could reduce the likelihood or consequences of failure. This makes FMEA particularly valuable within a multidimensional technology evaluation framework because it connects technical reliability with human behaviour, data quality, cybersecurity, maintenance, organizational processes, environmental conditions, supply-chain dependencies, resilience, and governance.

Ultimately, FMEA reinforces the principle that technology risk should be evaluated before, during, and after deployment. A technology should not be considered adequately evaluated simply because it performs its intended function under normal conditions. A comprehensive evaluation should also consider how the technology behaves when components deteriorate, information becomes incomplete, users make mistakes, networks fail, suppliers become unavailable, cyber threats occur, environmental conditions change, maintenance is delayed, or several failures occur simultaneously. FMEA provides a structured way of bringing these questions into the evaluation process and thereby strengthens the preventive, evidence-based, and lifecycle-oriented character of technology risk management.