Part III — Evaluating Risk, Intelligent Technologies, Digital Systems, Biotechnology and Human-Centred Technologies

Chapter 11. Evaluating Digital, Connected and Data-Driven Technologies — Part 2 of 2

11.13 Blockchain and Distributed Ledger Technologies

Blockchain systems can provide distributed records and mechanisms for validating transactions. Distributed ledger technologies represent a broader category of systems in which records are maintained across multiple participating nodes rather than relying exclusively on a single centralized database administrator.

Blockchain has attracted considerable attention because of its potential to support shared records, transaction verification, traceability and automated processes. Applications have been proposed or implemented in areas including financial services, supply-chain management, digital identity, asset management, healthcare information, certification, intellectual property and other forms of digital record keeping.

However, evaluation should not assume that decentralization automatically creates superior performance.

Assessment should examine:

scalability;

energy use;

governance;

transaction speed;

security;

privacy;

interoperability;

legal status;

operational complexity.

The technology should be evaluated against the specific problem rather than its promotional claims (Swan, 2015).

The first question in blockchain evaluation should therefore be whether a distributed ledger is actually necessary for the problem being addressed. A conventional centralized database may provide faster processing, lower costs, simpler governance and easier modification in circumstances where a trusted organization already exists and is responsible for maintaining the records.

The value of blockchain is more closely associated with situations in which multiple parties need to share records while operating with limited trust in one another or where a common, tamper-evident history provides meaningful value. Even in these situations, the characteristics of the participants, governance arrangements, transaction requirements and legal environment must be carefully examined.

Scalability is a major consideration. A blockchain architecture may experience limitations relating to transaction throughput, storage requirements, consensus mechanisms or network participation. A technology that performs adequately during a small pilot may behave differently when transaction volumes increase substantially.

Transaction speed should therefore be evaluated in relation to the intended application. A financial or operational system requiring immediate confirmation may have different requirements from a system recording certificates, ownership histories or supply-chain events where confirmation can occur over a longer period.

Energy use is particularly relevant to certain blockchain architectures. The environmental implications depend on the consensus mechanism, hardware requirements, network size, transaction activity and energy sources involved. It is therefore inappropriate to characterize all blockchain systems as environmentally equivalent. Evaluation should consider the specific technical architecture and its lifecycle resource requirements.

Governance is another critical dimension. Decentralization does not mean the absence of governance. Decisions still need to be made concerning protocol changes, participation, access, software updates, dispute resolution, security incidents and changes to operating rules.

The distribution of governance authority should therefore be examined. A system may be technically distributed while decision-making remains concentrated among a small group of developers, organizations, validators or other participants.

This creates an important distinction between technical decentralization and institutional decentralization. A distributed technical architecture does not necessarily imply that power is equally distributed among all stakeholders.

Security must also be examined at multiple levels. Blockchain records may be designed to be resistant to unauthorized modification, but surrounding components can remain vulnerable. Wallets, private keys, applications, smart contracts, interfaces, exchanges, identity systems and user devices may all introduce risks.

Consequently, the security of the overall blockchain ecosystem should not be inferred solely from the cryptographic properties of the underlying ledger.

Private-key management is particularly important. Loss or compromise of cryptographic credentials can result in loss of control over digital assets or permissions. Evaluation should therefore examine key generation, storage, recovery, access control and incident procedures.

Smart contracts introduce another evaluation dimension. Automated contractual logic can execute actions according to programmed conditions, but software errors can produce unintended outcomes. The use of smart contracts therefore requires appropriate testing, auditing, monitoring and mechanisms for responding to discovered vulnerabilities.

Privacy also requires careful consideration. The fact that transactions are recorded on a distributed ledger does not automatically mean that the information is appropriate for public disclosure. Some systems may expose transaction metadata or permit information to remain accessible for long periods.

This creates a potential tension between transparency and privacy. A system designed to provide permanent traceability may create difficulties when information should later be corrected, restricted or removed.

Interoperability is another important issue. Blockchain systems may need to communicate with conventional databases, cloud systems, payment systems, IoT devices and other distributed ledgers. Without suitable interoperability mechanisms, organizations may create isolated technological environments rather than genuinely integrated ecosystems.

Legal status should also be examined because the enforceability and regulatory treatment of blockchain-based records, digital assets, smart contracts and decentralized arrangements can differ across jurisdictions and applications. Organizations operating internationally should therefore assess the applicable legal and regulatory environment rather than assuming that technical functionality establishes legal recognition.

Operational complexity can also reduce the practical value of blockchain. Organizations may require specialized technical expertise, cryptographic key management, governance procedures, infrastructure and monitoring capabilities. These requirements should be included in total cost and risk assessments.

Blockchain evaluation should also consider data quality. A blockchain can provide a reliable record of information that has been entered into the system, but it cannot automatically establish that the original information was true. This is sometimes described as the distinction between the integrity of the record and the truth of the underlying event.

For example, if an incorrect quantity is entered into a supply-chain ledger, the system may preserve that incorrect entry accurately. Distributed recording therefore does not eliminate the need for trustworthy data collection, verification and governance.

The relationship between physical and digital assets creates an additional challenge. A blockchain may record that a particular physical product was entered into a system, but the ledger alone does not necessarily establish that the physical product corresponds to the digital record. Reliable links between physical events and digital records may require sensors, human verification, trusted intermediaries or other controls.

Lifecycle considerations should also be examined. Organizations should assess how the blockchain system will be maintained, upgraded, governed and eventually replaced or retired. A technology with permanent records may create long-term storage and governance obligations.

A useful evaluation sequence is:

Problem definition → justification for distributed architecture → participant structure → consensus mechanism → data architecture → transaction requirements → security → privacy → governance → interoperability → legal environment → operational cost → lifecycle management → resilience → retirement or transition.

The broader principle is that blockchain should be evaluated as one technological architecture among several alternatives. Its value depends on whether its particular characteristics solve a genuine organizational or societal problem better than feasible alternatives while maintaining acceptable levels of performance, security, privacy, governance, cost and sustainability.

11.14 Smart Cities

Smart-city technologies combine sensors, communications, analytics and public infrastructure.

Applications may include:

traffic management;

energy management;

waste management;

public safety;

environmental monitoring.

However, smart-city evaluation must consider privacy, surveillance, digital exclusion and governance.

Technology should not be evaluated solely on its ability to collect data.

Smart-city systems represent one of the clearest examples of technology operating as an interconnected socio-technical infrastructure. A smart city may integrate transportation systems, energy networks, telecommunications, environmental sensors, public services, emergency systems, buildings, payment platforms and citizen-facing applications.

The purpose of smart-city technology should therefore be defined before the technology itself is evaluated. The mere availability of sensors and data does not establish that a city has become more efficient, sustainable, inclusive or resilient.

Traffic management provides an example. Sensors and cameras may provide information about traffic flows, while analytics may identify congestion patterns and optimize traffic signals. However, evaluation should examine whether these changes actually reduce travel time, fuel consumption, emissions and congestion rather than merely increasing the amount of information available to city authorities.

Energy management provides another example. Smart meters and connected infrastructure can improve visibility of electricity consumption and support demand management. Yet the evaluation should consider whether these capabilities produce measurable improvements in efficiency, affordability, reliability and sustainability.

Waste-management technologies may use sensors to determine when containers require collection. Such systems may reduce unnecessary collection journeys, but evaluation should consider equipment costs, maintenance, connectivity, data quality and whether the resulting operational changes actually improve waste-management outcomes.

Public safety applications require particularly careful evaluation because they may involve extensive surveillance, facial recognition, location tracking or behavioral analysis. The potential operational benefits must be considered alongside privacy, proportionality, accuracy, discrimination risks, governance and mechanisms for accountability.

Digital exclusion is another important consideration. Smart-city services may assume access to smartphones, broadband connectivity, digital identification or digital payment systems. Residents who lack these resources may experience reduced access to public services.

This creates an important distinction between digital availability and social accessibility. A service may be technically available while remaining practically inaccessible to particular populations.

Smart-city evaluation should therefore consider whether technology complements existing services or unintentionally creates barriers to participation.

Governance is also central. City governments may depend on technology providers to collect, process and analyze information about residents and infrastructure. This raises questions concerning ownership, access, data use, security, retention, procurement, accountability and long-term dependency.

Public-sector technology procurement can create substantial long-term commitments. A city may invest heavily in a proprietary platform and subsequently find it difficult to replace the provider. Interoperability and exit planning should therefore be incorporated into smart-city architecture.

Smart-city systems also create cybersecurity risks because critical infrastructure becomes increasingly connected. A vulnerability in one system may affect transportation, energy, communications or emergency services.

Systemic risk should therefore be included in evaluation. The evaluator should identify dependencies among infrastructures and examine what could happen if several systems fail simultaneously.

Environmental implications should also be considered. Smart-city technologies may reduce resource consumption through better monitoring and optimization, but they also require sensors, communication infrastructure, data centres, devices and energy. Environmental benefits should therefore be assessed through lifecycle analysis rather than assumed.

The social distribution of benefits should also be examined. Improvements in transportation, energy management or public services may benefit some neighborhoods more than others. Technology evaluation should therefore examine geographic and demographic distribution rather than relying only on city-wide averages.

Smart-city projects should also include meaningful mechanisms for public accountability. Residents may need information about what data are collected, why they are collected, how automated systems are used and how decisions can be challenged.

The evaluation process can therefore be understood as:

Urban problem → stakeholder needs → alternative solutions → technology selection → infrastructure integration → data collection → analysis → public service → social and environmental outcomes → monitoring → governance → reassessment.

The central principle is that a smart city should not be defined by the quantity of technology it deploys. The relevant question is whether technology improves legitimate public objectives while maintaining privacy, inclusion, security, accountability, affordability, sustainability and democratic governance.

11.15 Connected Vehicles

Connected and autonomous vehicles create a particularly strong interaction between physical and digital systems.

Evaluation must include:

safety;

cybersecurity;

communications;

sensor reliability;

human interaction;

infrastructure;

liability;

emergency procedures.

Connected vehicles can exchange information with other vehicles, road infrastructure, cloud services, navigation systems and external platforms. Autonomous systems may additionally use sensors, artificial intelligence, high-definition maps and real-time computational systems to perceive and respond to their environment.

This creates a complex cyber-physical system in which software decisions can produce immediate physical consequences.

Safety must therefore remain a central evaluation dimension. Evaluation should consider how the vehicle responds to unexpected road conditions, sensor failures, communication interruptions, software errors, adverse weather, road construction and unusual human behavior.

Sensor reliability is particularly important. Cameras, radar, lidar, GPS and other sensors may experience limitations under particular environmental conditions. Evaluation should therefore consider sensor uncertainty, redundancy, calibration and the system's ability to recognize when information is unreliable.

Connected vehicles also depend on communication systems. Vehicles may exchange information with infrastructure or other vehicles, but communication cannot be assumed to be continuously available. A resilient design should consider what happens when communication is delayed, corrupted, unavailable or deliberately disrupted.

Cybersecurity is another major concern because connected vehicles may provide multiple pathways for unauthorized access. Evaluation should include authentication, secure communications, software updates, access control, vulnerability management, intrusion detection and incident response.

Software updates introduce their own risks and benefits. Updating software may correct vulnerabilities and improve functionality, but an improperly implemented update could create new problems. Evaluation should therefore consider update validation, rollback capability, version control and monitoring.

Human interaction remains important even in highly automated vehicles. Drivers may be required to supervise automated systems, respond to warnings or take control under certain conditions. Evaluation should therefore examine whether users understand system limitations and whether sufficient time and information are available for appropriate intervention.

Automation can create challenges if human users become excessively dependent on system outputs. Human oversight must therefore be evaluated in terms of actual capability rather than merely the presence of a human in the system.

Infrastructure is also significant. Connected and autonomous vehicles may depend on road markings, digital maps, traffic signals, communication networks, charging infrastructure and maintenance systems. Vehicle capability should therefore be evaluated in relation to the infrastructure environment in which the technology will operate.

Liability becomes more complex when decisions are distributed among vehicle manufacturers, software developers, infrastructure providers, owners, operators and users. Evaluation should therefore identify responsibility for system design, maintenance, software updates, monitoring and incident response.

Emergency procedures are particularly important. A vehicle should have appropriate responses to situations involving system failure, communication loss, sensor malfunction, unexpected obstacles or medical emergencies.

Connected vehicles also raise privacy considerations because they can generate detailed information about location, travel patterns, driving behavior and vehicle use. Data governance should therefore address collection, retention, access, sharing and secondary use.

Environmental evaluation should also be considered. Connected and autonomous technologies may influence fuel consumption, traffic efficiency, vehicle utilization and transportation patterns. However, additional sensors, processors, communication systems and computing infrastructure also have resource and energy requirements.

Evaluation should therefore examine both direct and indirect effects.

A useful sequence is:

Vehicle purpose → operating environment → sensor capability → perception → communication → decision-making → human interaction → physical action → incident response → monitoring → software updating → lifecycle review.

The central principle is that connected vehicles cannot be evaluated solely as automobiles or solely as software systems. They must be evaluated as integrated cyber-physical systems operating within transportation ecosystems.

11.16 Digital Resilience

Digital resilience is the ability of an organization or system to continue operating, recover and adapt during disruption.

Resilience should be evaluated through:

redundancy;

backups;

disaster recovery;

incident response;

alternative suppliers;

offline procedures;

staff capability.

Digital resilience extends beyond cybersecurity. A system can be secure against known cyber threats and still fail because of power interruptions, telecommunications failure, software defects, natural disasters, supplier disruption, hardware failure, human error or unexpected changes in operating conditions.

Resilience therefore concerns the ability to absorb disruption without losing essential functions, recover critical operations within an acceptable period and learn from the disruption to reduce future vulnerability.

Redundancy can support resilience by providing alternative components or pathways. However, redundancy has costs and should be evaluated according to the importance of the function being protected. Critical systems may require greater redundancy than systems where temporary interruption has limited consequences.

Backups are similarly important, but the existence of backups does not guarantee recoverability. Organizations should test whether backups are complete, accessible, sufficiently recent and protected from the same threats that could affect the primary system.

Disaster recovery should therefore be tested rather than assumed. Recovery exercises can reveal dependencies, inadequate documentation, insufficient staff capability and unrealistic recovery assumptions.

Recovery time objectives and recovery point requirements can help organizations determine how quickly systems must be restored and how much information loss is acceptable. These requirements should be linked to the consequences of disruption.

Incident response is another important element. Organizations should know how incidents will be detected, classified, contained, investigated, communicated and resolved.

Alternative suppliers can reduce dependency, particularly where a single external provider supports a critical function. However, supplier diversification may introduce additional integration and management costs. The appropriate level of diversification should therefore depend on criticality and consequence.

Offline procedures can be particularly important for systems that normally depend heavily on digital infrastructure. Organizations should determine whether essential functions can continue temporarily when networks, cloud services or digital applications are unavailable.

Staff capability is also central to resilience. Technology cannot provide resilience if employees do not know how to respond when systems fail. Training, role clarity, communication procedures and decision authority should therefore be incorporated into resilience evaluation.

Digital resilience should also include adaptability. Recovery should not simply restore the previous state if that state contains vulnerabilities that contributed to the disruption. Lessons from incidents should feed into system redesign, governance improvement and future risk assessment.

A useful resilience lifecycle is:

Prepare → protect → detect → respond → continue → recover → learn → adapt.

Resilience should also be evaluated at several levels. An individual application may be resilient while the organization remains vulnerable because it depends on an unavailable external service. Similarly, an organization may be resilient while its broader supply chain remains fragile.

This demonstrates the relationship between organizational resilience and systemic risk.

Digital resilience should also be evaluated under stress scenarios. Organizations can examine what would happen during prolonged cloud outages, cyber incidents, telecommunications failures, power interruptions, extreme weather events, supplier failures or simultaneous disruptions.

The purpose of scenario analysis is not to predict exactly what will happen but to identify vulnerabilities and improve preparedness.

Environmental and geographic factors may also affect digital resilience. Data centres, communication infrastructure and physical computing systems can be affected by flooding, extreme temperatures, storms, water shortages and other environmental conditions.

Resilience therefore connects digital technology with physical infrastructure and climate conditions.

An important distinction should also be made between resilience and redundancy. Redundancy is one mechanism for improving resilience, but resilience also depends on adaptability, organizational learning, governance, human capability and recovery processes.

The central evaluation question is therefore not simply whether a system can prevent failure, but whether it can continue essential operations, recover from disruption and adapt after failure occurs.

11.17 Data Sovereignty

Data sovereignty concerns the legal and governance implications of where data are stored, processed and transferred.

Organizations operating internationally must consider:

jurisdiction;

privacy law;

contractual obligations;

cross-border transfer;

government access;

localization requirements.

Data sovereignty becomes increasingly important as cloud computing, international data centres, digital platforms and global service providers allow information to move across geographical boundaries.

The physical location of a server does not necessarily provide the complete picture of data governance. Data may be collected in one country, processed in another, stored in a third jurisdiction and backed up in several additional locations.

Organizations must therefore understand the complete data flow.

Jurisdiction is important because different legal systems may impose different requirements concerning privacy, security, retention, disclosure, access and government authority.

Privacy law is particularly relevant when personal information crosses borders. Organizations should determine whether data transfers are legally permitted and what safeguards are required.

Contractual obligations may impose additional requirements. A customer, government agency, research institution or business partner may require information to remain within a specified jurisdiction or may impose restrictions on secondary processing.

Cross-border transfer therefore requires both legal and operational evaluation. An organization should know not only where information is currently stored but also where it could be transferred through backups, disaster recovery, analytics or third-party services.

Government access is another consideration. Different jurisdictions may provide authorities with different legal mechanisms for obtaining access to data. Organizations operating internationally should understand the implications of such arrangements for confidentiality and regulatory compliance.

Localization requirements may require certain categories of information to be stored or processed within specified geographical boundaries. Compliance with such requirements may affect cloud architecture, vendor selection and system design.

Data sovereignty also has implications for resilience. Restricting all data to a single geographical location may simplify governance in some circumstances but can create concentration risk if that location becomes unavailable.

Conversely, distributing information across multiple jurisdictions may improve resilience while increasing legal and governance complexity.

The appropriate architecture therefore requires balancing resilience, privacy, legal requirements, security and operational needs.

Data sovereignty should also be considered in relation to data classification. Not every category of information requires the same level of geographical control. Public information, confidential corporate information, personal information, financial information and highly sensitive information may require different governance arrangements.

Data provenance and data lineage are therefore important. Organizations should be able to identify where information originated, where it has travelled, who has processed it and where copies may exist.

Deletion and retention requirements can also become more complex in international environments. An organization may be required to retain certain information for regulatory purposes while simultaneously facing obligations concerning deletion or limitation of personal information.

Data sovereignty therefore connects closely with data governance, cybersecurity, privacy, cloud computing, contractual management and regulatory compliance.

A useful sequence is:

Data classification → data origin → processing requirements → storage location → transfer pathways → applicable jurisdictions → contractual controls → access rights → retention → deletion or archival → monitoring.

The broader principle is that data location is not merely an infrastructure decision. It can influence legal obligations, organizational control, privacy, security, resilience and strategic risk.

11.18 Digital Dependency

Modern organizations can become highly dependent upon digital infrastructures.

A technology evaluation should therefore ask:

What happens if this technology becomes unavailable?

The answer should include both short-term operational consequences and long-term strategic consequences.

Digital dependency occurs when organizational activities become reliant on particular technologies, platforms, networks, data sources, vendors or digital infrastructures. Some dependency is unavoidable in modern organizations. The objective of evaluation is therefore not necessarily to eliminate dependency but to understand its nature, magnitude and consequences.

Short-term consequences may include interruption of communication, inability to access information, disruption of transactions, loss of operational visibility or reduced employee productivity.

Long-term consequences may be more significant. Prolonged dependence on a particular platform may affect strategic flexibility, bargaining power, innovation options, data control and organizational autonomy.

Dependency can occur at multiple levels. An organization may depend on a cloud provider, telecommunications network, software vendor, payment platform, identity service, API, data provider or specialized hardware supplier.

Dependencies may also be hidden. An organization may believe that it operates independently while several of its systems rely on the same underlying cloud infrastructure or external service provider.

Dependency mapping is therefore an important component of technology evaluation. Organizations should identify direct and indirect dependencies and determine which dependencies are critical.

A useful distinction can be made between replaceable and difficult-to-replace dependencies. A service may be technically replaceable but operationally difficult to replace because migration would require significant time, cost or organizational change.

Critical dependencies should be evaluated according to the consequences of failure. A technology supporting a non-critical administrative process may tolerate substantial downtime, whereas a technology supporting emergency response, healthcare, financial transactions or essential infrastructure may require very high resilience.

Digital dependency should also be examined in relation to concentration. If multiple organizational functions depend on the same provider, a single provider-level failure can produce multiple simultaneous disruptions.

This creates a systemic dependency.

Interdependency can be even more complex. Organization A may depend on a cloud provider, while the cloud provider depends on telecommunications infrastructure, electricity providers, hardware manufacturers and software suppliers. A disruption in one part of this chain may therefore propagate through several layers.

Supply-chain evaluation should consequently include digital dependencies as well as physical suppliers.

Dependency can also arise through data. An AI system may depend on a particular dataset. A research organization may depend on an external data provider. A financial system may depend on external market information. If the data source becomes unavailable or changes its structure, the organization's operations may be affected.

Human expertise can also become a form of dependency. If only a small number of employees understand a critical system, the organization may become vulnerable when those individuals leave, become unavailable or move to other responsibilities.

Digital dependency therefore has both technological and organizational dimensions.

The evaluation should also consider the possibility of deliberate or accidental service withdrawal. A provider may discontinue a product, change pricing, alter technical architecture, restrict access or impose new contractual conditions.

Organizations should therefore maintain sufficient awareness of strategic alternatives.

Digital dependency can sometimes create positive value. Specialization allows organizations to use capabilities that would be difficult or expensive to develop internally. Cloud computing, specialized analytics, cybersecurity services and digital platforms can provide significant benefits.

The issue is therefore not dependency itself but uncontrolled dependency.

A useful evaluation sequence is:

Technology function → dependency identification → criticality assessment → consequence analysis → concentration assessment → alternative identification → resilience measures → monitoring → contingency planning → exit or substitution capability.

Digital dependency also connects directly with vendor lock-in. Vendor lock-in represents one mechanism through which dependency becomes difficult to reverse. However, dependency can exist even when no formal lock-in exists.

Digital dependency also connects with interoperability. Greater interoperability and open standards may increase the ability to replace components, although they do not eliminate all switching costs.

Resilience is another important dimension. Organizations should determine whether critical functions can continue during temporary technology unavailability and whether alternative procedures can be activated.

The evaluation should also consider the human dimension. Excessive dependence on automated systems can reduce human familiarity with manual processes. Over time, employees may lose the knowledge required to operate without the technology. This creates a potential form of capability dependency.

Consequently, organizations should maintain appropriate levels of human understanding and contingency capability for critical systems.

Digital dependency should also be considered dynamically. A dependency that is relatively minor at the beginning of implementation may become critical as more organizational functions are integrated with the technology.

For this reason, technology evaluation should not end at procurement or deployment. Dependency should be reassessed throughout the technology lifecycle.

The central principle is that technological convenience should not obscure strategic dependence. A technology may provide substantial efficiency and functionality while simultaneously becoming essential to the organization's continued operation.

The final question is therefore broader than "Does the technology work?"

It should also ask:

What other systems depend upon it?

What does it depend upon?

Who controls those dependencies?

What happens if one dependency fails?

How quickly can essential operations recover?

Can the organization operate temporarily without it?

Can the technology be replaced?

How much would replacement cost?

Does the organization retain sufficient knowledge and capability to manage the transition?

These questions place digital dependency within the broader multidimensional evaluation framework. Technology creates value not only when it performs effectively under normal conditions, but also when the organization retains sufficient control, resilience and strategic flexibility when conditions change.

11.19 Digital Technology Evaluation Matrix

11.20 Case Study: Digital Healthcare Platform

A national healthcare platform connects hospitals, laboratories, pharmacies and patients.

Potential benefits include:

faster information exchange;

reduced duplication;

improved continuity of care;

better public-health monitoring.

Potential risks include:

privacy breaches;

cyberattacks;

incorrect records;

system outages;

digital exclusion;

vendor dependency.

The AMTEF evaluation should therefore examine the entire healthcare ecosystem.

A national digital healthcare platform provides a useful example of why interconnected technology cannot be evaluated solely by examining the performance of individual applications. The platform may include hospital information systems, laboratory information systems, pharmacy systems, patient portals, electronic health records, diagnostic services, insurance or payment systems, public-health databases and communication infrastructure. Each component may perform satisfactorily in isolation while weaknesses in the relationships among components create significant risks.

The primary purpose of such a platform should be clearly established before technological performance is assessed. The purpose may include improving continuity of care, reducing duplication of diagnostic tests, increasing access to patient information, improving public-health surveillance, supporting clinical decision-making or increasing administrative efficiency. Each objective creates different evaluation requirements.

Faster information exchange can potentially improve healthcare coordination. A clinician may be able to access laboratory results or previous treatment information without waiting for paper records or separate communication between institutions. However, faster information exchange is beneficial only when the information is accurate, appropriately interpreted and available to authorized users.

Reduced duplication is another potential benefit. If healthcare providers can access reliable records, patients may not need to repeat diagnostic tests unnecessarily. This may reduce costs, inconvenience and resource consumption. However, inaccurate or incomplete records can create the opposite effect by causing clinicians to distrust the digital system or repeat procedures.

Improved continuity of care is particularly important when patients move between hospitals, clinics, laboratories and pharmacies. The platform should therefore be evaluated according to whether relevant information follows the patient appropriately while maintaining necessary privacy and security protections.

Public-health monitoring may also benefit from integrated data. Aggregated information can potentially help identify disease patterns, resource requirements or emerging public-health conditions. However, public-health analytics depend heavily on data quality, timeliness, representativeness and appropriate interpretation.

Privacy breaches represent a major risk because healthcare information can be highly sensitive. Evaluation should therefore examine identity management, authentication, authorization, encryption, access monitoring, data minimization, retention and incident response.

The principle of least privilege is particularly important. Healthcare personnel should receive access appropriate to their legitimate responsibilities rather than unrestricted access to all available information.

Auditability is also important. The platform should provide sufficient records to establish who accessed information, what information was accessed, what changes were made and when those activities occurred. Auditability can support both security and accountability.

Cybersecurity should be evaluated at the ecosystem level. Hospitals, laboratories, pharmacies and other connected organizations may have different cybersecurity capabilities. A vulnerability in one participating institution could potentially create pathways into shared systems.

Incorrect records create another major risk. A digital platform may increase the speed at which information is distributed, but speed can amplify the consequences of incorrect information. An inaccurate medication record, allergy record or laboratory result could potentially influence subsequent decisions.

Data quality should therefore be evaluated before assuming that integration improves healthcare outcomes.

System outages represent another important concern. Healthcare services may need to continue even when the digital platform is unavailable. Evaluation should therefore examine redundancy, backup systems, disaster recovery, alternative communication procedures and offline processes.

The appropriate level of resilience depends on clinical criticality. A temporary outage affecting a non-critical administrative function may have relatively limited consequences, whereas an outage affecting emergency or medication-related information may create much more serious consequences.

Digital exclusion should also be considered. Patients may have different levels of digital literacy, access to smartphones, internet connectivity, language capability, disability access and confidence in digital services. A platform that assumes universal digital access may unintentionally disadvantage some populations.

Digital healthcare should therefore complement rather than unnecessarily eliminate appropriate non-digital pathways.

Vendor dependency is another strategic issue. A national healthcare platform may depend on one or several external technology providers. Evaluation should examine contractual arrangements, data portability, interoperability, migration capability, service continuity and the ability to change providers if necessary.

Interoperability is especially important because hospitals, laboratories and pharmacies may use different information systems. The platform must allow information to be exchanged without creating ambiguity concerning terminology, identifiers, dates, units or clinical meanings.

The human dimension is equally important. Healthcare professionals must understand what information the system provides, what its limitations are and how they should respond when information appears incomplete or contradictory. Technology should support professional judgment rather than create inappropriate automation or excessive dependence.

The platform may also incorporate artificial intelligence for diagnosis support, risk prediction, triage or administrative decision-making. If so, the AI components should be evaluated separately and as part of the broader healthcare ecosystem. Technical accuracy alone would not establish clinical appropriateness.

Legal and regulatory requirements should also be integrated into the evaluation. The platform may be subject to requirements relating to privacy, health information, professional responsibility, cybersecurity, data retention, research use and cross-border data transfer.

Environmental considerations may also be relevant at national scale. Data centres, network infrastructure, connected devices and electronic equipment consume energy and require physical resources. A national platform should therefore consider infrastructure efficiency and lifecycle sustainability.

The evaluation should also consider stakeholder consequences. Patients, doctors, nurses, pharmacists, laboratory personnel, hospital administrators, policymakers, technology providers and regulators may experience different benefits and risks.

A useful AMTEF sequence for this case is:

Healthcare objective → stakeholder requirements → alternative approaches → system architecture → interoperability → data quality → privacy → cybersecurity → clinical and operational performance → human interaction → accessibility → governance → resilience → vendor dependency → monitoring → incident management → continuous improvement.

The case demonstrates that successful digital healthcare cannot be established simply by showing that information moves faster. The more important question is whether the complete ecosystem produces better, safer, more equitable and more reliable healthcare processes while maintaining appropriate privacy, security, accountability and resilience.

The case also illustrates the distinction between capability and outcome. The platform may possess the technical capability to exchange information instantly, but this does not automatically establish that healthcare outcomes improve. The pathway from technological capability to societal benefit must be demonstrated through evidence.

11.21 Case Study: Smart Agriculture

IoT sensors monitor:

soil moisture;

temperature;

rainfall;

crop conditions.

Analytics recommend irrigation.

Technical evaluation may demonstrate improved water management.

However, broader evaluation should examine:

farmer skills;

connectivity;

equipment costs;

data ownership;

vendor dependency;

maintenance;

environmental effects.

Technology adoption depends not only on technical performance but also on organizational and social conditions.

Smart agriculture provides an important example of technology operating within a complex environmental, economic and social system. Agricultural production is influenced by soil conditions, rainfall, temperature, crop type, farm size, labor availability, market conditions, infrastructure, farmer knowledge and access to financial resources. Digital technology therefore operates within a broader system that cannot be reduced to sensor performance.

The purpose of the technology should first be defined. The objective might be reducing water consumption, improving crop yields, reducing fertilizer use, identifying disease earlier, improving resource efficiency or increasing farmer income. Different objectives require different measures of success.

Soil-moisture sensors may provide valuable information concerning irrigation requirements. However, sensor accuracy depends on calibration, placement, soil characteristics, environmental conditions and maintenance.

Temperature and rainfall information may similarly support agricultural decision-making, but local measurements may need to be interpreted alongside broader weather patterns and forecasts.

Crop-condition monitoring may involve sensors, cameras, satellite imagery or artificial intelligence. Such systems can potentially identify stress or disease, but their reliability may vary according to crop type, growth stage, environmental conditions and image quality.

Analytics can transform data into irrigation recommendations. However, a recommendation is not automatically an appropriate action. Farmers may need to consider water availability, labor, electricity costs, crop value, weather forecasts and market conditions before implementing the recommendation.

This demonstrates the importance of human-AI and human-technology interaction. Technology should support agricultural decision-making while recognizing the knowledge and contextual expertise of farmers.

Farmer skills are therefore central to evaluation. A technically sophisticated system may have limited value if users do not understand how to interpret its information, identify sensor errors or respond to system warnings.

Digital literacy should therefore be evaluated alongside technical capability.

Connectivity can also determine whether smart agriculture is practically feasible. Rural agricultural areas may experience limited broadband or mobile connectivity. A system that depends continuously on cloud connectivity may therefore be unreliable in locations where network access is intermittent.

Edge or offline capabilities may provide alternative approaches. Some information could potentially be processed locally and synchronized when connectivity becomes available.

Equipment costs must also be evaluated. Sensors, gateways, communication equipment, software subscriptions, maintenance and replacement components may create substantial costs, particularly for small-scale farmers.

The economic evaluation should therefore examine total cost of ownership rather than the purchase price alone.

Data ownership is another important issue. Agricultural technologies can generate detailed information concerning farm operations, crop performance, land conditions and production practices. Farmers should understand who owns or controls this information and how it may be used.

Data may potentially be valuable to technology providers, agricultural suppliers, financial institutions, insurers or other organizations. Appropriate governance is therefore required to prevent unintended or unauthorized secondary uses.

Vendor dependency can develop when sensors, software and analytics are available only through one provider. If replacement devices or services are difficult to obtain, the farmer may become dependent on the original supplier.

Interoperability can reduce some of this dependency by allowing data and devices to work across different platforms.

Maintenance is another critical factor. Agricultural environments can expose equipment to dust, moisture, heat, mechanical damage and other physical stresses. Sensor calibration and replacement may therefore be required regularly.

A system that performs well under laboratory conditions may perform differently in actual agricultural environments. Field testing should therefore be an essential part of evaluation.

Environmental effects must also be considered. If smart irrigation reduces unnecessary water use, it may contribute to resource efficiency. However, the technology itself requires electronic equipment, batteries, communication infrastructure and replacement components.

Lifecycle environmental assessment should therefore examine the complete system rather than assuming that digital agriculture is automatically sustainable.

There may also be rebound effects. If improved irrigation efficiency reduces the cost of water use, farmers may increase production or expand cultivated areas, potentially changing the overall resource demand. Such effects should be considered when evaluating sustainability outcomes.

Distributional effects are also important. Larger commercial farms may be better positioned to afford sophisticated digital systems than small-scale farmers. Technology adoption could therefore increase productivity while simultaneously widening differences in technological capability if access is unequal.

The evaluation should consider whether appropriate financing, training, shared infrastructure or cooperative models can improve accessibility.

The smart-agriculture evaluation sequence can be represented as:

Agricultural problem → farmer requirements → environmental context → technology selection → sensing → data quality → connectivity → analytics → recommendation → farmer decision → agricultural outcome → environmental outcome → economic outcome → monitoring → adaptation.

This case demonstrates that technology adoption depends not only on technical performance but also on organizational and social conditions.

A sensor that accurately measures soil moisture does not automatically produce improved agricultural outcomes. The farmer must have access to the information, understand it, trust it, have the resources to act upon it and operate within an environment in which the recommended intervention is feasible.

The AMTEF approach therefore evaluates the entire pathway from technological capability to practical and sustainable agricultural outcomes.

11.22 Data Governance Framework

A digital technology evaluation should establish:

Data ownership

Who has legitimate authority over the data?

Data stewardship

Who maintains quality and integrity?

Data access

Who can use the information?

Data security

How are data protected?

Data retention

How long are data retained?

Data deletion

How are data securely removed?

Data accountability

Who is responsible when errors occur?

Data governance provides the organizational framework through which information is controlled, maintained, protected and used throughout its lifecycle. As organizations become increasingly dependent on interconnected technologies, data governance becomes inseparable from technology evaluation.

Data ownership should be clearly distinguished from data stewardship and data access. Ownership may involve legal, contractual or organizational rights, whereas stewardship concerns the operational responsibility for maintaining data quality and integrity. Access concerns who is authorized to use information and under what circumstances.

The question of ownership can become complicated in digital ecosystems. Information may be generated by a customer, collected through an organization's system, processed by a cloud provider and analyzed by a third-party service. Several parties may therefore have legitimate interests in the information.

Technology evaluation should identify these relationships before deployment.

Data stewardship concerns the practical management of information. Someone must be responsible for ensuring that data remain accurate, complete, consistent and appropriately maintained. Without identifiable stewardship responsibilities, data-quality problems can persist across interconnected systems.

Data access should be based on legitimate purpose and appropriate authorization. Not everyone who can technically access information should necessarily be permitted to use it.

Role-based or attribute-based access arrangements can help align access with responsibilities. Access rights should also be reviewed when employees change roles or leave an organization.

Data security encompasses both technical and organizational safeguards. Encryption, authentication, authorization, monitoring, network security and backup systems should operate alongside policies, training, procedures and accountability.

Data retention should be determined according to legal, operational, research, historical and business requirements. Retaining information indefinitely can increase storage costs, cybersecurity exposure and privacy risk. At the same time, premature deletion may prevent legitimate operational, legal or research activities.

Retention decisions should therefore be purpose-driven and documented.

Data deletion requires particular attention in interconnected environments because information may exist in multiple databases, backups, analytical environments, cloud systems and replicated systems. Organizations should understand where copies exist and whether deletion requirements can be implemented effectively.

Data accountability is necessary when errors occur. If incorrect information leads to an inappropriate decision, responsibility should not disappear simply because several systems and organizations were involved.

Accountability should therefore be distributed according to identifiable roles and decision authority.

Data governance should also address data lineage. Organizations should be able to trace important information from its origin through transformation, storage, analysis and use. This becomes particularly important when data support AI systems or high-consequence decisions.

Data quality should be incorporated into governance rather than treated as a separate technical activity. Governance should establish who defines quality requirements, who measures them, what thresholds apply and what happens when requirements are not met.

Data classification is another important component. Information should be categorized according to sensitivity and use so that appropriate security, access, retention and handling requirements can be established.

Data governance should also address data sharing. When information moves between organizations, the receiving organization should understand the permitted purpose, security requirements, retention expectations and accountability arrangements.

Cross-border data transfers require additional consideration because legal and regulatory requirements may differ between jurisdictions.

AI creates further data-governance requirements. Training data, validation data, operational data, prompts, generated outputs and feedback information may all form part of an AI lifecycle. Organizations should determine how these different categories are governed.

Data drift should also be monitored. Information that was representative when a system was developed may become less representative over time. Governance should therefore include periodic reassessment.

A mature data governance framework can be understood as:

Data creation → collection → classification → ownership → stewardship → validation → access → processing → sharing → monitoring → retention → correction → archival or deletion → accountability.

The framework should also be proportional to risk. Not all data require identical governance controls. Publicly available information may require different controls from sensitive personal, financial, healthcare or security-related information.

The AMTEF perspective therefore treats data governance as an enabling condition for responsible technology. Good governance supports reliable analytics, trustworthy AI, cybersecurity, privacy, regulatory compliance and organizational decision-making.

The deeper principle is that data should never be treated simply as an organizational asset without considering the responsibilities associated with collecting, controlling and using it. Data value and data responsibility exist simultaneously.

11.23 Evaluating Digital Ecosystems

A digital ecosystem should be evaluated at three levels:

Level 1: Component

Does the individual technology work?

Level 2: Organizational system

Does it integrate into organizational processes?

Level 3: Ecosystem

How does it affect suppliers, customers, regulators, communities and other systems?

This three-level approach prevents the common mistake of evaluating a component in isolation.

The first level focuses on the technical component. Evaluation may examine functionality, performance, reliability, security, usability, accuracy and compliance with specified requirements. This level is necessary because an individual technology must provide the capabilities for which it was designed.

However, component-level performance is insufficient to establish overall technological value. A technically excellent component may provide little organizational value if it does not integrate effectively with existing systems or processes.

The second level therefore evaluates the technology as part of the organizational system. This includes integration with workflows, databases, people, policies, management structures, suppliers and decision-making processes.

At this level, evaluators should consider whether employees can use the technology effectively, whether processes need to change, whether new skills are required and whether organizational responsibilities are clearly defined.

A technology may therefore pass component-level testing while failing organizational-level evaluation.

For example, a cloud-based application may demonstrate excellent technical performance but require extensive manual data entry because it cannot communicate with the organization's existing systems. The component performs well, but the organizational system may experience limited benefit.

The third level examines the wider ecosystem. Digital technologies rarely operate within organizational boundaries. Suppliers, customers, regulators, partners, communities, infrastructure providers and other technological systems may all be affected.

At this level, evaluation should consider interoperability, network effects, systemic dependency, supply-chain exposure, regulatory consequences, social impacts, environmental effects and resilience.

The ecosystem perspective is particularly important where technologies are highly interconnected. A failure in one organization can sometimes affect other organizations through shared platforms, cloud services, communication networks or data systems.

The three levels should not be treated as independent stages that are completed once. They should be evaluated iteratively because changes at one level can affect the others.

For example, increasing the technical capability of an AI system may require changes to organizational processes. Those organizational changes may subsequently affect customers, suppliers and regulators. New ecosystem requirements may then require modifications to the technology itself.

The relationship can therefore be represented as:

Component capability → organizational integration → ecosystem interaction → stakeholder consequences → feedback → component and organizational adaptation.

This creates a continuous evaluation cycle rather than a one-time technology assessment.

The first level asks whether the technology works.

The second level asks whether the technology works within the organization.

The third level asks whether the resulting digital ecosystem functions appropriately within its wider environment.

The distinction is particularly important because a technology may produce different outcomes at each level.

A connected sensor may function correctly at the component level. It may integrate successfully into an agricultural monitoring system at the organizational level. Yet at the ecosystem level, it may create data-ownership issues, vendor dependency, environmental costs or unequal access.

Similarly, a digital healthcare application may function reliably as software while creating problems when integrated with hospital systems or when used across a national healthcare ecosystem.

The three-level model therefore reinforces the central principle of multidimensional technology evaluation: technological value cannot be established by technical performance alone.

Evaluation should also consider time. A system may perform well immediately after deployment but become less effective as data volumes increase, vendors change their products, cybersecurity threats evolve, users change their behavior or the external environment changes.

Consequently, ecosystem evaluation should include lifecycle monitoring.

Risk should also be assessed across all three levels. Component-level risks may include software defects or hardware failure. Organizational risks may include process disruption or inadequate training. Ecosystem risks may include systemic concentration, cascading failures or widespread privacy consequences.

Resilience should similarly be assessed across the three levels. A component may have redundancy while the organization has no alternative process. An organization may have contingency arrangements while its critical suppliers remain dependent on a single external infrastructure.

Governance must therefore extend across organizational boundaries when technology operates as an ecosystem.

Accountability should be mapped accordingly. The relevant actors may include technology developers, vendors, organizational owners, administrators, operators, users, regulators and infrastructure providers.

This is particularly important when responsibility is distributed. A technology failure may involve several contributing factors, and effective evaluation should identify responsibilities before an incident occurs rather than attempting to determine accountability only after failure.

The three-level approach can therefore be integrated into AMTEF as follows:

Level 1: Component evaluation → technical capability, functionality, reliability, security and usability.

Level 2: Organizational evaluation → integration, processes, people, governance, cost, risk and operational outcomes.

Level 3: Ecosystem evaluation → interoperability, dependencies, suppliers, customers, regulators, communities, environmental effects, systemic risk and societal consequences.

The model can then be extended into a continuous lifecycle:

Component → organization → ecosystem → outcomes → monitoring → incident or change → reassessment → adaptation.

This approach prevents the common mistake of evaluating a component in isolation and assuming that its performance represents the performance of the larger technological system.

The central principle is that digital ecosystems should be evaluated according to relationships as well as components. A technology creates value through the functions it performs, the organizational processes it enables and the wider ecosystem within which it operates.

Accordingly, a comprehensive digital-technology evaluation should ask three interconnected questions:

Does the technology work?

Does it work appropriately within the organization?

Does the wider ecosystem remain effective, secure, resilient, sustainable and accountable as a result?

These questions provide a bridge between conventional technology evaluation and the broader AMTEF perspective. The evaluation moves from technical capability to organizational integration and finally to ecosystem-level consequences, thereby recognizing that modern digital technologies operate within networks of people, organizations, infrastructures, data and other technologies rather than as isolated products.

11.24 Digital Technology and AMTEF

CONCEPTUAL FRAMEWORK

Digital Technology and AMTEF

1. DIGITAL TECHNOLOGY
2. AMTEF DIMENSION — DIGITAL TECHNOLOGY EVALUATION FOCUS — TECHNICAL — Performance and reliability — FUNCTIONAL — Usefulness — ECONOMIC — Cost and value — ORGANIZATIONAL — Readiness — HUMAN — Usability and agency — SOCIAL — Inclusion and relationships — ETHICAL — Privacy and fairness — ENVIRONMENTAL — Energy and materials — RISK — Security and resilience — GOVERNANCE — Accountability and regulation
3. MULTIDIMENSIONAL EVALUATION OF DIGITAL TECHNOLOGY

The digital domain therefore provides an especially strong demonstration of the value of multidimensional evaluation. Digital technologies activate nearly every AMTEF dimension, requiring evaluation that extends beyond technical performance to economic, organizational, human, social, ethical, environmental, risk, and governance considerations.

Figure: Digital Technology and the Adams Multidimensional Technology Evaluation Framework (AMTEF)

The digital domain therefore provides an especially strong demonstration of the value of multidimensional evaluation.

11.25 Research Questions

How does digital dependency affect organizational resilience?

How can organizations evaluate vendor lock-in?

What criteria should be used to evaluate digital ecosystems?

How should data governance influence technology adoption?

How can organizations evaluate the social consequences of smart-city technologies?

11.26 Chapter Summary

Digital technologies create value through connectivity, information and integration. Their evaluation therefore requires attention to interoperability, cybersecurity, data quality, dependency, resilience and governance.

The most important lesson is that digital technologies should be evaluated as ecosystems rather than isolated products.

11.27 Discussion Questions

Why is interoperability important?

What are the strategic risks of vendor lock-in?

How can data quality affect technology performance?

What makes digital ecosystems different from individual technologies?

How should digital resilience be measured?