Chapter 12. Evaluating Biotechnology, Healthcare and Human-Centred Technologies — Part 2 of 2
12.28 Technology and Human Agency
Human agency refers to people's capacity to understand, choose and act. Technology can strengthen human agency when it increases people's ability to obtain information, communicate, make decisions and participate in activities that might otherwise be difficult or impossible.
For example, technology may strengthen agency by improving access to information and enabling individuals to make more informed decisions. Digital accessibility tools can support people with disabilities, while assistive devices can reduce physical limitations and increase independence. Decision-support systems can provide additional information to professionals and patients, potentially strengthening the quality of decision-making when appropriately designed and interpreted.
At the same time, technology can weaken human agency. Excessive automation may reduce opportunities for meaningful human judgment. Surveillance systems may constrain behaviour by creating the perception that activities are continuously observed. Manipulative technologies may influence choices without sufficient awareness on the part of users. Excessive technological dependency may reduce people's ability to function independently when the technology becomes unavailable. Opaque automated systems may also make decisions that users cannot adequately understand or challenge.
The evaluation question is therefore not simply whether technology assists people. The deeper question is whether technology preserves meaningful human control. A system that performs tasks efficiently but prevents individuals from understanding, questioning or rejecting important decisions may create a different form of technological dependence.
Human agency is particularly important in healthcare, education, employment, finance and public administration because technological decisions in these environments may affect important aspects of people's lives. Evaluation should therefore examine whether individuals retain meaningful opportunities to understand decisions, provide input, correct errors and challenge outcomes.
12.29 Assistive Technologies
Assistive technologies are designed to support individuals who experience disabilities, functional limitations or age-related challenges. They include mobility devices, communication technologies, hearing technologies, visual-support technologies, adaptive interfaces and other systems designed to improve independence and participation.
The evaluation of assistive technology should not be based solely on technical specifications. The central question is whether the technology actually improves the user's ability to participate in everyday activities and whether it does so in a safe, affordable, accessible and dignified manner.
Users with disabilities should be meaningfully involved in the design and testing process. Their experiences can reveal barriers that developers may not identify through conventional technical testing. User participation may also improve the relevance, usability and acceptability of the final technology.
Accessibility should not be treated merely as a matter of regulatory compliance. Accessibility represents a fundamental component of effective human-centred design. A technology may technically comply with a minimum accessibility requirement while remaining difficult to use in practice. Evaluation should therefore examine actual user experiences and determine whether people can access and use the technology independently and effectively.
Affordability is also important. A highly effective assistive technology that is financially inaccessible may provide limited social benefit. Maintenance, replacement, technical support and availability of trained professionals should also be considered because assistive technologies often provide continuing rather than one-time support.
12.30 Neurotechnology
Neurotechnology creates especially complex ethical questions because it can interact directly or indirectly with brain activity and the nervous system. Potential applications include medical treatment, neurological rehabilitation, communication, assistive technologies and scientific research.
The potential benefits of neurotechnology are significant, particularly for individuals affected by neurological conditions or physical limitations. Technologies that allow individuals to control assistive devices or communicate through neural signals may provide new forms of independence and participation.
At the same time, neurotechnology raises questions that extend beyond conventional medical-device assessment. Because information concerning neurological activity may reveal highly personal characteristics, mental privacy becomes an important consideration. The protection of neural data may therefore require particularly strong safeguards.
Autonomy is another central issue. Individuals should understand the nature of the technology, its capabilities, limitations and risks before consenting to its use. Questions concerning identity may also arise where technology substantially changes how individuals communicate, move or interact with their environment.
Evaluation should therefore consider mental privacy, autonomy, identity, informed consent, data security, long-term effects, psychological consequences and the possibility of unintended uses. Neurotechnology demonstrates particularly clearly why technological evaluation must include human, ethical and social dimensions alongside technical performance.
12.31 Human-Centred Evaluation Framework
A human-centred evaluation framework should examine whether a technology preserves meaningful human agency. The evaluator should determine whether users retain meaningful opportunities to understand, choose, influence and challenge technological processes and decisions.
Accessibility should be evaluated by determining whether diverse users can meaningfully access and use the technology. This includes consideration of physical accessibility, digital accessibility, language, affordability, infrastructure and individual differences.
Usability should examine whether users can operate the technology effectively and efficiently within the environment for which it was designed. Evaluation should consider whether users can understand the system, complete relevant tasks and recover from errors.
Dignity should be considered by examining whether the technology respects people as individuals and avoids unnecessary humiliation, exclusion, dependency or dehumanization. This dimension is particularly important in healthcare, social care, disability services and technologies involving vulnerable populations.
Privacy should address whether sensitive personal information is appropriately collected, stored, processed and shared. Safety should examine foreseeable harms and determine whether appropriate safeguards have been implemented.
Inclusion requires consideration of marginalized, disadvantaged or underrepresented users. The evaluator should determine whether the design unintentionally creates barriers for groups whose needs may not have been sufficiently represented during development.
Autonomy concerns whether users can meaningfully accept, reject, modify or challenge technological decisions. Trust concerns whether users receive sufficient information to understand the technology and make informed decisions concerning its use.
Accountability requires that responsibility can be identified when something goes wrong. A technology should not create an environment in which responsibility becomes so distributed among developers, organizations, algorithms and users that no identifiable party can be held accountable.
Taken together, these dimensions provide a structured way to examine whether a technology is genuinely human-centred rather than merely technologically sophisticated.
12.32 Ethical Trade-Offs
Technology evaluation frequently involves trade-offs because improvements in one dimension may create costs or risks in another. Greater personalization, for example, may require the collection and analysis of additional personal data. More automation may improve efficiency while reducing direct human involvement. Increased security may require additional authentication steps and consequently increase user friction. Greater transparency may improve accountability but may also reveal sensitive information or proprietary technical details. More intensive monitoring may improve safety while simultaneously reducing privacy.
These trade-offs should not be hidden or simplified. The purpose of AMTEF is not to eliminate every trade-off because some trade-offs may be unavoidable. Rather, the purpose is to make trade-offs visible, identify the affected stakeholders, examine the available evidence and determine what safeguards can reduce unnecessary harm.
Ethical evaluation should therefore document not only expected benefits but also the costs and risks associated with achieving those benefits. Decision-makers should be able to understand why particular compromises exist and what mechanisms are available for reviewing them.
Trade-offs may also change over time. A privacy compromise that appears reasonable during a limited pilot may become less acceptable when a system is scaled nationally. Similarly, a technology initially requiring substantial human supervision may become more automated as evidence and technical capabilities develop. Continuous evaluation is therefore necessary.
12.33 Human-Centred Technology Decision Rule
A human-centred technology should begin by addressing a legitimate human need rather than simply demonstrating technological novelty. The need should be clearly defined, and the relationship between the technology and the identified problem should be supported by appropriate evidence.
The expected benefits should be supported by credible evidence appropriate to the maturity and risk profile of the technology. Preliminary laboratory evidence should not automatically be treated as equivalent to extensive real-world evidence.
Foreseeable harms should be actively identified, assessed and managed. Risk management should continue throughout the technology lifecycle because new risks may emerge during implementation and widespread use.
Human agency should be preserved. Users should retain meaningful opportunities to understand, influence and, where appropriate, reject technological decisions that materially affect them.
Finally, accountability should remain identifiable. Organizations should establish clear responsibility for development, deployment, monitoring, incident response and correction.
These conditions should always be interpreted alongside the specific legal, regulatory, ethical and professional requirements applicable to the relevant sector.
12.34 Biotechnology and AMTEF
Biotechnology demonstrates the importance of integrating all ten AMTEF dimensions. Technical performance must be considered alongside functional effectiveness, economic value, organizational readiness, human impact, social consequences, ethical responsibility, environmental sustainability, risk and security, and governance and regulation.
Technical performance examines whether the biotechnology functions as intended. Functional effectiveness determines whether that technical capability produces the desired practical or clinical outcome. Economic evaluation considers development, production, implementation and maintenance costs. Organizational evaluation examines whether institutions possess the expertise, infrastructure and procedures required for safe and effective implementation.
Human impact considers how individuals experience the technology, including effects on health, dignity, autonomy and quality of life. Social evaluation examines broader consequences for communities and populations. Ethical evaluation considers questions of responsibility, consent, justice and acceptable use. Environmental evaluation considers effects across the technology lifecycle.
Risk and security evaluation addresses foreseeable failures, misuse, biological risks, cybersecurity and other threats. Governance and regulatory evaluation examines whether appropriate institutional, legal and professional structures exist to supervise the technology.
No single dimension is sufficient. A biotechnology may perform extremely well technically while presenting significant unresolved questions concerning safety, equity, environmental consequences or governance. AMTEF therefore provides a multidimensional structure through which these dimensions can be considered together.
12.35 Research Questions
Research into healthcare, biotechnology and human-centred technologies can examine how healthcare technologies should be evaluated across clinical, ethical and organizational dimensions. Such research can investigate which forms of evidence are appropriate for emerging biotechnology and how evidence standards should change as technologies move from laboratory development to real-world implementation.
Research can also investigate how AI-enabled medical technologies should be monitored after deployment, particularly where algorithms interact with changing populations, clinical practices and data environments. Another important research area concerns how technology evaluation can incorporate health equity so that differences in access, outcomes and risks among populations are identified.
Further research is needed concerning how human agency should influence the evaluation of neurotechnology and assistive technologies. Such research may examine whether users retain meaningful control, whether technologies support independence and whether privacy and dignity are sufficiently protected.
12.36 Chapter Summary
Healthcare, biotechnology and human-centred technologies require rigorous multidimensional evaluation because their consequences can directly affect individuals, communities, organizations and society. Technical performance is important, but technical capability alone does not establish technological success.
Clinical effectiveness, safety and functional performance must be considered alongside privacy, equity, autonomy, accessibility, evidence quality, economic feasibility, organizational readiness, environmental sustainability and governance.
The evaluation of biotechnology also requires attention to biological uncertainty, long-term consequences, ecological effects, ethical responsibility and social acceptance. Healthcare technologies require particular attention to patient welfare, informed consent, professional accountability and real-world clinical outcomes.
Human-centred technologies require evaluators to examine whether technology strengthens rather than unnecessarily constrains human agency. Assistive technologies demonstrate the importance of accessibility and user participation, while neurotechnology demonstrates the importance of mental privacy, autonomy, identity and neural-data protection.
The central principle is therefore that responsible technology evaluation must examine not only what technology can do, but also what it does to people, organizations, communities and the environment.
12.37 Discussion Questions
Why is clinical effectiveness different from technical performance, and why is this distinction important when evaluating healthcare technologies? What characteristics make biotechnology evaluation particularly difficult compared with conventional technology evaluation? How should AI diagnostic systems be evaluated when their performance may vary across different populations and clinical environments? In what ways can health technology evaluation incorporate equity and accessibility into the evaluation process? What does human-centred technology mean in practical terms, and how can evaluators determine whether a technology genuinely preserves human agency?
INTEGRATIVE SYNTHESIS OF PART III
1. Integrating Risk, AI, Digital Systems and Human-Centred Technologies
The preceding chapters demonstrate that different technology domains create different types of risks, opportunities and consequences, but these domains can be evaluated through a common multidimensional architecture. Artificial intelligence, digital systems, biotechnology and healthcare technologies differ considerably in their technical characteristics, but they share many evaluation requirements.
The AMTEF dimensions provide a broad structure consisting of technical performance, functional effectiveness, economic value, organizational readiness, human impact, social consequences, ethical responsibility, environmental sustainability, risk and security, and governance and regulation.
These dimensions are deliberately broad enough to accommodate a wide range of emerging and established technologies. They provide a common language through which technologies can be examined even when their technical characteristics differ substantially.
2. Integrated AMTEF Evaluation Architecture
The integrated AMTEF evaluation architecture begins with the technology and proceeds through problem and purpose definition, evidence collection, technical evaluation, functional evaluation, economic evaluation, organizational evaluation, human and social evaluation, ethical evaluation, environmental evaluation, risk and security evaluation, governance and regulatory evaluation, stakeholder deliberation, decision-making, implementation, monitoring and re-evaluation.
The architecture can therefore be understood as a continuous sequence:
Technology → Problem and Purpose → Evidence → Technical Evaluation → Functional Evaluation → Economic Evaluation → Organizational Evaluation → Human and Social Evaluation → Ethical Evaluation → Environmental Evaluation → Risk and Security Evaluation → Governance and Regulatory Evaluation → Stakeholder Deliberation → Decision → Implementation → Monitoring → Re-evaluation
The central principle is that evaluation is iterative. It should not occur once at the beginning of the technology lifecycle and then disappear. New evidence, changing circumstances, emerging risks and stakeholder experiences may require the technology and its implementation to be reassessed.
3. Evaluation Across the Technology Lifecycle
Technology evaluation should occur throughout the complete technology lifecycle. During research, evaluation should focus on feasibility, scientific validity, potential benefits and possible consequences. During development, attention should shift toward technical performance, design quality, reliability and safety.
During pilot implementation, evaluation should examine real-world usability, user experiences, organizational fit and initial outcomes. During deployment, evaluators should examine operational performance, governance, safety and stakeholder experiences. During scaling, attention should expand to economic, social, environmental and systemic consequences.
As a technology reaches maturity, evaluation should consider maintenance, resilience, long-term value and changing user requirements. At retirement, evaluation should address data management, infrastructure decommissioning, environmental disposal, replacement technologies and transition arrangements.
This lifecycle perspective reflects the broader movement toward continuous technology assessment and responsible innovation (OECD, 2023; Stilgoe et al., 2013).
4. Importance of Evidence
Not all evidence has equal strength. A technology may initially appear theoretically plausible without having been demonstrated experimentally. Laboratory evidence may demonstrate functionality under controlled conditions without establishing real-world effectiveness. A prototype may demonstrate feasibility without demonstrating operational reliability at scale.
A useful evidence progression therefore begins with conceptual evidence, in which a technology appears theoretically plausible. This can be followed by laboratory evidence demonstrating that the technology functions under controlled conditions. Prototype evidence demonstrates the existence of a functioning technological system. Pilot evidence examines performance within a limited real-world environment. Operational evidence examines performance under normal conditions, while longitudinal evidence examines performance and consequences over an extended period.
The evaluator should avoid treating early conceptual or laboratory evidence as equivalent to extensive operational or longitudinal evidence. Evidence maturity should correspond to the magnitude and reversibility of the decision being considered.
5. Evidence Confidence
AMTEF should distinguish between different levels of confidence in available evidence. High-confidence evidence exists when multiple independent sources demonstrate consistent results and methodological limitations are relatively limited. Moderate-confidence evidence exists when useful evidence is available but significant limitations remain. Low-confidence evidence exists when findings are preliminary, context-specific or based on limited data.
There may also be circumstances in which evidence is insufficient to support a meaningful conclusion. Such circumstances should be described as unknown rather than converted into an assumption of either safety or effectiveness.
This distinction is particularly important for emerging technologies because enthusiasm concerning technological innovation can sometimes develop faster than the evidence necessary to establish long-term effectiveness and safety.
6. Uncertainty Register
Technology evaluations should maintain an explicit uncertainty register. Long-term performance represents one important area of uncertainty because technologies may degrade or perform differently over extended periods. This uncertainty can be addressed through longitudinal monitoring and continuing performance assessment.
User behaviour represents another important uncertainty because actual use may differ from developer assumptions. Field research, observation and user studies can therefore provide evidence concerning how technologies are actually used.
Environmental impact may also remain uncertain when lifecycle data are incomplete. Lifecycle assessment can help address this uncertainty by examining environmental consequences from material extraction through manufacturing, transportation, operation and disposal.
Security represents another continuing uncertainty because new vulnerabilities may emerge after deployment. Red-team testing, penetration testing, vulnerability assessment and continuous security monitoring can therefore provide additional evidence.
Regulation may also change as technologies develop. Regulatory monitoring is therefore necessary to ensure that technology remains aligned with evolving legal and professional requirements.
Finally, social consequences may be difficult to predict because some effects are indirect or emerge only after widespread adoption. Social-impact research, stakeholder consultation and longitudinal research can help identify such effects.
Uncertainty should not be hidden simply because it complicates decision-making. Explicitly identifying uncertainty can improve transparency and encourage appropriate monitoring and further evidence generation.
7. Integrated Technology Evaluation Scorecard
The AMTEF scorecard should not be interpreted as a universal numerical ranking system. Its purpose is to structure evidence, identify weaknesses, expose trade-offs and clarify what additional action may be required.
For the technical dimension, evidence may include technical test results. The evaluator should determine the strength of the evidence and identify technical risks requiring attention. Functional evaluation may draw upon user testing and operational evidence. Economic evaluation may use total-cost-of-ownership analysis and financial modelling.
Organizational evaluation may use readiness assessments to determine whether appropriate capabilities, infrastructure and human resources are available. Human evaluation may draw upon user research, interviews and observations. Social evaluation may involve impact studies and stakeholder research.
Ethical evaluation may incorporate formal ethics review and stakeholder deliberation. Environmental evaluation may use lifecycle assessment. Security evaluation may incorporate penetration testing and other forms of security assessment. Governance evaluation may involve policy review, legal analysis and institutional oversight assessment.
The scorecard should therefore be used to determine whether evidence is sufficient for the intended decision rather than to generate a simplistic overall technology score.
MULTIDIMENSIONAL TECHNOLOGY EVALUATION IN PRACTICE
Stage One: Define the Technology
The first stage requires the evaluator to document the technology being assessed. This includes its name, developer, purpose, maturity, intended users, operating environment, dependencies and current lifecycle stage. Defining the technology clearly prevents ambiguity concerning what is actually being evaluated.
The evaluator should distinguish between the technological artifact itself and the broader system within which it operates. For example, evaluating an AI healthcare system requires consideration not only of the software but also the data, hardware, clinical professionals, institutional processes and patients that interact with it.
Stage Two: Define the Problem
The problem should be distinguished from the technology. The evaluator should first identify what problem exists independently of the proposed technological solution.
The central question should be: What problem exists independently of the technology?
This prevents organizations from adopting technology merely because it is fashionable, commercially promoted or perceived as innovative. A technology should be evaluated in relation to an identified need rather than assumed to be valuable simply because it is technologically sophisticated.
Stage Three: Identify Stakeholders
Stakeholder identification is an essential part of multidimensional evaluation. Stakeholders may include customers, employees, managers, regulators, communities, suppliers, investors, patients, students, citizens, technology developers and other groups affected by the technology.
Stakeholder mapping is important because different groups may experience different benefits, costs and risks. A technology may benefit organizational managers while creating additional burdens for employees, or it may improve service delivery for some patients while creating accessibility barriers for others.
The evaluator should therefore identify who is affected, how they are affected and what mechanisms exist for their concerns to be incorporated into the evaluation.
Stage Four: Establish Evaluation Criteria
Evaluation criteria should be established before evidence is interpreted. Establishing criteria in advance reduces the risk of changing standards after results become available.
Criteria should reflect the purpose, risk profile and maturity of the technology. High-risk technologies may require more extensive evidence and stronger safeguards than low-risk technologies. Criteria should also reflect applicable legal, professional, ethical and organizational requirements.
Stage Five: Collect Evidence
Evidence should include both quantitative and qualitative information. Quantitative evidence can include performance metrics, cost, error rates, energy use, reliability and adoption. Qualitative evidence can include interviews, observations, expert assessments, stakeholder perceptions and ethical analysis.
Neither form of evidence should automatically be treated as superior. Quantitative evidence can demonstrate measurable patterns, while qualitative evidence can explain experiences, meanings, motivations and contextual factors that numerical measures may not capture.
Stage Six: Evaluate Trade-Offs
Technology decisions rarely maximize every dimension simultaneously. Higher security may reduce convenience. Greater privacy may limit personalization. Higher performance may require greater computational resources. Lower cost may increase vendor dependency.
The purpose of evaluation is therefore to make such trade-offs explicit. Evaluators should identify which values are being prioritized, which interests may be affected and what safeguards can reduce negative consequences.
Stage Seven: Determine Readiness
Technology readiness and adoption readiness are different concepts. A technology may be technically mature while an organization remains unprepared to implement it effectively.
For example, an organization may possess a highly developed technological system but have insufficient staff training, inadequate infrastructure, weak governance or limited financial resources. In such a situation, implementation readiness may be low even though technological readiness is high.
Evaluation should therefore distinguish between whether the technology is ready and whether the organization is ready to use it responsibly.
Stage Eight: Pilot
Pilot projects should be designed to generate evidence rather than simply demonstrate technology. A meaningful pilot should establish defined objectives, measurable outcomes, risk controls, stakeholder feedback mechanisms, stop criteria and evaluation procedures.
A pilot should also be sufficiently representative of the environment in which the technology is ultimately expected to operate. A highly controlled pilot may provide limited evidence concerning real-world conditions.
Stage Nine: Monitor
Monitoring should continue after implementation. Performance, cost, user experience, incidents, security, environmental impact and equity should be observed over time.
Monitoring is particularly important where technologies are adaptive, connected, software-driven or dependent upon changing data. Performance at the time of initial deployment may not remain unchanged.
Stage Ten: Re-evaluate
Re-evaluation should occur when the technology changes, data change, users change, regulations change, major incidents occur, performance declines or the technology is expanded into new contexts.
A technology that was considered appropriate for one population or environment may require additional evaluation before being introduced elsewhere. Re-evaluation therefore represents an essential component of responsible technology governance.
INTEGRATED CASE STUDY: GENERATIVE AI IN A UNIVERSITY
Consider a university introducing generative artificial intelligence across teaching, administration and research. The evaluation of such an initiative should begin by identifying the institutional problems the technology is intended to address. These may include administrative workload, access to information, educational support, research assistance or other clearly defined needs.
From a technical perspective, the university should evaluate system availability, response quality, factual reliability, integration with existing systems, cybersecurity and data protection. Technical performance should be assessed using evidence appropriate to the intended applications rather than assuming that general-purpose AI performance automatically translates into institutional effectiveness.
From an economic perspective, the university should consider licensing, infrastructure, staff training, technical support, productivity and opportunity costs. The financial assessment should include both direct and indirect costs because implementation may require changes in infrastructure, staffing and professional development.
From an educational perspective, the university should examine learning outcomes, student engagement, assessment validity, academic integrity and staff workload. The evaluation should consider whether AI contributes to learning or merely accelerates the production of educational outputs.
From a human perspective, the university should assess student autonomy, faculty roles, digital literacy, accessibility and trust. The introduction of AI may alter relationships between students, teachers and information. These changes should therefore be evaluated rather than assumed to be automatically beneficial.
Ethical considerations may include bias, privacy, transparency, authorship and academic responsibility. UNESCO's AI ethics framework emphasizes human rights, dignity, transparency, fairness, inclusiveness and human oversight, all of which are relevant to higher-education AI adoption (UNESCO, 2021).
From a security perspective, the university should assess data leakage, account compromise, prompt injection, malicious content and unauthorized access. The institution should also consider how confidential academic, administrative, research and personal information is handled.
Governance should include acceptable-use policies, responsibility structures, review procedures, data policies, academic-integrity requirements and incident-management procedures. Governance should clarify who is responsible for approving applications, monitoring risks, responding to incidents and reviewing policy.
Environmental evaluation should consider computational resources, energy consumption and infrastructure requirements. Although digital technologies may reduce certain physical activities, their computational infrastructure can have significant resource requirements.
The final decision should not simply state that AI is good or bad. Instead, it should document the identified problem, expected benefits, evidence, risks, stakeholder concerns, safeguards, unresolved uncertainties, implementation conditions, monitoring requirements and review arrangements.
This approach improves transparency and institutional accountability because it establishes an evidence-based record of how the technology is being evaluated and governed.
RESEARCH DESIGN FOR MULTIDIMENSIONAL TECHNOLOGY EVALUATION
Quantitative Evaluation
Quantitative methods can measure reliability, accuracy, costs, productivity, adoption, user satisfaction and environmental indicators. Statistical analysis can identify differences, relationships and patterns among variables.
However, quantitative methods alone may not explain why particular outcomes occur. A system may demonstrate a measurable decline in performance, for example, without quantitative data explaining whether the decline resulted from poor training, organizational resistance, technical limitations or changes in user behaviour.
Qualitative Evaluation
Qualitative methods can explore stakeholder experiences, ethical concerns, organizational resistance, human-technology interaction and unintended consequences. Interviews, focus groups, observation, document analysis and case-study methods can provide detailed contextual evidence.
Qualitative research is particularly valuable when evaluating emerging technologies because many important consequences may not yet have established numerical indicators. Stakeholders may identify concerns that were not anticipated during initial technology design.
Mixed-Methods Evaluation
Mixed-methods evaluation combines quantitative outcomes with qualitative explanation. For example, a researcher may use performance analysis to determine whether a technology improves operational outcomes and then conduct user interviews to understand why the improvement occurred or why some users experienced different outcomes.
A study examining whether technology performance improves could therefore combine quantitative performance analysis with qualitative user interviews. An investigation of whether costs decline could combine financial analysis with interviews involving managers and operational personnel. An examination of user acceptance could combine survey data with focus groups. Research into whether harms are distributed unequally could combine subgroup analysis with stakeholder narratives. Evaluation of implementation success could combine key performance indicator analysis with organizational case-study evidence.
A mixed-methods design is particularly appropriate when technology evaluation involves both measurable outcomes and complex human, ethical and organizational consequences. It allows the researcher to examine not only whether a change occurred but also how, why, for whom and under what circumstances it occurred.
The integration of quantitative and qualitative evidence can therefore strengthen AMTEF-based technology evaluation by connecting measurable technological performance with the experiences and consequences associated with implementation.
The overall principle is that multidimensional technology evaluation should not be reduced to a single metric, a single research method or a single stakeholder perspective. Emerging technologies require evidence that is technically credible, contextually relevant, ethically informed and sufficiently broad to identify both intended benefits and unintended consequences.
AMTEF TECHNOLOGY EVALUATION WORKSHEET
A. Technology Identification
Technology:
Developer:
Technology category:
Lifecycle stage:
Intended users:
Primary purpose:
B. Technical Evaluation
Performance:
Reliability:
Scalability:
Interoperability:
Security:
Maintainability:
C. Functional Evaluation
Problem addressed:
Expected outputs:
Expected outcomes:
Evidence of effectiveness:
D. Economic Evaluation
Initial cost:
Operating cost:
Total cost of ownership:
Expected benefit:
Opportunity cost:
Financial risks:
E. Organizational Evaluation
Leadership support:
Staff capability:
Infrastructure:
Processes:
Culture:
Change readiness:
F. Human Evaluation
Usability:
Accessibility:
Human agency:
Training:
Workforce impact:
G. Social Evaluation
Inclusion:
Equity:
Community effects:
Social relationships:
Digital divide:
H. Ethical Evaluation
Privacy:
Fairness:
Transparency:
Accountability:
Human oversight:
Dignity:
I. Environmental Evaluation
Energy:
Materials:
Water:
Emissions:
Waste:
Circularity:
J. Risk and Security
Threats:
Vulnerabilities:
Potential consequences:
Controls:
Residual risk:
K. Governance
Regulatory requirements:
Policies:
Responsible officers:
Auditability:
Monitoring:
Review date:
L. Final Evidence Assessment
Strong evidence:
Moderate evidence:
Weak evidence:
Unknowns:
Major unresolved issues:
Required safeguards:
Conditions for adoption: