Part II — Dimensions of Technology Evaluation

Chapter 7. Human, Social and Ethical Evaluation of Emerging Technologies

7.1 Introduction

Technology is ultimately used by people and affects people.

Consequently, technology evaluation cannot be complete without examining human, social and ethical consequences.

These dimensions are interconnected but distinct.

Human evaluation examines effects on individuals.

Social evaluation examines effects on groups, communities and society.

Ethical evaluation examines questions of values, rights, responsibilities and acceptable conduct.

The importance of these dimensions has increased as technologies increasingly influence decisions involving employment, education, healthcare, communication, finance and public administration.

UNESCO's Recommendation on the Ethics of Artificial Intelligence emphasizes human rights and dignity, fairness, transparency, human oversight and environmental and social responsibilities in the development and use of AI (UNESCO, 2021).

The underlying principle applies beyond AI:

Technology should be evaluated according to its relationship with human beings.

7.2 Human-Centred Technology

Human-centred technology begins with human needs rather than technological capability.

The evaluation asks:

What do people need?

What capabilities does the technology provide?

What burdens does it create?

Does it increase or decrease autonomy?

Does it improve accessibility?

Does it respect dignity?

Human-centred design emphasizes understanding users and their contexts rather than designing solely from technical requirements.

7.3 Human Agency

Human agency refers broadly to the ability of people to make meaningful decisions and act intentionally.

Automation can strengthen agency by:

reducing repetitive work;

providing information;

improving accessibility.

But automation can also reduce agency if:

decisions become difficult to challenge;

people are required to follow automated recommendations;

systems constrain available choices;

workers lose control over how tasks are performed.

Therefore, technology evaluation should examine both augmentation and substitution.

7.4 Automation and Human Work

Technology can:

replace tasks;

augment tasks;

transform tasks;

create new tasks.

The simplistic question:

“Will technology replace jobs?”

should therefore be complemented by:

“How will technology change tasks, occupations, skills and organizational structures?”

Autor (2015) argues that automation should be understood in relation to tasks rather than treating occupations as indivisible units.

This distinction is highly relevant to contemporary AI.

7.5 Human Oversight

For technologies involved in consequential decisions, human oversight may be important.

Examples include:

medical diagnosis;

employment decisions;

education;

credit;

public services.

Human oversight does not automatically guarantee responsible outcomes.

A human reviewer may:

overtrust the system;

lack technical understanding;

lack authority to challenge it;

simply approve automated recommendations.

Therefore, evaluation should examine whether human oversight is meaningful rather than merely formal.

7.6 Privacy

Privacy is a central issue in data-intensive technologies.

Evaluation should examine:

what data are collected;

why they are collected;

how long they are retained;

who can access them;

how they are secured;

whether individuals understand data use;

whether data can be deleted or corrected.

Privacy should be assessed throughout the technology lifecycle.

7.7 Surveillance

Digital technologies can increase organizational and social surveillance.

Examples include:

workplace monitoring;

facial recognition;

location tracking;

biometric systems;

online behaviour analytics.

The evaluation should consider:

purpose;

proportionality;

transparency;

consent;

access;

retention;

potential misuse.

7.8 Fairness

Technologies can distribute benefits and burdens unevenly.

AI systems may reproduce patterns contained in historical datasets.

Automated decision systems may therefore generate different outcomes for different populations.

Fairness evaluation requires careful definition.

Possible concepts include:

equal treatment;

equal opportunity;

outcome parity;

procedural fairness.

Different fairness principles can conflict.

Therefore, evaluators should explicitly state the fairness criterion being used.

7.9 Bias

Bias can enter a technological system through:

data;

sampling;

labels;

algorithms;

system design;

deployment context;

human interpretation.

The presence of bias does not necessarily mean that an entire technology is unusable.

It means the sources and consequences of bias must be identified and managed.

7.10 Transparency

Transparency concerns whether relevant information about the system is accessible.

Transparency may include:

data documentation;

model documentation;

decision processes;

limitations;

intended use;

prohibited use;

update history.

Transparency requirements should be proportional to risk.

7.11 Explainability

Explainability concerns whether system outputs can be meaningfully explained.

This is particularly relevant when people are affected by decisions.

The evaluation should ask:

Who needs an explanation?

What level of explanation is necessary?

What decisions require explanation?

Can users challenge the output?

The objective is not necessarily to make every computational process completely understandable to every person.

Rather, the goal is appropriate accountability.

7.12 Ethical Responsibility

Ethical responsibility concerns who is accountable for technological consequences.

Possible actors include:

developers;

vendors;

managers;

users;

regulators;

institutions.

Responsibility can become unclear when multiple actors contribute to a system.

A robust technology evaluation should therefore map responsibility explicitly.

7.13 Social Inclusion

Technology can improve inclusion by:

providing remote access;

supporting people with disabilities;

lowering service costs;

improving communication.

But technology can also create exclusion.

Examples include:

digital-only services;

inaccessible interfaces;

high subscription costs;

lack of connectivity;

language limitations.

Therefore, evaluators should ask:

“Who cannot use this technology?”

This question can be as important as:

“Who can use it?”

7.14 Digital Divide

The digital divide concerns differences in access to and effective use of digital technologies.

Differences may involve:

income;

geography;

infrastructure;

education;

age;

disability;

digital literacy.

Technology evaluation should therefore consider both access and meaningful use.

7.15 Social Relationships

Technologies can alter:

communication;

family relationships;

workplace relationships;

educational interaction;

professional identity;

community participation.

Some effects may be positive.

Others may be negative.

The evaluator should therefore avoid assuming that increased digital interaction automatically equals improved social connection.

7.16 Technology and Education

Technology evaluation in education should consider:

learning outcomes;

student autonomy;

teacher roles;

accessibility;

academic integrity;

privacy;

digital inequality.

The introduction of generative AI illustrates the complexity.

The technology may support:

brainstorming;

language development;

tutoring;

feedback;

research assistance.

It may also create concerns involving:

inaccurate information;

overdependence;

assessment validity;

authorship;

academic misconduct.

A multidimensional evaluation is therefore required.

7.17 Technology and Employment

Technology can alter employment in several ways.

Substitution

A human task is automated.

Augmentation

Technology helps a worker perform a task.

Transformation

The task changes substantially.

Creation

New occupations or tasks emerge.

Elimination

Some roles may decline.

The evaluation should therefore consider transition requirements.

These may include:

retraining;

reskilling;

redeployment;

new professional standards.

7.18 Psychological and Cognitive Considerations

Technology can influence:

attention;

workload;

cognitive effort;

decision-making;

dependence;

stress.

These effects should be evaluated carefully and empirically.

Technology should not automatically be assumed to improve human cognition simply because it increases information availability.

7.19 Human Dependency

Technology may create dependency.

Examples include:

dependence on cloud infrastructure;

dependence on AI recommendations;

dependence on digital payment systems;

dependence on navigation systems.

Dependency is not automatically negative.

The key questions are:

Is the dependency understood?

Is it manageable?

Are alternatives available?

What happens during failure?

7.20 Ethical Evaluation Matrix

Table 7.1 — AMTEF Human, Social and Ethical Matrix

7.21 Ethical Impact Assessment

An ethical impact assessment may follow:

identify affected individuals;

identify rights and values;

identify potential harms;

identify potential benefits;

identify competing ethical principles;

identify responsible actors;

identify safeguards;

establish monitoring.

This should be documented before high-impact technologies are deployed.

7.22 Case Study: AI Recruitment

Consider an AI recruitment platform.

Potential benefits:

faster screening;

reduced administrative workload;

standardized processing.

Potential concerns:

historical bias;

opaque criteria;

privacy;

lack of applicant appeal;

overreliance on automated recommendations.

A comprehensive assessment might require:

bias testing;

documentation;

human review;

applicant notification;

audit trails;

appeal mechanisms.

7.23 Social Impact Assessment

Social assessment should consider distribution.

Table 7.2 — Distributional Impact Matrix

7.24 Ethical Trade-Offs

Ethical technology evaluation frequently involves competing values.

For example:

Privacy versus personalization.

Security versus anonymity.

Automation versus employment.

Efficiency versus human interaction.

Transparency versus protection of proprietary information.

Innovation speed versus precaution.

There may not be a universal solution.

The role of evaluation is to make the trade-off explicit.

7.25 Principles for Human-Centred Technology Evaluation

AMTEF recommends the following principles:

human dignity;

meaningful human agency;

proportionality;

fairness;

inclusion;

privacy;

transparency;

accountability;

accessibility;

contestability.

Contestability is particularly important.

People affected by technology should, where appropriate, have mechanisms through which decisions can be questioned or challenged.

7.26 Chapter Research Questions

How does technology affect human agency?

How should algorithmic fairness be evaluated?

What constitutes meaningful human oversight?

How can technology evaluation identify digital exclusion?

How should privacy be evaluated throughout a technology lifecycle?

How does technology transform rather than simply eliminate work?

How can ethical trade-offs be made transparent?

7.27 Chapter Summary

Human, social and ethical evaluation examines how technologies affect people, groups and values.

Important considerations include:

human agency;

privacy;

surveillance;

fairness;

transparency;

accountability;

inclusion;

employment;

social relationships;

accessibility.

Technology evaluation should therefore move beyond the question of whether a system works and ask whether it works in ways that respect human dignity, social inclusion and ethical responsibility.

7.28 Discussion Questions

What does human-centred technology mean?

Can automation strengthen human agency?

How should algorithmic fairness be defined?

Why is meaningful human oversight difficult?

What are the social consequences of digital exclusion?

How should competing ethical values be handled?