Chapter 16. Technology Evaluation in Education and Higher Education — Part 1 of 2
16.1 Introduction
Education represents one of the most important environments in which emerging technologies are developed, adopted, evaluated and debated. Schools, colleges, universities, professional institutions and research organizations increasingly depend upon digital technologies to support teaching, learning, assessment, administration, research, communication and institutional management. The rapid development of cloud computing, learning management systems, learning analytics, artificial intelligence, generative AI, virtual and augmented reality, educational applications, digital libraries, automated assessment systems and research technologies has transformed the technological environment of education.
Higher education is particularly significant because universities occupy several roles simultaneously. A university is a user of technology, a producer of knowledge, a research institution, an employer, an educator of future professionals, an institutional decision-maker, a custodian of academic standards and, in many circumstances, a regulator of academic practice within its own institutional environment. Universities also influence wider society through research, professional education, innovation, policy engagement and the preparation of graduates who will use technology in government, business, healthcare, science, engineering and other sectors.
This combination of functions makes higher education a particularly important context for the application of AMTEF. A university cannot evaluate educational technology only according to whether a system works technically. It must also consider whether the technology contributes to educational objectives, whether faculty and students can use it effectively, whether it protects privacy and academic integrity, whether it is accessible, whether it is financially sustainable, whether it creates unacceptable risks and whether it is consistent with institutional values and responsibilities.
Digital learning platforms, learning analytics, artificial intelligence, automated assessment, educational data systems, virtual reality, simulation technologies and generative AI are increasingly changing educational practice. These technologies may create opportunities for greater access, personalization, flexibility and administrative efficiency. At the same time, they may create new risks involving surveillance, privacy, inequality, inaccurate information, academic misconduct, algorithmic bias, excessive automation and weakening of human interaction.
Technology adoption in education should therefore not automatically be equated with educational improvement. The presence of sophisticated technology does not demonstrate that meaningful learning has improved. Similarly, increased system usage does not necessarily demonstrate increased educational quality. A technology may be widely used while producing limited learning benefits, or it may reduce administrative workload while creating new burdens elsewhere.
The central AMTEF question can therefore be expressed as follows: Does the technology improve meaningful learning and institutional outcomes while protecting academic integrity, human agency, equity, privacy, accessibility, security and educational quality?
This question requires a multidimensional evaluation. Technical performance is necessary, but it is only one component. Educational value, human experience, organizational readiness, financial sustainability, ethical implications, social consequences, environmental requirements, cybersecurity and governance must also be considered.
Technology evaluation in education should consequently begin with educational purpose rather than technological novelty. The evaluator should first identify what educational or institutional problem exists and then determine whether technology provides an appropriate means of addressing that problem.
16.2 Technology and the Purpose of Education
Education is not simply the transmission of information from an instructor to a learner. Higher education has broader purposes that include developing knowledge, critical thinking, research capability, communication, creativity, ethical judgement, professional competence, problem-solving ability and the capacity to engage independently with complex issues. Universities also contribute to intellectual development, social development, innovation and the creation of new knowledge.
Technology should therefore be evaluated against educational purposes rather than against technological capabilities alone. A system may contain numerous advanced functions but provide limited educational value if those functions do not contribute meaningfully to learning. Conversely, a relatively simple technology may provide considerable value if it directly addresses an important educational need.
For example, an AI writing assistant may be capable of producing sophisticated text, but its educational value depends upon how it is used. If students simply submit machine-generated answers without engaging with the underlying concepts, the technology may reduce opportunities for learning. If, however, students use the technology to compare alternative explanations, identify weaknesses, critique generated material and improve their own reasoning, it may support deeper learning.
The same principle applies to learning management systems, online courses, simulations and educational applications. The evaluator should ask whether the technology improves comprehension, engagement, feedback, problem-solving, collaboration, research capability or other defined educational outcomes.
Educational technology should also be evaluated in relation to different levels of learning. Some technologies may be highly effective for information retrieval but less effective for developing practical competence. Others may support procedural learning but provide limited opportunities for ethical reasoning or interpersonal development. Evaluation should therefore consider the specific learning outcomes associated with the educational programme.
The purpose of education also introduces a human dimension into technology evaluation. Universities are communities of students, faculty, researchers, administrators and professional staff. Education involves relationships, mentorship, discussion, collaboration and intellectual exchange. Technology should therefore be evaluated not only according to what it automates but also according to what it enables, changes or potentially displaces.
AMTEF consequently requires evaluators to begin with the educational problem, desired learning outcomes and institutional objectives. Technology should then be examined as one possible means of achieving those objectives.
16.3 Learning Management Systems
Learning Management Systems, commonly known as LMS platforms, have become central components of modern educational infrastructure. An LMS can support content distribution, communication, assessment, grading, discussion, attendance management, feedback, student records and learning analytics. In many institutions, the LMS functions as a central digital environment connecting students, instructors, academic programmes and administrative systems.
The technical evaluation of an LMS should include reliability, availability, response time, scalability, integration capability, security and system performance. A platform that frequently becomes unavailable during assessment periods may create significant educational disruption even if its general functionality is extensive. Scalability is also important because a system that performs adequately for a small cohort may experience difficulties when used by thousands of students simultaneously.
Usability represents another major dimension. Students and faculty should be able to perform essential tasks without unnecessary complexity. Navigation, assignment submission, grading, feedback, communication and content access should be evaluated from the perspective of actual users. Usability testing can identify problems that may not be visible from technical specifications.
Accessibility must also be considered. An LMS should provide reasonable access for students with different abilities and different technological environments. Compatibility with assistive technologies, alternative content formats, captioning, keyboard navigation and appropriate visual design can influence educational participation.
Integration is another important evaluation dimension. Universities frequently use multiple systems for student information, finance, identity management, library services, examination management and learning analytics. Poor integration can create duplicate data entry, inconsistent records and additional administrative work.
Privacy and security are particularly important because LMS platforms contain substantial amounts of student and faculty information. Evaluation should consider authentication, authorization, data storage, retention, access control, encryption, incident response and third-party data processing.
Student engagement and faculty workload should also be evaluated. A platform may provide extensive functionality but create excessive administrative demands for faculty. Similarly, students may technically access a platform while making limited meaningful use of its learning resources.
An AMTEF evaluation should therefore consider the LMS as an institutional socio-technical system rather than merely as software. Technical performance, educational usefulness, organizational integration, user experience, privacy, security, accessibility and financial sustainability should be considered together.
16.4 Online Learning
Online learning can substantially increase geographical access to education by allowing students to participate without being physically present on a university campus. It can provide flexibility for working professionals, international students, geographically dispersed learners and students whose personal circumstances make conventional attendance difficult. Online delivery can also increase scalability by allowing educational institutions to reach larger populations.
However, increased access does not automatically guarantee educational effectiveness. The quality of online learning depends upon course design, instructor engagement, student support, digital infrastructure, assessment methods, communication, feedback and learner characteristics.
A technically sophisticated online platform may produce limited educational value if courses are simply converted from classroom lectures into static digital materials. Effective online education requires deliberate instructional design. Learning activities should be aligned with learning outcomes, students should receive appropriate feedback and opportunities should exist for interaction, collaboration and application of knowledge.
Student support is another important factor. Online learners may experience difficulties related to time management, isolation, technology access, digital literacy or competing responsibilities. Evaluation should therefore examine whether institutions provide adequate academic, technical and administrative support.
Digital access is also a significant consideration. Students may have different levels of internet connectivity, hardware availability and access to reliable electricity. A technology-dependent educational model may therefore unintentionally disadvantage students with limited infrastructure.
Online assessment requires particular attention. Institutions must consider authentication, academic integrity, assessment validity, accessibility and the suitability of different forms of assessment for online environments. The technology used for examination monitoring may itself create privacy, accessibility and ethical concerns.
AMTEF therefore encourages evaluation of online learning as an educational ecosystem. The evaluator should examine not simply whether the online platform functions but whether the complete educational system produces meaningful and equitable learning outcomes.
16.5 Blended Learning
Blended learning combines face-to-face educational activities with online and digital learning components. It can allow universities to use physical interaction where it provides particular educational value while using digital technologies to support flexibility, preparation, collaboration, assessment and access to resources.
The central evaluation question is whether technology adds meaningful educational value or simply reproduces existing teaching processes digitally. Uploading a lecture recording to an online platform, for example, does not necessarily create a meaningful blended-learning environment. The technology should contribute to the design of the learning experience.
Blended learning may enable students to review instructional material at their own pace, participate in online discussions, complete preparatory activities before classroom sessions and use face-to-face time for problem-solving, discussion, practical activities and higher-order learning.
Evaluation should therefore consider the relationship between online and face-to-face components. The two environments should ideally complement one another rather than operate as disconnected educational systems.
Faculty workload is also important. Poorly designed blended courses may require instructors to maintain two separate teaching systems without reducing other responsibilities. Technology adoption may therefore increase rather than reduce workload.
Student experience should similarly be evaluated. Excessive fragmentation between platforms can create confusion, while insufficient integration can increase cognitive and administrative burden.
An AMTEF evaluation should consequently consider educational outcomes, usability, accessibility, faculty workload, infrastructure, student engagement, cost, integration and sustainability when assessing blended learning.
16.6 Artificial Intelligence in Education
Artificial intelligence is increasingly being incorporated into education through intelligent tutoring systems, automated feedback, learning analytics, recommendation systems, adaptive learning, generative AI, administrative automation and research support. AI has the potential to personalize learning, provide rapid feedback, support instructional planning, assist with administrative tasks and provide students with new forms of interactive learning.
AI may also support educators by assisting with content development, lesson planning, language support, coding, data analysis and research exploration. Administrative applications may include scheduling, document processing, student support and information management.
However, AI introduces important risks. AI-generated information may be inaccurate, incomplete or fabricated. Algorithms may reproduce biases contained within training data or system design. AI systems may encourage dependency if users rely on generated answers rather than developing their own reasoning. Data-intensive systems may create privacy concerns, while automated decision-making may raise questions concerning accountability and transparency.
UNESCO's guidance on generative AI in education emphasizes human-centred use, institutional capacity, data protection and pedagogical considerations. These principles are relevant to AMTEF because AI should be evaluated not merely according to technical capability but according to its educational, ethical, organizational and social implications.
AI evaluation should therefore examine the relationship between automation and human agency. An important question is not simply whether an AI system can perform a task but whether that task should be automated and what role should remain with the educator or student.
The quality of AI-generated information should also be evaluated. Students and faculty require mechanisms for verification, particularly where AI-generated information is used in academic research, professional education or assessment.
AI evaluation should additionally consider equity. Students may have different levels of access to AI tools, different levels of AI literacy and different abilities to identify inaccurate outputs. Institutions should therefore consider whether AI adoption creates new forms of educational inequality.
Within AMTEF, AI in education should consequently be assessed across technical performance, educational effectiveness, human agency, academic integrity, privacy, security, equity, organizational readiness, environmental requirements and governance.
16.7 Generative AI and Academic Work
Generative AI has introduced significant changes in the way students, faculty, researchers and administrators can create and process academic information. Such systems can assist with brainstorming, language support, summarization, coding, research exploration, drafting, translation, administrative communication and other forms of knowledge work.
The educational question, however, is not simply whether students or faculty use AI. The more important questions concern how the technology is used, what learning objective is involved, what human intellectual work remains, how generated information is verified, how authorship is represented and how academic integrity is maintained.
For example, using AI to generate alternative explanations of a difficult concept may support learning if the student critically evaluates the explanations. Using AI to identify possible research questions may support research development if the researcher independently verifies the resulting ideas. Using AI to improve grammar may assist students whose primary challenge is language expression. In contrast, submitting an unverified AI-generated response as original intellectual work may undermine the educational objective of the assessment.
Universities should therefore distinguish between assistance and substitution. AI assistance may support human intellectual activity, whereas excessive substitution may remove the very cognitive processes that an educational programme is designed to develop.
Faculty also need guidance concerning appropriate use. Academic staff may use generative AI for lesson planning, administrative communication, coding assistance or preliminary drafting, but they remain responsible for verifying accuracy, protecting confidential information and ensuring that the resulting material is appropriate for students.
Research use requires similar care. AI-generated references, summaries or interpretations should be independently verified because generative systems may produce incorrect or nonexistent references. Researchers should maintain responsibility for the accuracy and integrity of their work.
The evaluation of generative AI should therefore focus on educational purpose rather than simple usage volume. High usage does not necessarily represent high educational value. The key question is whether AI contributes to the development of knowledge, critical thinking, creativity, research capability and other intended educational outcomes.
16.8 AI and Academic Integrity
Academic integrity requires students and academic staff to represent intellectual work honestly and responsibly. Generative AI complicates traditional assumptions concerning authorship because AI systems can generate text, code, images, analyses and other forms of content that may resemble human-created work.
Universities therefore need clear policies concerning permitted and prohibited uses of AI, disclosure expectations, citation practices, verification responsibilities and assessment requirements. However, policies should not focus exclusively on punishment. Students and faculty also need education concerning responsible AI use.
A university policy may distinguish between acceptable assistance, restricted use and prohibited substitution depending upon the educational context. For example, an instructor may permit AI-assisted brainstorming while requiring students to disclose its use and independently develop the final analysis. Another assessment may prohibit generative AI because independent production of the work is itself the learning objective.
Disclosure can provide transparency concerning how AI has been used. However, disclosure requirements should be clear and practical. Students should understand what constitutes AI assistance and how it should be acknowledged.
Verification is also essential. AI-generated material should not automatically be assumed to be accurate simply because it appears authoritative. Students and faculty should be encouraged to verify important factual claims against appropriate sources.
Academic-integrity policies should also recognize disciplinary differences. Appropriate AI use in computer science may differ from appropriate use in literature, medicine, engineering, law or theology. Institutional policies should therefore establish broad principles while allowing programmes and instructors to define context-specific requirements.
Assessment design is closely connected to academic integrity. If an assessment primarily measures the production of generic written text, generative AI may make it increasingly difficult to determine whether the assessment is measuring the intended capability. Assessment redesign can therefore be more educationally productive than relying exclusively upon AI-detection technologies.
AI-detection systems themselves should be evaluated carefully. False positives and false negatives can create significant consequences for students. A detection score should not automatically be treated as definitive evidence of misconduct. Academic decisions should be based upon appropriate evidence and institutional procedures.
16.9 Assessment Redesign
Technology evaluation in education should include assessment design because changes in technology can change the validity of traditional assessment methods. If an assessment can be completed effectively by simply prompting an AI system, educators may need to reconsider whether the assessment continues to measure the intended learning outcome.
Assessment should begin with the question of what students are expected to demonstrate. If the objective is critical thinking, an assessment should provide evidence of critical thinking. If the objective is professional competence, the assessment should provide evidence of the ability to apply knowledge in realistic contexts. If the objective is research capability, the assessment should provide evidence of research design, evidence evaluation, analysis and interpretation.
Possible approaches include oral defence, project-based assessment, process documentation, reflective analysis, authentic tasks, supervised assessments and staged assignments. These approaches can provide greater visibility into how students develop their work and how they reason through problems.
Process-oriented assessment may be particularly valuable. Instead of evaluating only a final product, instructors can examine research notes, drafts, methodological decisions, revisions, reflections and oral explanations. This provides evidence concerning the student's intellectual process.
Authentic assessment can also connect academic learning with real-world contexts. Students may be required to analyse a genuine organizational problem, design a technology evaluation, conduct a field investigation or defend a professional recommendation. Such tasks can be more difficult to complete meaningfully through simple AI prompting because they require contextual understanding and personal engagement.
Oral assessment can provide additional evidence of understanding. A student who can explain, defend and critically evaluate submitted work demonstrates a different form of evidence from a student who can merely submit a polished written document.
Assessment redesign should nevertheless avoid assuming that AI is inherently incompatible with learning. In some professional environments, responsible use of AI may itself be an important competence. Students may need to learn how to formulate effective prompts, verify AI-generated information, identify limitations, protect confidential information and exercise professional judgement.
The objective is therefore not necessarily to eliminate technology from assessment. Rather, assessment should be designed so that technology use is consistent with the intended educational outcome.
16.10 Learning Analytics
Learning analytics refers to the collection, analysis and interpretation of data concerning learners, learning processes and educational environments. Universities may use learning analytics to understand patterns of participation, identify students who may require support, evaluate course engagement and improve educational design.
Potential benefits include earlier identification of students experiencing difficulties, improved academic advising, more personalized feedback and evidence-based course development. For example, a university may identify patterns showing that students who stop accessing learning materials during particular stages of a course are more likely to experience difficulties. Such information could allow academic staff to intervene earlier.
However, learning analytics also creates important risks. Extensive monitoring can create perceptions of surveillance. Students may not fully understand what data are being collected or how they are being used. Predictive systems may also classify students according to risk categories, potentially affecting how they are treated by institutions.
Prediction accuracy must therefore be carefully examined. A system that predicts that a student is at risk does not establish that the student will actually fail. Incorrect predictions may result in unnecessary interventions or inappropriate assumptions about students.
Profiling also raises questions concerning fairness. If historical data contain structural inequalities, predictive models may reproduce those patterns. Evaluation should therefore examine performance across relevant student groups and investigate whether particular groups experience systematically different outcomes.
Transparency is another important consideration. Students should, where appropriate, receive understandable information concerning the purposes of data collection and the use of analytics. Institutions should establish clear governance regarding access, retention, security and permissible uses.
Learning analytics should therefore be evaluated according to both educational benefit and student rights. Its value should be demonstrated through meaningful improvements in learning or student support rather than through the volume of data collected.
16.11 Educational Data Privacy
Educational institutions collect substantial quantities of student and staff information. Student data may include grades, attendance, assessment records, learning activity, communications, demographic information, behavioural patterns, academic progress and other forms of institutional information.
The collection of such information can support educational planning and student services, but it also creates responsibilities concerning privacy, security and responsible data governance. Institutions should establish clear purposes for data collection and should avoid collecting information merely because it is technically possible to do so.
Access controls should specify who can access different categories of information and under what circumstances. Faculty may require access to information necessary for teaching and assessment, while administrative personnel may require different levels of access. Excessive access increases the consequences of potential misuse or security compromise.
Data retention is another important issue. Institutions should establish how long different categories of data need to be retained and when information should be securely deleted or anonymized where appropriate.
Transparency is essential. Students and staff should understand, in an appropriate manner, what information is collected, why it is collected, how it is used and with whom it may be shared.
Security should include technical and organizational measures. Authentication, authorization, encryption, monitoring, incident response and staff training may all be required to protect educational information.
Third-party providers create additional considerations. When universities use cloud-based LMS platforms, AI systems, analytics platforms or external educational applications, data may be processed outside the university's direct infrastructure. Contracts, data-processing arrangements, access rights, retention provisions and security responsibilities should therefore be evaluated.
Privacy evaluation should also consider secondary uses of data. Information originally collected for teaching may later be used for analytics, research, commercial services or system development. Such uses should be governed appropriately and should not be assumed to be automatically acceptable simply because the original collection was legitimate.
Within AMTEF, educational data privacy should therefore be evaluated as an integrated technical, legal, ethical, organizational and governance issue.
16.12 Adaptive Learning
Adaptive learning systems use information about student responses, performance or progress to modify learning activities, content, sequencing or difficulty. The underlying objective is to provide students with learning experiences that respond more closely to individual needs.
Adaptive systems may potentially support students who progress at different rates. A learner who demonstrates mastery may receive more advanced material, while a learner experiencing difficulties may receive additional explanation, practice or feedback.
Evaluation should examine whether personalization actually improves learning outcomes. A system may claim to be adaptive while making relatively superficial changes to content or difficulty. The evaluator should therefore establish what is being adapted, how adaptation decisions are made and whether those adaptations produce measurable educational benefits.
Accuracy is another important issue. If the system incorrectly estimates a student's level of knowledge, the resulting recommendations may be inappropriate. The evaluator should therefore examine the quality of the data and algorithms used to make adaptation decisions.
Student autonomy should also be considered. Excessive personalization may result in students having limited control over their learning pathways. Students should understand, where appropriate, how recommendations are generated and should have opportunities to seek human guidance.
The teacher's role is also important. Adaptive learning should not necessarily be viewed as a replacement for educators. Teachers can interpret student needs, provide encouragement, address misconceptions, facilitate discussion and make professional judgements that automated systems may not reproduce.
Accessibility should also be evaluated because adaptive systems may perform differently for students with different needs, languages or abilities.
AMTEF therefore requires adaptive learning to be evaluated not simply as an algorithmic technology but as part of a wider educational system involving students, educators, curriculum, data and institutional objectives.
16.13 Automated Assessment
Automated assessment can increase efficiency by supporting automated marking, feedback generation, question selection, plagiarism analysis, coding evaluation and other assessment processes. In large educational systems, automation may reduce administrative workload and provide students with faster feedback.
However, automated assessment introduces important questions concerning accuracy, fairness, transparency and accountability. An automated system may perform well for certain types of responses but poorly for nuanced reasoning, creative work, interdisciplinary answers or language varieties that were not adequately represented in its development data.
Evaluation should therefore examine the accuracy of automated assessment against appropriate human assessment. Differences between automated and human judgements should be analysed rather than assuming that the automated result is correct.
Fairness should also be considered. Assessment systems may perform differently across languages, disciplines, writing styles, student populations or accessibility requirements. An evaluation should therefore investigate whether systematic differences occur.
Transparency is particularly important when automated assessment influences grades or progression. Students should have appropriate information about the nature of the assessment process and, where significant consequences are involved, access to meaningful review or appeal mechanisms.
High-stakes automated assessment requires particularly strong evidence. The consequences of an incorrect assessment can include failure, delayed graduation, exclusion from opportunities or reputational harm. Human oversight may therefore be particularly important when automated systems are used in consequential academic decisions.
The evaluator should also distinguish formative and summative uses. Automated feedback used to help students improve their work may involve different risks from automated systems used to determine final grades.
Automated assessment should therefore be evaluated according to the educational purpose, the consequences of error, the nature of the assessment construct, the quality of the data and the degree of human oversight.
16.14 Virtual and Augmented Reality
Virtual Reality and Augmented Reality technologies can create immersive learning environments in which students interact with simulations, three-dimensional environments and digital representations of physical systems. These technologies have applications in medical education, engineering, laboratory simulation, professional training, architecture, geography and other disciplines.
The educational value of VR and AR should be evaluated in relation to specific learning outcomes. Immersion itself is not evidence of educational effectiveness. A highly sophisticated simulation may be unnecessary if a simpler technology achieves the same learning outcome at substantially lower cost.
For medical education, immersive simulation may allow students to practise procedures in controlled environments. Engineering students may explore complex systems or hazardous environments without being exposed to physical risks. Professional training may use simulated scenarios to develop decision-making and practical skills.
Evaluation should examine learning outcomes, usability, accessibility, cost, instructional value and technical reliability. Motion sickness, visual fatigue, physical discomfort and other user effects may also need to be considered.
Accessibility requires particular attention because some students may not be able to use specific VR or AR devices comfortably or effectively. Alternative learning pathways may therefore be necessary.
Infrastructure requirements can also be significant. VR systems may require specialized hardware, high-performance computing, technical support and dedicated physical space. Total cost of ownership should therefore be considered.
The evaluator should also consider whether immersive technology improves learning sufficiently to justify its additional complexity. AMTEF encourages comparison with realistic alternatives rather than evaluating the technology in isolation.
16.15 Digital Accessibility
Educational technology should be accessible to students with different abilities, technological circumstances and learning needs. Accessibility is therefore not simply a technical feature but a component of educational equity and participation.
Evaluation should consider compatibility with screen readers, captioning, keyboard navigation, alternative text, visual accessibility, audio accessibility, appropriate colour contrast, alternative content formats and other relevant accessibility requirements.
Accessibility should be considered during procurement rather than after a technology has already been purchased. If accessibility is treated as an afterthought, institutions may discover that major modifications are required or that some students cannot participate fully.
Accessibility should also be evaluated in the context of actual educational activities. A platform may technically support screen readers while individual course materials remain inaccessible because documents, diagrams, videos or assessments have not been prepared appropriately.
Digital accessibility should therefore be assessed across the complete learning environment, including the LMS, electronic documents, video content, assessment systems, communication tools, digital libraries and external applications.
Universities should also consider differences in devices and connectivity. Accessibility is not limited to disability. Students may access educational resources through mobile devices, low-bandwidth connections or older hardware. A technology that requires expensive high-performance equipment may create barriers for students with limited resources.
AMTEF therefore treats accessibility as an integrated technical, human, organizational and social dimension of technology evaluation. The relevant question is not merely whether the technology contains accessibility features but whether students can meaningfully participate in the educational experience.
16.16 Faculty Readiness
Faculty readiness is a critical component of educational technology adoption. Even technically advanced systems may produce limited value if educators lack the knowledge, skills, confidence, time or institutional support required to use them effectively.
Faculty readiness includes technical knowledge, pedagogical understanding, digital literacy, AI literacy, assessment design capability, data awareness, cybersecurity awareness and digital ethics. Different technologies require different combinations of these capabilities.
For example, an LMS requires faculty to understand course organization, digital communication, online assessment and feedback. Learning analytics requires some understanding of data interpretation and privacy. Generative AI requires awareness of prompt design, verification, intellectual property, academic integrity and appropriate disclosure. AI-supported assessment requires additional knowledge concerning reliability, fairness and human oversight.
Faculty development should therefore not be treated as a one-time training event. Effective capability development may require introductory training, practical workshops, peer support, instructional-design assistance, technical help, communities of practice and continuing professional development.
Time is also an important resource. Faculty may resist or underuse technology when implementation requires substantial additional work without adequate institutional support. Technology evaluation should therefore examine workload implications rather than assuming that faculty can absorb new responsibilities without consequence.
Institutional incentives also influence readiness. If promotion, recognition and workload systems do not acknowledge digital course development or educational innovation, faculty may have limited incentives to invest substantial time in technology adoption.
Leadership support is similarly important. Faculty are more likely to engage meaningfully with educational technology when institutional leadership provides clear expectations, resources, training and appropriate support.
Readiness should therefore be evaluated at both individual and institutional levels. A university may have technically capable individual faculty members while lacking institution-wide infrastructure, policies or support mechanisms. Conversely, an institution may possess strong infrastructure while faculty capability remains uneven.
AMTEF consequently treats faculty readiness as a multidimensional organizational capability rather than simply as an individual's willingness to use technology. Effective adoption requires alignment among technology capability, faculty competence, pedagogical objectives, institutional support, governance, resources and educational purpose.
Ultimately, the evaluation of educational technology should ask not merely whether faculty can operate a system but whether they can use it in ways that improve learning while maintaining academic standards, human judgement, student wellbeing and institutional quality.
16.17 Student Readiness
Student readiness is a central component of educational technology evaluation because students are the primary participants in many technology-enabled learning environments. A technology may be technically sophisticated and institutionally well supported, yet its educational value may be limited if students are not adequately prepared to use it. Student readiness encompasses digital literacy, access to appropriate devices and connectivity, confidence in using technology, previous experience, learning preferences, language capability, and the availability of technical and academic support. These characteristics can vary substantially within the same institution, particularly among students from different socioeconomic, geographical, cultural and educational backgrounds.
Digital literacy is broader than the ability to operate a computer or mobile device. Students increasingly need to understand how digital platforms function, how information is generated and evaluated, how to protect personal information, how to communicate appropriately online, and how to distinguish reliable information from misleading or fabricated material. In environments where artificial intelligence is increasingly integrated into learning and research, students also require AI literacy. This includes understanding what AI systems can and cannot reliably do, recognizing the possibility of fabricated or inaccurate outputs, understanding issues of bias and privacy, and knowing when human verification is necessary.
Student confidence is another important dimension. Students who have limited previous exposure to digital learning may experience anxiety or frustration when confronted with unfamiliar systems. A university may therefore interpret low technology usage as evidence that students do not value the technology when the underlying issue is inadequate orientation or support. Evaluation should distinguish between lack of interest, lack of capability, lack of access and lack of confidence. These are different problems and require different institutional responses.
Learning preferences also deserve consideration. Some students may benefit from asynchronous learning because it allows them to study at flexible times, whereas others may benefit from synchronous interaction, classroom discussion, practical activities or direct interaction with academic staff. Technology evaluation should therefore avoid assuming that a single technological format is equally effective for all learners. The objective should be to determine whether the technology supports the intended educational outcomes across relevant student groups.
Access to technological resources is equally important. Students may differ in the availability of computers, smartphones, reliable electricity, broadband connections, quiet study environments, assistive technologies and suitable software. A technology that assumes continuous high-speed internet access may function effectively for some students while creating substantial barriers for others. Consequently, student readiness should be evaluated as part of the wider institutional readiness framework rather than being treated simply as an individual student characteristic.
From an AMTEF perspective, student readiness involves technical, educational, human, social, economic, ethical and organizational considerations. Universities should establish whether students have the resources, skills, confidence and support necessary to use a technology meaningfully. Technology adoption should ultimately improve educational opportunity rather than create new barriers to participation.
16.18 Digital Divide
The digital divide refers to unequal access to digital technologies, connectivity, skills, resources and opportunities to benefit from digital systems. In higher education, the digital divide is not limited to whether a student possesses a device. It can involve the quality of the device, reliability and affordability of internet access, availability of electricity, digital skills, language capability, technical support, accessible software, learning environments and the ability to use technology effectively for educational purposes.
The first-dimension concerns physical access. Students may possess different types of devices, ranging from advanced computers to smartphones with limited processing capability. A university platform that functions effectively on a high-performance computer may provide a substantially different experience on a mobile phone. Software requirements, file sizes, video streaming, specialized applications and virtual laboratories can therefore create hidden inequalities.
The second-dimension concerns connectivity and affordability. High-speed internet may be available in urban areas but less reliable or more expensive in rural or geographically isolated communities. Even when connectivity exists, students may face financial constraints associated with data packages, software subscriptions, device maintenance, electricity and other technology-related expenses. The nominal availability of an online service therefore does not necessarily mean that all students can use it equally.
The third-dimension concerns skills. Students may have access to identical devices but possess very different levels of digital competence. A student who can efficiently search academic databases, use reference-management software, analyze data and navigate learning platforms may gain substantially more value from technology than a student who has only basic operational skills. Digital inequality can therefore persist even when hardware access becomes widespread.
The fourth-dimension concerns support. Students may require technical assistance, academic guidance, disability support, language support and training. Institutions that introduce new technologies without establishing appropriate support mechanisms may unintentionally transfer the costs of implementation from the institution to the student.
Technology evaluation should therefore ask whether an intervention expands educational opportunity or reinforces existing inequalities. Universities should examine participation rates, completion rates, usage patterns, support requests and learning outcomes across relevant student groups. Equity should not be treated as an additional consideration after implementation; it should form part of the original technology evaluation and procurement process.
The digital divide also has an important international dimension. Transnational education, distance doctoral education and global online programmes connect learners across countries with very different technological infrastructures. A technology that performs well in one national environment may be difficult to use in another. Consequently, universities operating internationally should evaluate technology across the actual contexts in which students are expected to use it.
16.19 Technology and the University Workforce
Technology increasingly changes the nature of academic, administrative and professional work within universities. Digital systems can automate administrative processes, support student advising, assist research, provide automated feedback, manage records, facilitate communication and improve institutional reporting. However, technological change should not automatically be interpreted as simple job replacement. In many cases, technology changes the composition of work by transferring particular tasks from humans to machines while creating new responsibilities for human workers.
For academic staff, technology may reduce the time required for certain repetitive activities while increasing the importance of interpretation, mentoring, curriculum design, assessment development and student support. An AI system may assist with preliminary feedback, for example, but academic staff may still need to determine whether the feedback is appropriate, whether it reflects disciplinary expectations and whether it addresses the particular learning needs of the student.
Administrative work may similarly be transformed. Automated admissions systems, digital document processing, chatbots, scheduling systems and workflow platforms can reduce repetitive tasks. At the same time, staff may become responsible for monitoring exceptions, handling complex cases, maintaining data quality and resolving situations that automated systems cannot adequately address.
Technology evaluation should therefore focus on task transformation. The relevant question is not simply whether a technology eliminates jobs, but how it changes tasks, responsibilities, skills, decision-making authority, workload, accountability and professional relationships. Some technologies may eliminate particular activities while creating new activities elsewhere in the organization.
Workload is also an important consideration. A technology may appear to increase efficiency while generating additional hidden work. Faculty may have to learn new systems, redesign assessments, respond to additional digital communications, monitor AI-generated outputs or maintain multiple platforms. Administrative staff may need to reconcile data between systems or address errors generated by automation. These indirect workload effects should be included in technology evaluation.
The human consequences of technology therefore form an important component of AMTEF evaluation. Universities should consider not only productivity and cost but also professional autonomy, workload, competence, job redesign, training requirements, accountability and the quality of academic and administrative services.
16.20 AI-Assisted Research
Artificial intelligence is increasingly being incorporated into research processes. Researchers may use AI systems to discover literature, organize information, generate search terms, assist with coding, explore datasets, support statistical analysis, improve language, translate material, identify themes and generate preliminary ideas. These applications can potentially reduce some forms of repetitive work and assist researchers in navigating increasingly large bodies of information.
However, the use of AI in research requires careful evaluation because AI systems can generate plausible but incorrect information. A system may produce a citation that does not exist, attribute an argument to the wrong author, misunderstand research finding or generate an apparently reasonable interpretation that is unsupported by the underlying evidence. The fluency of an AI-generated response should therefore never be confused with scholarly validity.
Source verification is particularly important. Researchers remain responsible for determining whether sources exist, whether they are relevant, whether the source actually supports the statement being made and whether the source is appropriate for the research question. AI can assist with discovery and organization, but the scholarly researcher remains responsible for evidence evaluation.
Confidentiality is another important issue. Researchers may work with unpublished manuscripts, proprietary organizational information, participant data, commercially sensitive information or research material subject to ethical restrictions. Entering such material into an external AI system without appropriate authorization may create confidentiality and data-protection concerns. Universities should therefore establish clear rules concerning which categories of information may be entered into AI systems.
Intellectual property is also relevant. Researchers need to consider ownership, licensing, copyright, contractual obligations and the terms under which AI services process uploaded or generated material. These questions become particularly important when research involves third-party data, unpublished material, proprietary datasets or collaborative intellectual property.
Research integrity remains the overarching principle. AI may support research activity, but it should not be treated as an independent scholarly authority. The researcher must retain responsibility for the research question, methodological choices, evidence interpretation, conclusions and reporting of results. AI-assisted research should therefore remain human-supervised, evidence-based and transparent.
16.21 Research Integrity
The increasing use of digital technologies and artificial intelligence creates new opportunities for research while also introducing new forms of research-integrity risk. Researchers may encounter fabricated information, inaccurate citations, manipulated or incorrectly interpreted data, inappropriate attribution of authorship, unauthorized disclosure of confidential information and insufficient disclosure of technology-assisted research processes.
Fabricated information is particularly significant in generative AI systems. A system may produce statements that appear scholarly but cannot be supported by reliable evidence. Researchers must therefore verify factual claims and references before incorporating AI-generated material into scholarly work. Citation verification is not merely a technical exercise; it is part of the researcher's responsibility for the integrity of the scholarly record.
Data integrity is equally important. AI-assisted analysis may introduce errors through incorrect coding, inappropriate model assumptions, data transformation problems or misinterpretation of statistical outputs. Researchers should therefore retain appropriate records of analytical procedures and verify important results independently.
Authorship and contribution also require careful consideration. AI systems do not possess scholarly responsibility in the same sense as human researchers. Decisions concerning authorship should therefore remain governed by the relevant academic and publication standards. The central issue is not simply whether AI was used but whether human researchers can take responsibility for the intellectual contribution and integrity of the work.
Confidentiality is particularly important in research involving human participants. Research data may contain personal, sensitive or identifiable information. Researchers must follow applicable ethical approvals, institutional policies and data-protection requirements when using digital or AI systems.
Research integrity therefore requires a culture of verification. Researchers should check AI-generated information against authoritative sources, maintain appropriate research records, disclose relevant AI use where required, protect confidential material and critically assess all outputs. The use of sophisticated technology does not transfer scholarly responsibility from the researcher to the technology.