Chapter 4. What AI Can and Cannot Do

4.1 Strong use cases

AI is often useful for first drafts, summarisation, translation, classification, brainstorming, extracting fields from documents, generating test cases, explaining code, converting formats, creating outlines, and identifying patterns for human review. It can be especially valuable when a person can judge the output and when mistakes are easy to detect and correct.

AI can also support accessibility through captioning, speech recognition, text simplification, and alternative ways to interact with digital content. Such support should be tested with the people who will use it, because errors can undermine accessibility rather than improve it.

4.2 Tasks that require extra caution

High-impact tasks include medical diagnosis, legal determinations, hiring, school admissions, credit decisions, public benefits, employee monitoring, safety-critical operations, and decisions affecting fundamental rights. AI may assist with limited parts of these workflows, but should not be treated as an unreviewed substitute for qualified judgement or legally required procedures.

Where errors could cause serious harm, require validated performance evidence, representative testing, documented responsibility, human review, appeal or correction routes, and ongoing monitoring. In some cases, the appropriate decision is not to use AI.

4.3 Reliability and uncertainty

AI reliability is task-specific. A model may perform well on routine questions and fail on unusual cases, ambiguous instructions, rare languages, outdated facts, or situations that differ from its training data. The confidence or tone of an answer is not a dependable measure of correctness.

Use a verification ladder: identify which claims matter; compare them with authoritative sources; test calculations independently; inspect quotations and references; seek a domain expert for specialised issues; and disclose uncertainty where evidence remains incomplete.

4.4 A simple decision rule

Use AI when it provides a measurable advantage and the risks can be managed. Use conventional software when the task is deterministic and well specified. Use a human expert when interpretation, accountability, empathy, or specialised judgement is essential. Use a combined workflow when AI can accelerate a task but a person must verify or approve the result.

4.5 The limits of anthropomorphism

Conversational systems can sound friendly, thoughtful, or certain. These features are interface behaviours, not proof of human consciousness, feelings, intentions, or moral responsibility. Treat the system according to its demonstrated capabilities and documented limitations, rather than assuming that fluent conversation means human-like understanding.