Chapter 7. Cybersecurity, Fraud, and AI-Enabled Abuse

7.1 AI changes the scale and style of abuse

AI can help defenders summarise alerts, classify suspicious activity, review code, and improve incident response. The same broad capabilities can help malicious actors produce more convincing messages, impersonate trusted people, automate reconnaissance, or adapt scams to a target. The central security lesson is not that every attack is AI-generated; it is that familiar threats can become cheaper, faster, more personalised, or harder to recognise.

Phishing remains dangerous because it exploits trust, urgency, routine, and authority. A message with perfect grammar may still be fraudulent. A familiar voice or face may not be reliable proof of identity. Verification must depend on trusted procedures, not merely on whether the communication looks or sounds authentic.

7.2 Defensive practices for individuals

• Use unique passwords and a reputable password manager; enable multi-factor authentication where available.
• Keep devices, browsers, and applications updated and use reputable security protection.
• Verify unusual payment, account-change, password-reset, or confidential-data requests through a second, known channel.
• Never disclose passwords, one-time codes, recovery phrases, or private authentication information in response to an unexpected message.
• Use privacy settings and limit public posting of personal details that could support impersonation.
• Back up important information and know how to report suspected account compromise.
• Teach family members and colleagues that a voice call or video appearance alone may not establish identity.

7.3 Organisational controls

Organisations should maintain an asset inventory, access controls, patching, backups, incident response, supplier oversight, and staff awareness. AI-specific controls include restricting what connected assistants can read or change, preventing untrusted content from overriding system instructions, validating tool calls, logging consequential actions, and requiring approval for sensitive operations.

Prompt injection is a class of risk in which instructions embedded in untrusted content attempt to manipulate an AI system’s behaviour. It matters especially when a system reads external documents and also has access to tools or private data. Defences include separating trusted instructions from untrusted content, limiting permissions, treating retrieved material as data rather than authority, validating outputs, and testing the complete workflow. No single prompt or filter should be treated as a complete security boundary.

7.4 Responding to a suspected incident

Act quickly but preserve evidence. Disconnect or disable affected access when appropriate, contact the organisation’s security team or service provider through known channels, change compromised credentials from a trusted device, revoke suspicious sessions or tokens, and follow the relevant reporting process. Do not investigate beyond your authority or share sensitive evidence publicly. Record what happened, when it was detected, what systems and data may be affected, and which actions were taken.