AI and cybersecurity: how attackers and defenders use it

Cybersecurity is an arms race, and AI is now on both sides. Attackers use it to make scams more convincing and operations more efficient; defenders use it to detect threats faster, find vulnerabilities before attackers do and cope with a shortage of skilled staff.

How attackers use AI

The UK’s National Cyber Security Centre assessed in January 2024 that AI will almost certainly increase the volume and impact of cyberattacks in the following years, mainly by improving existing tactics rather than creating entirely new ones. Key uses:

• Phishing and social engineering: fluent, personalized emails and messages in any language, without the errors that once revealed scams.
• Voice and video deepfakes: impersonating executives or relatives to request money transfers or credentials. In a widely reported 2024 case in Hong Kong, an employee transferred about US$25 million after a video call in which other participants, including the apparent chief financial officer, were deepfakes.
• Reconnaissance: summarizing information about targets from public sources.
• Malware development: helping less skilled attackers write or modify code, though safety measures in mainstream models and the need for expertise limit this.
• Vulnerability discovery: analyzing code for exploitable flaws.

How defenders use AI

• Threat detection: machine learning has long been used to spot malware, spam and anomalous network behavior. Modern systems analyze vast logs to flag suspicious patterns.
• Security copilots: assistants that help analysts investigate alerts, explain suspicious scripts, write detection queries and summarize incidents, speeding up security operations centers.
• Phishing defense: models that analyze emails’ content, sender behavior and links.
• Vulnerability discovery: Google’s Project Zero and DeepMind reported in 2024 that their Big Sleep agent found a previously unknown exploitable bug in the widely used SQLite database engine, which was fixed before it appeared in an official release — an early example of AI finding real-world vulnerabilities.
• Automated patching: DARPA’s AI Cyber Challenge (AIxCC) asked teams to build AI systems that find and fix vulnerabilities in open-source software automatically; finalists demonstrated systems that discovered and patched many bugs.

Securing AI itself

AI systems introduce new attack surfaces:

• prompt injection that hijacks AI assistants and agents — see our prompt injection explainer;
• data poisoning of training data;
• model theft and extraction;
• leaking sensitive data through AI tools;
• supply chain risks in models and libraries downloaded from public hubs.

Government agencies, including the NCSC and US CISA with international partners, have published guidelines for secure AI system development.

Defensive advantage?

Experts debate whether AI favors attackers or defenders. Attackers need to succeed once; defenders must protect everything. But defenders own their systems and data, and AI that finds and fixes bugs at scale could shift the balance toward security — if organizations adopt it quickly.

What organizations should do

1. Train staff to verify unusual requests through separate channels, regardless of how authentic they seem.
2. Use phishing-resistant authentication such as passkeys or hardware keys.
3. Adopt AI-assisted detection and response tools where they demonstrably help.
4. Inventory and secure AI tools used internally.
5. Patch quickly, since AI may shorten the time between disclosure and exploitation.

Skills for security teams

Security professionals increasingly need to understand how AI systems work, how to evaluate AI security tools, and how to secure AI applications against new attack types. Familiarity with AI-assisted investigation and with threat models for agents is becoming part of the job.

What to watch next

• AI agents finding and patching vulnerabilities at scale.
• Deepfake-enabled fraud and verification countermeasures.
• Security standards for AI agents and tool integrations.
• Regulatory cybersecurity requirements for AI systems.

Key takeaways

• The NCSC expects AI to increase the volume and impact of cyberattacks.
• Attackers use AI for phishing, deepfakes, reconnaissance and code.
• Defenders use AI for detection, security copilots and vulnerability discovery.
• Big Sleep found a real SQLite vulnerability before release in 2024.
• AI systems themselves need protection against prompt injection, poisoning and data leaks.

Frequently asked questions

Can AI write malware?

Mainstream models have safeguards, but AI can help with parts of malicious development. The bigger near-term impact is on phishing and social engineering.

How do I protect against deepfake scams?

Verify requests for money or credentials through a separate, known channel, use code words for sensitive requests, and require multiple approvals for large transfers.

Will AI replace security analysts?

AI automates parts of investigation and triage, but skilled analysts remain essential to judge context, respond and improve defenses.