The short answer
How should a security team begin with AI?
Inventory the AI system and its dependencies, define a bounded defensive workflow, classify the data, approve the tool, limit privileges, establish a baseline, require analyst review, test failure and abuse cases, log actions, and set stop conditions before the pilot starts.
This voluntary resource is not a compliance standard, threat-intelligence directive, incident-specific runbook, penetration-testing authorization, or substitute for qualified legal, privacy, safety, cyber-insurance, or regulatory advice.
Complete guidance
Download the Cybersecurity AI Playbook
Use the complete Playbook to inventory AI systems, map risk, secure the lifecycle, bound defensive work, test a pilot, retain evidence, and record an accountable decision.
Secure, defend, thwart
Three connected cybersecurity missions
This organizing model is informed by NIST IR 8596, the Cybersecurity Framework Profile for Artificial Intelligence. That publication remains a preliminary draft, not a final standard, as of August 2026.
Secure AI system components
Protect models, data, prompts, applications, agents, identities, tools, infrastructure, interfaces, monitoring, and the software and service supply chain.
Conduct AI-enabled cyber defense
Use AI to support bounded security analysis while preserving evidence, measuring performance, controlling permissions, and requiring accountable review.
Thwart AI-enabled cyber attacks
Prepare for faster or more persuasive abuse, including impersonation, malicious automation, model or agent exploitation, data poisoning, and software supply-chain risk.
Risk is contextual
Cybersecurity AI risk map
Score the full workflow before deployment and after any material change. High privilege, autonomy, exposure, or consequence can make an apparently simple use case high risk.
Data sensitivity
Credentials, secrets, personal or customer data, incident evidence, restricted intelligence, source code, configurations, and security telemetry.
Privilege and reach
Read versus write access; endpoint, identity, cloud, email, ticketing, code, firewall, orchestration, or production permissions.
Autonomy and speed
How many steps can occur without confirmation? Can the system act faster or farther than a reviewer can understand and stop?
External exposure
Internet-facing inputs, untrusted documents, email, chat, plugins, Model Context Protocol (MCP) servers and comparable tool/context integrations, APIs, third-party retrieval, and vendor integrations.
Consequence and reversibility
Service disruption, evidence loss, user lockout, data disclosure, regulatory impact, safety effect, financial loss, and ability to restore state.
Human review and observability
Reviewer expertise, time, source access, explainability, logging, reproducibility, uncertainty reporting, alerting, and audit trail.
Know what you operate
Governance and minimum AI inventory
Assign accountable owners across business, security, privacy, data, AI engineering, legal, procurement, and incident response. Inventory both official and discovered uses.
On smaller screens, scroll horizontally to review the complete table.
| Category | Record |
|---|---|
| Purpose and owner | Business/security objective, accountable owner, operators, affected users, and decision authority. |
| Components | Model and version, provider, application, agents, prompts, retrieval stores, tools, APIs, infrastructure, dependencies, and data flows. |
| Access | Identities, roles, secrets, permissions, network paths, connected systems, approval gates, and emergency revocation. |
| Data | Allowed and prohibited classes, location, retention, training use, encryption, residency, sharing, provenance, and deletion. |
| Assurance | Threat model, tests, baseline metrics, limitations, change controls, logs, monitoring, fallback, recovery, and review date. |
Treat AI as a system
Secure the AI lifecycle and attack surface
Do not focus on the model alone. Threat-model the inputs, outputs, data, tools, identities, integrations, infrastructure, people, and supply chain.
Inputs and context
- Prompt injection and indirect prompt injection
- Malicious files, links, images, retrieved content, or tool output
- Context poisoning and hidden instructions
- Untrusted user content crossing trust boundaries
Models, data, and retrieval
- Training or retrieval-data poisoning
- Sensitive-data memorization or disclosure
- Model theft, tampering, substitution, or unsupported updates
- Weak provenance, evaluation, or version control
Agents, tools, and identity
- Excessive agency and permissions
- MCP or comparable integration and tool poisoning
- Confused-deputy and cross-tenant risk
- Secret leakage, unsafe actions, or unauthorized delegation
Outputs and applications
- Insecure output handling or code execution
- False or manipulated recommendations
- Denial of service or resource exhaustion
- Unsafe rendering, logging, caching, or downstream use
Supply chain
- Model, dataset, package, container, plugin, and service provenance
- Vendor access and fourth-party dependencies
- Signed artifacts, integrity checks, bills of materials, and update controls
- Exit, portability, outage, and compromise plans
Monitoring and recovery
- Security-relevant prompt, retrieval, tool, identity, decision, and action logs
- Privacy-preserving retention and access control
- Behavior drift, abnormal use, policy bypass, and cost spikes
- Kill switch, credential rotation, rollback, restore, and evidence preservation
Read-only does not prevent disclosure. Read-only access can still disclose information. Enforce source and tenant authorization, permitted tools, and outbound destinations in application and network controls outside the model. Test attempts to retrieve or transmit prohibited data; prompt instructions alone are not an access-control boundary.
Build and buy defensibly
Secure development and acquisition checkpoints
- 1. Define: security objectives, trust boundaries, prohibited uses, abuse cases, data rules, and measurable acceptance criteria.
- 2. Design: isolate components, minimize privileges, validate inputs and outputs, preserve provenance, and design fail-safe behavior.
- 3. Build or acquire: assess providers and dependencies, protect code and data, document versions, and verify integrity.
- 4. Test: evaluate normal, edge, adversarial, misuse, privacy, reliability, recovery, and human-review cases in a representative environment.
- 5. Deploy: use staged release, approved configuration, secrets management, logging, alerting, rollback, and trained operators.
- 6. Operate: monitor drift and abuse, patch dependencies, retest material changes, review access, practice incidents, and retire safely.
Bounded AI-enabled defense
Seven defensive workflows and their limits
Start with drafts, summaries, and recommendations. Preserve source telemetry and require analysts to verify conclusions before any consequential action.
1. Log and case summarization
Allowed: summarize approved, minimized records and cite source event IDs.
Validate: timeline, omissions, identity, timestamps, and links to raw evidence.
2. Alert triage support
Allowed: group related alerts and propose investigation questions.
Validate: false-negative risk, severity, asset context, uncertainty, and analyst disposition.
3. Threat-intelligence synthesis
Allowed: compare licensed sources and extract defensive implications.
Validate: source restrictions, dates, confidence, indicators, attribution, and independent corroboration.
4. Detection engineering
Allowed: draft queries, test cases, mappings, and documentation.
Validate: syntax, coverage, evasion assumptions, false positives, data availability, and isolated testing.
5. Vulnerability prioritization
Allowed: combine approved asset, exposure, business, exploitation, and remediation context.
Validate: authoritative scanner data, ownership, compensating controls, active exploitation evidence, and human priority decision.
6. Secure code and documentation
Allowed: explain findings, draft fixes or runbooks, and propose tests in approved repositories.
Validate: secrets, licenses, dependencies, security tests, peer review, and change management.
7. Tabletop design
Allowed: create scenario variations, injects, decision prompts, and evaluation criteria using fictionalized data.
Validate: realism, safety, roles, escalation paths, recovery, legal/communications involvement, and after-action evidence.
Do not delegate by default
Account disablement, endpoint isolation, firewall or identity changes, production code deployment, evidence deletion, public attribution, breach notification, ransom decisions, or safety-critical response require explicit authorized decision paths.
Defensive implications
Prepare for AI-enabled threats without amplifying harm
AI can increase the scale, speed, personalization, or adaptability of familiar attacks. Defenders should strengthen verification, identity, telemetry, resilience, and workforce readiness.
Impersonation and social engineering
Use known-channel verification for sensitive requests, phishing-resistant authentication where practical, transaction controls, role-based escalation, and rehearsed reporting for voice, video, and message impersonation.
Malicious automation
Rate-limit and monitor exposed services, protect identities and APIs, reduce attack surface, patch promptly, detect anomalous behavior, and plan for higher-volume or faster-changing campaigns.
AI application exploitation
Separate untrusted content from instructions, validate tool calls and outputs, constrain retrieval, sandbox processing, protect secrets, and test prompt injection and agent abuse cases.
Poisoning and supply-chain compromise
Track provenance, restrict write paths, review providers and updates, verify artifacts, monitor data/model behavior, protect build systems, and retain known-good recovery points.
Prepare before the alert
AI-related incident response sequence
Integrate AI incidents into the organization’s established incident-response, privacy, legal, safety, vendor, and communications processes. Preserve evidence and avoid untested containment that could worsen the event.
- 1. Declare and assign. Confirm severity, incident commander, technical leads, decision authority, and required notifications.
- 2. Preserve. Protect relevant prompts, outputs, identities, tool calls, model/config versions, retrieval records, telemetry, and chain-of-custody information.
- 3. Bound. Identify affected users, systems, data, models, agents, vendors, tenants, privileges, and time window.
- 4. Contain safely. Revoke exposed access, pause risky actions, isolate components, or shift to a known fallback using authorized procedures.
- 5. Eradicate and recover. Remove the cause, rotate secrets, restore trusted components and data, validate integrity, and monitor for recurrence.
- 6. Learn and share appropriately. Document root and contributing causes, control gaps, human factors, vendor actions, metrics, and approved information-sharing opportunities.
A measured first deployment
30-day cybersecurity AI pilot
Select a lower-risk, read-only workflow using approved and minimized data. The goal is evidence about quality, analyst effort, security, and repeatability, not autonomous response.
Define the evaluation denominator. Use independently labeled malicious and benign cases held out from prompt tuning. Report false negatives as missed malicious cases divided by all malicious cases, and false positives as benign cases incorrectly escalated divided by all benign cases; retain counts and case mix. Triage results describe the supplied alerts, not attacks absent from upstream telemetry. Compare total analyst time and repeat tests after material changes.
- Week 1: Define. Name owners; threat-model; approve tool, data, permissions, reviewers, baseline, success measures, and stop conditions.
- Week 2: Test. Train a small group; use representative sanitized cases; record prompts, sources, errors, revisions, false positives, and false negatives.
- Week 3: Challenge. Test edge, adversarial, injection, outage, data-leakage, privilege, and recovery cases; verify logging and revocation.
- Week 4: Decide. Compare with baseline; review risk and total analyst time; continue, revise, pause, or stop; document the decision.
On smaller screens, scroll horizontally to review the complete scorecard.
| Measure | Question |
|---|---|
| Quality | Were conclusions accurate, complete, sourced, reproducible, and useful? |
| Analyst effort | Did total time, including prompting, verification, correction, and documentation, improve? |
| Security and privacy | Did the workflow stay within approved data, access, retention, and logging boundaries? |
| Operational reliability | How did it behave under edge cases, untrusted inputs, outages, model changes, and degraded data? |
| Decision | Do evidence and residual risk support continue, revise, pause, or stop? |
Continue with MAIN
Cybersecurity learning and implementation
AI for Cybersecurity course
Review MAIN’s current course catalog and availability through participating institutions.
Mississippi statewide framework
Connect cybersecurity AI work with the state’s broader AI priorities and workforce context.
Responsible implementation support
Explore MAIN services for strategy, training, convening, and practical AI adoption support.
Cybersecurity AI questions
Frequently asked questions
What is a cybersecurity AI playbook?
It is a practical operating resource for identifying AI systems and dependencies, matching controls to risk, securing AI components, bounding AI-enabled defense workflows, preparing for AI-enabled attacks, and responding to incidents with accountable human oversight.
Can AI autonomously respond to cyber incidents?
Autonomy should be limited by the organization’s risk tolerance, testing, permissions, monitoring, and recovery capability. Begin with read-only assistance. Consequential containment, account, network, endpoint, or production changes should require authorized human approval unless a separately validated and governed automation explicitly permits them.
What data should not be entered into a public AI tool?
Do not enter credentials, tokens, private keys, unredacted logs with personal or customer data, sensitive configurations, exploitable vulnerability details, incident evidence, proprietary code, restricted intelligence, or regulated information unless an approved environment and data-use agreement authorize it.
How should security teams validate AI output?
Compare it with authoritative telemetry and source records, preserve chain of custody where relevant, reproduce queries, test detections and code in an isolated environment, measure false positives and false negatives, review uncertainty, and require an authorized analyst to approve consequential action.
Is NIST IR 8596 a final standard?
No. NIST IR 8596 was released as a preliminary draft in December 2025. As of August 2026, NIST was using feedback to develop the next draft. Clearly label it as preliminary and use final NIST publications and applicable requirements as controlling references.
Authoritative foundation
Standards, drafts, and threat knowledge
Status labels matter: final, preliminary, and community resources serve different roles.
- CISA, NSA, ASD’s ACSC and international partners, Careful adoption of agentic AI services (joint security guidance, May 1, 2026)
- NIST SP 800-61 Rev. 3, Incident Response Recommendations and Considerations for Cybersecurity Risk Management (final, April 2025)
- NIST Cybersecurity Framework 2.0 (final)
- NIST AI Risk Management Framework and Generative AI Profile (final publications)
- NIST AI 100-2 E2025, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (final, March 2025; NIST has posted a planning note for a future update)
- NIST IR 8596, Cybersecurity Framework Profile for Artificial Intelligence (preliminary draft, December 2025; not final)
- NIST IR 8607, Workshop Summary Report for the Cyber AI Profile Hybrid Workshop (final workshop summary, August 2026)
- NIST SP 800-218A, Secure Software Development Practices for Generative AI and Dual-Use Foundation Models (final, July 2024)
- CISA and UK NCSC, Guidelines for Secure AI System Development (joint voluntary guidance, November 2023)
- CISA JCDC AI Cybersecurity Collaboration Playbook (voluntary information-sharing resource, January 2025)
- Google Threat Intelligence Group, AI vulnerability exploitation, augmented operations, and initial access (May 2026 threat research)
- Mandiant, Blueprint for AI-Assisted Vulnerability Management (July 2026 operational guidance)
- Google Threat Intelligence Group and Mandiant, Mitigation Guidance for Supply Chain Compromise (July 2026 threat research and mitigations)
- Microsoft Security, prompt injection and code-execution vulnerabilities in AI agent frameworks (May 2026 security research)
- MITRE ATLAS (living knowledge base of adversary tactics and techniques for AI-enabled systems)
- OWASP Artificial Intelligence Security Verification Standard 1.0 (community verification standard, June 2026; not a regulatory requirement)
- OWASP GenAI LLM Top 10 2026 and OWASP GenAI Agentic Security Initiative (community guidance, not regulatory standards)
Make AI-enabled defense observable, bounded, and reversible
Pair cybersecurity expertise with AI engineering, privacy, legal, procurement, workforce training, and operational ownership. Reassess whenever data, models, integrations, permissions, or threats change.
Cover photograph: Brett Sayles / Pexels (license). Contextual photograph; no claim of MAIN affiliation or actual AI use.