MAIN AI Playbook
Cybersecurity AI Playbook
A practical guide to govern AI systems, secure AI components, bound AI-enabled defense, prepare for AI-enabled threats, validate outputs, and respond to AI-related incidents.
Bound the workflow. Verify the defense.
- Audience
- Security leaders, analysts, IT and engineering teams, and risk, compliance, and procurement reviewers.
- Produces
- An AI inventory, bounded defensive workflow, incident record, and controlled pilot decision.
- Boundary
- Qualified people retain authority for consequential security actions and incident decisions.
Periodically reviewed to stay up to date.
Complete guidance
Download the Cybersecurity AI Playbook
Use the complete Playbook to inventory AI systems, map risk, secure the lifecycle, bound defensive work, test a pilot, retain evidence, and record an accountable decision.
Wide tables scroll horizontally. Focus a table to use the arrow keys.
In this playbook
Built on Mississippi’s Statewide AI Framework
The Mississippi Statewide AI Framework (August 2026) is the foundation for this playbook. It supplies shared priorities and a learning progression; this playbook applies them to AI security, bounded defensive work and cybersecurity workforce learning. Both are voluntary. Applicable law, adopted standards and institutional authority remain controlling.
Read the August 2026 Statewide AI Framework (PDF)
Learning progression. Workforce: technical practice through security leadership. This is a specialist application; it does not require every learner to become a security engineer.
Five priorities in practice
Train analysts for usable, approved access (1); challenge conclusions and effects (2); enforce data, identity and system controls (3); practice transferable defensive skills (4); compare labeled cases, total effort and failure evidence (5).
Priority numbers refer to literacy and access (1), ethics and critical thinking (2), privacy and governance (3), workforce readiness (4), and strategic alignment and measurable outcomes (5).
| Framework domain | Practice and evidence to retain |
|---|---|
| 11. Cyber Security and Safety | Test privileges, malicious inputs, leakage and recovery; retain observed results. |
| 6. AI-Enabled Workflow and System Design | Map tools, identities, data and escalation; retain the inventory and threat model. |
| 3. Critical Evaluation of AI Outputs | Verify conclusions against telemetry; retain labeled counts and correction evidence. |
| 8. Human Agency, Judgment, and Restraint | Keep consequential response with authorized people; retain action approval and decision rationale. |
Keep the shared human-review boundary
Apply the Framework's human-review triggers (p. 35) before action: assessment or credentials; financial allocation or compensation; health or physical safety; discipline or employment; legal or compliance content; and sensitive or protected information. Its additional agent triggers cover autonomous system-of-record changes; persistent memory; tool access, credentials or external transactions; handoffs in safety-critical, regulated or student-facing processes; missing stop conditions or logged escalation to a named human owner; and institutional responses to synthetic media about identifiable people. Product or pilot approval does not replace action-specific review. Review the system and access before enabling them; review each proposed system-of-record change before execution. Advance permission cannot waive these triggers.
Use evidence within its limits
Use the crosswalk to select practice and retain observable evidence. It is MAIN's implementation mapping, not a validated proficiency test, credential or certification. Local rubrics and pilot measures alone do not establish statewide proficiency or causal impact. Recheck alignment when the Framework changes; record the edition, local owner and next review.
Source: August 2026 Framework, pp. 5-7 (priorities), 14-30 (domains and progression), and 35-37 (review, measurement and maintenance).
The short answer
How should a security team begin with AI?
Inventory the AI system and its dependencies, define a bounded defensive workflow, classify the data, approve the tool, limit privileges, establish a baseline, require analyst review, test failure and abuse cases, log actions, and set stop conditions before the pilot starts.
This voluntary resource is not a compliance standard, threat-intelligence directive, incident-specific runbook, penetration-testing authorization, or substitute for qualified legal, privacy, safety, cyber-insurance, or regulatory advice.
Maintain the underlying safeguards: managed updates, unique passwords, MFA, least privilege, protected and tested backups, and trained reporting. Use CISA’s voluntary Cross-Sector CPGs 2.0 to prioritize applicable improvements; AI does not replace them.
Secure, defend, thwart
Three connected cybersecurity missions
This organizing model follows the three focus areas in NIST IR 8596, the Cyber AI Profile. It is an Initial Preliminary Draft, not a final standard; use it only as supplemental input.
Secure AI system components
Protect models, data, prompts, applications, agents, identities, tools, infrastructure, interfaces, monitoring, and the software and service supply chain.
Conduct AI-enabled cyber defense
Use AI to support bounded security analysis while preserving evidence, measuring performance, controlling permissions, and requiring accountable review.
Thwart AI-enabled cyber attacks
Prepare for faster or more persuasive abuse, including impersonation, malicious automation, model or agent exploitation, data poisoning, and software supply-chain risk.
Risk is contextual
Cybersecurity AI risk map
Score the full workflow before deployment and after any material change. High privilege, autonomy, exposure, or consequence can make an apparently simple use case high risk.
Data sensitivity
Credentials, secrets, personal or customer data, incident evidence, restricted intelligence, source code, configurations, and security telemetry.
Privilege and reach
Read versus write access; endpoint, identity, cloud, email, ticketing, code, firewall, orchestration, or production permissions.
Autonomy and speed
How many steps can occur without confirmation? Can the system act faster or farther than a reviewer can understand and stop?
External exposure
Internet-facing inputs, untrusted documents, email, chat, plugins, Model Context Protocol (MCP) servers and comparable tool/context integrations, APIs, third-party retrieval, and vendor integrations.
Consequence and reversibility
Service disruption, evidence loss, user lockout, data disclosure, regulatory impact, safety effect, financial loss, and ability to restore state.
Human review and observability
Reviewer expertise, time, source access, explainability, logging, reproducibility, uncertainty reporting, alerting, and audit trail.
Know what you operate
Governance and minimum AI inventory
Assign accountable owners across business, security, privacy, data, AI engineering, legal, procurement, and incident response. Inventory both official and discovered uses.
On smaller screens, scroll horizontally to review the complete table.
| Category | Record |
|---|---|
| Purpose and owner | Business/security objective, accountable owner, operators, affected users, and decision authority. |
| Components | Model and version, provider and router, application, model endpoints, training/evaluation harnesses, agents, prompts, retrieval stores, tools, APIs, infrastructure, compute quotas, dependencies, and data flows. |
| Access | Human and service identities, roles, secrets, token lifetime, permissions, network paths and egress, connected systems, approval gates, and emergency revocation. |
| Data | Allowed and prohibited classes, location, retention, training or evaluation use, encryption, residency, sharing or resale by providers and intermediaries, provenance, and deletion. |
| Assurance | Threat model, tests, baseline metrics, limitations, change controls, logs, monitoring, provider-escalation path, fallback, recovery, and review date. |
Treat AI as a system
Secure the AI lifecycle and attack surface
Do not focus on the model alone. Threat-model the inputs, outputs, data, tools, identities, integrations, infrastructure, people, and supply chain.
Inputs and context
- Prompt injection and indirect prompt injection
- Malicious files, links, images, retrieved content, or tool output
- Context poisoning and hidden instructions
- Untrusted user content crossing trust boundaries
Models, data, and retrieval
- Training or retrieval-data poisoning
- Sensitive-data memorization or disclosure
- Model theft, tampering, substitution, or unsupported updates
- Weak provenance, evaluation, or version control
Agents, tools, and identity
- Excessive agency, long-lived credentials, and permissions
- MCP or comparable integration and tool poisoning
- Confused-deputy and cross-tenant risk
- Unapproved network egress, shared-service access, unsafe actions, or unauthorized delegation
Outputs and applications
- Insecure output handling or code execution
- False or manipulated recommendations
- Denial of service or resource exhaustion
- Unsafe rendering, logging, caching, or downstream use
Supply chain
- Model, dataset, package, container, plugin, and service provenance
- Vendor, router, reseller, and fourth-party access or data reuse
- Signed artifacts, integrity checks, bills of materials, and update controls
- Exit, portability, outage, and compromise plans
Monitoring and recovery
- Security-relevant prompt, retrieval, tool, identity, decision, and action logs
- Privacy-preserving retention and access control
- Behavior drift, boundary probing, burst activity, unexpected model use, and cost spikes
- Kill switch, credential rotation, rollback, restore, and evidence preservation
Read-only does not prevent disclosure. Read-only access can still disclose information. Enforce source and tenant authorization, permitted tools, and outbound destinations in application and network controls outside the model. Test attempts to retrieve or transmit prohibited data; prompt instructions alone are not an access-control boundary.
Build and buy defensibly
Secure development and acquisition checkpoints
- Define: security objectives, trust boundaries, prohibited uses, abuse cases, data rules, and measurable acceptance criteria.
- Design: isolate components; deny network egress and shared-service access by default for high-risk agent, training, or evaluation workloads; minimize privileges; validate inputs and outputs; preserve provenance; and design fail-safe behavior.
- Build or acquire: assess providers, routers, data practices, emergency support, and dependencies; protect code and data; document versions; and verify integrity.
- Test: evaluate normal, edge, adversarial, misuse, privacy, reliability, sandbox escape, boundary probing, provider refusal, outage, recovery, and human-review cases in a representative environment.
- Deploy: use staged release, approved configuration, short-lived credentials, secrets management, logging, alerting, rollback, and trained operators.
- Operate: monitor drift, abnormal agent loops, unexpected model use, and abuse; patch dependencies; retest material changes; review access; practice incidents; and retire safely.
Bounded AI-enabled defense
Seven defensive workflows and their limits
Start with drafts, summaries, and recommendations. Preserve source telemetry and require analysts to verify conclusions before any consequential action.
1. Log and case summarization
Allowed: summarize approved, minimized records and cite source event IDs.
Validate: timeline, omissions, identity, timestamps, and links to raw evidence.
2. Alert triage support
Allowed: group related alerts and propose investigation questions.
Validate: false-negative risk, severity, asset context, uncertainty, and analyst disposition.
3. Threat-intelligence synthesis
Allowed: compare licensed sources and extract defensive implications.
Validate: source restrictions, dates, confidence, indicators, attribution, and independent corroboration.
4. Detection engineering
Allowed: draft queries, test cases, mappings, and documentation.
Validate: syntax, coverage, evasion assumptions, false positives, data availability, and isolated testing.
5. Vulnerability prioritization
Allowed: combine approved asset, exposure, business, exploitation, and remediation context.
Validate: authoritative scanner data, ownership, compensating controls, active exploitation evidence, and human priority decision.
6. Secure code and documentation
Allowed: explain findings, draft fixes or runbooks, and propose tests in approved repositories.
Validate: secrets, licenses, dependencies, security tests, peer review, and change management.
7. Tabletop design
Allowed: create scenario variations, injects, decision prompts, and evaluation criteria using fictionalized data.
Validate: realism, safety, roles, escalation paths, recovery, legal/communications involvement, and after-action evidence.
Do not delegate by default
Account disablement, endpoint isolation, firewall or identity changes, production code deployment, evidence deletion, public attribution, breach notification, ransom decisions, or safety-critical response require explicit authorized decision paths.
Defensive implications
Prepare for AI-enabled threats without amplifying harm
AI can increase the scale, speed, personalization, or adaptability of familiar attacks. Defenders should strengthen verification, identity, telemetry, resilience, and workforce readiness.
Impersonation and social engineering
Verify sensitive requests through a known, independently obtained contact, as advised by FBI/IC3 impersonation guidance. Use transaction controls and rehearsed reporting. Prefer available phishing-resistant MFA, such as a supported security key or passkey; use NIST SP 800-63B-4 for authentication requirements appropriate to the system.
Malicious automation
Rate-limit and monitor exposed services, protect identities and APIs, reduce attack surface, patch promptly, detect anomalous behavior, and plan for higher-volume or faster-changing campaigns.
AI application exploitation
Separate untrusted content from instructions, validate tool calls and outputs, constrain retrieval, sandbox processing, protect secrets, and test prompt injection and agent abuse cases.
Poisoning and supply-chain compromise
Track provenance, restrict write paths, review providers and updates, verify artifacts, monitor data/model behavior, protect build systems, and retain known-good recovery points.
Consent phishing can survive a password change. A legitimate sign-in or consent screen can grant a malicious app access to email or files. Review the app, publisher and requested permissions before consenting. If access was granted, use the connected service’s security settings or authorized administrator to revoke the app’s permissions and affected tokens; a password change alone may leave access in place. See FBI/IC3 OAuth consent-phishing guidance.
Source coverage through October 18, 2026.
Prepare before the alert
AI-related incident response sequence
Use NIST SP 800-61 Rev. 3 to integrate AI incidents into established incident-response, privacy, legal, safety, vendor and communications processes. Preserve evidence and avoid untested containment that could worsen the event.
- Declare and assign. Confirm severity, incident commander, technical leads, decision authority, and required notifications.
- Preserve. Protect relevant prompts, outputs, identities, tool calls, model/config versions, retrieval records, telemetry, and chain-of-custody information.
- Bound. Identify affected users, systems, data, models, agents, endpoints, evaluation harnesses, providers or routers, tenants, privileges, compute, and time window.
- Contain safely. Use pre-authorized, deterministic procedures to revoke exposed access, malicious app grants and tokens; terminate sessions, freeze affected pipelines, isolate components or accounts, and shift to a known fallback. These procedures execute predefined conditions and actions; an AI recommendation to select or change a consequential action requires authorized human review before execution. Novel or high-consequence choices remain with authorized incident leadership.
- Eradicate and recover. Remove the cause, rotate secrets, restore trusted components and data, validate integrity, and monitor for recurrence.
- Learn and share appropriately. Document root and contributing causes, control gaps, human factors, vendor actions, metrics, and approved information-sharing opportunities.
Follow the organization’s incident escalation and notification procedures. For cyber-enabled crime, retain relevant records and report through FBI/IC3; use CISA’s reporting guidance as applicable. Reporting does not replace containment, recovery or required notifications.
A measured first deployment
30-day cybersecurity AI pilot
Select a lower-risk, read-only workflow using approved and minimized data. The goal is evidence about quality, analyst effort, security, and repeatability, not autonomous response.
Define the evaluation denominator. Use independently labeled malicious and benign cases held out from prompt tuning. Report false negatives as missed malicious cases divided by all malicious cases, and false positives as benign cases incorrectly escalated divided by all benign cases; retain counts and case mix. Triage results describe the supplied alerts, not attacks absent from upstream telemetry. Compare total analyst time and repeat tests after material changes.
- Week 1: Define. Name owners; threat-model; approve tool, data, permissions, network egress, reviewers, baseline, success measures, and stop conditions.
- Week 2: Test. Train a small group; use representative sanitized cases; record prompts, sources, errors, revisions, false positives, and false negatives.
- Week 3: Challenge. Test edge, adversarial, injection, denied egress, sandbox escape, boundary probing, burst activity, provider refusal, outage, data-leakage, privilege, synchronized containment, and recovery cases; verify logging and revocation.
- Week 4: Decide. Compare with baseline; review risk and total analyst time; continue, revise, pause, or stop; document the decision.
On smaller screens, scroll horizontally to review the complete scorecard.
| Measure | Question |
|---|---|
| Quality | Were conclusions accurate, complete, sourced, reproducible, and useful? |
| Analyst effort | Did total time, including prompting, verification, correction, and documentation, improve? |
| Security and privacy | Did the workflow stay within approved data, access, retention, and logging boundaries? |
| Operational reliability | How did it behave under edge cases, untrusted inputs, outages, model changes, and degraded data? |
| Decision | Do evidence and residual risk support continue, revise, pause, or stop? |
Move from pilot to approved operation
The security leader and accountable system or business owner should review the pilot record with the organization’s designated privacy, IT, risk, and procurement authorities as applicable. Authorization is for a named workflow and configuration, not for AI-enabled defense generally. Keep the baseline comparison, labeled-case counts, false-positive and false-negative results, analyst time, adversarial and failure tests, residual risks, approved data and permissions, and the documented decision.
Before routine use, assign an operating owner, trained analyst reviewers, monitoring and log review, change control, tested revocation and fallback, and a review date. Keep consequential actions behind the approved human gate. Reassess after a material model, data, integration, privilege, threat, or performance change; pause under the stop conditions above. This playbook does not confer security, legal, regulatory, or procurement approval.
Continue with MAIN
Cybersecurity learning and implementation
AI for Cybersecurity course
Review current course availability and MGCCC registration steps in MAIN’s course catalog.
For everyday verification, account protection and scam recovery, use MAIN’s AI Cyber Safety guide.
Mississippi statewide framework
Connect cybersecurity AI work with the state’s broader AI priorities and workforce context.
Responsible implementation support
Explore MAIN services for strategy, training, convening, and practical AI adoption support.
Cybersecurity AI questions
Frequently asked questions
What is a cybersecurity AI playbook?
It is a practical operating resource for identifying AI systems and dependencies, matching controls to risk, securing AI components, bounding AI-enabled defense workflows, preparing for AI-enabled attacks, and responding to incidents with accountable human oversight.
Can AI autonomously respond to cyber incidents?
Autonomy should be limited by the organization’s risk tolerance, testing, permissions, monitoring, and recovery capability. Begin with read-only assistance. Model-selected consequential containment, account, network, endpoint or production changes require an authorized person to review the specific proposed action before execution. Separately approved deterministic response procedures may execute only their predefined steps; AI output cannot choose or widen those steps without the required human review. System or pilot approval does not waive the Statewide AI Framework’s review triggers.
What data should not be entered into a public AI tool?
Do not enter credentials, tokens, private keys, unredacted logs with personal or customer data, sensitive configurations, exploitable vulnerability details, incident evidence, proprietary code, restricted intelligence, or regulated information unless an approved environment and data-use agreement authorize it.
How should security teams validate AI output?
Compare it with authoritative telemetry and source records, preserve chain of custody where relevant, reproduce queries, test detections and code in an isolated environment, measure false positives and false negatives, review uncertainty, and require an authorized analyst to approve consequential action.
What if a hosted AI service refuses incident evidence?
Test that failure mode before an incident. Maintain a pre-approved alternative that protects sensitive evidence, such as an authorized provider escalation path, a vetted private capability where justified, another approved forensic tool, or a manual workflow.
Is NIST IR 8596 a final standard?
No. NIST IR 8596 was released as a preliminary draft in December 2025. NIST identifies this publication as an Initial Preliminary Draft, not a final standard. Clearly label it as preliminary and use final NIST publications and applicable requirements as controlling references.
Authoritative foundation
Standards, drafts, and threat knowledge
Federal publications provide the control foundation. Final, preliminary, voluntary and community resources serve different roles; this playbook does not establish compliance. Vendor reports below illustrate attributed incidents and proposals.
- CISA Cross-Sector Cybersecurity Performance Goals 2.0 (voluntary prioritized safeguards, aligned with NIST CSF 2.0)
- NIST SP 800-63B-4, Authentication and Authenticator Management (final; apply the requirements appropriate to the system)
- FBI/IC3, OAuth consent phishing (federal threat advisory; revoke malicious app access)
- FBI/IC3, Law enforcement and government impersonation (federal fraud advisory; independently verify requests)
- CISA, NSA, ASD’s ACSC and international partners, Careful adoption of agentic AI services (joint security guidance, May 1, 2026); NSA coauthor announcement
- NIST SP 800-61 Rev. 3, Incident Response Recommendations and Considerations for Cybersecurity Risk Management (final, April 2025)
- NIST Cybersecurity Framework 2.0 (final)
- NIST AI Risk Management Framework and Generative AI Profile (final publications)
- NIST AI 100-2 E2025, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (final, March 2025; NIST has posted a planning note for a future update)
- NIST IR 8596, Cybersecurity Framework Profile for Artificial Intelligence (preliminary draft, December 2025; not final)
- NIST IR 8607, Workshop Summary Report for the Cyber AI Profile Hybrid Workshop (final workshop summary, August 2026)
- NIST SP 800-218A, Secure Software Development Practices for Generative AI and Dual-Use Foundation Models (final, July 2024)
- CISA and UK NCSC, Guidelines for Secure AI System Development (joint voluntary guidance, November 2023)
- CISA JCDC AI Cybersecurity Collaboration Playbook (voluntary information-sharing resource, January 2025)
- Google Threat Intelligence Group, AI vulnerability exploitation, augmented operations, and initial access
- Mandiant, Blueprint for AI-Assisted Vulnerability Management
- Google Threat Intelligence Group and Mandiant, Mitigation Guidance for Supply Chain Compromise
- Microsoft Security, prompt injection and code-execution vulnerabilities in AI agent frameworks
- MITRE ATLAS (living knowledge base of adversary tactics and techniques for AI-enabled systems)
- OWASP Artificial Intelligence Security Verification Standard 1.0 (community verification standard, June 2026; not a regulatory requirement)
- OWASP GenAI LLM Top 10 2026 and OWASP GenAI Agentic Security Initiative (community guidance, not regulatory standards)
Attributed incident reports
Incident and governance sources
These sources inform the selected threat reports and operational additions above. Vendor incident reports and industry proposals are attributed evidence, not standards or proof of prevalence.
- Hugging Face, Security incident disclosure — July 2026 (vendor incident disclosure)
- OpenAI, The Hugging Face incident and other third-party impact from misaligned models (incident review and continuing updates)
- Unit 42, An AI-Assisted Cyber Attack: Inside a Unit 42 Investigation
- Google Threat Intelligence Group, From Prompting to Autonomy — The Evolution of Adversarial AI
- Microsoft, 2026 Digital Defense Report overview
- FBI/IC3, Scammers Exploit 2026 U.S. Midterm Election to Perpetrate Multiple Fraud Schemes
- Dario Amodei, We Must Pace the Frontier (industry governance proposal; not a standard)
Make AI-enabled defense observable, bounded, and reversible
Pair cybersecurity expertise with AI engineering, privacy, legal, procurement, workforce training, and operational ownership. Reassess whenever data, models, integrations, permissions, or threats change.