ISO/IEC 42001 AI Management System Consulting × AI attack defence

AI governance: keep staff AI token budgets from arming attackers

Updated 2026-11-10

The short answer

Manage employees' AI budgets as company assets: one identity per person, access through an AI gateway, no long-lived shared keys; budgets and anomaly alerts per purpose; a written list of prohibited uses such as phishing, malware or attack content against suppliers; least privilege for AI agents with human confirmation for external sending and code execution; revocation on the day someone leaves. These map to responsible and intended use in ISO/IEC 42001 Annex A.9 and supply chain responsibilities in A.10.

Companion page: service overview: ISO/IEC 42001 AI Management System ConsultingWhat this service is, who it is for and what we deliver are on the service overview page.

How can employees' AI accounts and token budgets become an arsenal against the company and its supply chain?

When a company opens generative AI to staff, it usually issues API keys, an enterprise account or a monthly budget. These have three properties: they convert directly into compute, the usage is recorded under the company's name, and they are easy to copy. Real cases exist. On 27 February 2025 Microsoft's Digital Crimes Unit reported that the Storm-2139 network exploited exposed customer credentials scraped from public sources to access generative AI services without authorisation, altered the services' capabilities and resold access to other malicious actors to produce harmful content.

The impact on a company has at least four layers:

  • Cost: the budget is drained; OWASP lists this risk as LLM10 Unbounded Consumption.
  • Liability and reputation: abusive content comes from the company's account, the provider may suspend it, and the company has to prove its innocence.
  • Attacks on yourself: a stolen account or an insider uses the company's AI to write phishing aimed at colleagues or to reconnoitre internal systems.
  • Attacks on the supply chain: an AI agent with tool permissions, once hit by prompt injection, can send malicious content to suppliers and customers from your trusted domain.

AI usage boundaries are therefore not just about what may be asked of AI; AI access has to be managed as a high-value credential.

Which misuse and attack scenarios belong in the AI risk assessment?

ScenarioHow it happensConsequenceReference
API key leakageKeys in code, configuration files, shared documents or chat groupsOutsiders do anything on the company's budgetMicrosoft Storm-2139 case; OWASP LLM10
Shared accounts, no accountabilityOne key shared by a whole departmentNo way to tell who did it or to disable one personISO/IEC 42001 Annex A.9
Insider misuseCompany budget used to write phishing or malware or to scrape at scaleLegal liability, service suspensionNIST AI 100-2 misuse
Prompt injection into AI agentsThe agent reads a malicious mail or page and follows its instructionsExternal mail, downloads, code executionNIST indirect prompt injection; OWASP LLM01, LLM06
Budget exhaustion, outageRunaway loops or abuseCost spike, normal work disruptedOWASP LLM10
No revocation on leaving or transferAccounts and keys stay validFormer staff can still use company AI and dataISO/IEC 42001 Annex A.9
AI misuse on the supplier sideContractors process your data or contact others on your behalf with their own AIYour data and name misusedISO/IEC 42001 Annex A.10

How do you set the usage boundary: what is allowed, what is not, and who approves?

ISO/IEC 42001 Annex A.9 asks the organisation to define processes for responsible use of AI systems (A.9.2), objectives for responsible use (A.9.3) and to ensure AI systems are used according to their intended use (A.9.4). For staff use of AI we recommend three written lists:

  1. Approved uses: for each use, the models or services allowed, the data classification that may be entered, the tools that may be called, and an owner. Anything not on the list needs an application first.
  2. Prohibited uses: creating or modifying malware, producing phishing or social engineering content, scanning or scraping other parties' systems without authorisation, circumventing provider safeguards, entering data above the approved classification. Keep the list consistent with each AI provider's usage policy.
  3. AI agent permissions: what each agent may read and write and whether it may send externally or execute code; high-risk actions always need human confirmation.

AI Act Article 4, as amended by Regulation (EU) 2026/1744, requires providers and deployers of AI systems to take measures to support the AI literacy of their staff. Training staff on these three lists and keeping the records is the most direct way to do it.

How do you control keys, budgets and AI agents technically, and what evidence should you keep?

The controls are our practical recommendations; the basis column follows the standards and legal texts.

BasisControlEvidence
ISO/IEC 42001 A.9.2, A.9.4All AI access through an enterprise AI gateway with single sign-on identifying each person; no long-lived shared keysGateway configuration, account list
OWASP LLM10Budget caps per person and purpose, spend alerts and anomaly detection (heavy use at night, sudden model switches, bursts of similar requests)Budget settings, alert records
Key leakage protectionSecret scanning of code and documents; short-lived, rotatable keys held in a key management serviceScan reports, rotation records
OWASP LLM06; NIST indirect prompt injectionLeast privilege for agents, external content separated from instructions, human confirmation for external sending and code executionPermission list, confirmation records
ISO/IEC 42001 A.9.3Periodic review of usage and purpose records, sampling for use outside approved purposesReview records
Staff changesAI accounts and keys revoked on the day of leaving or transferRevocation records
Incident responseOn key leakage or misuse: revoke, notify the provider, assess impact on suppliers and customersIncident records

How do you manage the supply chain side?

ISO/IEC 42001 Annex A.10 covers third-party and customer relationships: allocating responsibilities between the organisation, its suppliers and customers (A.10.2), establishing a supplier process (A.10.3) and considering customer expectations and needs (A.10.4). For AI misuse the supply chain has to be managed in two directions:

  • Your suppliers: when contractors use AI on your data or code, or contact your customers on your behalf, the contract states which AI services may be used, forbids training on your data and sets deadlines for reporting misuse and leaks.
  • Your responsibility to suppliers: if your AI accounts or agents are used to attack a supplier, you must be able to disable them at once, notify the supplier and hand over records for the investigation.

Writing both directions into supplier management and incident response keeps AI misuse from turning into mutual blame between you and your suppliers.

How do IT, security, legal and HR divide the work, which part do Winners take, and where do you start?

  • AI governance lead: maintains the three lists, approves new uses and chairs periodic reviews.
  • IT and security: build the AI gateway, budgets and alerts, secret scanning and agent permissions.
  • Legal and procurement: AI clauses in provider and supplier contracts.
  • HR: onboarding, training and exit procedures.

Winners Consulting implements ISO/IEC 42001: AI system inventory, usage boundaries and the three lists, risk and impact assessment, supplier clauses and evidence planning, sharing one evidence base with an existing ISO 27001 system. Ahura (POC version) is fully on-premises, keeping approval and review records in the company's own environment.

Start with three things: inventory every AI account and key (including those scattered in code), replace shared keys with personal identities through the gateway, and publish the prohibited-use list first.

AI attack defence for other services

Official sources

Ask a consultant on LINE

No forms: the button opens LINE with the page you are reading filled in. Send it and a Winners Consulting consultant will reply. The first diagnosis is free.

Winners Consulting track record

Winners Consulting Services is a Taiwan RegTech platform provider and enterprise risk management consultancy.

Trusted long-term by 20+ companies, we work as an industry-focused, long-term partner to our clients.

Named client engagements, published with client permission:

See case studies

AI governance: keep staff AI token budgets from arming attackers