Small and mid-sized businesses now use AI in marketing, HR, customer support and software development, often through tools that staff adopted without a formal decision. Regulators do not excuse a company because its AI arrived as a subscription. This guide explains how to build a proportionate AI governance framework: which standards to look at, the six working parts, how to handle AI literacy and human oversight, and a 90-day plan. It is general information, not legal advice.
Why governance cannot wait
The EU AI Act's AI literacy duty has applied since 2 February 2025. After the 2026 Digital Omnibus, providers and deployers must take measures that support the development of AI literacy, without guaranteeing a specific level for every individual, according to Praxikon. High-risk obligations follow from 2 December 2027 (see our compliance calendar), and the GDPR already applies to any AI that touches personal data, as we explain in where the GDPR and the AI Act overlap.
Governance is how a company meets all of this without a large compliance team. It means knowing which AI you use, who is accountable, which rules apply and how you can prove it. The Omnibus also extends relief to “small mid-caps” with fewer than 750 employees, including simplified documentation templates and tailored penalty caps, per Orrick. Proportionate effort is expected; none is not an option.
Three reference points: ISO/IEC 42001, NIST AI RMF and the AI Act
A small business does not need to adopt every framework. It needs to know what each one is for.
| ISO/IEC 42001 | NIST AI RMF | EU AI Act | |
|---|---|---|---|
| Nature | International standard for an AI management system, published 2023 | Voluntary US framework | Binding EU regulation |
| Certifiable | Yes, through accredited certification bodies | No | No; conformity assessment applies to high-risk systems |
| Structure | Clauses 4 to 10 share the harmonised structure of ISO 27001; Annex A offers reference controls | Four functions: Govern, Map, Measure, Manage | Risk tiers with duties for providers, deployers and others |
| Best used for | A repeatable, auditable programme | Structuring risk thinking and vocabulary | Defining what you legally must do |
ISO/IEC 42001 suits companies that already run ISO 27001 or a similar system, because the structure is familiar and a certificate can reassure enterprise customers. TCSA notes that Annex A is a reference set from which you select controls in a Statement of Applicability, not a mandatory checklist. The NIST framework is lighter and free to use, and it fits teams that want a risk method without certification.
Neither replaces legal compliance. Framework publishers map their content to AI Act articles such as risk management (Article 9), data governance (Article 10) and human oversight (Article 14), as Ampcus Cyber does, but those mappings are guidance rather than legal advice. A sensible path for an SMB is to start with the four NIST functions as a working structure and adopt ISO 42001 later if customers ask for certification.
Six building blocks
Whichever framework you pick, the working parts are the same.
- AI inventory. Record each system, its owner, purpose, data used, vendor and users. Reconcile the register against what staff actually use, because the gap between the two is where risk hides.
- Roles and accountability. Name an executive owner, a day-to-day lead and reviewers from legal, security and the business. ISO 42001 expects defined roles, although in a small firm one person may hold several.
- AI policy. Keep it short. Cover approved tools, prohibited uses, data that must never enter public tools, and when human review is mandatory. TCSA suggests tying review depth to a verification threshold by risk tier; that model is its own proposal, not a standard.
- Risk classification. Screen each use case against the AI Act's prohibited practices and high-risk categories, and against the GDPR triggers for a DPIA.
- Vendor due diligence. Ask suppliers for instructions for use, data handling and retention terms, whether customer data trains their models, sub-processor lists and incident notification commitments. Reflect the answers in the data processing agreement.
- Monitoring and incident handling. Define how problems are reported, who can pause a system, and how serious incidents reach regulators where the law requires it.
The same inventory feeds the GDPR record of processing activities, so building it once serves both regimes. For a company with a dozen AI tools, a spreadsheet and a named owner is a legitimate starting point; sophistication can follow risk.
AI literacy and human oversight
Training is the most immediate obligation. A practical programme has three layers: a short baseline for everyone who uses AI, role-specific modules for people in HR, marketing, finance or engineering, and refreshers whenever a system or its use changes, as TCSA recommends. Because the legal test is now that you take measures that support literacy, documented content, attendance and role mapping become your evidence.
Human oversight is formally a high-risk requirement, yet it is also a sound design principle for lower-risk uses. Under Article 14(4), as TCSA summarises it, the person overseeing a system must understand its capacities and limits, stay aware of automation bias, and be able to disregard, override or reverse its output. Article 26 requires deployers of high-risk systems to assign oversight to people with the necessary competence, training and authority.
Oversight fails when it is nominal. A reviewer who sees fifty AI-generated decisions an hour, with no time and no authority to challenge them, provides oversight on paper only. Test the arrangement with three questions:
- Can the reviewer see why the system recommended an outcome?
- Does the reviewer have the time and standing to disagree?
- Has anyone actually overridden a recommendation this quarter?
If the answers are no, no, and never, the control needs redesign before an auditor or a regulator finds the same thing. The same logic supports the GDPR: Article 22 limits decisions based solely on automated processing, and a rubber-stamp review does not take a decision outside that limit.
A 90-day rollout plan
A small company can stand up a working programme in a quarter. The plan below assumes one accountable owner and part-time input from legal, IT and the business.
| Period | Actions | Output |
|---|---|---|
| Days 1-30 | Inventory AI use across teams, name an owner, publish a one-page interim policy, stop clearly risky uses | AI register and interim policy |
| Days 31-60 | Classify use cases, run assessments for the riskiest, review top vendors' terms, launch baseline training | Risk register, DPIAs, vendor findings, training log |
| Days 61-90 | Define oversight and incident procedures, test them in a tabletop exercise, set a review cadence, decide whether to pursue ISO 42001 | Oversight procedure, incident playbook, leadership report |
The bottom line
AI governance in a smaller company is a habit, not a project. A short register, a clear owner, a proportionate policy and evidence of training will answer most of the questions that customers, auditors and regulators ask. Start with what exists today, document it, and let the framework grow with the risk.
To see where your AI tools stand today, a good first step is GDPRGard's audit tool.
Sources
- Praxikon: Digital Omnibus and high-risk postponement
- Orrick: Digital Omnibus finalises 8 compliance changes
- EC-Council: EU AI Act, NIST AI RMF and ISO/IEC 42001 compared
- Cloud Security Alliance: how ISO 42001 and NIST AI RMF help with the AI Act
- Ampcus Cyber: NIST AI RMF vs ISO 42001 vs EU AI Act
- TCSA: AI competence and training obligations
Several sources are vendor-authored, and the framework mappings are guidance rather than legal analysis. Verify the ISO/IEC 42001 clause numbers and Annex A figures against the standard, and the NIST documents on nist.gov, before relying on them. The rewritten Article 4 wording rests mainly on Praxikon and TCSA and should be checked against Regulation (EU) 2026/1744.
Also read:
- GDPR and AI governance in 2026: one half of the reform is already law. The other half is stuck.
- The AI Act delay doesn't cover your chatbot
- The AI Act Was Delayed, Not Cancelled: Your 2026 to 2028 Compliance Calendar