Prefer to listen? Click play for AI narration

Small and mid-sized businesses now use AI in marketing, HR, customer support and software development, often through tools that staff adopted without a formal decision. Regulators do not excuse a company because its AI arrived as a subscription. This guide explains how to build a proportionate AI governance framework: which standards to look at, the six working parts, how to handle AI literacy and human oversight, and a 90-day plan. It is general information, not legal advice.

Why governance cannot wait

The EU AI Act's AI literacy duty has applied since 2 February 2025. After the 2026 Digital Omnibus, providers and deployers must take measures that support the development of AI literacy, without guaranteeing a specific level for every individual, according to Praxikon. High-risk obligations follow from 2 December 2027 (see our compliance calendar), and the GDPR already applies to any AI that touches personal data, as we explain in where the GDPR and the AI Act overlap.

Governance is how a company meets all of this without a large compliance team. It means knowing which AI you use, who is accountable, which rules apply and how you can prove it. The Omnibus also extends relief to “small mid-caps” with fewer than 750 employees, including simplified documentation templates and tailored penalty caps, per Orrick. Proportionate effort is expected; none is not an option.

Three reference points: ISO/IEC 42001, NIST AI RMF and the AI Act

A small business does not need to adopt every framework. It needs to know what each one is for.

ISO/IEC 42001NIST AI RMFEU AI Act
NatureInternational standard for an AI management system, published 2023Voluntary US frameworkBinding EU regulation
CertifiableYes, through accredited certification bodiesNoNo; conformity assessment applies to high-risk systems
StructureClauses 4 to 10 share the harmonised structure of ISO 27001; Annex A offers reference controlsFour functions: Govern, Map, Measure, ManageRisk tiers with duties for providers, deployers and others
Best used forA repeatable, auditable programmeStructuring risk thinking and vocabularyDefining what you legally must do

ISO/IEC 42001 suits companies that already run ISO 27001 or a similar system, because the structure is familiar and a certificate can reassure enterprise customers. TCSA notes that Annex A is a reference set from which you select controls in a Statement of Applicability, not a mandatory checklist. The NIST framework is lighter and free to use, and it fits teams that want a risk method without certification.

Neither replaces legal compliance. Framework publishers map their content to AI Act articles such as risk management (Article 9), data governance (Article 10) and human oversight (Article 14), as Ampcus Cyber does, but those mappings are guidance rather than legal advice. A sensible path for an SMB is to start with the four NIST functions as a working structure and adopt ISO 42001 later if customers ask for certification.

Six building blocks

Whichever framework you pick, the working parts are the same.

The same inventory feeds the GDPR record of processing activities, so building it once serves both regimes. For a company with a dozen AI tools, a spreadsheet and a named owner is a legitimate starting point; sophistication can follow risk.

AI literacy and human oversight

Training is the most immediate obligation. A practical programme has three layers: a short baseline for everyone who uses AI, role-specific modules for people in HR, marketing, finance or engineering, and refreshers whenever a system or its use changes, as TCSA recommends. Because the legal test is now that you take measures that support literacy, documented content, attendance and role mapping become your evidence.

Human oversight is formally a high-risk requirement, yet it is also a sound design principle for lower-risk uses. Under Article 14(4), as TCSA summarises it, the person overseeing a system must understand its capacities and limits, stay aware of automation bias, and be able to disregard, override or reverse its output. Article 26 requires deployers of high-risk systems to assign oversight to people with the necessary competence, training and authority.

Oversight fails when it is nominal. A reviewer who sees fifty AI-generated decisions an hour, with no time and no authority to challenge them, provides oversight on paper only. Test the arrangement with three questions:

If the answers are no, no, and never, the control needs redesign before an auditor or a regulator finds the same thing. The same logic supports the GDPR: Article 22 limits decisions based solely on automated processing, and a rubber-stamp review does not take a decision outside that limit.

A 90-day rollout plan

A small company can stand up a working programme in a quarter. The plan below assumes one accountable owner and part-time input from legal, IT and the business.

PeriodActionsOutput
Days 1-30Inventory AI use across teams, name an owner, publish a one-page interim policy, stop clearly risky usesAI register and interim policy
Days 31-60Classify use cases, run assessments for the riskiest, review top vendors' terms, launch baseline trainingRisk register, DPIAs, vendor findings, training log
Days 61-90Define oversight and incident procedures, test them in a tabletop exercise, set a review cadence, decide whether to pursue ISO 42001Oversight procedure, incident playbook, leadership report

The bottom line

AI governance in a smaller company is a habit, not a project. A short register, a clear owner, a proportionate policy and evidence of training will answer most of the questions that customers, auditors and regulators ask. Start with what exists today, document it, and let the framework grow with the risk.

To see where your AI tools stand today, a good first step is GDPRGard's audit tool.

Sources

Several sources are vendor-authored, and the framework mappings are guidance rather than legal analysis. Verify the ISO/IEC 42001 clause numbers and Annex A figures against the standard, and the NIST documents on nist.gov, before relying on them. The rewritten Article 4 wording rests mainly on Praxikon and TCSA and should be checked against Regulation (EU) 2026/1744.

Share this article

Also read:

⚠️ This article is for informational purposes only and does not constitute legal advice. For complex compliance situations, consult a qualified data protection professional. GDPR and EU AI Act requirements are subject to ongoing regulatory guidance — verify current obligations with your legal adviser.