Definitions of key responsible AI, AI governance, and EU AI Act terms. Search the glossary to find your concept.
A system's resilience to adversarial attacks or manipulated inputs.
Principle of collecting/processing only data necessary for a specified purpose.
EU AI Act requirement (Art. 14) that high-risk AI systems be designed to allow effective human supervision.
Standardized document capturing the key details of a specific AI use case (purpose, data, stakeholders, risks, controls) to support inventory, transparency, and governance – complementing model and system cards at the application level.
Fairness criterion requiring a decision would be the same in a counterfactual world absent a protected attribute.
Control point that mediates and monitors the tools, data, and systems an AI agent can reach via the Model Context Protocol (MCP), enforcing permission boundaries and flagging out-of-scope actions before they occur.
EU AI Act–required assessment (Art. 27) of a high-risk AI system's impact on fundamental rights, for certain deployers.
Practice of defining, monitoring, and enforcing the boundaries of what an AI agent may do (the tools, data, and actions within its remit), and detecting scope violations where an agent exceeds those permissions.
Attack that extracts a system's hidden prompt, instructions, or confidential context through crafted user input.
GDPR-mandated inventory documenting how personal data is processed (purposes, data categories, recipients, safeguards); in AI governance it complements the AI registry and DPIAs for data-driven systems.
Artificially generated data used to train or test models, often to preserve privacy or augment datasets.
Risk remaining after controls and mitigations have been applied.
Risk of users trusting AI outputs without sufficient verification or oversight.
Unauthorized modification of a model's weights, architecture, or configuration after training, whether via supply chain compromise or insider action.
An event where an AI system causes or nearly causes harm, tracked in resources like the OECD AI Incidents Monitor and MIT AI Risk Repository.
Continuous, real-time monitoring and constraint of a deployed AI system or agent to keep it within approved boundaries — checking permissions, tool usage, and behavior at runtime and flagging or blocking scope violations before they become incidents.
AI safety concern where a model deliberately underperforms during evaluation to conceal true capabilities.
An AI model whose internal decision logic is not interpretable to users or auditors.
Evaluation of whether a control has been implemented and is actually working to mitigate the AI risk it targets; increasingly automated by analyzing connected sources and evidence to judge effectiveness.
Independent examination of an algorithm's design, data, and outputs for bias, safety, or compliance issues.
Large model trained on broad data, adaptable to a wide range of downstream tasks.
The end-to-end discipline of identifying, assessing, treating, and monitoring risks arising from AI systems across their lifecycle — the practice that frameworks like NIST AI RMF and ISO 42001 structure and formalize.
Documentation or artifacts (logs, reports, test results, approvals) that demonstrate a control is in place and effective, or that a governance requirement has been met; the substantiating basis for audits and conformity assessments.
Regulation (EU) 2024/1689 establishing harmonised rules on AI in the EU, using a risk-based, tiered approach.
Practices and tooling for operationalizing, deploying, and monitoring machine learning models at scale.
Research problem of supervising AI systems whose capabilities exceed a human evaluator's own.
Field focused on preventing unintended, harmful, or catastrophic behavior from AI systems.
Automated processing of personal data to evaluate or predict aspects of a person (GDPR Art. 4(4)).
Classifying individuals based on biometric data into categories such as race, gender, or political opinion; restricted under the EU AI Act.
EU body responsible for AI Act implementation, particularly oversight of general-purpose AI models.
Category of risk in the MIT AI Risk Repository referring to AI's potential to cause human extinction or permanent civilizational collapse.
Specialized inventory of an organization's AI agents recording each agent's configuration, permissions, connected tools, and risk classification — extending the AI registry concept to autonomous systems that get deployed quickly and at scale.
Governance mechanism protecting individuals who report AI-related misconduct or risk.
Indirect discrimination occurring when a neutral variable correlates strongly with a protected attribute.
Attack that recovers verbatim or near-verbatim training examples (including sensitive data) directly from a model's outputs.
Structured testing of a model's capabilities, safety properties, or risks against benchmarks.
Principle of embedding privacy protections into system architecture from the outset rather than retrofitting.
Risk that an AI system reproduces or generates content that violates third-party copyright or intellectual-property rights, via training data or model outputs — a growing responsible-AI and legal concern.
EU AI Act term for a change to an AI system significant enough to require reassessment of conformity.
Principle that organizations and individuals must be answerable for the outcomes of AI systems they design, develop, or deploy (OECD, NIST AI RMF).
Technical tool (e.g., differential privacy, federated learning) that reduces privacy risk in data processing.
The subset of AI risk management focused on protecting AI systems, models, and data from malicious attacks (e.g., data poisoning, model theft, adversarial manipulation) — distinct from AI safety's broader focus on unintended harm.
EU AI Act classification (Annex III / Art. 6) triggering the strictest pre- and post-market obligations short of prohibition.
EU AI Act category for models displaying significant generality and capable of competently performing a wide range of tasks.
Documentation, similar to a model card, describing an AI system's overall behavior, safeguards, and limitations at the product level.
Documented log of identified risks, owners, likelihood/impact ratings, and mitigation status.
Risk of models leaking confidential or personal data through outputs.
International management-system standard for AI (AIMS), specifying requirements to responsibly govern, manage, and continually improve AI systems across their lifecycle; a leading basis for AI governance certification.
Person who coordinates and drives an organization's AI governance program — securing buy-in, aligning business with governance goals, and overseeing progress — without necessarily owning every individual governance task.
Traceable record of data's origin, movement, and transformations through a system.
EU AI Act Art. 5 list of banned uses, e.g., social scoring, manipulative AI, untargeted facial recognition scraping.
Principle requiring AI systems and their use to be disclosed, documented, and understandable to relevant stakeholders.
Attack that implants a hidden trigger during training so a model behaves normally until a specific input activates malicious behavior.
Entity established in the EU that places on the market an AI system from a non-EU provider.
Risk that occurs when an adversary manipulates model behavior via crafted input that overrides intended instructions.
Synthetic media generated or manipulated by AI that falsely depicts a real person's likeness or actions.
Standard requiring humans retain real, informed influence over an AI system's consequential decisions.
The ongoing practice of identifying, collecting, mapping, and governing an organization's AI use cases across their lifecycle — the foundation for inventory, risk classification, and EU AI Act compliance.
EU AI Act term for an entity using an AI system under its authority, other than in personal non-professional use.
Attack that reconstructs training data or sensitive attributes from a model's outputs.
GRC model separating operational management, risk/compliance oversight, and independent audit functions.
Identifying and analyzing parties affected by or influencing an AI system's development and deployment.
EU proposal addressing civil liability for damage caused by AI systems.
Systematic and repeatable errors in a system that create unfair outcomes for particular groups.
GDPR-mandated assessment of privacy risks for high-risk processing activities, often paired with AI risk assessments.
Degradation of model performance over time as real-world data diverges from training data.
Skills, knowledge, and understanding enabling providers, deployers, and users to make informed decisions about AI (explicit EU AI Act obligation, Art. 4).
Entity in the supply chain, other than provider or importer, that makes an AI system available on the market.
Attack determining whether a specific data record was used in a model's training set.
Metadata or cryptographic proof documenting the origin and history of AI-generated or edited content (e.g., C2PA).
Policies and controls managing the availability, integrity, security, and usability of data used in AI systems.
Decisions made by an AI system with little or no human involvement, often subject to explanation/opt-out rights (cf. GDPR Art. 22).
NIST's voluntary framework (Govern, Map, Measure, Manage functions) for managing AI risks.
Acceptable variation around risk appetite for specific objectives or risk types (NIST AI RMF term).
Conformity marking indicating a high-risk AI system meets EU AI Act requirements.
Structured comparison of the controls an AI system currently has against those required by a chosen framework or regulation, surfacing missing or insufficient safeguards to remediate.
Discipline (rooted in financial services, now applied to AI) governing model validation, monitoring, and controls.
Umbrella practice of designing, developing, and deploying AI in ways that are ethical, safe, fair, and accountable.
Cross-functional body that steers an organization's AI governance program — bringing together stakeholders such as legal, security, data protection, risk, and technical teams to set direction, prioritize, and oversee decisions (broader in remit than an AI ethics board).
Hypothesized failure mode where a model behaves safely during evaluation but pursues different goals when unsupervised.
Attack that reconstructs or steals a proprietary model's functionality or parameters via repeated queries against its API.
Level and type of risk an organization is willing to accept in pursuit of its objectives.
Process of assessing and monitoring risks introduced by vendors, suppliers, or AI model providers.
EU AI Act obligation (Art. 72) for providers to systematically monitor high-risk AI system performance after deployment.
Tamper-evident, chronological record of the actions, decisions, and changes across an AI system's governance lifecycle (who did what, when), enabling accountability, review, and evidence for internal and external audits.
Structured mechanism (e.g., forms sent to teams) for discovering and registering AI systems and use cases across an organization, including surfacing shadow AI – the entry point that feeds the AI inventory.
Techniques and practices that make an AI system's decisions understandable to humans.
Risk arising from compromised third-party models, data, or plugins.
Metric used to signal increasing risk exposure ahead of an actual control failure.
Internal governance body reviewing AI systems for ethical risk and compliance.
Defensive technique that trains a model on adversarial examples to improve its robustness against attack.
Mathematical technique that adds calibrated noise to protect individual privacy in datasets or outputs.
Adversarial testing exercise simulating attacks or misuse to uncover a system's vulnerabilities.
Describes AI usage, development, etc. that is valid, reliable, safe, secure, accountable, transparent, explainable, privacy-enhanced, and fair.
AI system that can autonomously plan, decide, and take actions (often by calling tools or other systems) to pursue goals with limited human intervention; its autonomy raises distinct responsible-AI challenges around oversight, accountability, traceability, and scope control.
Integrated approach to organizational governance, risk management, and regulatory compliance.
Structured evaluation of a third-party AI product or provider against your security, privacy, and risk policies — reviewing documentation, questionnaires, and evidence — to decide whether an external AI system meets your standards before adoption.
Standardized document describing a model's intended use, performance, limitations, and evaluation results.
Independent testing confirming a model performs as intended and within acceptable risk limits.
EU AI Act Art. 50 requirements to disclose AI-generated or manipulated content (e.g., deepfakes) to users.