trail is now part of
Read more

Responsible AI Glossary

Definitions of key responsible AI, AI governance, and EU AI Act terms. Search the glossary to find your concept.

Sort by
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Adversarial Robustness

A system's resilience to adversarial attacks or manipulated inputs.

Data Minimization

Principle of collecting/processing only data necessary for a specified purpose.

Human Oversight

EU AI Act requirement (Art. 14) that high-risk AI systems be designed to allow effective human supervision.

Use Case Card

Standardized document capturing the key details of a specific AI use case (purpose, data, stakeholders, risks, controls) to support inventory, transparency, and governance – complementing model and system cards at the application level.

Counterfactual Fairness

Fairness criterion requiring a decision would be the same in a counterfactual world absent a protected attribute.

MCP Gateway

Control point that mediates and monitors the tools, data, and systems an AI agent can reach via the Model Context Protocol (MCP), enforcing permission boundaries and flagging out-of-scope actions before they occur.

Fundamental Rights Impact Assessment (FRIA)

EU AI Act–required assessment (Art. 27) of a high-risk AI system's impact on fundamental rights, for certain deployers.

Scope Violation / Scope Management

Practice of defining, monitoring, and enforcing the boundaries of what an AI agent may do (the tools, data, and actions within its remit), and detecting scope violations where an agent exceeds those permissions.

Prompt Leaking

Attack that extracts a system's hidden prompt, instructions, or confidential context through crafted user input.

RoPA (Records of Processing Activities)

GDPR-mandated inventory documenting how personal data is processed (purposes, data categories, recipients, safeguards); in AI governance it complements the AI registry and DPIAs for data-driven systems.

Synthetic Data

Artificially generated data used to train or test models, often to preserve privacy or augment datasets.

Residual Risk

Risk remaining after controls and mitigations have been applied.

Overreliance

Risk of users trusting AI outputs without sufficient verification or oversight.

Model Tampering

Unauthorized modification of a model's weights, architecture, or configuration after training, whether via supply chain compromise or insider action.

AI Incident

An event where an AI system causes or nearly causes harm, tracked in resources like the OECD AI Incidents Monitor and MIT AI Risk Repository.

Runtime Enforcement / Runtime Monitoring

Continuous, real-time monitoring and constraint of a deployed AI system or agent to keep it within approved boundaries — checking permissions, tool usage, and behavior at runtime and flagging or blocking scope violations before they become incidents.

Sandbagging

AI safety concern where a model deliberately underperforms during evaluation to conceal true capabilities.

Black Box (Model)

An AI model whose internal decision logic is not interpretable to users or auditors.

Control Assessment / Control Effectiveness

Evaluation of whether a control has been implemented and is actually working to mitigate the AI risk it targets; increasingly automated by analyzing connected sources and evidence to judge effectiveness.

Further resources →

Algorithmic Auditing

Independent examination of an algorithm's design, data, and outputs for bias, safety, or compliance issues.

Foundation Model

Large model trained on broad data, adaptable to a wide range of downstream tasks.

AI Risk Management

The end-to-end discipline of identifying, assessing, treating, and monitoring risks arising from AI systems across their lifecycle — the practice that frameworks like NIST AI RMF and ISO 42001 structure and formalize.

Further resources →

Evidence

Documentation or artifacts (logs, reports, test results, approvals) that demonstrate a control is in place and effective, or that a governance requirement has been met; the substantiating basis for audits and conformity assessments.

Further resources →

AI Act (EU AI Act)

Regulation (EU) 2024/1689 establishing harmonised rules on AI in the EU, using a risk-based, tiered approach.

Further resources →

MLOps / ModelOps

Practices and tooling for operationalizing, deploying, and monitoring machine learning models at scale.

Scalable Oversight

Research problem of supervising AI systems whose capabilities exceed a human evaluator's own.

AI Safety

Field focused on preventing unintended, harmful, or catastrophic behavior from AI systems.

Profiling

Automated processing of personal data to evaluate or predict aspects of a person (GDPR Art. 4(4)).

Biometric Categorization

Classifying individuals based on biometric data into categories such as race, gender, or political opinion; restricted under the EU AI Act.

AI Office

EU body responsible for AI Act implementation, particularly oversight of general-purpose AI models.

Existential Risk (X-Risk)

Category of risk in the MIT AI Risk Repository referring to AI's potential to cause human extinction or permanent civilizational collapse.

Agent Registry

Specialized inventory of an organization's AI agents recording each agent's configuration, permissions, connected tools, and risk classification — extending the AI registry concept to autonomous systems that get deployed quickly and at scale.

Further resources →

Whistleblowing / Non-Retaliation Policy

Governance mechanism protecting individuals who report AI-related misconduct or risk.

Proxy Discrimination

Indirect discrimination occurring when a neutral variable correlates strongly with a protected attribute.

Training Data Extraction

Attack that recovers verbatim or near-verbatim training examples (including sensitive data) directly from a model's outputs.

Evaluation (Model Eval)

Structured testing of a model's capabilities, safety properties, or risks against benchmarks.

Privacy by Design

Principle of embedding privacy protections into system architecture from the outset rather than retrofitting.

Copyright / IP Infringement

Risk that an AI system reproduces or generates content that violates third-party copyright or intellectual-property rights, via training data or model outputs — a growing responsible-AI and legal concern.

Substantial Modification

EU AI Act term for a change to an AI system significant enough to require reassessment of conformity.

Accountability

Principle that organizations and individuals must be answerable for the outcomes of AI systems they design, develop, or deploy (OECD, NIST AI RMF).

Privacy-Enhancing Technology (PET)

Technical tool (e.g., differential privacy, federated learning) that reduces privacy risk in data processing.

AI Security

The subset of AI risk management focused on protecting AI systems, models, and data from malicious attacks (e.g., data poisoning, model theft, adversarial manipulation) — distinct from AI safety's broader focus on unintended harm.

High-Risk AI System

EU AI Act classification (Annex III / Art. 6) triggering the strictest pre- and post-market obligations short of prohibition.

Further resources →

General-Purpose AI (GPAI) Model

EU AI Act category for models displaying significant generality and capable of competently performing a wide range of tasks.

System Card

Documentation, similar to a model card, describing an AI system's overall behavior, safeguards, and limitations at the product level.

Risk Register

Documented log of identified risks, owners, likelihood/impact ratings, and mitigation status.

Sensitive Information Disclosure

Risk of models leaking confidential or personal data through outputs.

ISO/IEC 42001

International management-system standard for AI (AIMS), specifying requirements to responsibly govern, manage, and continually improve AI systems across their lifecycle; a leading basis for AI governance certification.

AI Governance Lead / Responsible AI Champion

Person who coordinates and drives an organization's AI governance program — securing buy-in, aligning business with governance goals, and overseeing progress — without necessarily owning every individual governance task.

Data Lineage

Traceable record of data's origin, movement, and transformations through a system.

Prohibited AI Practice

EU AI Act Art. 5 list of banned uses, e.g., social scoring, manipulative AI, untargeted facial recognition scraping.

Further resources →

Transparency

Principle requiring AI systems and their use to be disclosed, documented, and understandable to relevant stakeholders.

Backdoor Attack / Trojan Attack

Attack that implants a hidden trigger during training so a model behaves normally until a specific input activates malicious behavior.

Importer (EU AI Act)

Entity established in the EU that places on the market an AI system from a non-EU provider.

Prompt Injection

Risk that occurs when an adversary manipulates model behavior via crafted input that overrides intended instructions.

Deepfake

Synthetic media generated or manipulated by AI that falsely depicts a real person's likeness or actions.

Meaningful Human Control

Standard requiring humans retain real, informed influence over an AI system's consequential decisions.

AI Use Case Management

The ongoing practice of identifying, collecting, mapping, and governing an organization's AI use cases across their lifecycle — the foundation for inventory, risk classification, and EU AI Act compliance.

Further resources →

Deployer

EU AI Act term for an entity using an AI system under its authority, other than in personal non-professional use.

Model Inversion Attack

Attack that reconstructs training data or sensitive attributes from a model's outputs.

Three Lines of Defense

GRC model separating operational management, risk/compliance oversight, and independent audit functions.

Stakeholder Mapping

Identifying and analyzing parties affected by or influencing an AI system's development and deployment.

Liability (AI Liability Directive, proposed)

EU proposal addressing civil liability for damage caused by AI systems.

Algorithmic Bias

Systematic and repeatable errors in a system that create unfair outcomes for particular groups.

Data Protection Impact Assessment (DPIA)

GDPR-mandated assessment of privacy risks for high-risk processing activities, often paired with AI risk assessments.

Model Drift

Degradation of model performance over time as real-world data diverges from training data.

AI Literacy

Skills, knowledge, and understanding enabling providers, deployers, and users to make informed decisions about AI (explicit EU AI Act obligation, Art. 4).

Further resources →

Distributor (EU AI Act)

Entity in the supply chain, other than provider or importer, that makes an AI system available on the market.

Membership Inference Attack

Attack determining whether a specific data record was used in a model's training set.

Content Provenance

Metadata or cryptographic proof documenting the origin and history of AI-generated or edited content (e.g., C2PA).

Data Governance

Policies and controls managing the availability, integrity, security, and usability of data used in AI systems.

Automated Decision-Making (ADM)

Decisions made by an AI system with little or no human involvement, often subject to explanation/opt-out rights (cf. GDPR Art. 22).

AI Risk Management Framework (AI RMF)

NIST's voluntary framework (Govern, Map, Measure, Manage functions) for managing AI risks.

Risk Tolerance

Acceptable variation around risk appetite for specific objectives or risk types (NIST AI RMF term).

CE Marking

Conformity marking indicating a high-risk AI system meets EU AI Act requirements.

Control Gap Analysis

Structured comparison of the controls an AI system currently has against those required by a chosen framework or regulation, surfacing missing or insufficient safeguards to remediate.

Model Risk Management (MRM)

Discipline (rooted in financial services, now applied to AI) governing model validation, monitoring, and controls.

Responsible AI

Umbrella practice of designing, developing, and deploying AI in ways that are ethical, safe, fair, and accountable.

Further resources →

AI Governance Committee

Cross-functional body that steers an organization's AI governance program — bringing together stakeholders such as legal, security, data protection, risk, and technical teams to set direction, prioritize, and oversee decisions (broader in remit than an AI ethics board).

Deceptive Alignment

Hypothesized failure mode where a model behaves safely during evaluation but pursues different goals when unsupervised.

Model Extraction / Theft / Model Stealing

Attack that reconstructs or steals a proprietary model's functionality or parameters via repeated queries against its API.

Risk Appetite

Level and type of risk an organization is willing to accept in pursuit of its objectives.

Third-Party Risk Management (TPRM)

Process of assessing and monitoring risks introduced by vendors, suppliers, or AI model providers.

Further resources →

Post-Market Monitoring

EU AI Act obligation (Art. 72) for providers to systematically monitor high-risk AI system performance after deployment.

Audit Trail

Tamper-evident, chronological record of the actions, decisions, and changes across an AI system's governance lifecycle (who did what, when), enabling accountability, review, and evidence for internal and external audits.

AI Intake / Intake Form

Structured mechanism (e.g., forms sent to teams) for discovering and registering AI systems and use cases across an organization, including surfacing shadow AI – the entry point that feeds the AI inventory.

Further resources →

Explainability / Explainable AI (XAI)

Techniques and practices that make an AI system's decisions understandable to humans.

Supply Chain Vulnerability

Risk arising from compromised third-party models, data, or plugins.

Further resources →

Key Risk Indicator (KRI)

Metric used to signal increasing risk exposure ahead of an actual control failure.

AI Ethics Board / Committee

Internal governance body reviewing AI systems for ethical risk and compliance.

Adversarial Training

Defensive technique that trains a model on adversarial examples to improve its robustness against attack.

Differential Privacy

Mathematical technique that adds calibrated noise to protect individual privacy in datasets or outputs.

Red Teaming

Adversarial testing exercise simulating attacks or misuse to uncover a system's vulnerabilities.

Trustworthy AI

Describes AI usage, development, etc. that is valid, reliable, safe, secure, accountable, transparent, explainable, privacy-enhanced, and fair.

Further resources →

AI Agent / Agentic AI

AI system that can autonomously plan, decide, and take actions (often by calling tools or other systems) to pursue goals with limited human intervention; its autonomy raises distinct responsible-AI challenges around oversight, accountability, traceability, and scope control.

Further resources →

Governance, Risk, and Compliance (GRC)

Integrated approach to organizational governance, risk management, and regulatory compliance.

Vendor Assessment / Third-Party AI

Structured evaluation of a third-party AI product or provider against your security, privacy, and risk policies — reviewing documentation, questionnaires, and evidence — to decide whether an external AI system meets your standards before adoption.

Further resources →

Model Card

Standardized document describing a model's intended use, performance, limitations, and evaluation results.

Further resources →

Model Validation

Independent testing confirming a model performs as intended and within acceptable risk limits.

Transparency Obligations (AI-Generated Content)

EU AI Act Art. 50 requirements to disclose AI-generated or manipulated content (e.g., deepfakes) to users.

Cookies
By clicking “Yes”, you agree to the storing of cookies on your device to enhance site navigation, and to improve our marketing. View our Privacy Policy for more information.
trailis now part of