Responsible AI Glossary

Classifying individuals based on biometric data into categories such as race, gender, or political opinion; restricted under the EU AI Act.

Ability to reconstruct and follow an AI system's decisions, data, and actions back to their sources — models, datasets, prompts, and human approvals — a prerequisite for accountability, debugging, and audits, and an especially acute challenge for autonomous agents.

Embedding detectable markers in AI-generated content to indicate its synthetic origin.

Any entity involved in an AI system's lifecycle: designer, developer, deployer, operator, evaluator (NIST AI RMF terminology).

Input deliberately crafted to cause a model to make a mistake (OWASP ML Top 10).

Attack where malicious instructions are embedded in third-party content later processed by an LLM.

Emerging executive role responsible for enterprise AI strategy, ethics, and governance.

Process of verifying that a high-risk AI system meets applicable EU AI Act requirements before market placement.

The set of ongoing functions, processes, and structures that operationalize AI governance day-to-day — typically spanning use-case management, risk and compliance management, third-party management, security, data governance, and incident management.

Controlled environment allowing testing of innovative AI systems under regulatory supervision before market entry (EU AI Act Art. 57–63).

Umbrella practice of designing, developing, and deploying AI in ways that are ethical, safe, fair, and accountable.

Independent testing confirming a model performs as intended and within acceptable risk limits.

Technical or procedural controls constraining an AI system's outputs or actions within acceptable bounds.

Research field aiming to reverse-engineer neural network internals into human-understandable algorithms.

Traceable record of data's origin, movement, and transformations through a system.

Continuous, real-time monitoring and constraint of a deployed AI system or agent to keep it within approved boundaries — checking permissions, tool usage, and behavior at runtime and flagging or blocking scope violations before they become incidents.

Cross-functional body that steers an organization's AI governance program — bringing together stakeholders such as legal, security, data protection, risk, and technical teams to set direction, prioritize, and oversee decisions (broader in remit than an AI ethics board).

Governance mechanism protecting individuals who report AI-related misconduct or risk.

Risk of models leaking confidential or personal data through outputs.

European standard, once adopted, that provides a presumption of conformity with EU AI Act requirements.

Research problem of supervising AI systems whose capabilities exceed a human evaluator's own.

Core AI governance function of ensuring AI systems and processes meet applicable legal, regulatory, and framework requirements on an ongoing basis – a key pillar of the AI governance operating model and a subset of GRC.

GDPR-mandated inventory documenting how personal data is processed (purposes, data categories, recipients, safeguards); in AI governance it complements the AI registry and DPIAs for data-driven systems.

Integrated approach to organizational governance, risk management, and regulatory compliance.

Degradation of model performance over time as real-world data diverges from training data.

Principle that organizations and individuals must be answerable for the outcomes of AI systems they design, develop, or deploy (OECD, NIST AI RMF).

Attack that implants a hidden trigger during training so a model behaves normally until a specific input activates malicious behavior.

The ongoing practice of identifying, collecting, mapping, and governing an organization's AI use cases across their lifecycle — the foundation for inventory, risk classification, and EU AI Act compliance.

International standard for information security management systems (ISMS); in AI governance it underpins the security of data and models and is often pursued alongside ISO/IEC 42001.

Structured mechanism (e.g., forms sent to teams) for discovering and registering AI systems and use cases across an organization, including surfacing shadow AI – the entry point that feeds the AI inventory.

Technique for bypassing an AI model's safety controls to elicit prohibited content or behavior.

GPAI model meeting a high-impact capability threshold, subject to additional EU AI Act obligations (Art. 51–55).

Attack that extracts a system's hidden prompt, instructions, or confidential context through crafted user input.

Practice of defining, monitoring, and enforcing the boundaries of what an AI agent may do (the tools, data, and actions within its remit), and detecting scope violations where an agent exceeds those permissions.

Framing of AI systems as inseparable combinations of technology, people, and organizational context.

Policies and controls managing the availability, integrity, security, and usability of data used in AI systems.

Principle of embedding privacy protections into system architecture from the outset rather than retrofitting.

Classifying AI applications (e.g., low, limited, high, unacceptable risk) to determine applicable controls.

EU AI Act requirement (Art. 14) that high-risk AI systems be designed to allow effective human supervision.

Skills, knowledge, and understanding enabling providers, deployers, and users to make informed decisions about AI (explicit EU AI Act obligation, Art. 4).

A policy, process, or technical safeguard put in place to manage a specific risk to an acceptable level; the basic building block of any GRC program.

Structured comparison of the controls an AI system currently has against those required by a chosen framework or regulation, surfacing missing or insufficient safeguards to remediate.

Conformity marking indicating a high-risk AI system meets EU AI Act requirements.

Category in the MIT AI Risk Repository covering AI-generated false or misleading content, distinguishing unintentional (mis-) from intentional (dis-) spread.

Regulation (EU) 2024/1689 establishing harmonised rules on AI in the EU, using a risk-based, tiered approach.

EU AI Act category for models displaying significant generality and capable of competently performing a wide range of tasks.

Unauthorized or unsanctioned use of AI tools within an organization, outside governance oversight.

Structured evaluation of the potential effects of an algorithmic system before/during deployment.

Discrimination that occurs when a facially neutral policy disproportionately harms a protected group.

EU AI Act Art. 50 requirements to disclose AI-generated or manipulated content (e.g., deepfakes) to users.

Live biometric identification in public spaces by law enforcement; heavily restricted under the EU AI Act.

Risk arising from compromised third-party models, data, or plugins.

EU regulation governing the processing of personal data; it underpins many AI governance obligations (lawful basis, data minimization, automated-decision rights) and frequently intersects with AI-specific requirements like DPIAs and profiling limits.

Training technique that fine-tunes models using human preference judgments.

Unauthorized modification of a model's weights, architecture, or configuration after training, whether via supply chain compromise or insider action.

Change in the statistical relationship between model inputs and outputs over time, degrading performance.

Prohibited EU AI Act practice of evaluating individuals based on behavior/characteristics leading to unjustified detrimental treatment.

Risk that occurs when an adversary manipulates model behavior via crafted input that overrides intended instructions.

Entity in the supply chain, other than provider or importer, that makes an AI system available on the market.

Attack that reconstructs or steals a proprietary model's functionality or parameters via repeated queries against its API.

Structured foundation of standards, requirements, and best practices an organization adopts and adapts to govern AI – often built on established frameworks (e.g., NIST AI RMF, ISO/IEC 42001) plus legal obligations and industry practices.

Intergovernmental standards (2019) for trustworthy AI: inclusive growth, human-centered values, transparency, robustness, and accountability.

Attack that reconstructs training data or sensitive attributes from a model's outputs.

Technical tool (e.g., differential privacy, federated learning) that reduces privacy risk in data processing.

Specialized inventory of an organization's AI agents recording each agent's configuration, permissions, connected tools, and risk classification — extending the AI registry concept to autonomous systems that get deployed quickly and at scale.

Risk that an AI system reproduces or generates content that violates third-party copyright or intellectual-property rights, via training data or model outputs — a growing responsible-AI and legal concern.

Stages from design and data collection through deployment, monitoring, and retirement (ISO 42001, NIST AI RMF).

EU AI Act–required assessment (Art. 27) of a high-risk AI system's impact on fundamental rights, for certain deployers.

Formal system of policies and processes for governing AI per ISO/IEC 42001.

Process of assessing and monitoring risks introduced by vendors, suppliers, or AI model providers.

Model-generated output that is factually incorrect or fabricated but presented as if true.

Formal internal document that sets the rules, principles, roles, and responsibilities for how an organization develops, procures, and uses AI — the practical formalization of an AI governance program that makes guidelines actionable for employees.

Risk of users trusting AI outputs without sufficient verification or oversight.

The overarching structures, policies, roles, and processes by which organizations direct, manage, oversee, and hold accountable the design, development, procurement, and use of AI — the umbrella discipline covering AI risk management, ethics, safety, and compliance.

Change in the statistical distribution of input data over time relative to training data, a common cause of model performance decay (distinct from concept drift, which is a change in input-output relationships).

EU AI Act–defined event (Art. 3(49)) involving death, serious harm, infrastructure disruption, or fundamental rights infringement linked to an AI system, triggering mandatory reporting.

Decisions made by an AI system with little or no human involvement, often subject to explanation/opt-out rights (cf. GDPR Art. 22).

Continual improvement cycle underlying ISO management system standards, including ISO 42001.

Tamper-evident, chronological record of the actions, decisions, and changes across an AI system's governance lifecycle (who did what, when), enabling accountability, review, and evidence for internal and external audits.

Level and type of risk an organization is willing to accept in pursuit of its objectives.

Evaluation of whether a control has been implemented and is actually working to mitigate the AI risk it targets; increasingly automated by analyzing connected sources and evidence to judge effectiveness.

Entity established in the EU that places on the market an AI system from a non-EU provider.

Documentation, similar to a model card, describing an AI system's overall behavior, safeguards, and limitations at the product level.

Discipline (rooted in financial services, now applied to AI) governing model validation, monitoring, and controls.

Entity that develops an AI system/GPAI model and places it on the market under its own name.

Acceptable variation around risk appetite for specific objectives or risk types (NIST AI RMF term).

Structured testing of a model's capabilities, safety properties, or risks against benchmarks.

Attack that corrupts training data to manipulate model behavior (OWASP LLM/ML Top 10).

Techniques applied to data, models, or processes to reduce unfair bias in AI outcomes.

GDPR-mandated assessment of privacy risks for high-risk processing activities, often paired with AI risk assessments.

Structured evaluation of a third-party AI product or provider against your security, privacy, and risk policies — reviewing documentation, questionnaires, and evidence — to decide whether an external AI system meets your standards before adoption.

Independent organization designated to assess conformity of certain high-risk AI systems under the EU AI Act.

Fairness metric requiring equal true-positive and false-positive rates across groups.

Risk of resource-exhaustion attacks against a hosted model.

Required EU AI Act records (Annex IV) describing a high-risk AI system's design, purpose, and risk controls.

GRC model separating operational management, risk/compliance oversight, and independent audit functions.

Standardized document describing a model's intended use, performance, limitations, and evaluation results.

Adversarial testing exercise simulating attacks or misuse to uncover a system's vulnerabilities.

Control point that mediates and monitors the tools, data, and systems an AI agent can reach via the Model Context Protocol (MCP), enforcing permission boundaries and flagging out-of-scope actions before they occur.

EU body responsible for AI Act implementation, particularly oversight of general-purpose AI models.