Responsible AI Glossary

EU AI Act classification (Annex III / Art. 6) triggering the strictest pre- and post-market obligations short of prohibition.

Risk that occurs when an adversary manipulates model behavior via crafted input that overrides intended instructions.

Core AI governance function of ensuring AI systems and processes meet applicable legal, regulatory, and framework requirements on an ongoing basis – a key pillar of the AI governance operating model and a subset of GRC.

International standard for information security management systems (ISMS); in AI governance it underpins the security of data and models and is often pursued alongside ISO/IEC 42001.

Live biometric identification in public spaces by law enforcement; heavily restricted under the EU AI Act.

A policy, process, or technical safeguard put in place to manage a specific risk to an acceptable level; the basic building block of any GRC program.

Techniques and practices that make an AI system's decisions understandable to humans.

A system's resilience to adversarial attacks or manipulated inputs.

AI system that can autonomously plan, decide, and take actions (often by calling tools or other systems) to pursue goals with limited human intervention; its autonomy raises distinct responsible-AI challenges around oversight, accountability, traceability, and scope control.

Category of risk in the MIT AI Risk Repository referring to AI's potential to cause human extinction or permanent civilizational collapse.

Describes AI usage, development, etc. that is valid, reliable, safe, secure, accountable, transparent, explainable, privacy-enhanced, and fair.

Controlled environment allowing testing of innovative AI systems under regulatory supervision before market entry (EU AI Act Art. 57–63).

GPAI model meeting a high-impact capability threshold, subject to additional EU AI Act obligations (Art. 51–55).

Formal internal document that sets the rules, principles, roles, and responsibilities for how an organization develops, procures, and uses AI — the practical formalization of an AI governance program that makes guidelines actionable for employees.

Discipline (rooted in financial services, now applied to AI) governing model validation, monitoring, and controls.

Predefined procedure for detecting, escalating, and remediating AI system failures or harms.

Cross-functional body that steers an organization's AI governance program — bringing together stakeholders such as legal, security, data protection, risk, and technical teams to set direction, prioritize, and oversee decisions (broader in remit than an AI ethics board).

Field focused on preventing unintended, harmful, or catastrophic behavior from AI systems.

Practices and tooling for operationalizing, deploying, and monitoring machine learning models at scale.

Structured foundation of standards, requirements, and best practices an organization adopts and adapts to govern AI – often built on established frameworks (e.g., NIST AI RMF, ISO/IEC 42001) plus legal obligations and industry practices.

Embedding detectable markers in AI-generated content to indicate its synthetic origin.

Independent testing confirming a model performs as intended and within acceptable risk limits.

Risk of resource-exhaustion attacks against a hosted model.

AI safety concern where a model deliberately underperforms during evaluation to conceal true capabilities.

Tool for assigning ownership and accountability across AI governance tasks (Responsible, Accountable, Consulted, Informed), clarifying who does what across the AI lifecycle and enabling governance as a joint, traceable effort.

Category in the MIT AI Risk Repository covering AI-generated false or misleading content, distinguishing unintentional (mis-) from intentional (dis-) spread.

Unauthorized or unsanctioned use of AI tools within an organization, outside governance oversight.

Process of verifying that a high-risk AI system meets applicable EU AI Act requirements before market placement.

An AI model whose internal decision logic is not interpretable to users or auditors.

Formal system of policies and processes for governing AI per ISO/IEC 42001.

Any entity involved in an AI system's lifecycle: designer, developer, deployer, operator, evaluator (NIST AI RMF terminology).

Systematic and repeatable errors in a system that create unfair outcomes for particular groups.

Specialized inventory of an organization's AI agents recording each agent's configuration, permissions, connected tools, and risk classification — extending the AI registry concept to autonomous systems that get deployed quickly and at scale.

Risk of users trusting AI outputs without sufficient verification or oversight.

Ensuring an AI system's goals and behaviors match human intentions and values.

EU AI Act category for models displaying significant generality and capable of competently performing a wide range of tasks.

The end-to-end discipline of identifying, assessing, treating, and monitoring risks arising from AI systems across their lifecycle — the practice that frameworks like NIST AI RMF and ISO 42001 structure and formalize.

Degree to which a human can understand the cause of a model's decision or prediction.

EU AI Act term for an entity using an AI system under its authority, other than in personal non-professional use.

Risk of models leaking confidential or personal data through outputs.

Evaluation of whether a control has been implemented and is actually working to mitigate the AI risk it targets; increasingly automated by analyzing connected sources and evidence to judge effectiveness.

Metric used to signal increasing risk exposure ahead of an actual control failure.

Tamper-evident, chronological record of the actions, decisions, and changes across an AI system's governance lifecycle (who did what, when), enabling accountability, review, and evidence for internal and external audits.

Independent examination of an algorithm's design, data, and outputs for bias, safety, or compliance issues.

Practice of defining, monitoring, and enforcing the boundaries of what an AI agent may do (the tools, data, and actions within its remit), and detecting scope violations where an agent exceeds those permissions.

Ability to reconstruct and follow an AI system's decisions, data, and actions back to their sources — models, datasets, prompts, and human approvals — a prerequisite for accountability, debugging, and audits, and an especially acute challenge for autonomous agents.

Entity established in the EU that places on the market an AI system from a non-EU provider.

Attack determining whether a specific data record was used in a model's training set.

An event where an AI system causes or nearly causes harm, tracked in resources like the OECD AI Incidents Monitor and MIT AI Risk Repository.

Skills, knowledge, and understanding enabling providers, deployers, and users to make informed decisions about AI (explicit EU AI Act obligation, Art. 4).

Stages from design and data collection through deployment, monitoring, and retirement (ISO 42001, NIST AI RMF).

The ongoing practice of identifying, collecting, mapping, and governing an organization's AI use cases across their lifecycle — the foundation for inventory, risk classification, and EU AI Act compliance.

Identifying and analyzing parties affected by or influencing an AI system's development and deployment.

Attack that reconstructs training data or sensitive attributes from a model's outputs.

Attack that reconstructs or steals a proprietary model's functionality or parameters via repeated queries against its API.

Frontier AI lab commitment to gate model capability increases behind corresponding safety and security measures.

Principle of embedding privacy protections into system architecture from the outset rather than retrofitting.

GRC model separating operational management, risk/compliance oversight, and independent audit functions.

The effect of uncertainty on objectives, typically expressed as a function of likelihood and impact (ISO 31000 / ISO 42001 framing, adapted for AI harms).

European standard, once adopted, that provides a presumption of conformity with EU AI Act requirements.

Traceable record of data's origin, movement, and transformations through a system.

Governance pattern where an AI agent's proposed changes are staged in a safe copy rather than written directly to live systems, so a human can review, edit, and approve them before they take effect — keeping humans in the loop without sacrificing automation.

Entity that develops an AI system/GPAI model and places it on the market under its own name.

Change in the statistical distribution of input data over time relative to training data, a common cause of model performance decay (distinct from concept drift, which is a change in input-output relationships).

Entity in the supply chain, other than provider or importer, that makes an AI system available on the market.

Artificially generated data used to train or test models, often to preserve privacy or augment datasets.

Input deliberately crafted to cause a model to make a mistake (OWASP ML Top 10).

Mathematical technique that adds calibrated noise to protect individual privacy in datasets or outputs.

Control point that mediates and monitors the tools, data, and systems an AI agent can reach via the Model Context Protocol (MCP), enforcing permission boundaries and flagging out-of-scope actions before they occur.

Attack that extracts a system's hidden prompt, instructions, or confidential context through crafted user input.

Ability that appears in a model unpredictably as scale increases, not present in smaller versions.

Technical or procedural controls constraining an AI system's outputs or actions within acceptable bounds.

Emerging executive role responsible for enterprise AI strategy, ethics, and governance.

EU AI Act term for a change to an AI system significant enough to require reassessment of conformity.

Unauthorized modification of a model's weights, architecture, or configuration after training, whether via supply chain compromise or insider action.

EU AI Act–defined event (Art. 3(49)) involving death, serious harm, infrastructure disruption, or fundamental rights infringement linked to an AI system, triggering mandatory reporting.

Documented log of identified risks, owners, likelihood/impact ratings, and mitigation status.

Metadata or cryptographic proof documenting the origin and history of AI-generated or edited content (e.g., C2PA).

Standardized document capturing the key details of a specific AI use case (purpose, data, stakeholders, risks, controls) to support inventory, transparency, and governance – complementing model and system cards at the application level.

Regulation (EU) 2024/1689 establishing harmonised rules on AI in the EU, using a risk-based, tiered approach.

Internal governance body reviewing AI systems for ethical risk and compliance.

Risk remaining after controls and mitigations have been applied.

GDPR-mandated assessment of privacy risks for high-risk processing activities, often paired with AI risk assessments.

Classifying individuals based on biometric data into categories such as race, gender, or political opinion; restricted under the EU AI Act.

Attack that recovers verbatim or near-verbatim training examples (including sensitive data) directly from a model's outputs.

LLM risk from failing to validate/sanitize LLM outputs before downstream use.

Risk that an AI system reproduces or generates content that violates third-party copyright or intellectual-property rights, via training data or model outputs — a growing responsible-AI and legal concern.

Attack that crafts inputs at inference time to cause a deployed model to misclassify or malfunction, without altering the model itself.

EU body responsible for AI Act implementation, particularly oversight of general-purpose AI models.

Process of assessing and monitoring risks introduced by vendors, suppliers, or AI model providers.

Adversarial testing exercise simulating attacks or misuse to uncover a system's vulnerabilities.

GDPR-mandated inventory documenting how personal data is processed (purposes, data categories, recipients, safeguards); in AI governance it complements the AI registry and DPIAs for data-driven systems.

Decisions made by an AI system with little or no human involvement, often subject to explanation/opt-out rights (cf. GDPR Art. 22).

Research field aiming to reverse-engineer neural network internals into human-understandable algorithms.

Synthetic media generated or manipulated by AI that falsely depicts a real person's likeness or actions.

Standard requiring humans retain real, informed influence over an AI system's consequential decisions.

The subset of AI risk management focused on protecting AI systems, models, and data from malicious attacks (e.g., data poisoning, model theft, adversarial manipulation) — distinct from AI safety's broader focus on unintended harm.

Acceptable variation around risk appetite for specific objectives or risk types (NIST AI RMF term).

Attack that corrupts training data to manipulate model behavior (OWASP LLM/ML Top 10).

Structured testing of a model's capabilities, safety properties, or risks against benchmarks.