Responsible AI Glossary

Structured evaluation of the potential effects of an algorithmic system before/during deployment.

Independent organization designated to assess conformity of certain high-risk AI systems under the EU AI Act.

Attack that implants a hidden trigger during training so a model behaves normally until a specific input activates malicious behavior.

Attack that reconstructs training data or sensitive attributes from a model's outputs.

EU body responsible for AI Act implementation, particularly oversight of general-purpose AI models.

Predefined procedure for detecting, escalating, and remediating AI system failures or harms.

Degree to which a human can understand the cause of a model's decision or prediction.

Integrated approach to organizational governance, risk management, and regulatory compliance.

Regulation (EU) 2024/1689 establishing harmonised rules on AI in the EU, using a risk-based, tiered approach.

Indirect discrimination occurring when a neutral variable correlates strongly with a protected attribute.

Principle that organizations and individuals must be answerable for the outcomes of AI systems they design, develop, or deploy (OECD, NIST AI RMF).

Input deliberately crafted to cause a model to make a mistake (OWASP ML Top 10).

Live biometric identification in public spaces by law enforcement; heavily restricted under the EU AI Act.

Attack that reconstructs or steals a proprietary model's functionality or parameters via repeated queries against its API.

EU AI Act Art. 50 requirements to disclose AI-generated or manipulated content (e.g., deepfakes) to users.

Emerging executive role responsible for enterprise AI strategy, ethics, and governance.

The effect of uncertainty on objectives, typically expressed as a function of likelihood and impact (ISO 31000 / ISO 42001 framing, adapted for AI harms).

The subset of AI risk management focused on protecting AI systems, models, and data from malicious attacks (e.g., data poisoning, model theft, adversarial manipulation) — distinct from AI safety's broader focus on unintended harm.

Automated processing of personal data to evaluate or predict aspects of a person (GDPR Art. 4(4)).

Tool for assigning ownership and accountability across AI governance tasks (Responsible, Accountable, Consulted, Informed), clarifying who does what across the AI lifecycle and enabling governance as a joint, traceable effort.

Stages from design and data collection through deployment, monitoring, and retirement (ISO 42001, NIST AI RMF).

Attack that corrupts training data to manipulate model behavior (OWASP LLM/ML Top 10).

Entity established in the EU that places on the market an AI system from a non-EU provider.

Metadata or cryptographic proof documenting the origin and history of AI-generated or edited content (e.g., C2PA).

Systematic and repeatable errors in a system that create unfair outcomes for particular groups.

Decisions made by an AI system with little or no human involvement, often subject to explanation/opt-out rights (cf. GDPR Art. 22).

LLM risk from failing to validate/sanitize LLM outputs before downstream use.

EU AI Act classification (Annex III / Art. 6) triggering the strictest pre- and post-market obligations short of prohibition.

EU AI Act–required assessment (Art. 27) of a high-risk AI system's impact on fundamental rights, for certain deployers.

Technical tool (e.g., differential privacy, federated learning) that reduces privacy risk in data processing.

Prohibited EU AI Act practice of evaluating individuals based on behavior/characteristics leading to unjustified detrimental treatment.

Mathematical technique that adds calibrated noise to protect individual privacy in datasets or outputs.

Documentation, similar to a model card, describing an AI system's overall behavior, safeguards, and limitations at the product level.

Governance pattern where an AI agent's proposed changes are staged in a safe copy rather than written directly to live systems, so a human can review, edit, and approve them before they take effect — keeping humans in the loop without sacrificing automation.

Formal internal document that sets the rules, principles, roles, and responsibilities for how an organization develops, procures, and uses AI — the practical formalization of an AI governance program that makes guidelines actionable for employees.

Risk remaining after controls and mitigations have been applied.

Adversarial testing exercise simulating attacks or misuse to uncover a system's vulnerabilities.

Formal system of policies and processes for governing AI per ISO/IEC 42001.

Structured testing of a model's capabilities, safety properties, or risks against benchmarks.

Person who coordinates and drives an organization's AI governance program — securing buy-in, aligning business with governance goals, and overseeing progress — without necessarily owning every individual governance task.

Attack determining whether a specific data record was used in a model's training set.

Techniques applied to data, models, or processes to reduce unfair bias in AI outcomes.

Fairness criterion requiring a decision would be the same in a counterfactual world absent a protected attribute.

Standardized document capturing the key details of a specific AI use case (purpose, data, stakeholders, risks, controls) to support inventory, transparency, and governance – complementing model and system cards at the application level.

Unauthorized or unsanctioned use of AI tools within an organization, outside governance oversight.

Level and type of risk an organization is willing to accept in pursuit of its objectives.

Independent testing confirming a model performs as intended and within acceptable risk limits.

Structured evaluation of a third-party AI product or provider against your security, privacy, and risk policies — reviewing documentation, questionnaires, and evidence — to decide whether an external AI system meets your standards before adoption.

Structured comparison of the controls an AI system currently has against those required by a chosen framework or regulation, surfacing missing or insufficient safeguards to remediate.

Acceptable variation around risk appetite for specific objectives or risk types (NIST AI RMF term).

Oversight model where a human actively participates in or approves an AI system's decisions.

Defensive technique that trains a model on adversarial examples to improve its robustness against attack.

Traceable record of data's origin, movement, and transformations through a system.

Technical or procedural controls constraining an AI system's outputs or actions within acceptable bounds.

Metric used to signal increasing risk exposure ahead of an actual control failure.

International standard for information security management systems (ISMS); in AI governance it underpins the security of data and models and is often pursued alongside ISO/IEC 42001.

Risk of users trusting AI outputs without sufficient verification or oversight.

AI system that can autonomously plan, decide, and take actions (often by calling tools or other systems) to pursue goals with limited human intervention; its autonomy raises distinct responsible-AI challenges around oversight, accountability, traceability, and scope control.

Large model trained on broad data, adaptable to a wide range of downstream tasks.

Embedding detectable markers in AI-generated content to indicate its synthetic origin.

Intergovernmental standards (2019) for trustworthy AI: inclusive growth, human-centered values, transparency, robustness, and accountability.

Principle of collecting/processing only data necessary for a specified purpose.

Documentation or artifacts (logs, reports, test results, approvals) that demonstrate a control is in place and effective, or that a governance requirement has been met; the substantiating basis for audits and conformity assessments.

Controlled environment allowing testing of innovative AI systems under regulatory supervision before market entry (EU AI Act Art. 57–63).

Risk arising from compromised third-party models, data, or plugins.

Skills, knowledge, and understanding enabling providers, deployers, and users to make informed decisions about AI (explicit EU AI Act obligation, Art. 4).

Internal governance body reviewing AI systems for ethical risk and compliance.

EU AI Act requirement (Art. 14) that high-risk AI systems be designed to allow effective human supervision.

EU AI Act term for an entity using an AI system under its authority, other than in personal non-professional use.

Category in the MIT AI Risk Repository covering AI-generated false or misleading content, distinguishing unintentional (mis-) from intentional (dis-) spread.

Technique for bypassing an AI model's safety controls to elicit prohibited content or behavior.

Artificially generated data used to train or test models, often to preserve privacy or augment datasets.

International management-system standard for AI (AIMS), specifying requirements to responsibly govern, manage, and continually improve AI systems across their lifecycle; a leading basis for AI governance certification.

Independent examination of an algorithm's design, data, and outputs for bias, safety, or compliance issues.

Standardized document describing a model's intended use, performance, limitations, and evaluation results.

European standard, once adopted, that provides a presumption of conformity with EU AI Act requirements.

Discipline (rooted in financial services, now applied to AI) governing model validation, monitoring, and controls.

Discrimination that occurs when a facially neutral policy disproportionately harms a protected group.

Entity in the supply chain, other than provider or importer, that makes an AI system available on the market.

A policy, process, or technical safeguard put in place to manage a specific risk to an acceptable level; the basic building block of any GRC program.

Policies and controls managing the availability, integrity, security, and usability of data used in AI systems.

NIST's voluntary framework (Govern, Map, Measure, Manage functions) for managing AI risks.

Synthetic media generated or manipulated by AI that falsely depicts a real person's likeness or actions.

The overarching structures, policies, roles, and processes by which organizations direct, manage, oversee, and hold accountable the design, development, procurement, and use of AI — the umbrella discipline covering AI risk management, ethics, safety, and compliance.

Frontier AI lab commitment to gate model capability increases behind corresponding safety and security measures.

Change in the statistical distribution of input data over time relative to training data, a common cause of model performance decay (distinct from concept drift, which is a change in input-output relationships).

An AI model whose internal decision logic is not interpretable to users or auditors.

Change in the statistical relationship between model inputs and outputs over time, degrading performance.

Techniques and practices that make an AI system's decisions understandable to humans.

Continuous, real-time monitoring and constraint of a deployed AI system or agent to keep it within approved boundaries — checking permissions, tool usage, and behavior at runtime and flagging or blocking scope violations before they become incidents.

Ensuring an AI system's goals and behaviors match human intentions and values.

Describes AI usage, development, etc. that is valid, reliable, safe, secure, accountable, transparent, explainable, privacy-enhanced, and fair.

GDPR-mandated assessment of privacy risks for high-risk processing activities, often paired with AI risk assessments.

Process of verifying that a high-risk AI system meets applicable EU AI Act requirements before market placement.

Ability that appears in a model unpredictably as scale increases, not present in smaller versions.

Practices and tooling for operationalizing, deploying, and monitoring machine learning models at scale.

Practice of defining, monitoring, and enforcing the boundaries of what an AI agent may do (the tools, data, and actions within its remit), and detecting scope violations where an agent exceeds those permissions.

Research problem of supervising AI systems whose capabilities exceed a human evaluator's own.

GRC model separating operational management, risk/compliance oversight, and independent audit functions.

GPAI model meeting a high-impact capability threshold, subject to additional EU AI Act obligations (Art. 51–55).