By clicking “Accept”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.

Regulating GenAI: Implications of the EU AI Act

The EU AI Act extensively regulates general purpose AI (GPAI), such as GenAI. This article explains how the EU classifies its risks and what obligations providers and integrators need to meet.

Regulating GenAI: Implications of the EU AI Act

Sleek v2.0 public release is here

Lorem ipsum dolor sit amet, consectetur adipiscing elit lobortis arcu enim urna adipiscing praesent velit viverra sit semper lorem eu cursus vel hendrerit elementum morbi curabitur etiam nibh justo, lorem aliquet donec sed sit mi at ante massa mattis.

  1. Neque sodales ut etiam sit amet nisl purus non tellus orci ac auctor
  2. Adipiscing elit ut aliquam purus sit amet viverra suspendisse potent i
  3. Mauris commodo quis imperdiet massa tincidunt nunc pulvinar
  4. Adipiscing elit ut aliquam purus sit amet viverra suspendisse potenti

What has changed in our latest release?

Lorem ipsum dolor sit amet, consectetur adipiscing elit ut aliquam, purus sit amet luctus venenatis, lectus magna fringilla urna, porttitor rhoncus dolor purus non enim praesent elementum facilisis leo, vel fringilla est ullamcorper eget nulla facilisi etiam dignissim diam quis enim lobortis scelerisque fermentum dui faucibus in ornare quam viverra orci sagittis eu volutpat odio facilisis mauris sit amet massa vitae tortor condimentum lacinia quis vel eros donec ac odio tempor orci dapibus ultrices in iaculis nunc sed augue lacus

All new features available for all public channel users

At risus viverra adipiscing at in tellus integer feugiat nisl pretium fusce id velit ut tortor sagittis orci a scelerisque purus semper eget at lectus urna duis convallis. porta nibh venenatis cras sed felis eget neque laoreet libero id faucibus nisl donec pretium vulputate sapien nec sagittis aliquam nunc lobortis mattis aliquam faucibus purus in.

  • Neque sodales ut etiam sit amet nisl purus non tellus orci ac auctor
  • Adipiscing elit ut aliquam purus sit amet viverra suspendisse potenti
  • Mauris commodo quis imperdiet massa tincidunt nunc pulvinar
  • Adipiscing elit ut aliquam purus sit amet viverra suspendisse potenti
Coding collaboration with over 200 users at once

Nisi quis eleifend quam adipiscing vitae aliquet bibendum enim facilisis gravida neque. Velit euismod in pellentesque massa placerat volutpat lacus laoreet non curabitur gravida odio aenean sed adipiscing diam donec adipiscing tristique risus. amet est placerat in egestas erat imperdiet sed euismod nisi.

“Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum”
Real-time code save every 0.1 seconds

Eget lorem dolor sed viverra ipsum nunc aliquet bibendum felis donec et odio pellentesque diam volutpat commodo sed egestas aliquam sem fringilla ut morbi tincidunt augue interdum velit euismod eu tincidunt tortor aliquam nulla facilisi aenean sed adipiscing diam donec adipiscing ut lectus arcu bibendum at varius vel pharetra nibh venenatis cras sed felis eget dolor cosnectur drolo.

The EU AI Act emerges as a pivotal legislation as it tries to regulate a technology that is rapidly advancing in its capabilities lately - and with it potential risks and harms. Initially, the 2021 proposal by the European Commission did not specifically address general purpose AI (GPAI) systems, like GenAI. However, recognizing the leaps in generative and foundation models, the Act was revised in late 2022 to encapsulate stringent rules for GPAI systems. This inclusion, met with broad endorsement from the European Parliament, underscored the trilogue negotiations. Yet, the GPAI regulation's stringent nature and technology-focused approach, instead of the original application-focused approach, elicited mixed reactions, notably from key EU Member States such as France, Germany, and Italy. In early 2024, all negotiating parties agreed upon the AI Act, including extensive regulations for GPAI systems akin to those of high-risk AI systems.

This article breaks down how the EU classifies the risks of GPAI models, as well as what responsibilities both model providers, e.g. OpenAI, Aleph Alpha, Mistral or those who modify models of such providers, and integrators, e.g. a finance company leveraging OpenAI’s GPT-4 for a customer-facing chatbot, have.

How the AI Act classifies GPAI models

The EU AI Act differentiates GPAI models based on their risk levels: systemic or non-systemic. This distinction is crucial as it dictates the regulatory scrutiny and obligations required from model providers, with systemic risk model providers facing heightened requirements.

What counts as systemic risk?

A GPAI model is deemed a systemic risk if it demonstrates “high capabilities”, i.e. the cumulative amount of the computing power used for its training is above 10^25 floating point operations per second (FLOPS).

Additionally, the European Commission may designate a GPAI model as having systemic risk based on various factors, such as the model's complexity of parameters, its input/output modalities, or its reach among businesses and consumers (refer to Annex IXc of the AI Act).

Can a GPAI model provider avoid the systemic risk class?

GPAI model providers can argue against a systemic risk classification by presenting "sufficiently substantiated arguments" to the Commission, a process mirroring that for high-risk AI system providers. Providers of systemic risk GPAI models can also request to have their model reevaluated in their risk classification at a later stage.

Obligations for GPAI Providers

All providers are mandated to disclose when users (i.e. natural persons) are interacting with an AI system (unless it's obvious), such as a generative AI chatbot, and to label all AI-generated outputs in a machine-readable format, which is referred to as transparency obligations.

Every GPAI model, including its training and testing process and the results of its evaluation, must also be well documented (see Annex IXa). This documentation must be kept up-tp-date and made available to downstream providers (i.e. those who want to integrate a third-party GPAI model into their products or AI systems) to assist them in adhering to their obligations under the AI Act (see Annex IXb)

GPAI providers also need to have a publicly available and detailed summary of the content that was used to train the model. Providers who are sitting outside the EU and who place their model on the Union’s market need to appoint an authorized representative who is established in the EU and who is responsible for the verification of the technical documentation or for providing the required information to the authorities, for instance.

The documentation requirements of GPAI models (seen in Annex IXa-b)

Additional obligations for systemic risk GPAI

For models classified as systemic risk, additional obligations are laid down by the AI Act. Systemic risk GPAI providers must immediately inform the European Commission about their model for public database inclusion, similar to high-risk AI systems, and report any serious incidents, including corrective actions taken. Comprehensive risk assessments, cybersecurity, and infrastructure security measures, along with a code of practice to demonstrate AI Act compliance, are also mandatory.

The documentation requirements are also extended and need to include detailed descriptions of the model evaluation strategies and results, the adversarial tests conducted and the system architecture explaining how software components build on or feed into each other and integrate into the overall processing.

The obligations for systemic risk foundation models (seen in Article 52c-e and Annex IXa-b)

Avoiding GPAI obligations

While you will not be able to avoid the obligations completely, providers can offer non-systemic risk GPAI models under a free open-source license to reduce the regulatory burdens. This is only possible if the model, parameters, including weights, information on model architecture and information on model usage are publicly accessible and if the modification and distribution of the model are possible.

Providers of such models only require having a publicly available and detailed summary of the training content of the GPAI model.

What does the EU AI Act imply if I am using GenAI for my products?

Numerous organizations are integrating foundation models, such as generative AI systems, into their own products or in other AI systems instead of building their very own models themselves. That is what is called a deployer, distributor or downstream provider in the EU AI Act. But does this mean that you are subject to any obligations set down by the EU AI Act? It depends on how you use these third-party systems.

If you are modifying an AI system, including a GPAI system, (given that it is already in operation and is not classified as high-risk) in its intended purpose such that it becomes a high-risk application, you would be considered as a high-risk AI system provider, i.e. that you need to fulfill the obligations for high-risk AI systems.

In the case that you need to demonstrate your compliance with the AI Act to the authorities, the GPAI system provider will have to support you with necessary information and documentation about their system, for which a written agreement is essential. This means there is a shared responsibility to ensure putting regulatory compliant AI systems on the market. However, the AI Act also states that the initial provider of the system does not need to make documentation available if the provider expressly excludes the change of its system into a high-risk system (see Article 28).

It is yet unclear how Article 28 will be interpreted, as it also states that a significant change to a third-party AI system would mean that there is a change in who is considered to be the AI system provider. So it remains unclear if, for example, the fine-tuning of a foundation model is considered a significant change and if this changes the obligations of operators along the AI value chain.

If you are using the GPAI system in a limited risk setting, the transparency obligations (i.e. labeling AI-generated content and informing users about the interaction with AI) apply.

Non-Compliance with the AI Act’s obligations

Violating the rules on general purpose, high-risk or limited risk AI systems can result in fines of up to €15 million or 3% of the organization’s global annual turnover. Those who fail to supply correct and complete information or provide misleading information to the authorities can expect fines of up to €7.5 million or 1.5% of the global annual revenue. SMEs and start-ups may receive smaller fines.

What’s next

The list of obligations is very long for providers of GPAI or GenAI models, and this article is solely supposed to give you a first overview of them. Organizations face a significant task in aligning with the AI Act, necessitating an extensive revamp of existing governance frameworks and documentation practices to meet compliance requirements.

You can expect the newly set up European AI Board and the AI Office to hand out more information, examples and guidance on how certain scenarios will be handled under the AI Act. Who will be liable for what responsibilities will be the key question in that process. However, the end-user-facing side will definitely have some sort of liability and notification duties.

It is also clear that there is an urgency in fulfilling the Act’s requirements, as it will likely enter into force in Q2 of 2024 and organizations have to comply with the regulation on prohibited AI systems 6 months after (end of 2024) and with the regulation on GPAI systems 12 months after (mid-2025).

Are you using an AI system in a high-risk use case, or are you building “traditional” AI models? Then feel free to read about the other requirements outlined in the AI Act here. We at trail want you to fully understand the whole AI development process, regardless of your background. Check out here, how we can help you with documentation, risk assessments, audits, and understanding your AI systems to be prepared for the EU AI Act.