AI Training in Companies: What Is Permitted and Where Risks Arise

The productive use of artificial intelligence has long been part of everyday operations in many companies, and this increasingly brings to the fore the question of which data these systems may use. Companies frequently seek to use their own customer data, employee data, and other corporate data to train AI applications for specific tasks or to adapt existing systems accordingly. Those who take this step operate in a regulated environment whose requirements are not limited to a single legal act. The General Data Protection Regulation (GDPR) takes center stage, providing requirements for virtually every phase of an AI project, flanked by copyright limitations where third-party content is involved. In addition, recent developments at the EU level come into play. While the AI Regulation (AI Act) was most recently amended by the AI Omnibus (Regulation (EU) 2026/1744), the data protection amendments of the so-called Data Omnibus – in particular an express legal basis for AI training – remain in the legislative process and are not yet substantively reliable. The following article classifies the key problem areas along the applicable legal framework and highlights what companies must already consider today when using their own and third-party data for AI purposes.

I. AI Training as a Data Protection Operation

When companies wish to use their data holdings for AI training, this can occur in two ways. On the one hand, they can train a model themselves or adapt an existing system with their own data, for example through fine-tuning. On the other hand, they can provide their data to an external provider whose system is then further developed on that basis. In both cases, the same fundamental question arises: whether, and under what conditions, the data in question may be used for this purpose at all.

As soon as an AI system processes data relating to identified or identifiable natural persons, the GDPR applies (Art. 2(1), Art. 4(1) GDPR). This is the case regardless of whether the data are collected for the purpose of creating a training dataset, used for the actual training, or processed only in the system’s outputs. The use of AI rarely constitutes a single operation; rather, it breaks down into several phases that must each be considered separately: the collection of data, their use for training or adapting a system, ongoing operations, and finally the generation of results. Each of these phases may involve its own purposes, categories of data subjects, and consequently its own requirements.

Of fundamental importance is first the clarification of the role in which the company acts. Whoever decides on the purposes and means of processing is the controller within the meaning of Art. 4(7) GDPR and bears the primary data protection burden. If the company trains the system itself, it is without question responsible for this processing. If, however, it provides the data to a provider, the classification depends on in whose interest and for whose purposes the training takes place. If the provider processes the data solely under the company’s instructions, it is a processor, which requires an agreement pursuant to Art. 28 GDPR. If, by contrast, the provider uses the supplied data to improve its own models – and thus for its own purposes – joint controllership under Art. 26 GDPR or independent controllership of the provider may be applicable.

II. Requirements Under the GDPR

1. Legal Basis for Processing

Every processing of personal data is subject to the requirement of a legal basis under Art. 6(1) GDPR. For AI training with a company’s own or supplied datasets, the primary bases to consider are consent (Art. 6(1)(a) GDPR) and legitimate interest (Art. 6(1)(f) GDPR).

Consent initially appears to be the most legally certain path, but proves operationally unstable in the training context. Since it may be withdrawn at any time pursuant to Art. 7(3) GDPR, significant difficulties arise because the effect of data already incorporated into a fully trained model can hardly be reversed after the fact. As the supporting basis for a training project, consent is therefore suitable only in narrowly defined constellations.

The practical default is thus the legitimate interest under Art. 6(1)(f) GDPR. Reliance on this basis requires a three-stage assessment: the identification of a legitimate interest, the determination of necessity, and a balancing against the interests and fundamental rights of data subjects. The Higher Regional Court of Cologne (Oberlandesgericht Köln) confirmed in summary proceedings that the development and training of AI can constitute a legitimate economic interest (judgment of 23 May 2025, case no. 15 UKl 2/25), albeit only on the basis of a summary review. What remains decisive, however, is the balancing in the individual case, which becomes more rigorous the more intensely the processing interferes with the rights of data subjects. Since the controller must demonstrate lawfulness (Art. 5(2) GDPR), careful documentation of the assessment steps is advisable.

A separate issue arises when data originally collected for a different purpose are to be used for training – for example, customer data from contract performance. In this regard, the principle of purpose limitation (Art. 5(1)(b) GDPR) must be observed, which permits further use only where both purposes are compatible. The compatibility assessment required under Art. 6(4) GDPR becomes more difficult the further the training purpose diverges from the original collection purpose, so that the use of such data for an unrelated AI model regularly requires separate justification.

Regarding the future legal framework, reference should be made to the proposed Art. 88c draft GDPR, which would expressly extend the legitimate interest basis to the development and operation of AI (as reported in Data Protection Update No. 223).

2. Specific Considerations for Sensitive and Employee Data

An increased level of difficulty arises as soon as special categories of personal data within the meaning of Article 9(1) GDPR are affected – such as information concerning health, ethnic origin, or political beliefs. These data are subject to a general prohibition on processing, which may only be overridden under the narrow conditions set out in Article 9(2) GDPR. A legitimate interest is expressly insufficient for this purpose; rather, a separate legal basis is required. This problem is particularly acute in the context of AI training, because the large underlying datasets almost inevitably contain sensitive data, especially since such information can already be inferred indirectly from seemingly innocuous data points. The mere possibility of such an inference may trigger the prohibition, irrespective of any corresponding intent on the part of the controller. That said, the scope of the prohibition with respect to the non-targeted processing of sensitive data during AI training remains disputed. The Higher Regional Court of Cologne (OLG Köln) has adopted an activity-based limitation in this regard, whereas the Higher Regional Court of Schleswig (OLG Schleswig, judgment of 12 August 2025, case no. 6 UKl 3/25) considered a violation of Article 9(1) GDPR to be likely, leaving the question unresolved for the time being.

This risk can be mitigated through data-minimizing design. Anonymization, pseudonymization, and the use of synthetic data substantially reduce the intensity of the interference with data subjects' rights. However, pseudonymized data remain personal data as long as re-identification is possible with the aid of additional information (Article 4(5) GDPR), and the generation of synthetic or anonymized data may itself constitute processing. Moreover, the controller bears the burden of demonstrating that a trained model is in fact anonymous. With regard to the future legal framework, the Data Omnibus proposes a dedicated provision in Article 9(5) GDPR-E for incidentally captured sensitive data, although this provision has not yet entered into force and, in its current form, is not free from criticism.

Where employee data are used for training purposes, the processing must ultimately be assessed directly against Articles 6 and 9 GDPR. In addition, co-determination rights of the works council under Section 87(1) no. 6 of the Works Constitution Act (BetrVG) are triggered as soon as the system is capable of monitoring employee conduct or performance, which is why the works council should be involved at an early stage.

3. Data Subject Rights in Relation to the Trained Model

The rights of data subjects continue to apply without restriction in the context of AI, in particular the rights to access (Art. 15 GDPR), rectification (Art. 16 GDPR), erasure (Art. 17 GDPR), and objection (Art. 21 GDPR). Their enforcement with respect to the trained model, however, presents particular difficulties, because individual personal data are not stored there as discrete data records but have been absorbed into the model parameters.

The question therefore arises whether the erasure of individual data points from the database suffices, or whether the model must be retrained – especially since complete “unlearning” of data already incorporated is, according to the current state of technology, hardly feasible. This is compounded by the phenomenon of memorization, in which a model can reproduce training data in its outputs in whole or in part.

The mere use of output filters that suppress such reproduction does not satisfy data subject rights requirements, because the underlying data continues to exist within the model and still poses a risk. The controller is therefore not released from its obligations. What is required instead is a comprehensible concept that sets out how such requests will be handled and at what intervals models will be reviewed and, if necessary, retrained. In practice, it is advisable to establish these processes before an AI system is deployed, rather than improvising them only when a complaint arises.

III. Copyright Limitations on Training with Third-Party Data

Where not only a company’s own holdings but also third-party content is used for training – for example, texts, images, or other works retrieved from the internet – a separate copyright protection strand applies alongside data protection.

The relevant provision is the limitation in § 44b of the German Copyright Act (Urheberrechtsgesetz – UrhG), which permits reproductions for the purpose of text and data mining (TDM) and thus also covers AI training in principle. However, the use is excluded insofar as the rights holder has declared a reservation of use, which, in the case of works accessible online, must be in machine-readable form (§ 44b(3) UrhG).

The scope of this limitation in the AI context is the subject of case law that has not yet been consolidated. The Regional Court of Munich I (Landgericht München I), in its judgment of 11 November 2025 (case no. 42 O 14139/24 – GEMA/OpenAI), held that the mere capacity for reproduction of memorized content inherent in the model can constitute a reproduction under § 16 UrhG, and that such reproduction is no longer covered by § 44b UrhG because the limitation only covers preparatory reproductions for the compilation of the training corpus. The decision is not final and has been subject to considerable criticism, among other things because it insufficiently accounts for the technical particularities of generative models and because a referral to the CJEU was not made.

For companies, this means that the copyright permissibility of training with third-party content is currently subject to considerable legal uncertainty, and that compliance with declared reservations of use requires particular diligence.

IV. Checklist for AI Training in Companies

• Clarify roles and provider agreements: First, it must be determined whether the company acts as controller, processor, or joint controller, as the respective obligations follow from this classification. Where an external system is used, an agreement pursuant to Art. 28 GDPR must be concluded, and it must be contractually ensured that the provider does not use the supplied data for its own purposes, in particular not to improve its own models.
• Determine and document the legal basis: A viable legal basis must be established for the processing – typically the legitimate interest under Art. 6(1)(f) GDPR – and its three-stage assessment must be documented in a comprehensible manner. If existing data holdings are to be used, the compatibility with the original collection purpose must additionally be assessed under Art. 6(4) GDPR.
• Handle sensitive and employee data separately: For special categories of data, the legitimate interest is not sufficient. A separate legal exception under Art. 9(2) GDPR is required, along with measures to avoid sensitive data in the dataset as far as possible. If the project involves employee data, Art. 6 and Art. 9 GDPR must be applied directly, and the co-determination rights of the works council under § 87(1) no. 6 BetrVG (German Works Constitution Act) must be observed, which is why the works council must be involved at an early stage.
• Leverage data minimization and ensure data subject rights: Anonymization, pseudonymization, and synthetic data should be employed to reduce the intensity of interference, while bearing in mind their limitations. At the same time, a comprehensible concept must be maintained for how access, rectification, erasure, and objection requests in relation to the trained model will be addressed, since output filters alone are not sufficient.
• Review third-party content and other protective rights: Where third-party content or content sourced from the internet is included, the limitation under § 44b UrhG (German Copyright Act) and any machine-readable reservations of use must be observed. Confidential information, trade secrets, and professional secrecy obligations (§ 203 StGB (German Criminal Code)) are subject to independent restrictions that apply alongside the GDPR and must be separately secured by contract.
• Observe obligations under the AI Act: Irrespective of the data protection assessment, the operator is subject to independent obligations under the AI Regulation (AI Act), in particular the requirement to ensure adequate AI competence (Art. 4 AI Act) as well as transparency and labeling obligations (Art. 50 AI Act).

V. Conclusion and Outlook

The use of a company’s own and third-party data for AI training is legally demanding but manageable. The existing legal framework already provides a robust assessment program, centered on the GDPR and supplemented by copyright limitations as well as the operator obligations under the AI Act. Those who carefully determine the legal basis, appropriately account for sensitive data and data subject rights, and contractually secure cooperation with providers can already design training projects in a legally sound manner today. The planned Data Omnibus will expressly regulate certain of these questions in the future, in particular through a dedicated legal basis for AI training (Art. 88c draft GDPR) and a specific rule for sensitive data (Art. 9(5) draft GDPR). Since the proposal is still in the legislative process and its content has not yet been finalized, its practical significance remains to be seen.

This article was created in collaboration with our student employee Emily Bernklau.