AI Regulation: ECRs’ Perspectives is a CREATe blog series featuring the work of early career researchers who are exploring the contemporary challenges of AI regulation. Drawing primarily on research presented at the AI Regulation ECR Conference, while also including related work on AI regulation, each post provides concise and accessible insights into emerging legal and policy debates around artificial intelligence.
The series continues with a blog post by Anna Katharina Suzuki-Klasen, LL.B. (Norwich, UK), M.L.B. (Hamburg/Vallendar, DE). Anna holds a PhD in law from the University of Hamburg and is an independent researcher. Her focus lies in contract and consumer law, with a particular interest in issues stemming from digital changes and comparative studies. She has considered the impact of smart contracts and, most recently, of AI on consumer contract law.
Can we CREATe a Pro-Consumer AI Regulation?
Most of us are connected to the internet 24/7 and surrounded by the Internet of Things (IoT). It is no longer a story of science-fiction to be woken by a smart alarm with music or podcasts, while a smart-machine brews our coffee (see Namase, 2025). Indeed, in the EU, some 9 out of 10 individuals use mobile devices to access the internet, while around 70% of 16 to 74-year-olds are using IoT devices (see Eurostat data here and here). As this data is from 2025 and 2024 and we are now in Q2 of 2026, these numbers will have increased.
Many of these technologies will be powered by Artificial Intelligence (AI), as the technology is deployed in a vast range of sectors. This includes food, transport, healthcare, media, education, and public services. This plethora of applications means that, as consumers, we are surrounded by AI. The question is: Does this improve our lives or are we being exposed to critical risks?
We may not always be aware that AI is being used, as with parking assistants (automatic parking), banks’ fraud detection systems, AI-powered medical diagnosis systems, personalised marketing, or electronic games and toys. Its employment is more obvious with voice assistants, chatbots, automated hotlines, recommendation systems, and autonomous vehicles.
In this blog post, I will explore the benefits and drawbacks of some AI applications and consider if EU law is giving us consumers sufficient protection. Spoiler alert: There’s room for improvement.
The Benefits of AI Applications
On the one – bright – side of AI, there are a range of benefits for us consumers. Generally, algorithms can help us be more efficient by reducing the time needed to complete tasks, like searching for products or information, translating texts, and calculating calories (see, among others, Durovic, 2024). Similarly, recommendations of products or services can be convenient and increase engagement with content (see Portinale and Abluton, 2024), and personalised advertisements may even act as inspiration, or as reminders of products that we wish to purchase. AI can also bring us an ‘added consumer value’ due to lower transaction costs and lower prices and a higher quality in goods and services (see Durovic, 2024).
Some applications also have specific benefits. For example, autonomous vehicles can potentially be safer and bring us to our destination faster than traditional vehicles (see Howells, 2024 and Durovic and Watson, 2022). We can save energy and money through AI-driven smart home applications like boilers that allow the heating to be set to run just before we return home. Emotional AI, a type of AI that uses affective computing to sense, learn, and respond to the user’s emotions, has a wide range of potential advantages (see Durovic and Watson, 2022): Personalised healthcare using AI technology, allowing for faster discovery of the causes of, and medicines and treatments for, health conditions (see Ho and others, 2022), and increased wellbeing through responses to the consumer’s emotion, like content recommendations based on inferences of the user’s emotions, are but two examples.
The Risks of AI Consumer Applications
On the other – dark- side of AI, we find several risks. Often raised issues are data protection and privacy concerns, which are inherent flaws, as all AI models require vast amounts of data to be trained and function. We consumers may not be aware of the fact or the extent to which our personal data is being collected and used. For example, digital assistants like Alexa or Siri collect large amounts of data through chat-interactions and might be seen to surveil users through their constantly activated microphones (see Durovic, 2024 and Durovic and Watson, 2022). This breach of privacy can affect all persons who interact with the assistant: the ‘owner’ (user) of the device and other members of the household – namely other adults and children – and even guests. Similar dangers exist for other internet-connected items such as toys (see Myrstad, 2016).
Equally well-known problems concern bias and discrimination. AI may reproduce biases acquired from training data (sampling bias, see Durovic, 2024), so that an image generation application prompted to create an image of a lawyer may produce pictures of white men (see Prince, 2023). Discrimination against consumers based on characteristics like their sex, ethnicity, disability, or age, and unfavourable treatment may follow (see Durovic and Watson, 2022). This can happen in situations like creditworthiness assessments, personalised marketing, and job applications, or where emotional AI misinterprets emotions due to, e.g., cultural differences between the user and the training data (see Durovic and Watson, 2022). Discrimination can in turn lead to consumers having negative emotions, feeling isolated or misunderstood (see Durovic, 2024).
There have been warnings of the business-consumer information asymmetry and the technology and bargaining power gap widening further through the employment of AI. This relates to the consumer’s lower level of knowledge or even ignorance of the workings of AI. Some AI features contribute to this gap, like random setting, semi-autonomous behaviour, and what is referred to as the black box effect, i.e., that AI algorithms do not provide an explanation for their decision (see Durovic, 2024). All of this puts consumers at a disadvantage and may lead to consumer vulnerabilities being exploited (see Ebers, 2020). One example is where AI is used in automatic contract term adjustments (see Scattarreggia, 2025).
AI may go a step further and manipulate consumer behaviour, which is strongly related to dark patterns and recommender systems that personalise advertisements, streaming content, prices, or contracts, and induce us consumers to act beyond our original intention. Similarly, emotional AI may interfere with our decisions by nudging us in a particular direction (see Durovic and Watson, 2022). The digital manipulation need not be coercive, like where the framing effect is exploited; however a recurring distortion of consumer decisions impacts our autonomy and freedom of choice (see Durovic, 2024), and brings the danger of the market devolving into a manipulation competition (see Scattarreggia, 2025).
Recently, instances of blatant technology misuse have become public, where AI was used to create and distribute discriminatory or illegal content, including sexualised images (see Gizem Yaşar, 2026). AI might also lead to physical danger, e.g., if an autonomous vehicle or a medical robot malfunction or make a ‘wrong’ decision/movement.
The Legal Framework on AI in the EU
After this grim assessment of the risks to AI, the protection framework for consumers found in EU law may provide some support. The core regulation dealing with AI is the AI Act (Regulation (EU) 2024/1689). It establishes a framework for ‘AI Systems’, defined in Art 3 point 1 AI Act as:
‘a machine-based system that is designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment, and that […] infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments’.
The aims of fostering human-centric and trustworthy AI (Rec 1 and Art 1(1) AI Act) and of supporting investment and innovation in AI make no explicit mention of consumers (see EU AI regulatory framework policy). So, while we consumers are not targeted, the regulation nevertheless has an impact on us. This makes it worth taking a closer look at these rules.
Three measures are foreseen by the Act. The first prohibits a range of AI practices that could negatively affect persons and are considered unacceptable. This includes certain uses of facial and emotional recognition systems (Art 5(1)(e) and (f) AI Act), distorting consumer behaviour through exploitation of vulnerabilities or ‘subliminal techniques’ or ‘purposefully manipulative or deceptive techniques’ (ibid (b) and (a)), and unfavourable treatment of persons based on behavioural classification (ibid (c)).
The second measure follows a risk-based approach and categorises AI systems as either high-risk (see Art 6 and Annex I and III AI Act) or non-high-risk. High-risk AI systems can only be put to use in the EU after fulfilling several technical requirements (Arts 1(2)(c), 8, 16). This includes applying data governance and management practices for quality control of training data, like checking for possible biases leading to discrimination (Art 10), ensuring ‘[a]n appropriate type and degree of transparency’ and providing instructions for use to deployers (Art 13), and allowing for human oversight by design to reduce the risk of physical harm or breach of fundamental rights (Art 14).
The third lays down transparency rules for ‘certain AI systems’ (Art 50 AI Act): Persons who interact directly with an AI system must be informed of the AI interaction (para 1), and media created by general-purpose AI must be clearly labelled as AI-generated or -manipulated content, whether manipulative (deep fake) or not (paras 2 and 4). Similarly, the use of biometric or emotional recognition AI systems has to be clearly notified to the persons affected (para 3). Limited exceptions exist, like for AI used to combat crime.
Taken together, these measures go in the right direction: The technical requirements should minimise several of the discussed risks, like the black-box effect, discrimination, or physical harm, while the prohibited practices tackle behaviour manipulation, discrimination, bias, and privacy issues. Moreover, the transparency obligations will help soften the information imbalance. The question is whether this will be enough: Can we as consumers use AI in the confidence that we are being protected? Some authors (like Howells, 2024) have noted that the limitation to ‘high-risk’ AI is not going far enough, as medium- or low-risk AI systems can equally pose risks to consumers. And there is the question of enforcement; the rules alone will not change business behaviour where it is profitable.
Harnessing the Consumer Protection Rules
Luckily, there are other rules that will protect us consumers, irrespective of the AI system’s risk level.
For instance, geographical discrimination is forbidden by the Geo-blocking Regulation (Regulation (EU) 2018/302). Furthermore, the Digital Services Act (Regulation (EU) 2022/2065) and the Unfair Commercial Practices Directive (Directive 2005/29/EC) can offer protection from manipulative strategies like dark patterns (see Scattarreggia, 2025, Kaprou, 2023, Ebers, 2020 and Isola and Esposito, 2025). This means that misleading or aggressive dark patterns are prohibited and must not be employed by businesses. (Caveat: this needs to be enforced).
When it comes to contracting, there are information requirements that should ensure a scale-tipping in the information asymmetry: An amendment to the Consumer Rights Directive (Directive 2011/83/EU) foresees that the trader must inform the consumer ‘that the price was personalised on the basis of automated decision-making’ (Art 6(1)(ea)). This will make the practise of price personalisation transparent; however, there is no general obligation to inform consumers of the employment of AI algorithms at the moment (on the difficulty of applying the requirements to AI, see Ebers, 2020 and Durovic, 2024). This may change under the upcoming Digital Fairness Act Proposal: This initiative aims to strengthen information requirements and tackle several problems consumers face in the digital sphere, including dark patterns, misleading marketing, and unfair personalisation practices.
Getting to a Pro-consumer Regulation of AI?
Where does this leave us consumers? While we are not being left to our own devices, pertinent measures on AI are scattered all over EU (consumer) law and may not be enough. The Commission aims for a balanced approach to regulating AI by protecting individuals from harm but not stifling innovation and development. It falls short on the former account. While there is a proposal underway to simplify the AI rules (the Digital Omnibus on AI Proposal), the intended amendments are technical, geared towards benefiting AI providers and employers. Again, consumers are not a priority. And likely will not be explicitly included in a specific AI regulation anytime soon. How much the Digital Fairness Act Proposal changes the situation remains open.
This means that we need to work with what we have and learn to be more aware of AI and its business applications. Because, in the end, the law cannot protect us from all the risks. Here, AI literacy schemes on a national level are desirable; however, it remains to be seen whether such schemes appear, as the Digital Omnibus on AI Proposal only foresees that Member States and the Commission ‘shall encourage providers and deployers of AI systems to take measures to ensure a sufficient level of AI literacy of their staff’ (Art 1 AI Omnibus Proposal).
On a bright note: AI literacy can be self-acquired. Even reading a blog post like this will help.
As AI Regulation: ECRs’ Perspectives draws to a close, the series will conclude with the final blog post from the AI Regulation ECR Conference Organising Committee (Qingqin Zhang, Weiwei Yi , Aline Iramina and Gabriele Cifrodelli). Throughout the series, contributors have highlighted the diversity and complexity of contemporary AI regulatory debates. Rather than attempting to summarise those discussions, we will offer our own perspectives on some of the challenges we believe will define the future of AI regulation. Drawing on our respective areas of research, we briefly reflect on workers’ privacy and labour rights, consumer protection, platform regulation, and intellectual property, identifying emerging issues that we believe policymakers, regulators and scholars will need to confront in the years ahead.