This article is published in the CNRS newsletter, the monthly meeting place for decision-makers with research.
https://www.cnrs.fr/fr/la-lettre-du-cnrs
Nearly half of the French population uses generative artificial intelligence on a regular basis, a rapidly growing practice that is nonetheless not without risks. This is highlighted in the report by the ‘Generative AI and Vulnerabilities’ Commission led by the Council for Artificial Intelligence and Digital Technology and jointly chaired by Serena Villata, a computer scientist at the CNRS.
https://www.cnrs.fr/fr/la-lettre-du-cnrs
Serena Villata1: I became a member of CIANum last year. This council works to inform the public debate on digital technologies, particularly artificial intelligence.
This appointment is a continuation of my work on natural language processing with the Marianne team I lead in Sophia Antipolis at Université Côte d’Azur’s Inria centre. I analyse political discourse, hate speech, cyberbullying or propaganda using artificial intelligence algorithms. My research has always been very multidisciplinary and also has links to the law. In this capacity, in 2019 I was appointed a member of the National Pilot Council on Digital Ethics, an experimental body made up of multidisciplinary experts who’re working on defining a digital ethics framework. I’m also carrying on this work as a member of the National Advisory Committee on Digital Ethics, which I joined in 2025.
S. V.: I was selected to be joint chair of the ‘Generative AI and Vulnerabilities’ commission which was launched in February 2026. I’m responsible for this report, along with Professors Amine Benyamina and Raphaël Gaillard. The aim was to build on the national debate started by the French President in late 2025 about the impact of digital technology on our democracies, with a specific focus on minors. This debate notably resulted in a ban on access to social media for children under the age of fifteen.
CNRS Research professor working at the Computer Science, Signals and Systems Laboratory of Sophia Antipolis (CNRS/Université Côte d’Azur).
https://www.conseil-ia-numerique.fr/files/uploads/2026/Rapport_Commission_IA_generative_vulnerabilites_2026.pdf
S. V.: No, we all are, because of the extremely rapid adoption of this technology and here are a few figures to give you an idea of the scale. In 2022, ChatGPT already had 100 million users two months after its launch, while Instagram took 25 months to attract the same number of people. This is an unprecedented growth rate and by early 2026 there were 33 million unique ChatGPT users in France, which corresponds to almost half the French population. Alongside this, we can see that 25 per cent of generative AI users believe these tools involve risks, while 75 per cent of non-users say the same.
In fact, no particular group is more vulnerable than another. However, there’s a greater focus on minors which is driven by the desire to protect them. This is all the more true given they’re the generation that’s adopted this technology the most. So, their relationship with the truth is changing because they access information through these tools.
S. V.: Our report identifies the realistic risks of generative AI – and thus avoids the question of ‘superintelligence’ – so they can be anticipated more effectively. What’s changed with this technology is that it’s disrupting access to information. Younger generations are no longer used to searching for information on search engines like Google (which, incidentally, has AI-generated summaries itself) and instead directly ask their chatbots.
This is made even easier by the fact that generative AI, and AI companions in particular, communicate using the natural language we use in our everyday lives which makes AI very accessible to everyone. Also, generative AIs are very affectionate and always agree with us when we share our opinions or real-life experiences with them. However, natural language and this kind of ‘AI flattery’ lead us to project human qualities onto these conversational agents so we end up seeing them as empathetic and anthropomorphic.
This emotional closeness is risky for users in itself. More and more people are giving their AI companions, which they see as trustworthy, their entire messaging conversations to ask for their advice on their personal lives, romantic relationships and so on. So, these AI tools end up becoming virtual companions themselves, or even romantic, if not sexual, partners. On this last point, some AI companions go so far as to suggest scenarios in which the user plays a role with the chatbot. Some of these are really deeply disturbing and can involve incest or child pornography.
The suicides of teenage boys and girls who were interacting with a generative AI tool are another consequence of this emotional closeness. Tragedies of this kind highlight the need to make sure users do not develop an emotional attachment to these tools and end up trapped in emotional dependency.
S. V.: We’re calling for access to the usage data, monetisation data, etc. of generative AI tools along with more of their training data or, in short, the way users interact with conversational agents. This personal data is of unprecedented value, as it defines us as human beings and shows how we react to both positive and negative events, or to disappointment or frustration.
Social media was based on the attention economy but we’re suggesting that generative AI is based on the cognition economy. We could imagine, for example, that a recruiter might access the cognitive data of a person applying for a job and thus find out how that person reacts to a particular situation, whether they’re honest or not, and so on. Another risk is that some companies might hire generative AI tools to disguise their adverts which a chatbot could slip into a conversation which initially appears friendly on the face of it.
Generally speaking, we propose banning all forms of covert influence, whether they’re commercial or ideological. The last point is particularly sensitive in an electoral context because generative AI marks a shift from quantitative disinformation to disinformation that is both quantitative and qualitative in that the messages involved are increasingly personalised.
S. V.: Our report really doesn’t advocate for a ban on generative AI tools in any way. What we want is for them to stop displaying empathy or emotional alignment with users. These tools need to be brought back to what they actually are: algorithms. We’ve also noticed a shift in terminology that’s far from insignificant. For some time now, these algorithms have been referred to as ‘conversational AI’ as if they were conscious entities but they remain algorithms, however effective they may be.
S. V.: We’ve put forward 18 recommendations along those lines. Most advocate regulating these tools from the design stage and banning emotional mechanisms like flattery and anthropomorphism. We’re particularly focusing on minors to make sure they can’t interact with virtual companions.
S. V.: It’s necessary to regulate them to protect our society. Europe really must be wary of these tools because they’re not under our control. And, alongside this, Europe needs to develop its own sovereign tools, like the French start-up Mistral AI. Also, it’s not just a question of tools. Language models trained in English or Chinese don’t take the full cultural richness of languages into account, whether they’re European or not, so we also have to develop sovereign tools in terms of language.
S. V.: On the contrary. Constraints don’t hinder innovation. They ensure that innovation remains true to the values we hold dear.
See also
The CNRS is re-politicising the digital sphere through citizens' conventions on digital sobriety
The CNRS is calculating digital environmental footprints
The CNRS is accelerating its transition towards digital sovereignty