Emotionally intelligent AI in the enterprise: uses, risks and governance for Project Managers (2026)

For years, in the AI projects I have led in banking, insurance and public administration, artificial intelligence was treated as a cold tool. It was logical, data-based and, moreover, detached from human experience. That view, however, is changing fast. Today many users feel that AI understands not only what they say, but how they say it. That is, it adjusts its tone, adds nuance and appears to “understand” emotional states. Hence the question that increasingly reaches me in committees: are we looking at an emotionally intelligent AI, and what does that mean for a regulated company?

My short answer is no, AI does not feel. The long answer, however, is more interesting. The perception of empathy is already changing how people interact with these systems. This therefore has concrete consequences in customer service, in human resources and in the governance of any project under the EU AI Act.

From automation to emotional connection

AI’s leap in recent years has not been only technical. It has also been communicative. While traditional systems were designed to execute specific tasks, today’s models interact in natural language with unprecedented sophistication. On one hand, they interpret the context of a conversation and detect implicit emotional signals. On the other, they adapt their style and generate more relatable responses. The result is clear: the interaction is no longer perceived as an exchange with a machine, but as a fluid and often satisfying conversation.

What emotionally intelligent AI really is

In humans, emotional intelligence has four legs: recognising one’s own emotions, understanding those of others, regulating emotional responses and acting with empathy. AI, however, possesses none of that. It lacks consciousness, emotions and subjective experience. So why do we talk about emotionally intelligent AI? Because what the model has learned is not to feel. Rather, it has learned to recognise emotional patterns and to respond in a way that is consistent with them.

The underlying mechanism is the analysis of large volumes of linguistic data. With them, the models identify words associated with emotional states, such as stress, frustration or joy. They also recognise structures that reflect intention or tone. In addition, they detect contexts in which certain responses are more appropriate. For example, faced with a message like “I’m exhausted, I can’t make progress with my project”, the system responds in an understanding tone. That does not imply feeling. Nonetheless, it generates an experience that, for the user, is equivalent to empathy.

A perception, not a property

This is the point I most insist on with teams. What matters is not whether the machine ‘feels’, but how the user interprets the interaction. When the person receives a response that validates their situation, adapts to their state and offers solutions with respect, a sense of being understood is generated. And that sense is enough to perceive the system as emotionally intelligent. In short, we are talking about an external perception, not an internal property. Understanding it this way avoids both excessive faith and automatic rejection.

Personalisation, for its part, is the real engine of that closeness. AI does not respond only on the basis of the current message. On the contrary, it adjusts its behaviour according to the user’s style, their goals, their level of knowledge and the context. Thus, communication stops being generic and becomes contextual, which reinforces the sense of proximity. In a way, it acts as an “intelligent mirror” that adapts to whoever is in front of it.

The impact of emotionally intelligent AI on the company

Customer service and human resources

In customer service, AI systems no longer just resolve incidents. They now detect frustration, defuse tension with empathetic responses and, in addition, hold more natural conversations. In human resources, for its part, AI is beginning to play a role in workplace climate, onboarding and employee assistance. In these cases, communicating with sensitivity increases team acceptance.

A new dimension for the Project Manager

For those of us who lead projects, this introduces a new dimension I call relational intelligence. It is no longer enough to build efficient systems; you also have to design experiences that connect with people. In practice, that means defining the tone from the design stage, weighing the emotional impact of each response and integrating ethics and transparency as part of the project, not as a final patch. It is, in short, the same “governance from day one” approach I apply with NIS2, DORA and the EU AI Act.

Risks of emotionally intelligent AI

The first risk is emotional dependence. If a system always responds with empathy, some users develop an excessive bond. This is especially delicate in sensitive contexts, such as psychological support, loneliness or personal decisions. The second risk is the illusion of understanding, since confusing simulation with real comprehension creates unrealistic expectations.

The third, without doubt the one I watch most in regulated projects, is emotional manipulation. Indeed, the ability to interpret emotions can be used to influence marketing or sales decisions. That is why clear limits must be set and ethical use guaranteed. In practice, it is wise to treat automated emotional persuasion with the same rigour as any other AI risk, as in a secure enterprise RAG project, within the European framework.

Ethics and transparency: the pillars of trust

An AI with advanced communicative capabilities requires, above all, a responsible approach. For me, there are four keys. The first is transparency: the user must always know they are interacting with an AI. Next come clear limitations, so that the system does not present itself as an entity with real emotions. Human control also counts, with supervision in critical contexts. Finally, ethical design rules out any manipulative practice. Trust will, consequently, be the decisive factor in adoption. And it is built with these four principles, not with marketing.

Do we humanise technology or technologise the human?

The phenomenon invites a deeper reflection. Not only does technology become more human; we too redefine our relationship with the human through it. While AI amplifies certain aspects of communication, it simplifies others. In that balance, two honest questions arise. On one hand, to what extent do we want technology to ‘look’ human? On the other, where is the line between usefulness and emotional substitution?

Conclusion: emotionally intelligent AI and trust

AI has not developed human emotions. Even so, it has reached a sophistication that lets it interact in an emotionally intelligent way. The turning point, therefore, is not in what AI can do, but in how it does it and how it makes us feel. Fundamentally, the real revolution is not that machines feel, but that we feel understood when we interact with them. For a company, that seemingly subtle nuance is where trust, governance and the responsibility of whoever leads the project are at stake.

Frequently asked questions about emotionally intelligent AI