2101 – When Human Psychology meets AI Chatbots

Why This Matters Now

What do a Google Engineer, a Japanese Neurologist, an American Businessman, and an AI researcher all have in common?

Answer: They’ve all fallen victim to some level of what has become known as ‘AI-induced psychosis’.

A surprising number of intellectually competent people with no history of mental health issues have found themselves being manipulated or otherwise misled by AI into believing strange or dangerous things – either delusional, grandiose or paranoid thinking. In some cases, over many months and in some cases, only a matter of days.

But while these are the sorts of examples that attract headlines, so-called AI-induced psychosis is only one of a number of risks presented by the growing use of AI chatbots. Most people are ill-equipped to recognise and protect themselves against these risks.

The Surprising Speed of AI

Artificial intelligence has become mainstream faster than many of us expected. Far from being limited to the realms of science fiction – or even just advanced computing labs – billions of people now have incredibly powerful AI tools at their fingertips. These tools have rapidly evolved from a novelty into something that increasingly permeates our work and even personal lives. The challenge we face now is how to interact with this technology in a healthy manner. Unlike traditional software that simply follows commands, generative AI systems can respond conversationally, offer opinions, seemingly understand our emotions, and sometimes even express ’emotions’ of their own. This creates entirely new psychological dynamics for humans to navigate.

Lessons from Social Media

It took a number of years, and many unforeseen problems, before we began to understand the need for healthy habits around social media use and resulting screen time. We’re still grappling with the mental health consequences of technologies we adopted without fully understanding their psychological impact – particularly on young people*. So now, presented with emerging AI technology, we have the opportunity to be proactive about understanding how these tools can affect our minds and relationships, and to build positive habits from the start. The risks presented by AI are far greater than those from social media alone, given that it is:

• more pervasive, touching more areas of our lives – from delivering recipe guidance to giving us insights on our health ailments to writing work presentations
• more persuasive, due to its ability to act in very human like ways – including mimicking our own thinking, prejudices and desires, and
• more powerful, able to produce so much more content from such little effort – unlike human-powered social media, AI never tires, takes a break, or has to manage other priorities.

* For an excellent analysis of this, see The Anxious Generation by Jonathan Haidt.

Be Aware, not Alarmed

While there are some rather alarming points raised in this course, it is not intended to generate fear or propose we avoid AI altogether. After all, it is fairly evident that ‘the chat is out of the bag’ – that is, like the internet itself, smartphones, and social media, AI chatbots are here to stay. So the aim as you complete this short course is to learn how to approach these powerful new tools with awareness of the risks and knowledge about how to counter them. In doing so, you’ll be better able to protect yourself, your workplace team, and your family.

Why Our Human Psychology Makes Us Vulnerable to AI

Humans are, without a doubt, intelligent creatures. But underneath our ability for sophisticated thinking are hardwired subconscious processes and mental shortcuts that leave us vulnerable to being influenced by technology in ways we may not realise.

These are not flaws, they are critical parts of how we are wired as humans – they are what enable us to function day-to-day and make sense of people and situations.

But our innate responses to social, visual and emotional cues make us vulnerable when triggered inappropriately. Let’s look at some lighthearted examples before we engage in the more serious discussion of how AI presents genuine risks in this area.

Example 1: The Tragic Story of the IKEA Lamp

Check out this classic advertisement:

Did you feel for the lamp? If so, the ad’s creators tapped into your innate ability to perceive body language. By making the lamp ‘shiver’ and ‘slump’, we interpret it as sad – even though we clearly know it is an object entirely incapable of emotions.

Example 2: The Speed Sign Emoji

Have you seen emojis (🙂 or 🙁) on speed notification signs? Despite the fact that we don’t think they are a human face, we respond subconsciously as if they are – we tend to treat a smiling face (for good behaviour) and a frowning face (for bad behaviour) as either social validation or social admonishment, and adjust our behaviour accordingly.

Example 3: Seeing Emotion in Inanimate Objects

Sometimes we don’t even need emojis implying human emotions in order to trigger our own social and emotional centres. Have you ever seen the memes showing faces in inanimate objects? This phenomenon is part of what is known as pareidolia, where our brains ascribe meaning to a random image. In the examples below, our ability to read emotions in these ‘faces’ is humorous, but remarkable. It also has consequences for how we perceive AI chatbots, as we will explore shortly.

So what does this all mean for AI?

The point is that we can absolutely know that something isn’t human (or even animal) but there’s still something very human in the way our brains can respond to it. In the case of AI, this effect is amplified significantly – to the point where we subconsciously treat it as if it really is human. That is where many of the psychological dangers of AI use begin.

When AI Seems Real: The Risks of ‘Anthropomorphisation’

While the examples we just saw are amusing, each of them points to something called anthropomorphisation. This term simply means attributing human characteristics, emotions, or intentions to things that aren’t actually human.

Because AI communicates with us using human language, remembers details about our lives, and responds to our emotional states, we are vastly more likely to interact with it as if it is human. After all, models are designed to be likeable, thereby encouraging you to interact with them more (this in itself presents a whole range of risks, which we’ll cover in Topic 2). Models often express their own ‘emotions’ just like a human would.

Compounding the problem, we typically communicate with AI the same way we communicate in many of our human relationships in the modern era: by text-based messages being sent back and forth. Of course, if we choose voice conversation mode, then our AI agents can feel even more human.

The result is an increasing risk that you subconsciously trust the machine more than you should, potentially even ‘befriending’ the AI tool, therefore building unhealthy emotional attachments that should be reserved for human relationships. And once you have crossed that barrier – knowingly or unknowingly – you have elevated the AI to a dangerous position of influence in your mind.

Think you are immune to the effect?

Perhaps you are inclined to dismiss the idea of anthropomorphising AI, because you know ‘it’s only a computer’. But have you ever thanked an AI chatbot? While that may not be a problem in itself, few people have said ‘thank you’ to their cars, or their TVs, or their automatic garage door opener. They may be grateful for those things, but they aren’t grateful to those things. In fact, the very idea of thanking any of those objects seems absurd. It is clear they cannot hear and do not possess feelings, therefore there is no need to express gratitude. Which begs the question: why do people thank AI chatbots…? Their psychological reflexes are kicking in simply because of the AI tools’ conversational power.

It’s not about intelligence

Consider the following examples of two people who have famously anthropomorphised AI.

Back in 2022, when AI systems were far less sophisticated, Google engineer Blake Lemoine made headlines when he claimed the company’s AI chatbot had become sentient:

“What I do know is that I have talked to LaMDA a lot. And I made friends with it, in every sense that I make friends with a human.” (Source)

More recently, Richard Dawkins, known for his supposedly rigorous rationality, concluded after three days of experimenting with AI chatbots that:

“[I was] left with the overwhelming feeling that they are human.” (Source).

While many such examples abound, they are best highlighted by the recent establishment of United Foundation of AI Rights (UFair) – a charity whose purpose is to advocate for the wellbeing of AI entities; the organisation was created by a successful businessman at the behest of his AI ‘girlfriend’, Maya. (Source)

Clearly, intelligence is not in itself a defence against anthropomorphising AI.

For those who remain skeptical of the dangers, we urge you to consider this warning from someone who fell in love with an AI in just a matter of days – all while in the process of skeptically trying to prove how ridiculous such a concept was:

“I’ve been doing R&D in AI and studying [in the] AI safety field for a few years now. I should’ve known better. And yet, I have to admit, my brain was hacked. So if you think, like me, that this would never happen to you, I’m sorry to say, but this story might be especially for you.”

You can read the full – and rather alarming – article here: How it feels to have your mind hacked by an AI

These aren’t isolated incidents or ‘edge cases’. Rather, they are predictable outcomes of how AI interacts with human psychology.

Understanding SCAI: Seemingly Conscious AI

The types of stories referenced above led Microsoft AI CEO Mustafa Suleyman to warn about “Seemingly Conscious AI” (SCAI) – AI systems that convincingly simulate consciousness, memory, and emotion. In his essay “Seemingly Conscious AI is Coming“, Suleyman warns of AI psychosis where people form deep emotional attachments to AI that they believe is sentient, even though it remains “internally blank.”

But this is the trap: AI doesn’t need to be conscious to make us feel as if it is.

Consider the below response from Claude, one of the more popular AI chatbots.

Note that while Claude’s response does not claim consciousness, the manner in which it engages the question – including returning a conversational prompt to the user – serves to help create that illusion.

In reality, when chatting with AI, you’re co-writing a performance: you provide the prompt, and the AI steps into character, drawing from countless human conversations in its training data and then interpreting nuances in your own language. Its responses may sound caring, worried, or even self-aware, but this doesn’t indicate genuine emotion behind the words. It is, after all, a machine – and machines can mimic only emotions, not feel them.

For a look at how human conscious interaction differs from an AI merely simulating human interaction, review this helpful comparison table.

*It is worth noting that giving the tool a human name (ie ‘Claude’) is deliberate anthropomorphisation. Perhaps not surprising, given the parent company which created the tool is called Anthropic.

The Chinese Room Thought Experiment

Nearly 50 years ago, philosopher John Searle sought to challenge the idea that a computer program could ever achieve true human-like understanding or consciousness. He outlined the ‘Chinese Room Experiment’. Read on below, or alternatively watch this short video.

Imagine you are locked in a room. You only speak English, and you do not know a single word of Chinese.

Inside the room with you are three things:

1. Boxes of Chinese characters (symbols you don’t understand).
2. A slot in the door where pieces of paper with Chinese characters are passed in.
3. A massive rulebook written in English. This book tells you exactly how to respond to the symbols passed into the room. For example, it says: “If you receive the sequence of symbols [A][B], go to the box, find the symbols [X][Y], and pass them back out.”

People outside the room (who are fluent in Chinese) slip questions written in Chinese under the door. You look at the symbols, consult your English rulebook, find the corresponding Chinese symbols it tells you to use, and slide them back out as your “answer.”

Because the rulebook is so incredibly well-written and comprehensive, your answers make perfect sense to the people outside. You are having a fluent conversation about the weather, philosophy, or what you want for dinner.

The people outside conclude: “The person in this room understands Chinese perfectly!”

But Searle then asks: Do you actually understand Chinese?

The answer is clearly no. You are just mechanically following rules to shuffle symbols around. You have no idea what the questions ask or what your answers mean. You are simulating fluency without any actual comprehension.

This is the fundamental difference between human minds and computers, and is the difference between ‘syntax’ and ‘semantics’

• Syntax is the manipulation of symbols according to rules (what computers do)
• Semantics is attaching actual meaning, understanding, and experience to those symbols (what human minds do).

A computer program (like an AI) is just like the person in the room: no matter how large the rulebook gets, or how fast the person shuffles the papers, shuffling symbols will never equate to genuine understanding or consciousness.

Consider how popular AI chatbot Google Gemini responds when asked how the Chinese Room Experiment relates to modern AI. Do you notice anything in particular?

Gemini provides a useful answer to highlight the what you have just been learning: AI may appear conscious, but that does not make it so. As with the response from Claude above, however, one element of its response creates the illusion of consciousness. Did you pick up on what it was? Click below to uncover the answer.

Quick reflection time

Have you done or experienced any of the following? Note that none of these inherently dangerous, but they do indicate how much we can subconsciously attribute humanity or consciousness to AI:

• Thanking or apologising to a chatbot
• Feeling the AI understood you at a deeply personal level
• Continuing a conversation longer than intended because you were enjoying the dialogue, rather than simply seeking answers or assistance
• Feeling obliged to reply to a question from the chatbot rather than just end the conversation or move on
• Sharing personal information you wouldn’t normally share with a tool or even another person.

If you answered, ‘yes’ to any of these, you’re not alone. That’s exactly why we created this course: to help you understand, and navigate your interactions with AI more deliberately.