auDigital Health Futures: how is Emotion AI reshaping experiences in healthcare?
Dawn Chu
The imagination about Emotion AI (Created with leonardo.ai)
Overture
The movie ‘Her (2013)’ is the story of a lonely, letter ghostwriter in the middle of a divorce, who falls in love deeply with Samantha, an empathic Operating System set in 2025. ‘Her’ represented an uncanny and meaningful way of interacting with Artificial Emotional Intelligence (AEI, Emotion AI). So, how advanced is Emotion AI in reality for 2025? Can AI truly understand nuanced emotional cues and support our well-being like ‘Her’? This article examines how Emotion AI is reshaping experiences in healthcare. I conducted secondary research focused on AI in healthcare case studies through multidisciplinary resources in medicine, medical humanities, and Human-computer interaction.
In this disruptive era of AI, multifaceted and balanced approaches are valued to create meaningful healthcare. Medical humanities strive to revive “human aspects of medicine” by encompassing various arts disciplines (Hooker, 2008). These balanced approaches can be further facilitated by a reflective, systemic collaboration between Humans and AI. However, one of the critical problems of AI in healthcare is that it lacks humanity or misses the clinical rapport between clinician and patient (Barnett et al., 2021; Altschuler et al., 2024). To alleviate this, Emotion AI as a part of Affective computing aims to recognise, analyse, interpret and echo human emotions. Through more understanding of our emotions, these technologies could foster clinical relationships and empathic interactions, particularly in mental health.
“Medicine is the most humane of sciences, the most empiric of arts, and the most scientific of humanities.” (Pellegrino, 1979, p. 17) There are approaches to creating the intersection of AI and Medical humanities. Altschuler et al. (2024) describe the new role of the clinician as editor of AI-generated clinical notes with narrative skills, and propose collaborative approaches to navigate this role shift. They provide an insightful view of how AI and Narrative medicine can improve healthcare practices by refocusing on the patient. Especially, the fact that AI can miss human aspects and clinical rapport can imply the importance of medical humanities, multidisciplinary approach and Emotion AI.
On the other hand, to enhance the depth of contextual understanding in Narrative medicine, AI-generated art images can offer a novel experience through the reflection process, creating and communicating (Huston & Kaminski, 2023). Huston and Kaminski (2023) created images of clinical scenarios that can explain the clinical culture better by incorporating certain styles of art like Van Gogh, Picasso Blue Period or Basquiat. By focusing on the narrative of each individual, including clinician and patient, it is possible to create a more humanistic and personalised clinical experience. This implies that Emotion AI can create novel scenes and diverse perspectives with humanistic approaches to healthcare.
Emotion AI and Empathic Interaction in Healthcare
Emotion AI can contribute to more empathic interactions in healthcare by analysing emotions. To begin with, Picard (1997, preface xi) coined the term Affective computing as a tool “to better understand how we are made, and so enhance our own humanity.” Similarly, Lupton (2017) cited the argument that digital health technologies “truly empower patients (and their families) and help them better understand their treatment and what’s going on with their bodies” (Lupton, 2017, p. 40, as cited in Husain, 2012).
Affective computing technically consists of ‘Emotion data capture’ from biomarkers such as voice, heart rate or eye movement, ‘Emotion analytics’, ‘Emotion response’ and ‘Emotion simulation’ powered by AI (Deloitte Insights, 2024). The Affective computing market size is expected to reach US$140.0 billion by 2025 from US$28.6 billion in 2020 based on Deloitte Insights (2024). MIT Media Lab (2025) presents this technology that aims to enhance people’s lives, such as supporting mental health and improving human-computer interaction and feedback for emotional conditions. Emotion has an important role in balanced and holistic decisions, creative problem-solving, and interactions (Picard, 1997), which will enhance the complex modern healthcare scene to be more humanistic and empathic.
Particularly, Emotion AI can be effective in mental health by interpreting human aspects better and supporting clinical rapport between clinician and patient. Several case studies proved the effectiveness of Emotion AI in mental health issues such as depression and anxiety (Somers, 2019; MIT Media Lab, 2020; Brooks, 2024; Deloitte Insights, 2024). As an example, Hume.ai is actively developing voice-voice AI called Empathic Voice Interface (EVI) and discovering its benefits for mental health, daily well-being, education and communication. He illustrated that empathic voice AI is creating a remarkable epoch in digital mental healthcare history which originated from the first mental health chatbot, Eliza in 1966 such as:
- Great “self-awareness” of mental health conditions.
- Identifying, tracking progress and managing mental health conditions with rich biomarkers such as speed or pitch of voice.
- Recommending clinical treatment or intervention as needed.
- Behavioural encouragement to enhance productivity or mental wellness.
- Voice assistant, journal or dialogue to create connection, and monitor daily well-being.
- Potential digital carer with accessibility for consistent support.
- Sophisticated emotional reactions based on the user’s context or emotional changes.
- “Offering conversational engagement, warmth, support, and empathy.”
(Brooks, 2024)
They consider ethical considerations such as data security, informed consent, and sociocultural diversity of emotions while using AI in healthcare accordingly. Brooks (2024) argues its potential to empathic support marginal groups, people who have difficulty accessing the mental healthcare system. These case studies represent the innovative progress of Emotion AI technologies and their benefits for our healthcare and well-being.
Finale
Given that Emotion AI is creating benefits for mental health and well-being with empathic interactions, it sounds promising to enhance healthcare with a well-designed human-AI collaboration system. However, it is also important to think critically about our responsibility to humanity including the ecosystem around us. Troika (2014) displayed installation art titled ‘The Weather Yesterday’, which ironically displayed the weather yesterday using cutting-edge technology, seeking the meaning of love of technology and obsession with progression. Stephens (2024) questioned the relationship between productivity and happiness by analysing Gilbreths’ photographic motion studies; chronocyclegraphs of the working body. These give us retrospective insights into progression of digital health technologies. Russell and Einstein (1976) published a manifesto when there was disruptive competition in the Cold War, “We appeal as human beings to human beings: Remember your humanity, and forget the rest.” While reflecting on our responsibility, understanding our own emotions better and collaborating with Emotion AI will be significant in creating Humanity-centred AI in Digital Health Futures.
Acknowledgements
I appreciate Elizabeth Stephens (The University of Queensland, Associate Professor of Cultural Studies in the School of Communication and Arts) for your passion, thoughtful guidance, and kindness and my fellow summer scholars, Gayathri Biju and Modesty Chang in the Digital Futures Lab for their generous collaboration.
Annotated Bibliography
Altschuler, S., Huntington, I., Antoniak, M., & Klein, L. F. (2024). Clinician as editor: Notes in the era of AI scribes. The Lancet, 404(10468), 2154–2155. https://doi.org/10.1016/S0140-6736(24)02568-6
This article describes the new role of the ‘clinician as editor’ of ‘AI scribes’ for ‘narrative medicine’ as well as its limitations and proposes collaborative approaches to navigate this shift. This article provides an insightful view of how AI and narrative medicine improve healthcare practices by refocusing on the patient. Especially, the fact that AI can miss human aspects and clinical rapport can imply the importance of medical humanities; multidisciplinary approach and Affective computing.
Barnett, A., Savic, M., Pienaar, K., Carter, A., Warren, N., Sandral, E., Manning, V., & Lubman, D. I. (2021). Enacting ‘more-than-human’ care: Clients’ and counsellors’ views on the multiple affordances of chatbots in alcohol and other drug counselling. The International Journal on Drug Policy, 94, 102910. https://doi.org/10.1016/j.drugpo.2020.102910
This article is interesting because it provides actual interviews with counsellors and patients to figure out their perspectives on AI in mental healthcare. This suggests ‘AI-human hybrid models’ for ‘more-than-human’ addiction care. This research is helpful for understanding the users’ perspectives and clarifying the importance of ‘humanity, ’empathy’ and ‘connecting’ in mental healthcare. Especially, this article can be directly connected to affective computing – Artificial Emotional Intelligence (AEI).
Brooks, J. (2024). How can emotionally intelligent voice AI support our mental health?. Hume AI. https://www.hume.ai/blog/voice-ai-mental-health
This article is essential to figure out the current progress of Affective computing, and an overview of its application in mental health. The author argues current period is remarkable as empathic voice-voice AI offers various benefits for mental health. Many case studies of empathic voice AI are explained which aims to enhance mental health, daily well-being, education and communication. These include Great “self-awareness” of mental health conditions and managing mental health conditions with rich biomarkers like speed or pitch of voice, Voice assistant, journal or dialogue to create connection and monitor daily well-being.
Hooker, C. (2008). The medical humanities—A brief introduction. ResearchGate. https://www.researchgate.net/publication/5386610_The_medical_humanities_-_A_brief_introduction
The author, Claire Hooker is one of the initiative members of medical humanities in Australia, she is an Associate Professor of Health Humanities and Arts and Health, Sydney Health Ethics at the University of Sydney. She presents the introduction, clear definition and goals of medical humanities. Medical humanities bridge the medicine and humanities disciplines. The main goals are presented as “to study human aspects of medicine”, “the intersection of medicine and the creative arts”, and “training of more insightful and compassionate doctors.” This article illustrates examples of medical humanities such as “illness narratives, death and dying, mental health and incarceration, semiotics of disease, how patients feel about and respond to illness and how they interpret and relate to their carers.” This article offers insights into how medical humanities enhance medicine with diverse perspectives. I appreciated the importance of focusing on human aspects in medicine and a balanced approach.
Holden, A., & Szuhaj, B. (2024). Affective computing: Emotion-aware technology and the evolution of human-centered design. Deloitte Insights.https://www2.deloitte.com/us/en/insights/industry/public-sector/affective-computing-in-government.html
As a consulting firm’s insight, this article shares the public sector perspective focusing on Affective computing. It is useful because it shares comprehensive information such as the importance of Affective computing, its market size, how it works, practical applications, case studies, effectiveness and ethical considerations. This article also provides how Affective computing can be combined with other innovative technologies, such as Gen AI, Mixed Reality and Digital Twins.
Huston, J. C., & Kaminski, N. (2023). A Picture Worth a Thousand Words, Created with One Sentence: Using Artificial Intelligence–created Art to Enhance Medical Education. ATS Scholar, 4(2), 145–151. https://doi.org/10.34197/ats-scholar.2022-0141PS
On the medical education aspect, AI-generated arts can offer a novel experience through reflection process, creating and communicating, enhancing the depth of understanding of the context in Narrative medicine. The authors created images of clinical scenarios with certain styles of art like Van Gogh, Picasso Blue Period or Basquiat style which explain the context better. By focusing on the narrative of each individual including clinicians and patients, it is possible to create a more humanistic and personalised clinical experience. However, the authors also share challenges such as the lack of understanding of medicine by AI, and repetition to create appropriate images. This article is insightful to see novel, practical ways of using AI in medicine. This implies that Affective computing can create novel possibilities and diverse perspectives with empathic and humanistic approaches to understanding emotions and human aspects better, specifically designing for medicine.
Marr, B. (2024). 7 Healthcare Trends That Will Transform Medicine In 2025. Forbes. https://www.forbes.com/sites/bernardmarr/2024/11/20/7-healthcare-trends-that-will-transform-medicine-in-2025/
This article mainly examines new opportunities in healthcare trends 2025 but presents new challenges. Those trends including ‘Technology In Mental Wellness’ and ‘The Personalized Healthcare Revolution; are significant in Affective computing for mental healthcare. These are seven trends; ‘The Personalized Healthcare Revolution, Future-Proofing Healthcare, Technology In Mental Wellness, Wearables 2.0 – BCIs And Implants, Genomics – Decoding The Secrets Of Life?, The Health Data Dilemma, Solving Healthcare’s Tech Skills Crisis’.
Sullivan, C., & Pointon, K. (2024). Artificial intelligence in health care: Nothing about me without me. The Medical Journal of Australia. https://www.mja.com.au/journal/2024/220/8/artificial-intelligence-health-care-nothing-about-me-without-me
This article emphasises one principle: “Nothing about me without me.” and shares the perspective of the Australian citizens’ jury on the application of AI in healthcare. The co-author, Professor Clair Sullivan is the Director of Queensland Digital Health Centre (QDHeC) and Keren Pointon is from Health Consumers Queensland. The jury welcomed the AI in healthcare and developed a consensus statement that emphasised “rigorous evaluation, fairness, patient rights, clinical governance and training, technical and data requirements, and community education and involvement.” This article is useful for understanding the responsible use of AI comprehensively in healthcare, and the importance of governance, voices of citizens as well as the current progress in various groups and another country.
Additional References
Advancing human wellbeing by developing new ways to communicate, understand, and respond to emotion. (n.d.). MIT Media Lab. https://www.media.mit.edu/groups/affective-computing/overview/
Leveraging artificial intelligence for the assessment of severity of depressive symptoms. (2020). MIT Media Lab. https://www.media.mit.edu/projects/leveraging-artificial-intelligence-for-the-assessment-of-severity-of-depressive-symptoms/overview/
Lupton, D. (2017). Digital Health: Critical and Cross-Disciplinary Perspectives. Routledge & CRC Press, Taylor & Francis Group. https://www.routledge.com/Digital-Health-Critical-and-Cross-Disciplinary-Perspectives/Lupton/p/book/9781138123458
Pellegrino, E. D. (1979). Humanism and the physician. Knoxville: University of Tennessee Press. Internet Archive. https://archive.org/details/humanismphysicia0000pell/page/n271/mode/2up
Picard, R. W. (1997). Affective computing. MIT Press.
Russell, B., & Einstein, A. (1976). The Russell-Einstein manifesto—UNESCO Digital Library. https://unesdoc.unesco.org/ark:/48223/pf0000017290
Somers, M. (2019). Emotion AI, explained | MIT Sloan. https://mitsloan.mit.edu/ideas-made-to-matter/emotion-ai-explained
Stephens, E. (2024). Efficiency and the Productive Body: The Gilbreths’ Photographic Motion Studies of Work. https://journals.sagepub.com/doi/full/10.1177/1357034X241262176
Troika (Director). (2014). Daelim Museum—Troika: Artwork Video 02—The Weather Yesterday [Video recording]. https://www.youtube.com/watch?v=ndDrak269tg