Multimodal Measurement of Internalizing Disorders in Clinical and Family Contexts [The Tianqiao and Chrissy Chen Talk]

Dr. Jeffrey Cohn
Professor of Psychology, Psychiatry, and Intelligent Systems
Department of Psychology
University of Pittsburgh, USA
Chief scientist at

Abstract:Reliable, valid, and efficient measurement of internalizing disorders (specifically, Major Depressive Disorder and Obsessive-Compulsive Disorder) is critical if we are to gauge patient improvement and reveal transmission within families.  With few exceptions, previous work in objective measurement is limited to detecting occurrence of internalizing disorders by single modalities in restricted settings and fails to address their severity, change with treatment, interpersonal mechanisms, or neural substrates. In this keynote, I first present what my interdisciplinary team has learned about multimodal and interpersonal indices of depression; then present our work in multimodal measurement of OCD and depression in patients undergoing deep brain stimulation for treatment-resistant disorders; and close with implications for precision and computational psychiatry.

Bio: Dr. Jeffrey Cohn is a professor of Psychology, Psychiatry, and Intelligent Systems at the University of Pittsburgh and chief scientist at He leads inter-disciplinary efforts to develop advanced methods of automatic analysis and synthesis of facial expression, body movement, and prosody and applies those tools to research and practice in human emotion, nonverbal communication, and computational psychiatry. His research has been supported by grants from the U.S. National Institutes of Health and the National Science Foundation among other sponsors.

Can intelligent agents effectively employ clinical empathy skills?

Elizabeth Broadbent
Professor of Health Psychology
Department of Psychological Medicine
Faculty of Medical and Health Sciences
The University of Auckland
New Zealand


 Abstract: Clinical empathy refers to the ability of a clinician to understand what a patient thinks and feels, and to communicate this understanding back to the patient. Greater clinician empathy has been associated with better patient outcomes. As intelligent agents move into healthcare applications, their clinical empathy skills become more salient. This talk critically reviews what we know to date from the medical literature, and covers several research questions: How can clinical empathy be built into intelligent agents? Does agent clinical empathy impact patient outcomes? What are the ethical considerations? What do we know so far and what remains to be understood? Borrowing from medical literature on communication skills training, this talk introduces the theoretical model of robot-patient communication. This model combines the effects of both patient and agent factors,  as well as communication factors, on patient outcomes. The talk includes a series of studies with both virtual agents and robots that test this model, and inform its further development. The results are discussed in terms of how they inform knowledge of the use of intelligent interactive agents in healthcare. Implications for best practice and future research are discussed.

Bio: Elizabeth initially trained as an electrical and electronic engineer at Canterbury University to pursue her interest in robotics. She then worked at Transpower, Électricité de Tahiti, and Robotechnology. After becoming interested in the psychological aspects of robotics and in psychoneuroimmunology, she obtained her MSc and PhD in health psychology, supported by a Bright Futures Top Achiever Doctoral Award. Her current research interests include how stress affects our health, how our body posture affects our mood, interventions to help patients make sense of and cope with illness, and human-robot interaction in health contexts. Elizabeth is well known for the development of the Brief Illness Perception Questionnaire, drawing assessments, and the development of illness perception interventions. In the technology space, Elizabeth is particularly interested in the emotional connections we form with robots, and how we can build emotional intelligence and empathy skills in robots, to help support patients. This research extends to Digital Humans, an advanced form of computer agent with artificial intelligence.


Affective computing: Value tensions in design engineering

Rafael A. Calvo
Faculty of Engineering, Dyson School of Design Engineering
Imperial College London, UK


Abstract: As researchers who “make things”, we like to focus on the positive impact that our work has on the world, and we occasionally look at its unintended consequences. But what we rarely do is ask “what are the values by which we judge the systems we create?”. I will start with an “archaeology of AI” using Weizenbaum’s ELIZA chatbot, created in 1966 to reflect on different schools of psychology, philosophy and engineering that today shape the world, and how the inconclusive debates at the dawn of AI still rage through our work on Affective Computing.

In this talk, I will be reflecting on the ethical considerations that arise from our affective computing research and how I have personally attempted to address the corresponding tensions within my own work.

This is not a talk on ethics, but rather a personal, and therefore partial, description of the dilemmas faced when looking at the ”big picture” impact of our work.

Bio:Rafael A. Calvo, PhD (2000) is a Professor and Director of Research at the Dyson School of Design Engineering, Imperial College London. He is also co-lead at the Leverhulme Centre for the Future of Intelligence, and co-editor of the IEEE Transactions on Technology and Society. He is the director of the Wellbeing Technology  Lab which focuses on the design of systems that support well-being in areas of mental health, medicine and education.