Session 3 – Speech and LLMs

Session Chair: Roland Goecke

• Jaya Narain, Amrit Romana, Vikram Mitra, Colin Lea and Shirley Ren, Affect Models Have Weak Generalizability to Atypical Speech (paper ID 28)
• Amrit Romana, Jaya Narain, Tien Dung Tran, Andrea Davis, Jason Fong, Ramya Rasipuram and Vikramjit Mitra, Switchboard-Affect: Emotion Perception Labels from Conversational Speech (paper ID 63)
• Abinay Reddy Naini and Carlos Busso, Multi-Dimensional Ordinal Embedding for Attribute Modeling in Speech Emotion Recognition (paper ID 73)
• Rui Amorim de Oliveira Pimenta, Tim Schlippe and Kristina Schaaff, Assessing Consciousness-Related Behaviors in Large Language Models Using the Maze Test (paper ID 14)
• Samad Roohi, Richard Skarbez and Hien Nguyen, Uncertainty-Aware Affect Prediction in Conversation Using Conformal Prediction (paper ID 22)
• Xiaojiang He, Yushan Pan, Nan Xiang, Yangbin Chen and Zhijie Xu, An Active Learning-based Global-Distribution Attention Model for Multimodal Sentiment Analysis (paper ID 03)
• Siyuan Zhang, Shun Chen, Zhuofan Wen, Hailiang Yao, Zheng Lian, Zhengqi Wen, Bin Liu and Jianhua Tao, LLM-Driven Multimodal Fusion: Enhancing Depression Detection through Textual Feature Augmentation (paper ID 80)
• Hiromi Narimatsu, Xijia Wei, Keishi Nomura, Nadia Berthouze and Shiro Kumano, Emotion in Art-Elicited Texts, Interpreted by GPT-4o: Self vs. Third-Person Perspectives (paper ID 85)

Ask me Anything

Panel: Prof Ursula Hess, Prof Flora Salim, Prof Yaser Sheikh, and A/Prof Dom Dwyer

Moderator: Marwa Mahmoud

• B Ashwini, Sanmay Sood, Pratyush Jain and Jainendra Shukla, Towards Context-Aware Multimodal AI for Assessment of Autism in Children (paper ID 60)
• Sunday David Ubur, Denis Gracanin and Eugenia H Rho, Adaptive Captioning with Emotional Cues: Supporting DHH and Neurodiverse Learners in STEM (paper ID 71)
• David Kalina, Umar Khan, Sumair Aziz, Julio Romero, Hugo Posada-Quintero, Roland Goecke and Raul Fernandez-Rojas, Assessing Pain Relief in Virtual Reality: A Multimodal and Machine Learning Approach (paper ID 98)

• Ruyi Wang, caseKG: A LLM-Based Knowledge Graph for Mental Schema Analysis (paper ID 21)
• Louis Simon and Mohamed Chetouani, Learning Longitudinal Stress Dynamics from Irregular Self-Reports via Time Embeddings (paper ID 70)
• Stan Meyberg, Gizem Sogancioglu and Heysem Kaya, Balancing Performance and Fairness: The Role of Text in Multimodal Depression Detection in Vlogs (paper ID 57)
• Hinako Kizawa and Yoshiko Arimoto, Physiological analysis on the effect of laughter contagion on interlocutors’ mental states (paper ID 05)
• Kexin Feng and Theodora Chaspari, The Impact of Explainable AI on Human Decision-Making in Speech-Based Depression Assessment (paper ID 11)

• Boning Zhao and Xiao Sun, Human Empathy as Encoder: AI-Assisted Depression Assessment in Special Education (paper ID 111)
• Amir Ansari, Jana Subirana, Bruna Silva, Sergio Escalera, David Gallardo-Pujol and Cristina Palmero, Enhancing Personality Recognition by Comparing the Predictive Power of Traits, Facets, and Nuances (paper ID 123)
• Hiroyuki Sakakibara and Kazunori Terada, Context- and Personality-Aware Emotional Expression Design via Creator-Embodied Agents: A Pilot Study (paper ID 115)
• Chenyu Zhang, Sharifa Alghowinem and Cynthia Breazeal, Ensembling Large-Language Models to Characterize Affective Dynamics in Student-AI Tutor Dialogues (paper ID 120)
• Hong Jia, Shiya Fu, Feng Xia, Vassilis Kostakos and Ting Dang, Beyond Scale: Small Language Models are Comparable to GPT-4 in Mental Health Understanding (paper ID 113)
• Pierrich Plusquellec, Sara Saint-Pierre-Côté, Marc-André Massé, Lesly Nzeusseu Kouamou, Pamela Lirio and Vincent Denault, Emotion AI in the Wild: Responsible Deployment of Affective Computing in Real-world Workplaces (paper ID 116)

Lennart Eing, Enhancing Machine Sign Language Understanding by the Use of Auxiliary Data

Łukasz Culer, The Heart Wants What it Wants: How Can ECG Foundation Models Help Us Understand Emotions?

Stan Meyberg, Fair, Explainable, and Personalized Treatment Outcome Modelling in Mental Healthcare 

Jiawei Wang, Affective Multimodal Modeling and Chatbot Design for Responsible Retail and Clothing Try-on 

Pritesh Contractor, Decoding Social Interaction to Understand Traumatic Behaviors in Social Dynamics