Triadic Healthcare Conversation

In a clinical consultation with a doctor, a patient, and a robotic assistant, who speaks when? And more importantly — when should the robot stay silent? This project investigated multimodal conversational cues in triadic healthcare settings: gaze direction, speech patterns, temporal gaps, and gestural signals that mark a turn as "claimable."

We analysed real doctor-patient-assistant conversations, building a framework for when appropriate intervention happens — and found that 10 out of 10 appropriate robot interventions occurred only when multiple cues converged simultaneously. No single signal is enough. The work contributed a nuanced model of collaborative context that goes beyond speech recognition, and laid the groundwork for our follow-on robot scenario simulation (TBD: see the Physical Therapy Robot case study).