Imagine testing hundreds of interaction designs for an XR application overnight—taking into account criteria that you deem important, such as physical toll, accuracy, and speed. Future biomechanical simulations could offer these possibil ities: acting as “virtual crash test dummies,” reinforcement learning (RL) agents controlling musculoskeletal models in physics-based interaction environments can already predict interaction behaviour and movement strategies to guide interaction design. “Simulated users,” i.e., biomechanical models of the upper or full body with muscles controlled via a pre-trained sensorimotor policy, have achieved remarkable performance across core HCI tasks, including mid-air pointing, keyboard typing, and mobile touch.

Current Challenges

However, several challenges limit the practical applicability of biomechanical RL as a research and prototyping method for interaction design. Biomechanical RL depends on computationally expensive training, with typically hand crafted rewards and learning curricula, and does not generalise well across tasks. Moreover, biomechanical models are often perceived technically complex and immature, requiring substantial domain expertise. Overall, biomechanical RL incurs significant computational costs and mental effort.

Open Questions

As a community, we must therefore ask: What role should biomechanical RL play in Computational Interaction? Which of its current limitations are we positioned and willing to address, and where to start? What qualities of biomechanical RL do we deem essential, and how to communicate this to the broader HCI community? This workshop seeks a critical, strategic reflection on biomechanical reinforcement learning as a method in Computational Interaction, and to define a roadmap for biomechanical RL in HCI.

Call for Participation

This S3CIX 2026 workshop seeks a critical, strategic reflection on biomechanical reinforcement learning as a method in Computational Interaction, and to define a roadmap for biomechanical RL in HCI. It invites all interested HCI researchers to discuss which limitations we as a community are positioned and willing to address, how to tackle them, and where to start best.

To let us know about your interest in this workshop, please fill this form, where you are asked to share your previous experiences with biomechanical RL (if any) and share your expectations for this workshop. All participants will be invited to contribute to an Interactions article, where we will share our insights on the potential of biomechanical user simulations for computational interaction and HCI research.

Schedule and Location

The workshop will take place on tbd. See full S³CIX workshop programm here.

Organizers

Florian Fischer

Florian Fischer is a Postdoc in the Intelligent Interactive Systems group at Cambridge University, with a background in mathematics. His research areas include optimal control methods and deep reinforcement learning (RL) for human-computer interaction, biomechanical user modelling, mid-air interaction techniques, and social VR.

Arthur Fleig

Arthur Fleig is a junior research group leader at ScaDS.AI Leipzig, Leipzig University. His research interests lie on the interface between Computer Science and Mathematics. He focuses on modelling, simulation, and optimal control of real-world-relevant dynamical systems within Human-Computer Interaction.

Patrick Ebel

Patrick Ebel is Assistant Professor for Computational Interaction at Hasso Plattner Institute. His research focuses on data-driven and RL-based simulated users that model human cognition, perception and motor control. These models aim to support the design and evaluation of interactive systems that are better aligned with human goals, abilities, and behavior.

Miroslav Bachinski

Miroslav Bachinski is an Associate Professor at the University of Bergen. His research focuses on the development and application of data-driven methods to improve post-desktop user interfaces within the large spaces of alternative designs (e.g., virtual reality, levitation).

Roderick Murray-Smith

Roderick Murray-Smith is professor in the School of Computing Science at the University of Glasgow, where he is a member of the Inference, Dynamics and Interaction group and works in the areas of human-computer interaction, machine learning, and control, leading the ERC project DIFAI on active inference in HCI.

Antti Oulasvirta

Antti Oulasvirta leads the User Interfaces research group at Aalto University and the Interactive AI research program at the Finnish Center for AI.

Per Ola Kristensson

Per Ola Kristensson is a Professor of Interactive Systems Engineering in the Department of Engineering at the University of Cambridge and a Fellow of Trinity College, Cambridge. He is a co-founder and co-director of the Centre for Human-Inspired Artificial Intelligence at the University of Cambridge.