Computational Modelling of Tangible Objects on Multisensory Interfaces

Introduction

Current interactive systems provide realistic audio-visual feedback while haptic feedback is limited, even though haptic interactions with devices are essential and touch plays a large role in conveying immersive sensations. Thus, usable high-fidelity haptic rendering has the potential to improve interactions with interactive interfaces and future metaverses.

Background

Knowledge has started emerging on haptic rendering and multisensory perception of tangible objects. Still, combining multiple techniques to stimulate the full bandwidth of tactile sensitivity remains a hard challenge, and little is known about the perceptual integration of tactile cues such as friction, temperature, or vibrations.

Challenges in Haptic Techniques

There is the intricate twofold challenge of developing new haptic techniques based on models of cognitive multisensory integration, whose investigation requires advanced stimulation techniques.

Objectives of TANGO

The originality of TANGO is to address the problem of multisensory rendering of tangible objects from a holistic view by considering simultaneously:

1. Multi-dimensional haptic techniques
2. Tactile acuity across its psychophysical dimensions
3. Mechanisms of multisensory integration

Specifically, TANGO aims to:

1. Characterize typical gestures and contact mechanics during interaction with real objects.
2. Understand how gestures and the dimension of haptic feedback influence perception of simulated tactile objects.
3. Create a computational model of multisensory integration that outputs perceived realism during interaction with objects.

Population Factors

Studies will also consider two main population factors:

• Age
• Sensory impairment

Expected Outcomes

Finally, the outcomes from objectives 1-3 will serve to design sensorily rich tangible 3D elements on displays, in virtual reality, and on maps for the visually impaired.

Interdisciplinary Approach

By a strongly interdisciplinary approach combining haptics, computer science, and neuroscience, TANGO will optimize haptic rendering in multisensory interactive interfaces to enhance its use in human-computer interactions.