In January 2026 I spent a month at the Institute for Cognitive Systems (ICS) at the Technical University of Munich, in Prof. Gordon Cheng's team, thanks to Erasmus+ funding. The goal was simple to state and hard to answer: if a robot is covered in artificial skin, is touching it actually a better way to control it?
The result is our paper "Tactile Guidance for Complex Mobile Manipulation: Performance and Design Implications", submitted to the IEEE-RAS International Conference on Humanoid Robots (Humanoids 2026).
The robot: TOMM
TOMM is ICS's dual-arm mobile manipulator: two UR5 arms on an omnidirectional base, with the arms and hands covered in ICS's modular robot skin. Each skin cell senses both proximity and force, so the whole upper body becomes a surface you can talk to with your hands.
Controlling it is not trivial. Moving a box with TOMM means coordinating the base, both arms and the grasp, about 15 degrees of freedom in total. That is exactly the kind of system where classic interfaces (joysticks, pendants, GUIs) start to feel abstract, and where "just push the robot where you want it" sounds appealing.
Turning skin into a controller
We built a gesture interface on top of the skin, using a few simple, body-mapped primitives:
- Double-tap on the shoulder: switch between driving the base and moving the arms
- Double-tap on the forearm: grasp or release
- Push with one hand: translate (the base, or the box held between the hands)
- Push with two hands at two places: rotate the base
Under the hood, contacts are clustered into wrenches that feed an admittance controller running at 200 Hz. When the robot is holding a box, your pushes move a virtual frame between the two hands, and both arms follow it while keeping the grasp intact.
For a fair comparison, the joystick baseline used the very same controller and logic. The only thing that changed was the interface.
The study
Twelve participants (6 women, 6 men, mostly with a technical background) did the same task twice, once with each interface, in counterbalanced order: drive TOMM to a first table, pick up a box with both hands, carry it to a second table (which requires turning the robot about 180°), and put it down.
There was deliberately no training session: we wanted to see what a first encounter with a touch-controlled robot really looks like. We measured phase-by-phase timing from video, interaction errors, how much people walked (with camera-based 3D pose tracking), NASA-TLX workload, questionnaires on control and trust, and finally whether people remembered the commands afterwards.
What we found
Honestly, on first contact the joystick won, and the numbers are clear:
- Slower: median total task time was 152 s with touch vs. 97 s with the joystick, with most of the gap coming from picking up the box and carrying it to the second table.
- More walking: people walked a median of 45 m vs. 16 m, moving around the robot to reach the right skin patch.
- Heavier workload: mental demand, physical demand and frustration were all significantly higher with touch, and people felt less in control.
Where participants spent their time. With the joystick (left, blue) they mostly stayed in one place; with touch (right, red) they spread all around the robot to reach the skin patches.
Each line is one participant. Touch made the transport and pickup phases longer, and made people walk much more.
But there was one result that points the other way: people remembered the touch commands much better. Command-recall accuracy was 73% for touch against 54% for the joystick. Mapping commands onto the robot's body, "tap the shoulder", "push the forearm", seems to build a much stickier mental model than remembering which button does what.
Why touch lost (for now)
Looking at the videos, the slowdown does not look like it is caused by touch itself. It comes from friction in how the interface was designed:
- Hidden state. People often did not know if TOMM was in "drive" or "arm" mode, so a push did something unexpected.
- One thing at a time. The joystick can translate and rotate the base simultaneously; our touch interface did these in separate modes, which is costly during a 180° turn with a box.
- Reach. The right patch was often on the other side of the robot, so people had to walk to it.
A small benchmark with two expert users suggests that the gap shrinks a lot with practice, so a good part of the novice penalty is learnable.
What we would change
From this, we propose a few concrete design guidelines for tactile interfaces on large robots:
- Make the robot's state obvious, with persistent visual or sound cues for the current mode.
- Treat patch placement as a design variable: bigger or duplicated key patches, and tolerant zones that map a near-miss to the intended command.
- Separate continuous guidance from discrete commands more clearly, both conceptually and physically on the body.
The aim is not to remove the embodiment, which is clearly what makes the commands memorable, but to remove the friction that makes the first minutes hard.
Thank you
A big thank you to Prof. Gordon Cheng and the ICS team for welcoming me and for the time spent with TOMM, to my supervisor Prof. Adriana Tapus at ENSTA, to the Erasmus+ programme for funding the visit, and to all the participants who patiently pushed a robot around for us.
This is a pilot study with 12 technically experienced participants and no training, so these are trends rather than final answers. Next steps: structured training, longer studies and more varied tasks, to see whether touch can catch up with, or overtake, the joystick once people know the robot.
See also: Publications · Haptic interface project · Tactile array project