As computing moves into the air around us, traditional metaphors of interaction break down — there are no glass, buttons, or surface to feel sensory feedback from. What’s left is the body and a need to rediscover how to gesture, press, connect, and play in a medium made of light and intention.

Using hands to interact with virtual objects, we are susceptible to an illusion of a ‘touchless touch’. Researchers find that audial and visual feedback from actions in VR can lead to a convincing illusion of a touch. In one experiment on Magic Leap, a user was interacting with an ‘electric’ cloud — when touched, it made a loud buzz and emitted light, and that resulted in a convincing tactile reaction for them.

The illusion is happening even when we are touching with virtual hands in the full VR mode. In one experiment, the participants were touching their virtual hand with a virtual stick, and many of them confirmed sensations in their hand that was being touched.

In this research, we weren’t trying to build a product, but to prototype and experiment based on intuition and feedback. Our goal was to uncover interaction models that feel natural, instinctive, and a little bit magical, like brushing your hand through bioluminescent water.

The approach of ‘useful magic’ that we selected as our guide in this experiment resulted in a few main principles that informed our work and helped find what felt right in space.

1. Touch should be as intuitive as possible. Aiming to create a feeling of a naturally occurring wonder, we were particularly interested in interactions that do not need to be taught — hands will know what to do if we design for the instinctive connections.

2. The system should invite experimentation. What happens if I press harder? Swipe faster? Wait longer? We wanted to think about interfaces that reward curiosity and don’t limit the dialogue between the technology and the user in the binary scenarios.

3. Delight is not optional. Surprise, playfulness, responsiveness — these create emotional memory and a difference between simple input and an inspiring interaction.

We started our exploration with mapping out various forms of organic interaction based on the natural instincts rather than habit and social conditioning. In our creative laboratory, each of the team members was approaching the topic based on their main tool: code, 3D software, animation, or text.

One of the examples was positive and negative feedback — how do we express it with just the gestures? Our answer to that included using open hand positions and lingering eye or hand contact as a way to say ‘yes’, and closed body language and avoidant contact to answer negatively.

A speed of hand movements can also be an important signal. Moving with slow hesitation, we are in contemplative mode and are more likely to want more order and support from our systems. With fast hand movements, we often act from the place of inspiration and can be more open to inspiring chaos and unexpected results of interaction.

We played with the potential of the natural gestures to add emotional and magical dimension to the communication — playing with the latency and subtlety, we used a slow press to create a glowing ripple and a fast swipe to burst into particles.

As the next step, we started imagining the systems that would support the kind of engaging and non-linear interaction with the digital systems. Our main medium was selected to be particles and dust — responsive, dynamic, and enticing.

Instead of limited points of contact, as in menus and buttons, particles gave us the freedom to support multiple limitless touch that could translate into more intricate messages. Floating fields of responsive dust moved, changed color, or followed the user’s hands, creating a sensory choreography of an interface.

We built prototypes where users could gather, blow, and mix dust with simple hand gestures. The dust responded with glow, density, or drift, depending on gesture strength and direction, turning each grain into a pixel of intention. It removed the sharpness from interaction — instead of clicking on something, you shape it in a way that allows for a surprise.

As we move into spatial computing, we believe that interfaces will become less like tools and more like companions — we won’t just use them, but play with them.

Glow served as another helpful visual metaphor — think of glowing dust, fireflies, electric jellyfish, synapses firing across a brain. That glow became the emotional entry point into touch, a visual response to presence.

Our inspiration came from bioluminescent plankton, simple organisms that emit light when disturbed. The purity of that reaction — movement becoming light — supplemented our visual language, centered around sensitive particles.

Glowing became a way for the particle structures to react to the touch. Using waves of energy to communicate the impulse coming from the movement, concentration of the dust, or the invisible processes of the digital system, interactive particles are directing the user’s attention and bring about the illusion of tactile feedback.

While our main focus was on purely virtual surfaces and objects, we experimented with integrating sensing particles into the interactions with the real-world objects.

Our idea was to use the touch as a key unlocking the hidden layer of the familiar objects and environments. In one of the XR prototypes, a wall turned into a sensitive surface when touched, revealing the potential for interactions that merge physical and digital. In these liminal applications, the virtual sensing creates an opportunity for ‘enchanting’ the ordinary and reminding about the simplified models of the world that we live by. It opens the door for a more playful and nuanced living in the physical space: being able to ‘listen’ to a tree or a stone, find secret written messages, or see the emotional feedback from your actions in the space.

Interfaces based on clouds and surfaces consisting of myriads of sensitive dots allow to invite more fluidity into virtual interactions. By accepting the in-between states, we create flows of information similar to natural ways of transmitting the data and break beyond the borders of linear and predefined pipelines of interaction.

Through this uncertainty and unpredictability, we bring more of our dynamic and always changing natural world into the digital universes that we populate. At the same time, we create conditions to add agency to the systems that are usually compliant and invisible. When our expectations from the system feedback change to let some uncertainty in, the foundation for a true communication with the technology appears in place of the rigid patterns, opening the horizon for new emotions and ideas.

In the next chapters, we will keep exploring this approach of nature-inspired patterns translated into the digital forms, as well as its applications and changes that it brings into our virtual worlds.