OPERATOR,
Spatial music-making experience

OPERATOR is our first application for Apple Vision Pro that turns the process of music production into a spatial modular experience. The composition is made by creating a virtual sculpture, and the sound in the experience is tightly connected to the form and arrangement.

Easy to use and playful, it features a sequencer, a set of ready-made electronic samples, and simple tools. With intuitive interactions, the application allows for an experience that is closer in its mechanics to arranging the building blocks than it is to using the audio production software.

Merging form and sound, OPERATOR acts as a system for audiovisual experiments. Each created composition exists as a sculpture integrated into a real space through the XR mode. Featuring realistic materials that light up gently together with the sound, each composition is a custom aesthetic object that can be saved into the library and brought along to various real-world spaces by its creator.

Experience

OPERATOR uses a set of cubes that act as samples, which include drums, synths, bass, and effects, and two simple tools, voice recorder and frequency filter. These blocks are stored in a inventory that acts as a portal — the ‘box’ with the blocks is visible only from the frontal side.

Put together on a sequencer that can be extended in any direction — like in a game of ‘Snake’ — the blocks form an electronic music composition. The tools connect to the same system and provide the possibility of customising the track and adding your own touch to it without a complicated setup — both of them are controlled by just one UI element. The system allows for shorter and longer creative sessions and is accessible for people without any prior knowledge or experience.

The visual component of OPERATOR is as important as its musical foundation. The visual effects within the app make the experience of building and arranging focused and highlight the form captured from the user’s movements and interactions. Integrated into the space and adaptable, the sound sculpture is a continuation of the user’s world.The approach of ‘useful magic’ that we selected as our guide in this experiment resulted in a few main principles that informed our work and helped find what felt right in space.

Influences
1.

The inspiration for OPERATOR comes from our interest in creating a fun modular system that would allow a user to experiment with music immediately — without spending too much time trying to understand the tool.

Gas Giant Studies, Jacob Hashimoto

Model of New Tretyakov Gallery, OMA

BRAUN Lectron System, Georg Greger

Possible, Plausible, Potential, Miguel Nóbrega

Ilot Caravane, MVRDV

Our approach to designing intuitive interactions was to use the language of simple spatial interactions and to move away from the traditional flat-screen experience.

The core of the experience is informed by the modular systems and projects. Lectron system manufactured and developed by Braun in the 1960s was a large source of inspiration for the interaction design. Timeless, intuitive, and versatile, it gave freedom to experiment and enabled learning by practice — the blocks attach to each other through magnets and form electric circuits without any additional assembly.

Architectural models and sketches informed the sculptural shapes that became the foundation for the application. Flexible systems that unite blocks of various colours and forms, they informed how different instruments and tools come together to form a single sound system.

Graphic design elements of the app were developed along the 3D components and were inspired by iconic Swiss design. Bold and distinct, the graphic design of the project is purposeful and informative at the same time and supports the clarity of the main shapes. The typography and UI elements in the application are integral parts of the experience that add to the visual expressiveness without distracting from the interactions with the system.

Process
11.

OPERATOR emerged from the first design tests of the studio and was developed into the app over the course of a few months.

Our design research started from an abstract idea of rethinking the music creation for space. The idea for the ‘player block’, a base that acts as a sequencer, came pretty early on. At the start, we tested more complicated shapes of the modules that formed beautiful intricate compositions. However, our decision was to create more minimalist blocks that would be easier to combine and put into a single system.

The ‘Snake’-like sequencer turned out to be one of the most important ideas for the app experience. When we built this first as a prototype, it was already fun to use, even without the music blocks on top. Adaptability of the shape and length of the player block became a central element of the app and added a component of irregularity into an otherwise uniform system.

An exploration of the graphic elements was a large part of the design stage. Testing dozens of prototypes with buttons, grids, and scales, we played with various shapes and ideas for the interaction. Integrating these ideas within the context of menus, names, lists, and other UI elements, we created a system that is essential and minimalist.

The graphic research informed our subsequent exploration of the integration of the effect block. We had a lot of ideas that used depth rather than a flat surface, and for the effect block, we had a lot of tests including three-dimensional handles and controllers. For the final app, we also decided to stick with the simpler and comprehensible option of having a flat control, but our experiments with virtual 3D devices were inspiring and will continue in one of our next projects.

In its first iteration, OPERATOR serves as a demonstration of a new approach to the music production — spatial and open for people with any level of experience.

As a modular experience and an evergoing experiment, it can endlessly grow by adding new portals with sounds, features, and tools.

Credits

Creative Director:

Maxim Zhestkov

Art Director, 3D Designer:

Vladimir Zhirnov

Developers:

Hankun Yu,

Theodore Cruickshank,

Kamen Dimitrov

Producer, Writer:

Anna Gulyaeva

Assistant:

Irina Kulinich

Animator:

Ivan Popov

Graphic Designer:

Anastasia Suryaninova

Musician:

David Guriev

Sound Designer:

ITSU

Voice:

Constantine Gregory

Creative Director:
Maxim Zhestkov

Developers:
Hankun Yu
Kamen Dimitrov

Animator:
Ivan Popov

Interaction Designer:
Anna Blewden

Art Director, 3D Designer:
Vladimir Zhirnov

3D Designer:

Diana Umiarova

Writers,
Creative Producers:
Irina Kulinich
Anna Gulyaeva