This system, developed in Unity, is based on a swarm model called “swarmalators“, which are “oscillators whose phase and spatial dynamics are coupled”. The model was adapted for interactivity, starting with an integrated version that involved a 3D virtual visualization and interaction environment in Unity, communicated via OSC messages to Max for sound synthesis, and a physical MIDI controller for music input. A demonstration of the system is given in the following video.
First version’s code: https://github.com/pedro-lucas-bravo/sync_swarm_experiments
Then, the system was implemented in the Meta Quest 3 to facilitate natural interactions with agents and swarm parameters, as demonstrated in the following video.