Audio Synthesis with Augmented Reality (AuSynthAR) is a personal project focused in the creation of a digital instrument based on Augmented Reality (AR) inspired by the iconic Reactable and based on a previous project related with electrical circuits and AR (Virtual Circuits). AuSynthAR allows sound synthesis modules to create simple sound networks. It only requires a mobile device, a set of tokens, a sound output device and, optionally, a MIDI controller, which makes it an affordable instrument. An application running on the device generates the sounds and the graphical augmentations over the tokens. The setup of a first iteration is depicted in the images below, together with a video explaining the main features when the touch screen was included to manipulate parameters for the very first prototype.
The prototype shown above evolved to avoid dependency on the touch screen and perform with the tokens. Thus, the next iteration was focused in using special components like-knobs to control parameters from modules such way the device screen becomes just a visor that interprets what is happening with the physical components. This new iteration considers visual aesthetics by including 3D characters that moves according to the sound interactions. The physical setup, application screenshot, and tokens, are presented in the next images, as well as a video that shows an improvised music performance.
This instrument was presented in a demo session at the 19th International Conference on New Interfaces for Musical Expression (NIME 2019) and the related paper with more details can be found here. This is one of my first personal projects as independent researcher. Also, this project has been shown in other local events like CITI Microemprendedor del Año (Quito-Ecuador) and Trend Hunter Future Festival (Guayaquil-Ecuador) with a high acceptance from the attendees.
Public Demo
Anyone who want to test AuSynthAR can download an .apk file for Android devices (versions >= 6.0) from here.