EmoSynth Real Time Emotion-Driven Sound Texture Synthesis via Brain-Computer Interface

In electroacoustic music composition, particularly in sound synthesis techniques, Deep Learning (DL) provides very effective solutions. However, these architectures generally have a high level of automation and use textual language for human interaction. To improve the relationship between composers and artificial intelligence systems, brain-computer interfaces (BCIs) are an effective and direct systems, which have led to considerable improvements in this area. The proposed system employs emotion recognition through electroencephalogram (EEG) signals to control four Variational Autoencoders (VAE) that generate new sound textures. A dataset was acquired using the MUSE2 headset to train four Machine Learning (ML) models capable of classifying human emotions based on Russell's circumplex model. VAEs were trained to produce different sound variations from an audio dataset that allows composers to integrate their sounds. In addition, a graphical user interface (GUI) was developed to facilitate the real-time generation of sound textures, with the support of an external MIDI controller. This GUI continuously provides visual information about the detected emotions and the activity of the left and right brain hemispheres.