Exploring the intersection between computational and human emotive music visualization.
What could be the future of the concert experience?
Cover Art Creator is an user interactive program that generates visuals for different songs as the music plays. It acts as both a performative audio visualizer but at the same the additive nature of the program provides user with an end product that can be used as an cover art for the song. The program uses audio frequency analysis and user input to generate one of a kind visuals .The program is created using Processing along with MINIM and ControlP5 external libraries. This project was completed for the RISD Intro to Computation course taught by artist Mattia Casalegno.
Sound + User Input + Code
This project was completed as part of an introductory course to computing. Over the course of four months we learned to code using Processing, an open source program that helps designers use and learn Java better. Processing allows us to take a more visual and interactive approach to coding. For my final project, I was really interested in experimenting with sound and how we can use sound as data to generate art.
While experimenting with sound on Processing, I noticed that it can only analyze frequency bands which doesn't generate enough visual differentiation that also captures the essence of the song. I decided that user input could be a solution to this. In the end I came up with a program that generates visuals for a piece of music based on the users perception of the song’s mood, emotional qualities and energy.
This project was completed as part of an introductory course to computing. Over the course of four months we learned to code using Processing, an open source program that helps designers use and learn Java better. Processing allows us to take a more visual and interactive approach to coding. For my final project, I was really interested in experimenting with sound and how we can use sound as data to generate art.
While experimenting with sound on Processing, I noticed that it can only analyze frequency bands which doesn't generate enough visual differentiation that also captures the essence of the song. I decided that user input could be a solution to this. In the end I came up with a program that generates visuals for a piece of music based on the users perception of the song’s mood, emotional qualities and energy.
For my second prototype, I decided to work from two sound samples. I took one music file and broke it down into two files based on the instruments. This allowed me to generate two different visuals elements while the music played. The interplay between the two created a more interesting composition.
For my third prototype, I worked from three different sound samples. In this prototype the biggest challenge I faced was the end result being too crowded with visual elements. Because there were more visual elements generated in this prototype, the over lapping of elements became problematic as the music played for a long time. I played around with having some elements cover previously generated elements completely. This solves the problem of an overcrowded composition but I still wasn't really satisfied with the end result.
For my second prototype, I decided to work from two sound samples. I took one music file and broke it down into two files based on the instruments. This allowed me to generate two different visuals elements while the music played. The interplay between the two created a more interesting composition.
For my third prototype, I worked from three different sound samples. In this prototype the biggest challenge I faced was the end result being too crowded with visual elements. Because there were more visual elements generated in this prototype, the over lapping of elements became problematic as the music played for a long time. I played around with having some elements cover previously generated elements completely. This solves the problem of an overcrowded composition but I still wasn't really satisfied with the end result.
Process: Experimenting with Additive Visuals
The biggest challenge when designing this program is making all visuals additive; meaning the generative visuals cannot disappear, instead the end result is an accumulation of all the visuals generated. Normal audio visualizers do not need to additive, however my goal is to have the music "print out" its own cover art so it requires all the generated visuals to remain on the board as part of a bigger composition.For my first prototype, I designed the program so that it prints out music in a linear fashion. The result is a long scroll of lines that wasn't really interesting compositionally.
The biggest challenge when designing this program is making all visuals additive; meaning the generative visuals cannot disappear, instead the end result is an accumulation of all the visuals generated. Normal audio visualizers do not need to additive, however my goal is to have the music "print out" its own cover art so it requires all the generated visuals to remain on the board as part of a bigger composition.For my first prototype, I designed the program so that it prints out music in a linear fashion. The result is a long scroll of lines that wasn't really interesting compositionally.
Program Design
Cover Art Creator
Program code on Processing
Generated cover art along with the song it was for
Live performative visualizer for different songs
Keep Going
Back to Home
Design Research and Methods
Designing solutions to prevent binge drinking amongst college students.
The Final Program takes in four different sound files – bass, drums, melody and other instruments. Each music sample is analyzed using MINIM and a numeric value is generated which changes the color, size and movement. Sliders made using controlP5 is also used to change this value that further changes the bahavior of the visual elements.
Urban Interactions
Rethinking urban housing models to increase housing affordability in Taipei City.
Possible Applications
This interactive visualizer opens up many interesting possibilities for shaping a different kind of live music experiences. Imagine an interactive concert exeperience where the users mood transforms the stage, the lighting and what’s diplayed on the screen.
This interactive visualizer opens up many interesting possibilities for shaping a different kind of live music experiences. Imagine an interactive concert exeperience where the users mood transforms the stage, the lighting and what’s diplayed on the screen.