Fresh off a 5 hour flight from my internship in Calgary, I went back to the University of Waterloo to compete in Hack the North, Canada's largest annual hackathon. Here, my team and I created HeyKanye, a project which would apply one artist's lyrics to the musical stylings of another, creating windows into parallel realities like one where Kanye's biggest hit is "You Belong with Yeezy".
Working with me were Ankai Jie, Neil Liu and Trevor Sookraj.
HeyKanye placed as one the 12 Grand Finalists at the hackathon and was the receipient of the Microsoft Azure API Prize.
A great way to spend 36 hours!
The Project (Technical Stuff)
We would select a track which contained the music stylings of an artist (host) as well as an artist whose lyrics we would be using (donor).
Generating Lyrics
For the host track, we would take the lyrics and analyze sentence structure, word-to-word relationships, grammar and rhyme schemes. By constructing parse trees and augmenting it with Microsoft Azure's Web Language Model API, we created a set of sentence structures for the host track. This was the scaffold which the donor artist's lyrics would be grafted.
For the donor artist, we scraped a lyrics API and stored all lyrics from all their songs into an Azure Database. By generating Markov chains on the collected data, we were able to create structures which gave us the probability of words following another for that artist. This gave us the "style" of the donor artist.
To generate new lyrics, we seeded the Markov chains with previously used words in the lyrics along with some new ones gathered using Azure's Web Language API to give it a little more differentiation. Generated lyrics which fit the parse trees were selected and collected.
Making the Beat
For the track, we used librosa to extract onset (pitch and beat changes) and applied edge detection to find the approximate timestamps of verses and choruses. This let us know where we would splice in our generated lyrics.
Our original hope was to record some convincing-sounding raps ourselves and attach them to the background track, but sadly lacked the talent and energy to do so.
Cause I thought it would be fun, we used linux's text-to-speech tool to "say" our generated lyrics and graft them to the host track, making a very robotic version but working version which was used in our final demo.
Results
The quality, as you can hear below was.... not so great.
Our primary problem was the Markov chains not doing a good enough job of predicting the next word. Even the overall feel was right, the logic was completely missing for which words followed which.
We also ran into a lot of other issues that took time to fix throughout:
- Database issues and using MS Azure
- How to implement HMM model to produce structurally correct data. We chose to construct the model based off word by word rather than line by line so we can have a truly random song implementation using grammar rules and parse trees.
- Insanely dirty lyrics generated by our program
- Connecting backend-to-frontend
Still it was a lot of fun and was a good introduction into the world of algorithmic song generation. While we used some really basic tools here, we think that more research and better techniques can really make this a cool tool for music creators in the future.