SubWoofer is an app that generates relevant lyrics for songs using image recognition and a library of song lyrics. You can add lyrics to lyric-less songs or you can generate better lyrics to accompany a song’s music video.
Have you ever listened to a song without words and wondered what words would be sung if the song had words? Have you ever seen the music video for a song and wondered why the video is showing whipped cream bras while the lyrics are talking about California Gurls? Have you ever wanted to replace nonsensical “work work work work work” lyrics with something that actually makes sense?
Well, we have. That’s why we’re so excited about SubWoofer, an app that uses image recognition and a library of song lyrics to generate suitable lyrics for whatever song you choose. Just copy and paste a Youtube URL into the box and go!
WHY WE ❤ IT
Besides being a super polished and usable hack, we also love that SubWoofer lets us experience music in a whole new way. Check out SubWoofer’s GitHub repo to try the app for yourself!
HOW YOU DO IT
Clarifai: What inspired your idea for SubWoofer?
To those in the know, Darude's Sandstorm might go down in history as the greatest musical composition of the 21st century. Despite its repute, its lack of lyrics could leave it untouched by future generations. What if we could generate lyrics for songs using image recognition and a library of other song lyrics? This is what inspired us t0 make SubWoofer at HackFSU.
What did you use to build your app?
We built this at HackFSU 16 using Node.js in the backend, and Vue.js in the frontend. We used FFMPEG from within Node.js to chop up the videos and generate freeze frames. We compiled a lot of rap and pop lyrics and removed all the adjectives and nouns from them to create templates, then we used WordPOS to classify the tags into parts of speech and Sentencer.js to put together sentences from the templates and the now categorized tags.
We ran into some issues generating the subtitle files. Initially, we generated SRT format subtitle files and attempted to embed them into the video itself via FFMPEG. When that failed, we had to modify the process to instead generate VTT subtitle files so we could serve them on the web using HTML5 video’s closed captioning feature. We then ran into an issue with tracks from previous videos being stacked on the video element. The solution was a single line of JQuery but we were far too exhausted to figure it out after being awake 20+ hours and with the submission deadline looming.
We also ran into the Clarifai request limit for the free tier by testing a bit too hard (Initially, we uploaded the entire video, rather than just a few freeze frames). We solved this by swapping keys a couple times, and by asking for a temporary limit increase.
What was the best part about working with Clarifai?
It was pretty easy to use. Tags were pretty solid for the most part. The fact that it takes video is pretty cool. But it does eat through the requests because it’s video.
Thanks for sharing, musical prodigies!
To learn more, check out our documentation and sign-up for a free Clarifai account to start using our API - all it takes is three lines of code to get up and running! We’re super excited to share all the cool things built by our developer community, so don’t forget to tweet @Clarifai to show us your apps.
And give the Triple J’s (Juan, Jacob, and Jess!) some props in the comments below. Until next time!