Google Created An AI That Can Generate Music From Text Descriptions, But Wont Release It
Google's awesome new AI system can generate any genre of music with a text description. However, fearing the risks, the company does not intend to launch it immediately.
Called MusicLM, Google certainly isn't the first AI system to create songs. There are other efforts, including Riffusion, an artificial intelligence that imagines music, as well as Dance Diffusion, Google's AudioML, and OpenAI's Jukebox. However, due to technical limitations and limited training data, no one has been able to create particularly sophisticated compositions or high-fidelity songs.
MusicLM may be the first.
In a detailed academic paper, MusicLM was trained on a database of 280,000 hours of music and learned to create composite songs for "highly complex" images (such as a solo singer or "90s Berlin techno"). they look like something a human artist might create, but not necessarily inventive or musically cohesive.
It's hard to judge how well the samples sound because there are no musicians or instrumentalists. Even when fed long, rushed descriptions, MusicLM manages to capture nuances like instrumental riffs, melodies, and moods.
The title of the show below, for example, includes the phrase "the experience of being lost in space," and it certainly delivers on that front (at least to my ears).
Here's another example created from a description starting with the phrase "Arcade game soundtrack". Probably right?
MusicLM's capabilities go beyond creating short snippets of songs. Google researchers have shown that the system can be built on existing melodies, whether humming, singing, whistling or playing an instrument. What's more, MusicLM can take different descriptions written in sequence (such as "time to meditate", "time to wake up", "time to run", "time to give yourself 100%" and some sort of melodic "story". Or perfect for a movie soundtrack. a few minutes of story that fits.
See below what stands out from the categories "electronic song played in a video game", "meditation song played along the river", "fire", "fireworks".
That's not all. MusicLM can also be instructed by a combination of description and title, or create a sound that is "played" by a specific type of instrument in a specific genre. Even the experience level of the AI "musician" can be set, and the system can create music inspired by places, times or requirements (for example, motivational music for workouts).
But MusicLM isn't perfect, far from it to be honest. Some examples demonstrate quality distortion, an inevitable side effect of the learning process. While MusicLM can technically produce vocals, including choral harmonies, they leave a lot to be desired. Most of the "lyrics" range from English to pure gibberish performed by synthesized voices that sound like mixes by different artists.
However, Google researchers point to many ethical issues with systems like MusicLM, including incorporating copyrighted material from training data into the generated songs. During testing, they found that about 1% of the music produced by the system is copied directly from the songs it runs on, a threshold that seems high enough to prevent them from releasing MusicLM in its current state.
"We accept the potential risk of misuse of creative content related to the use case," the paper's co-authors wrote. "We emphasize the need for further future work to address these risks associated with music generation."
If MusicLM or a similar system exists, major legal challenges seem inevitable, even if the systems present themselves as tools to help artists rather than replace them. They already have some, but around simpler AI systems. In 2020, Jay-Z's record company filed copyright claims against YouTube channel Vocal Synthesis for using artificial intelligence to cover Jay-Z songs such as Billy Joel's "We Didn't Start the Fire." After initially removing the videos, YouTube restored them and determined that the removal requests were "incomplete." But grassroots music still finds itself on murky legal ground.
A white paper written by Eric Sunray, a legal expert at the Association of Music Publishers, claims that AI music generators such as MusicLM infringe on music copyrights by "creating composite audio tapestries from compositions received during training, thereby violating US copyright law and fair reproduction." violates After Jukebox's release, critics also questioned whether it was appropriate to train AI models on copyrighted musical material. Similar concerns have been raised about the training data used in AI systems that generate images, code and text that are often scraped from the internet. without the knowledge of the creators.
From the user's perspective, Waxy's Andy Baio suggests that music created by an AI system should only be considered a derivative work protected by the copyright of the original elements. Of course, it is not clear what can be considered "original" in such music. To commercially exploit this music is to enter uncharted waters. Using the music created for purposes protected by fair use, such as parodies and interpretations, is a simpler matter, but Baio expects the courts to decide on a case-by-case basis.
It will not be long before the matter is clarified. Several lawsuits are likely to affect music-producing AI, including claims over the rights of artists whose work is used to power AI systems without their knowledge or consent. But time will tell.
