What would it sound like if Billie Eilish explained the GameStop short squeeze? What if Bob Dylan sang Britney Spears’ “…Baby One More Time”? Using artificial intelligence and machine learning, these bizarre queries are becoming a dystopian reality as fans, hobbyists and scammers mimic the voices of their favorite stars.
For singers and rappers, these so-called “deepfake” vocals also present a looming threat as the technology rapidly improves, and the line becomes less discernible between re-creation and reality. Currently, most musical deepfakes are created in jest or homage by music fans, but however well-intentioned, deepfake vocals are almost always unauthorized uses of an artist’s voice. Because the human voice is not covered under copyright law and the underriding technology is so novel, this places AI vocals in a state of legal limbo, leaving artists vulnerable to identity theft and with no compensation for the use of their likeness.
Though there is evidence to suggest “publicity” or “personality” rights may provide some pathway to legal recourse for the commercial use of a public figure’s voice (like Tom Waits v. Frito Lay, a 1992 lawsuit in which the singer sued Frito Lay for the unauthorized impersonation his voice for a commercial and won), these rights vary from state-to-state and still have yet to address a case specific to the new frontier of AI-rendered vocals.
But for experimental electronic musician Holly Herndon, who has been working at the intersection of music and artificial intelligence for years, including a doctoral stint at Stanford University, she sees this space as a brave new world for musicians. Instead of fighting to destroy the inevitable behemoth of AI, Herndon is showing others how to control it and even use it to their advantage by releasing her new AI voice instrument, Holly+.
Created with start-up Never Before Heard Sounds and longtime collaborator Mat Dryhurst, Holly+ is a cloud-based instrument that allows users to upload up to five minutes of audio and map a rendering of Herndon’s voice over top. Programmed through the amalgamation of hours of Herndon’s voice recordings, the Holly+ instrument solves a few of the biggest challenges with AI voice renderings, most notably, the ownership and monetization of one’s digital likeness. “We’re really trying to figure out robust, foundational logic for how to deal with this technology through Holly+,” says Dryhurst.
Instead of sticking a price tag to Herndon’s vocal likeness, the team decided to make the instrument available and free to all and to monetize the project in a way that wouldn’t exclude any users in the process. “We truly want people to play around with it,” Herndon explains. “Because I feel like for a lot of people, it’s kind of mysterious how this works. Of course, it still is somewhat mysterious, but people can at least get a better sense of the technology.”
The groundbreaking monetization process of Holly+ has already begun. For the past week, users of Holly+ are able to upload creations made with the tool to be considered for the official Holly+ NFT (non fungible token) collection. This will be the first of multiple Holly+ NFT collections, and the submission process closes tonight. Like a digital gallery of AI-based musical works, Herndon and other team members will sift through submissions and choose their favorites to showcase in the collection which will be sold on the NFT marketplace Zora. This allows Herndon and the team the power to choose which songs are given official value, something that Dryhurst compares to the current practice of authorizing remixes. “A similar model already exists there. The official remixes made, authorized by the artist themselves, are more valuable because they are recognized. I think this will become the standard.”
The NFT collection also offers a rare opportunity for users to be supported by the artist who created the tool. “It’s a marketplace provided by the artist themselves. I think that could also be an interesting future model for fan engagement,” says Herndon. Covering the costs of minting the officially selected NFTs and providing a platform for users, Holly+’s method of monetization lowers barriers of entry for users and allows Herndon control over the use of her voice. “There’s a couple ways of dealing with this. You could have crazy punitive digital rights management, or you could have a completely free tool for all. We’re trying to find the happy medium,” Herndon explains.
Proceeds from the sale of the Holly+ NFT collection would be distributed 50% to the creator, 40% to the investment of future tools (governed by a decentralized autonomous organization or “DAO”), and 10% to Herndon directly as a royalty for the use of her likeness. “This is no get rich quick scheme,” Herndon says.
“But this model promises to solve a bunch of issues in terms of official attribution and revenue,” Dryhurst adds.
Although this model compensates Herndon for the attribution of her voice and the creator using the instrument, it still does not remunerate Never Before Heard Sounds or Dryhust directly for their contributions to the project. “We’re still trying to figure out what the business model is,” admits Chris Deaner, co-founder of Never Before Heard Sounds. “A lot of this project is proof of concept and gaging what the appetite is from users. With Holly+ we’ve found there truly is a lot of interest.”
Instead, for now, the music tech up start will remain a “lean team,” of just Deaner and co-founder Yotam Mann, continually testing new products and determining the path to monetization along the way. Mann and Deaner will also participate, along with Dryhust, as members of the DAO and will have voting power in the creation of future tools and in the curation of the NFT collection.
The monetization of AI voice models is still in its infancy, but Holly+ offers novel solutions to the growing and inevitable use of machine learning in music. “This is a conversation starter, and this is something we are trying to figure out in public,” says Herndon. “I think there are going to be some uncomfortable moments that come up, and we’ll just have to figure it out together.” Mann adds, “eventually, we think it’s going to revolutionize music.” The auction for the first set of Holly+ NFTs will take place this September.
Source
