Techno storm as AI Stefanie Sun goes viral on Bilibili

Singaporean singer Stefanie Sun would probably never have imagined that she would bask in the limelight again in this manner.

Her voice is the centerpiece of a series of music videos generating a buzz on Bilibili.com, a gen-z-dominated Chinese video platform, recently. But she hasn't sung these songs. They were generated entirely by artificial intelligence software.

Such videos are generated by so-vits-svc fork, an open source software developed on GitHub that enables anyone to train their own AI model to speak in any voice and language.

One of the uploader 子鱼 has garnered millions of views on the few AI-generated videos he posted. Many left comments saying they were astonished by the fidelity of Sun's voice generated by AI.

子鱼 has previously made a series of AI-generated music videos of pop singer David Zee Tao and Jay Chou, but none of their video' views can quite match those of the AI Stefanie Sun series.

"It probably has something to do with Sun's distinctive voice and the way she articulates while singing. Her voice has a grainy quality and a unique breath. These qualities somehow matched the AI model, which is probably why videos using Sun's voice have gained the most views," 子鱼 speculated.

According to the introduction on Github, the current version of this project has iterated to version 4.0. Compared with previous projects such as VITS, soft-vc, and VISinger2, the use of Sovits is very simplified, and users can train the desired acoustic model with just a few audio segments.

However, the audio has to be a "dry sound". In a song, it is the "original sound" after removing the instruments, accompaniment, reverberation, and harmony.

In order to make the AI's singing more believable, the trainer must also collect the singer's "dry sound" performance at various pitches. "So generally speaking, the AI works of singers with a wide vocal range will be more similar," 子鱼 explained.

Out of curiosity, he has replaced many songs with his own voice, and sung in other languages such as English and Japanese.

"When I first heard my own voice, I had goosebumps all over my body. It was fun, but I also felt a little scared," 子鱼 said.

How singers and the music industry take it?

Many people, including the singers themselves, feel fearful. Taiwan crooner Sandee Chan revealed on social media last month that her new song "Teach Me How to Be Your Lover," released on March 14, was actually sung by "AI Sandee Chan".

The news shocked the music industry because no one had noticed it before. In fact, some fans even commented that Chan's voice seemed to have "regained her youth," and her singing level in the number was better than her recent performances.

A well-trained "AI singer" can deceive the vast majority of people. Chan later revealed her intention, saying that she had hoped the song could make people who care about creativity think.

If the AI era is inevitable, then maybe creators should not care more about "whether we will be replaced," but "what else we can do," she pointed out.

What if people intentionally use "AI singers" for evil purposes?

Fake singing, which was already rampant, becomes even harder to detect, the threshold for singers' profession becomes lower, and copyright infringements regarding a singer's voice become more frequent.

On the other hand, "AI singers" can be used positively.

By restoring the voices of deceased singers such as Teresa Teng, Leslie Cheung, and Michael Jackson, audiences can mourn them. At a broader level, this could also "resurrect" the voices of those who have passed away, providing comfort to their fans and loved ones.

The emergence of "AI singers" has the potential to serve the public or become an accomplice to crimes. Ultimately, it depends on the choices users make.

Do AI-generated music videos constitute infringement?

Li Qiang, a copyright lawyer who has worked with many music streaming platforms, believes that AI enthusiasts will argue that training AI is purely for fun and does not harm the rights of singers.

However, copyright holders consider using unauthorized singer or data to train AI as an act of infringement.

The use of AI to simulate a singer's voice without his or her authorization, and then displaying it publicly, should be considered an infringement of the singer's rights, Li argued.

Some musicians hold an open attitude towards "AI cloning." For example, Canadian singer Grimes openly welcomes people to use AI to imitate her voice for creative purposes, with a 50/50 split in royalties.

Chinese rapper Xiao Laohu also openly said that if someone cloned his voice to sing another song, "it's not a bad thing, maybe we can become friends."

"Compared to 'protection,' I am more interested in how to cooperate with AI. I think it will be difficult to 'create' without cooperating with AI in the future," he insisted.

Regarding further improvement of regulations, industry analyst An Laining believes that it will definitely require joint efforts from lawmakers and copyright owners, as well as countless lawsuits that follow, to promote the establishment of relevant AI legislation, regulations, and new industry norms.