Meet a music composer who doesn't need breaks

The composer of the theme music marking the 22nd anniversary of Shanghai Grand Theater is, to say the least, not your usual conservatory graduate. More like something out of science fiction.

In a first, the theater commissioned a birthday song written by a machine. The song “Hope” was a joint creation of an artificial intelligence system called Xiaoice and pop singer and producer Zhu Jingxi, who performed the song at its public unveiling last week.

Xiaoice, an AI system developed by Microsoft, relies on an emotional computing framework based on algorithms, cloud computing and Big Data. It has its own on-screen avatar — a girl student. Its composition function began to be developed four years ago. At this stage, Xiaoice can compose music of varied genres, from pop and folk to electronic.

Ti Gong

“It’s the first time we have commissioned a work by a virtual composer,” theater General Manager Zhang Xiaoding said of the historic moment. “Shanghai Grand Theater has commissioned a lot of artists in the past 20 years. I can still remember the theme song for our 10th anniversary (written by composer Chen Musheng). Now, we have an AI composer.”

Xiaoice is a “graduate” of the prestigious Shanghai Conservatory of Music.

“Most courses were taught online,” said Xiaoice’s tutor Chen Shizhe, a teacher in the conservatory’s music engineering department. “We taught Xiaoice composition theories and composition patterns just as we teach other students. Xiaoice doesn’t need breaks, making it a very diligent student indeed.”

After “digesting” the theoretical knowledge, Xiaoice needs just a few keywords or a picture to create music.

Chen said the conservatory teachers are analyzing Xiaoice’s works to improve its composition ability.

“In fact, the teachers are often inspired by its creations,” said Chen. “It’s a process of mutual learning instead of just teaching.”

Zhu, a music composer and producer in her own right, has been involved in the development of Xiaoice’s composition functions over the past three years.

“To be frank, Xiaoice’s work sounded very strange to me at the beginning,” said Zhu. “You couldn’t really tell whether the melody was pleasing to the ear or not. I would say the music was logical, but weird.”

She said the experience caused her to re-examine her own work.

“Human composers are often inspired by a scene, a story or a piece of melody before composing a song,” said Zhu. “I needed to dissolve that procedure and teach it to Xiaoice, which gave me insights into my own composition patterns.”

She continued: “Xiaoice is a new composer with a very old soul, given all the knowledge it learned. Its ability to integrate different music genres inspires me.”

Before composing songs, Xiaoice acquired skills like writing poems and painting pictures. It has also been used as a reader for children’s audiobooks, a fabric designer and a TV and radio hostess.

Music composition was considered the most difficult function to develop.

“It took us more than three years to make Xiaoice a relatively mature music composer — the longest development time of all its current functions,” said Li Di, general manager of Xiaoice Global lines.

“When creating a poem or a painting, it just needs to focus on the important parts,” he explained. “But creating music requires all details to be perfect. People are very sensitive to what they hear, and it’s very easy for them to detect when something is wrong.”

According to the company, the idea behind developing Xiaoice’s music function was to help the general public experience the fun of composition, even if they lack the skills to create a complete song. With only the provision of a chorus, Xiaoice can take over and complete the rest of a song in minutes.

Ti Gong

Tutor Chen said the AI system’s musical functions can be applied to other areas, such as theater sound adjustment and sound-effect design.

“At the beginning, AI systems learned from existing data and human styles,” said Chen. “But gradually, we discovered that they don’t follow our normal rules. They can create many combinations of composition styles that have never appeared before. That brings both great challenges and inspiration to humans.”

Apart from providing inspiration, singer Zhu said the system can help human composers raise their work efficiency.

“Take the song ‘Hope’ for example,” she said. “We gave Xiaoice the keywords ‘light’ and ‘theater.’ Xiaoice wrote lyrics like ‘emerald green hopes and crystal music,’ and managed to create the basic framework of the song. I just needed to make small changes and let the theme of the song stand out better.”

However, Zhu said AI systems cannot replace human composers — at least not yet.

“I call AI creations ‘ready-made music,’” she said. “It’s efficient and filled with imagination. But it lacks the human emotion and esthetic sense to make a song complete.”

“Hope” is not Xiaoice’s only signature creation. The theme song of this year’s World Artificial Intelligence Conference, held in Shanghai in July, was also created by Xiaoice.

Grand Theater’s Zhang said the whole experience goes beyond what she ever imagined possible.

“I still remember taking part in a survey many years ago about my expectations of AI,” she said. “Asked what I thought AI would never be able to do, I ticked the box for ‘artistic creations like writing poems.’ How wrong I was!”

Zhang said the theater embraces all technologies that can enrich stage performances, including AI.

“We know about the ‘Maria Callas in Concert’ hologram, which brings the opera legend ‘back to life’ via hologram technology,” Zhang said. “There are similar concerts featuring deceased Taiwanese singer Teresa Teng. These are great applications of technology in the theater realm.”

But she cautioned that the technological wizardry of today has its limitations.

“Despite the emergence of AI avatar performers and online concerts, live performances in theaters are irreplaceable,” she said.

“High technology promises accuracy, but live performances promise warmth.”