Microsoft introduced the VASA-1 neural network for “revitalizing” people’s portraits
VASA-1 allows you to animate a photo or even a drawing of a person’s face by overlaying facial expressions and voice. Microsoft has published many examples of the algorithm’s work on its website.
Upon close examination, some video artifacts and inconsistencies can be noticed, but even that is enough for malicious parties to create deepfakes with the intent to deceive people. Therefore, the company will not publish the API until it is confident that the technology “will be used responsibly and in accordance with the appropriate requirements”.
The authors of this technology believe it will be useful in education and for socializing people who have communication difficulties.
The VASA-1 neural network was trained on the VoxCeleb2 dataset, which includes “over 1 million statements from 6112 celebrities” extracted from YouTube videos.