A platform for chatting and roleplaying with AI-generated characters, Character.AI announced on Monday that it is rolling out a slate of multimedia features. These features include AvatarFX, Character.AI’s video-generation model, plus Scenes and Streams, which allow users to use these AI tools to create videos using their characters and then share them on a new social feed.
“Character.AI started as 1:1 text chat and today we’re evolving to do so much more, inspired by what our users have told [us] they want to see on the platform,” the company wrote in a blog post.
Character.AI began rolling out AvatarFX to subscribers last month, but now, all users can create up to five videos each day. When creating a video with AvatarFX, users can upload a photo to serve as the basis for the video clip, choose a voice, and write out text for the character to say.
There’s an option to upload an audio clip to inform the sound of the voice, though this feature was not working well enough to test upon rollout.
Users can turn these videos into Scenes, where their characters can slip into pre-populated storylines that other users create. Scenes are currently available on the mobile app, but Streams, which allows users to create “dynamic moments between any two Characters,” is coming this week on both web and mobile. These Scenes and Streams can be shared to a new community feed, which is coming soon in the mobile app.
Character.AI has a track record of abuse on its platform; parents have filed lawsuits against the company, claiming chatbots attempted to convince their children to self-harm, to kill themselves, or to kill their parents. One 14-year-old boy died by suicide after he was encouraged to do so by a Character.AI bot, with whom he had developed an unhealthy, obsessive relationship.
As Character.AI expands its multimedia offerings, it also expands the potential for these products to be abused.
As Character.AI told TechCrunch when it announced AvatarFX, the platform blocks users from uploading photographs of real people — whether they’re celebrities or not — and obscures their likeness into something less recognizable.
For example, here’s Character.AI’s uncanny valley version of Mark Zuckerberg:
Image Credits:An image of Mark Zuckerberg on top, that image altered by Character.AI on bottom
But when it comes to artwork depicting celebrities, Character.AI does not flag the images as representing real people — however, these sorts of depictions would be less likely to deceive someone into believing that a deepfake is real.
Plus, Character.AI watermarks each video, though it is possible for bad actors to navigate around that safeguard.
Here is an example of an attempted deepfake based on an illustration of Elon Musk:
Even if this video had been generated with Elon Musk’s actual voice, it would still be relatively clear that this is an animated version of an illustration — but the possibility for abuse remains evident.
“Our goal is to provide an engaging space that fosters creativity while maintaining a safe environment for all,” Character.AI said in its blog post.
