Facial expression mirroring
Overview
The ZegoAvatar SDK provides the facial expression mirroring
feature. Based on accurate recognition of face key points and 52 basic facial expression dimensions, including the face, tongue, and eyeball, the feature can capture facial expressions on users' faces and restore and render them on the virtual avatars in real time.
This feature can be widely used in social interaction and voice chat live streaming scenarios.
Prerequisites
Before you implement the facial expression mirroring
feature, ensure that the following conditions are met:
- Integrate the ZegoAvatar SDK into your project. For more information, see Integrate the SDK.
- Create a basic virtual avatar by referring to Create a virtual avatar.
Implementation process
Refer to the following procedures to implement the facial expression mirroring
feature.
1. Start facial expression detection
- Before you start the facial expression detection, ensure that the camera permission is enabled.
- If the ZegoCharacterHelper class is used, no APIs related to IZegoCharacter need to be called.
After the basic virtual avatar is created, call the startDetectExpression API, set the drive mode to Camera
, and use the front camera to detect facial expressions. Then, use the setExpression
API of ZegoCharacterHelper to set facial expressions and drive facial expression changes of the virtual avatar.
// Start facial expression detection.
ZegoAvatarService.getInteractEngine().startDetectExpression(ZegoExpressionDetectMode.Camera, expression -> {
// Ensure that mCharacterHelper is created, AvatarView is set, and a default avatar is set by running setDefaultAvatar or setAvatarJson.
// Drive facial expression changes of the virtual avatar.
mCharacterHelper.setExpression(expression);
});
2. Stop facial expression detection
Run the application in the backend or exit the current page. Call the stopDetectExpression API to stop facial expression detection.
// Stop facial expression detection.
ZegoAvatarService.getInteractEngine().stopDetectExpression();