Facial expression mirroring

• Overview

The ZegoAvatar SDK provides the facial expression mirroring feature. Based on accurate recognition of face key points and 52 basic facial expression dimensions, including the face, tongue, and eyeball, the feature can capture facial expressions on users' faces and restore and render them on the virtual avatars in real time. This feature can be widely used in social interaction and voice chat live streaming scenarios.

Prerequisites

Before you implement the facial expression mirroring feature, ensure that the following conditions are met:

• Integrate the ZegoAvatar SDK into your project. For more information, see Integrate the SDK.
• Create a basic virtual avatar by referring to Create a virtual avatar.

Implementation process

Refer to the following procedures to implement the facial expression mirroring feature.

1. Start facial expression detection

After the basic virtual avatar is created, call the startDetectExpression API, set the drive mode to Camera, and use the front camera to detect facial expressions. Then, use the setExpression API of ZegoCharacterHelper to set facial expressions and drive facial expression changes of the virtual avatar.

// Start facial expression detection.
ZegoAvatarService.getInteractEngine().startDetectExpression(ZegoExpressionDetectMode.Camera, expression -> {
     // Ensure that mCharacterHelper is created, AvatarView is set, and a default avatar is set by running setDefaultAvatar or setAvatarJson.
    // Drive facial expression changes of the virtual avatar.
     mCharacterHelper.setExpression(expression);
});

2. Stop facial expression detection

Run the application in the backend or exit the current page. Call the stopDetectExpression API to stop facial expression detection.

// Stop facial expression detection.
ZegoAvatarService.getInteractEngine().stopDetectExpression();