ZEGO Avatar
  • iOS : Objective-C
  • Android
  • Overview
  • Client SDKs
  • Demo app
  • Getting started
    • Integrate the SDK
    • Create a virtual avatar
    • ZegoCharacterHelper instructions
  • Guides
  • Best practice
  • Error codes
  • API Documents
  • FAQ

Facial expression mirroring

Last updated:2022-06-29 12:12

Overview

The ZegoAvatar SDK provides the facial expression mirroring feature. Based on accurate recognition of face key points and 52 basic facial expression dimensions, including the face, tongue, and eyeball, the feature can capture facial expressions on users' faces and restore and render them on the virtual avatars in real time. This feature can be widely used in social interaction and voice chat live streaming scenarios.

Prerequisites

Before you implement the facial expression mirroring feature, ensure that the following conditions are met:

Implementation process

Refer to the following procedures to implement the facial expression mirroring feature.

1. Start facial expression detection

  • Before you start the facial expression detection, ensure that the camera permission is enabled.
  • If the ZegoCharacterHelper class is used, no APIs related to IZegoCharacter need to be called.

After the basic virtual avatar is created, call the startDetectExpression API, set the drive mode to ZegoExpressionDetectModeCamera, and use the front camera to detect facial expressions. Then, use the setExpression API of ZegoCharacterHelper to set facial expressions and drive facial expression changes of the virtual avatar.

// Start facial expression detection.
___weak typeof(self) weakSelf = self;
BOOL ret = [[[ZegoAvatarService sharedInstance] getInteractEngine] startDetectExpression:ZegoExpressionDetectModeCamera callback:^(ZegoExpression *expression) {
    // Drive facial expression changes of the virtual avatar.
    __strong typeof(self) strongSelf = weakSelf;
    [strongSelf.characterHelper setExpression: expression];
}];

2. Stop facial expression detection

Run the application in the backend or exit the current page. Call the stopDetectExpression API to stop facial expression detection.

// Stop facial expression detection.
[[[ZegoAvatarService sharedInstance] getInteractEngine] stopDetectExpression];