ZegoCharacterHelper instructions
Overview
To simplify the integration process of the initialization, serialization, data caching, path splicing, and resource mapping features during virtual avatar creation, the ZegoAvatar SDK provides the open-source ZegoCharacterHelper class to help developers create virtual avatars quickly. When you use ZegoCharacterHelper, go to SDK downloads to obtain the related code of the ZegoCharacterHelper class.
The ZegoCharacterHelper class encapsulates IZegoCharacter. If the ZegoCharacterHelper class is used, no APIs related to IZegoCharacter need to be called.
Implementation steps
1 Optional: Preload resources
To preload resources, do this before initializing the ZegoCharacterHelper:
First, upload the absolute path of the resource packages. For details, see Import resource packages.
Call the
preload
method, and pass the absolute path of the model resource to preload the resource to reduce the time required to initialize ZegoCharacterHelper later.
- base.bundle: Head model
- human.bundle: Full-body model
The following is an example of the preloading of a header model. You can select the required model to be loaded according to the requirements of the scenario.
// Get the absolute path to the art resource (the path can be customized).
NSString *resPath = [[[NSBundle mainBundle] bundlePath] stringByAppendingString:@"/assets/base.bundle"];
// Preload resources.
[ZegoCharacterHelper preload:resPath];
2. Initialize ZegoCharacterHelper
- Upload the absolute paths of the aesthetics resource packages. For more information, see Import resource packages.
- Call the
init
API and introduce the absolute path of the resource packages to initialize the ZegoCharacterHelper class.
// Define ZegoCharacterHelper.
@property (nonatomic, strong) ZegoCharacterHelper *characterHelper;
// Obtain the absolute path of aesthetics resource packages. The path can be customized.
NSString *baseRes = [[[NSBundle mainBundle] bundlePath] stringByAppendingString:@"/assets/base.bundle"];
// Initialize ZegoCharacterHelper.
_characterHelper = [[ZegoCharacterHelper alloc] init:resPath];
3. Set the default avatar
When no virtual avatar data exists, you can call the setDefaultAvatar
method to set a default virtual avatar.
- Male role: MODEL_ID_MALE
- Female role: MODEL_ID_FEMALE
// MODEL_ID_MALE is used as an example to set the appearance.
[_characterHelper setDefaultAvatar:MODEL_ID_MALE];
4. Create a virtual avatar view
- After ZegoCharacterHelper is initialized, call the
createAvatarView
API to create an avatar view and configure the width, height, and other information of the view. - Call the
setCharacterView
API, introduce avatarView, and associate it with the view of the virtual avatar.
/// Create AvatarView.
_avatarView = [[ZegoAvatarService sharedInstance] createAvatarView:CGRectMake(0, 0, width, height)];
[self.view addSubview:_avatarView];
// Associate with AvatarView and display the avatar on the screen.
[_characterHelper setCharacterView:avatarView];
In this case, a default virtual avatar is created.
Adjust the appearance of the virtual avatar
After the basic virtual avatar is created, refer to the following content to adjust the appearance of the virtual avatar.
Set the face shape
Call the setFaceShape
API and introduce the faceshapeID (adjustable facial dimension), value (avatar creation coefficient), and other parameters to set or modify the dimensions of facial parts. For more information about adjustable facial dimensions, see Manual avatar creation. We also describe all adjustable facial dimensions (faceshapeID) in helper/ZegoCharacterHelper.h.
//** Set the eyebrow thickness.*/
[_characterHelper setFaceShape:FACESHAPE_BROW_SIZE_Y value:0.3];
Set and obtain the hair color
Call the setHairColor
API to set the hair color of the virtual avatar.
The first parameter indicates the color of the hair root, and the second parameter indicates the color of the hair tail. The valid values are red, green, blue, and alpha.
You can set the same or different colors for the hair root and tail. If different colors are set, the color gradually changes from the hair root to the hair tail.
[_characterHelper setHairColor:[ZegoColor redColor] endColor:[ZegoColor blueColor]];
To obtain the hair color, call the getHairColor
API. The hair color array is returned.
0 indicates the color of the hair root, and 1 indicates the color of the hair tail. The valid values are red, green, blue, and alpha.
NSArray<ZegoColor *> *colorList = [_characterHelper getHairColor];
Set and obtain the skin color
)
// Set the coordinates of the skin color for the virtual avatar. For coordinate values, see the color picking card.
[_characterHelper setSkinColorCoordinates:CGPointMake(0.5, 0.8)];
// Obtain the coordinates of the skin color for the virtual avatar.
CGPoint coodr = [_characterHelper getSkinColorCoordinates];
Automatic avatar generation
To automatically generate a virtual avatar based on an image, perform the following steps:
- After the image is prepared, call the detectFaceFeature API and introduce the UIImage to generate a virtual avatar.
- After a virtual avatar is created, call the
applyFaceFeature
API and introduce the feature parameter to set the dimensions of facial parts.
// Extract facial features based on the introduced image.
ZegoFaceFeature *feature = [[[ZegoAvatarService sharedInstance] getInteractEngine] detectFaceFeature:image];
// Set the appearance of the avatar based on facial features.
[_characterHelper applyFaceFeature:feature];
- During automatic avatar generation, you do not need to set the default gender or other JSON data in advance because the data will be generated automatically after the API is called.
- Before automatic avatar generation, you need to set
ExtendPackagesPath
in advance to prevent the related resources from failing to be found and the automatic avatar generation effect failing to be restored.
Set facial expressions
- Before you start the facial expression detection, ensure that the camera permission is enabled.
- If the ZegoCharacterHelper class is used, no APIs related to IZegoCharacter need to be called.
After the basic virtual avatar is created, call the startDetectExpression API, set the drive mode to ZegoExpressionDetectModeCamera
, and use the front camera to detect facial expressions. Then, use the setExpression
API of ZegoCharacterHelper to set facial expressions and drive facial expression changes of the virtual avatar.
// Start facial expression detection.
___weak typeof(self) weakSelf = self;
BOOL ret = [[[ZegoAvatarService sharedInstance] getInteractEngine] startDetectExpression:ZegoExpressionDetectModeCamera callback:^(ZegoExpression *expression) {
// Drive mouth shape changes of the virtual avatar.
__strong typeof(self) strongSelf = weakSelf;
[strongSelf.characterHelper setExpression: expression];
}];
Styles and materials
If the ZegoCharacterHelper class is used, no APIs related to IZegoCharacter need to be called. Even though IZegoCharacter also has the setPackage API, do not call it directly. If you call the IZegoCharacter API skipping the ZegoCharacterHelper class, cache at the ZegoCharacterHelper layer will become untrusted.
Before changing the makeup or clothing for the virtual avatar, perform the following operations:
- If you set the Packages import method to dynamic downloading, call the
setExtendPackagesPath
API of ZegoCharacterHelper to set the Packages download directory to the downloadPath parameter before using the packages to facilitate resource indexing. The downloadPath parameter needs to be specified to the folder of Packages, for example,/Documents/downloads/Package/
. - Call the setPackage API and introduce the packageID to adjust the appearance of the virtual avatar. The packageID parameter indicates the resource that needs to be set. For more information, see Styles and materials.
// Ensure that the external directory of Packages is set before the API is called.
[_characterHelper setExtendPackagesPath:downloadPath]
/** Call the Styles and materials API.*/
NSString *packageID = @"facepaint1"; // Ensure that this directory is under the downloadPath directory.
[_characterHelper setPackge:packageID];
Obtain appearance data
After appearance data is set, you can call the getAvatarJson
API of the ZegoCharacterHelper class to obtain JSON data of the virtual avatar and store the data to the user profile on the server end for restoration.
NSString *json = [_characterHelper getAvatarJson];
Restore appearance data
If you call the getAvatarJson
API of the ZegoCharacterHelper class to save the appearance data of the virtual avatar, you can call the setAvatarJson
API and introduce the data to restore the personalized virtual avatar.
[_characterHelper setAvatarJson:json];
Play animation
To play the animation, call the playAnimation API, and pass the corresponding Key parameter.
For more resources, you can go to SDK downloads for downloading.
[_characterHelper playAnimation:@"walk"];
The following shows the supported animation keys:
Animation Key | Description |
---|---|
relax | Relax |
walk | Walking |
run | Running |
think | Thinking |
Enable/Disable animation
To enable animation, call the enableAnimation API and pass YES
. To disable it, pass NO
instead.
[_characterHelper enableAnimation:NO];
Set view angles
To set view angles, call the setViewport API. The parameter description is as follows:
Parameter | Description |
---|---|
ZegoAvatarViewPortType_FullBody | Full-length portrait |
ZegoAvatarViewPortType_HalfBody | Half-length portrait |
ZegoAvatarViewPortType_Head | Head portrait |
// Set full-length portrait.
[_characterHelper setViewport:ZegoAvatarViewPortType_FullBody];
Set default expression
- First, call the
setDefaultExpression
method, and pass the default expression data file path (the path can be customized. The default expression data files can be obtained in the SDK download. - Call the
enableDefaultExpression
method to control whether to enable the default expression. **Please ensure that you have calledsetDefaultExpression
to pass in the correct default expression data, otherwise, the default expression will not be displayed properly. **
// Get the absolute path to the default emoji data file (path can be customized)
NSString *expressionPath = [[[NSBundle mainBundle] bundlePath] stringByAppendingString:@"/ZADefaultExpressionData.txt"];
// Set the default expression.
[_characterHelper setDefaultExpression: expressionPath];
// Enable the default expression.
[_characterHelper enabelDefaultExpression: YES];
Export Avatar texture
After creating a basic virtual avatar, you can directly call the startCaptureAvatar
method, and pass AvatarCaptureConfig (configures the width and height of the exported texture) to start exporting the Metal texture.
You can also refer to Using avatars when publishing streams to use avatars when publishing streams.
- To export the Avatar Texture, don’t use the
setAvatarView
method, that is, do not use AvatarView for rendering, otherwise it may be unable to obtain the Avatar texture to publish the stream. - When exporting the Avatar Texture, it is equivalent to doing offline rendering, the previous Avatar will not be rendered to the screen.
// Start exporting Avatar Texture.
// Set the [captureWidth] and [captureHeight] of the content returned by the Avatar as required.
AvatarCaptureConfig* config = [[AvatarCaptureConfig alloc] initWithWidth:captureWidth height:captureHeight];
@weakify(self);// Resolve Self intersecting loop problems.
[self.helper startCaptureAvatar:config callback:^(unsigned long long texture, int width, int height) {
// The Texture ID, width, and height.
}];
// Stop exporting Avatar Texture.
[self.helper stopCaptureAvatar];
- 1 Overview
- 2 Implementation steps
- 2.1 1 Optional: Preload resources
- 2.2 2. Initialize ZegoCharacterHelper
- 2.3 3. Set the default avatar
- 2.4 4. Create a virtual avatar view
- 3 Adjust the appearance of the virtual avatar
- 3.1 Set the face shape
- 3.2 Set and obtain the hair color
- 3.3 Set and obtain the skin color
- 3.4 Automatic avatar generation
- 3.5 Set facial expressions
- 3.6 Styles and materials
- 3.7 Obtain appearance data
- 3.8 Restore appearance data
- 3.9 Play animation
- 3.10 Enable/Disable animation
- 3.11 Set view angles
- 3.12 Set default expression
- 3.13 Export Avatar texture