Implement basic image processing

This document describes how to implement basic image processing with the ZegoEffects SDK.

Prerequisites

Before implementing the basic image processing functionality, make sure you complete the following steps:

• Integrate the ZegoEffects SDK into your project. For more information, see Quick starts - Integration.
• Get the unique license file of the SDK. For details, see Online privilege authentication.
• A project has been created in ZEGOCLOUD Console and applied for a valid AppID and AppSign. For details, please refer to Console - How to view project information .

Implementation steps

The following diagram shows the API call sequence of basic image processing with the ZegoEffects SDK:

Create a ZegoEffects object

1. Import the AI resources and models.

   To use the SDK's AI features, you must import the necessary AI resources or models by calling the setResources method. For details, see Quick starts - Import resources and models.

   // Specify the absolute path of the face recognition model, which is required for various features including Face detection, eyes enlarging, and face slimming.
   NSString *faceDetectionModelPath = [[NSBundle mainBundle] pathForResource:@"FaceDetectionModel" ofType:@"model"];
   
   // Specify the absolute path of the portrait segmentation model, which is required for AI portrait segmentation.
   NSString *segmentationModelPath = [[NSBundle mainBundle] pathForResource:@"SegmentationModel" ofType:@"model"];
   
   NSArray<NSString *> * modelList = @[faceDetectionModelPath, segmentationModelPath];
   
   // Set the list of model paths, which must be called before calling the create method.
   [ZegoEffects setResources:resourcesList];

2. Deploy Advanced Configuration

   Call the setAdvancedConfig interface to deploy advanced configuration items, such as configuring device performance levels. For details, please refer to Configure Device Performance Level.

   // Set advancedConfig
   ZegoEffectsAdvancedConfig *config = [ZegoEffectsAdvancedConfig alloc];
   // Configure device performance level
   [ZegoEffects setAdvancedConfig:config];

3. Create Effects Object

   Pass the AppID and AppSign obtained from the Prerequisites directly to the create interface. After internal authentication by the SDK, it will create an Effects object and return the corresponding error code.

   NSInteger appId = *******;
   NSString *appSign = @"********";
   self.effects = [ZegoEffects create:appid appSign:appSign];

Initialize the ZegoEffects object

1. Call the initEnv method to initialize the ZegoEffects object, passing in the width and height of the original image to be processed.

   // Initialize the ZegoEffects object, passing in the width and height of the original image to be processed.
   [self.effects initEnv:CGSizeMake(1280, 720)];

2. Call the following methods to enable the AI features you want to use.

   • enableWhiten

   • enableBigEyes

   • setPortraitSegmentationBackgroundPath

   • enablePortraitSegmentation

     // 1. Enable the skin tone enhancement feature.
     [self.effects enableWhiten:YES];
     
     // 2. Enable the eyes enlargeing feature.
     [self.effects enableBigEyes:YES];
     
     // 3. Enable the AI portrait segmentation feature, passing in the absolute path of the segmented background image.
     [self.effects setPortraitSegmentationBackgroundPath: @"MY_BACKGROUND_PATH" mode:ZegoEffectsScaleModeAspectFill];
     [self.effects enablePortraitSegmentation:YES];

Perform image processing

Call the processImageBuffer method to perform image processing, passing in the original video image to be processed, and the processed result will be written back to the original buffer.

// Pass in the original video image to be processed, and the processed result will be written back to the original buffer.
[self.effects processImageBuffer:pixelBuffer];