- Documentation
- AI Effects
- Guides
- Background segmentation
Background segmentation
The ZegoEffects SDK provides various background segmentation features that can be used in a wide range of use cases, such as online education, video conferencing, short videos, and more.
Prerequisites
Before you begin to use the SDK's background segmentation feature in your project, make sure you complete the following steps:
- Integrate the ZegoEffects SDK into your project. For details, see Integration.
- Implement the basic image processing functionality. For details, see Implementation.
- Import the
SegmentationModel
. For details, see Import models.
Implementation steps
The SDK supports two background segmentation methods: portrait segmentation and green screen segmentation. For each segmentation method, you can also enable the background blurring or background mosaicing feature as needed.
Portrait segmentation
To use the portrait segmentation feature, import the model file SegmentationModel
first; otherwise, this feature can't take effect. For details, see Import resources and models.
Do either one of the following:
Call the
setPortraitSegmentationBackgroundPath
method to set the background path,Call the
setPortraitSegmentationBackgroundBuffer
method to set the background buffer.// Method 1: Set the background path for portrait segmentation. [self.effects setPortraitSegmentationBackgroundPath: @"MY_BACKGROUND_PATH" mode:ZegoEffectsScaleModeAspectFill]; // Method 2: Set the background buffer for portrait segmentation. [self.effects setPortraitSegmentationBackgroundBuffer:pixelbuffer mode:ZegoEffectsScaleModeAspectFill];
Call the
enablePortraitSegmentation
method to enable the portrait segmentation feature.// Enable the portrait segmentation feature. [self.effects enablePortraitSegmentation:YES];
After enabling portrait segmentation, you can also enable background blurring or background mosaicing as needed.
Background blurring with portrait segmentation
- Call the
enablePortraitSegmentationBackgroundBlur
method to enable background blurring with portrait segmentation. - Call the
setPortraitSegmentationBackgroundBlurParam
method to set theintensity
property ofZegoEffectsBlurParam
to adjust the extent of background blurring.
// Enable background blurring with portrait segmentation.
[self.effects enablePortraitSegmentationBackgroundBlur:YES];
// Set the background blurring parameters.
ZegoEffectsBlurParam *param = [[ZegoEffectsBlurParam alloc] init];
param.intensity = 100;
[self.effects setPortraitSegmentationBackgroundBlurParam:param];
Background mosaicing with portrait segmentation
- Call the
enablePortraitSegmentationBackgroundMosaic
method to enable background mosaicing with portrait segmentation. - Call the
setPortraitSegmentationBackgroundMosaicParam
method to set theintensity
property ofZegoEffectsMosaicParam
to adjust the extent of background mosaicing.
// Enable background mosaicing with portrait segmentation.
[self.effects enablePortraitSegmentationBackgroundMosaic:YES];
// Set the background mosaicing parameters.
ZegoEffectsMosaicParam *param = [[ZegoEffectsMosaicParam alloc] init];
param.intensity = 100;
[self.effects setPortraitSegmentationBackgroundMosaicParam:param];
Green screen segmentation
Do either one of the following:
Call the
setChromaKeyBackgroundPath
method to set the background path.Call the
setChromaKeyBackgroundBuffer
method to set the background background buffer.// Method 1: Set the background path for green screen segmentation. [self.effects setChromaKeyBackgroundPath: @"MY_BACKGROUND_PATH" mode:ZegoEffectsScaleModeAspectFill]; // Method 2: Set the background buffer for green screen segmentation. [self.effects setChromaKeyBackgroundBuffer:pixelbuffer mode:ZegoEffectsScaleModeAspectFill];
Call the
enableChromaKey
method to enable the green screen segmentation feature, and then call thesetChromaKeyParam
method to set the chrome key parameters to adjust the color tolerance, edge smoothness index, and others.// Enable the green screen segmentation feature. [self.effects enableChromaKey:YES] // Set the green screen segmentation parameters. ZegoEffectsChromaKeyParam *param = [[ZegoEffectsChromaKeyParam alloc] init]; [self.effects setChromaKeyParam:param];
After enabling green screen segmentation, you can also enable background blurring or background mosaicing as needed.
Background blurring with green screen segmentation
- Call the
enableChromaKeyBackgroundBlur
method to enable background blurring with green screen segmentation. - Call the
setChromaKeyBackgroundBlurParam
method to set theintensity
property ofZegoEffectsBlurParam
to adjust the extent of background blurring.
// Enable background blurring with green screen segmentation.
[self.effects enableChromaKeyBackgroundBlur:YES];
// Set the background blurring parameters.
ZegoEffectsBlurParam *param = [[ZegoEffectsBlurParam alloc] init];
param.intensity = 100;
[self.effects setChromaKeyBackgroundBlurParam:param];
Background mosaicing with green screen segmentation
- Call the
enableChromaKeyBackgroundMosaic
method to enable the background mosaicing feature. - Call the
setChromaKeyBackgroundMosaicParam
method to set theintensity
property ofZegoEffectsMosaicParam
to adjust the extent of background mosaicing.
Enabling the background mosaicing feature will override any previously set custom background and background blurring feature.
// Enable background mosaicing with green screen segmentation.
[self.effects enableChromaKeyBackgroundMosaic:YES];
// Set the background mosaicing parameters.
ZegoEffectsMosaicParam *param = [[ZegoEffectsMosaicParam alloc] init];
param.intensity = 100;
[self.effects setChromaKeyBackgroundMosaicParam:param];