Set mixed-stream layout
When there are multiple users sending audio, video, and whiteboard streams in a room, our Cloud Recording service mixes the streams of all the users into one stream. This document describes how to set the video layout in the mixed-stream recording mode.
We currently support the following video layouts:
- Customized layout: You can customize the video layout as needed, and this canvas can display up to 25 users.
- Grid layout: The number of columns and rows and the grid size vary according to the number of users in the room. This layout supports up to 25 users.
- Horizontal layout: One large region is displayed on the bottom edge of the canvas, and several small regions are displayed along the top edge of the canvas. The space on the top supports up to two rows of small regions with eight regions per row. And when the small regions are more than two rows, the left regions are displayed on the right edge of the canvas. This layout supports up to 25 users.
- Vertical layout: One large region is displayed on the left edge of the canvas, and several small regions are displayed along the right edge of the canvas. The space on the right supports up to two columns of small regions with eight regions per column. And when the small regions are more than two columns, the left regions are displayed on the bottom edge of the canvas. This layout supports up to 25 users.
- Floating layout: The first user in the room occupies the full canvas, the other users occupy the small regions on the top of the canvas, starting from the bottom right corner (every small region is the same size). This layout supports up to 25 users.
For an audio-only stream, it still occupies a small region.
Implementation
To set video layout, set the MixMode
parameter when calling the StartRecord
method.
During the mixed-stream recording, you can change the layout by calling the UpdateLayout
method.
The following figures describe several layouts in detail. The numbers in the diagram represent the order of users joining the room.
Grid layout
- As shown in the diagrams, the region that is not occupied by any user shows the background color.
- If you have not set the fill-in parameter: The video region will be replaced by the next user when there is a user leaving the room. If you set the fill-in parameter: The video region will be kept and the image displays the last video image of the user when he leaves the room.
- If whiteboard recording is enabled, the whiteboard region will always occupy region 1.
- If the aspect ratio of a user's video does not match that of the user's region, the video is cropped to fit the region.
The following diagrams show the actual layouts with different number of users in the room:
)
)
)
)
)
In grid layout, video regions are layout by column preferentially.
Horizontal layout
To specify the large region in the horizontal layout, set the MixMaxResolutionStreamId
parameter when calling the calling the StartRecord
method and UpdateLayout
methods.
- If you don't specify a user (by setting the
MixMaxResolutionStreamId
), the large region shows the video of the user who joins the room first. - If you have specified a user and the specified user doesn't publish a stream (by setting the
MixMaxResolutionStreamId
), the large region shows the background color. - The region that is not occupied by any user shows the background color.
- If you have not set the fill-in parameter: The video region will be replaced by the next user when there is a user leaving the room. If you set the fill-in parameter: The video region will be kept and the image displays the last video image of the user when he leaves the room.
- If whiteboard recording is enabled, the whiteboard region will always occupy the large region when the
MixMaxResolutionStreamId
parameter is invalid. - If the aspect ratio of the specified user's video does not match that of the large region, the video is scaled to fit the region, ensuring the completeness of the video. If the aspect ratio of a user's video does not match that of the user's region, the video is cropped to fit the region.
The following diagrams show the actual layouts with different number of users in the room:
)
)
)
)
Vertical layout
To specify the large region in the vertical layout, set the MixMaxResolutionStreamId
parameter when calling the calling the StartRecord
method and UpdateLayout
methods.
For detailed notes, refer to the Horizontal layout.
The following diagrams show the actual layouts with different number of users in the room:
)
)
)
)
Floating layout
To specify the large region in the floating layout, set the MixMaxResolutionStreamId
parameter when calling the calling the StartRecord
method and UpdateLayout
methods.
For detailed notes, refer to the Horizontal layout.
The following diagrams show the actual layouts with different number of users in the room:
)
)
)
)
Customize the video layout
To set the size and position of each user's region on the canvas, set the MixInputList
parameter when calling the StartRecord
method and UpdateLayout
methods.
The MixInputList
is an array in JSON format that consists of the configurations of the user regions. You can configure up to 25 regions.
The parameters are defined as follows:
Parameter | Type | Required | Description |
---|---|---|---|
StreamId |
String | No | A string that contains the stream ID of the user you want to display in the region. If this parameter is not specified, the configurations apply in the order of the users joining the room. |
ViewType |
Int | No | The type of content displayed on canvas.
|
Top |
Int | Yes | The relative vertical position of the top-left corner of the region. The value range is [0, 1920]. |
Left |
Int | Yes | The relative vertical position of the top-left corner of the region. The value range is [0, 1920]. |
Bottom |
Int | Yes | The relative height of the region. The value range is [0, 1920]. |
Right |
Int | Yes | The relative width of the region. The value range is [0, 1920]. |
Layer |
Int | Yes | The layer property, determines the layer level. The larger the value, the higher the layer level. |
FillMode |
Int | No | The video display mode when the actual video aspect ratio is inconsistent with the canvas.
|
- If you have specified a user and the specified user doesn't publish a stream (by setting the
StreamId
), the region shows the background color. - The region that is not occupied by any user shows the background color.
- If you have not set the fill-in parameter and
StreamId
: The video region will be replaced by the next user when there is a user leaving the room. - If you have not set the fill-in parameter, but specified the
StreamId
: The video region displays the background color. - If you set the fill-in parameter: The video region will be kept and the image displays the last video image of the user when he leaves the room.
- If whiteboard recording is enabled, the whiteboard occupies the region that with the
ViewType
is "2".
As shown in the following coordinate system figure:
)
Sample request
You can update the video layout during a recording.
Content-type
: application/json; charset=utf-8
The request example of UpdateLayout
is as follows:
{
"TaskId": "xxxx",
"MixMode": 1,
"MixInputList": [
{
"StreamId": "xxxx",
"ViewType": 1,
"Top": 0,
"Left": 0,
"Bottom": 720,
"Right": 540,
"Layer": 1
},
{
"StreamId": "",
"ViewType": 1,
"Top": 0,
"Left": 540,
"Bottom": 360,
"Right": 1080,
"Layer": 1
},
{
"StreamId": "",
"ViewType": 1,
"Top": 360,
"Left": 540,
"Bottom": 720,
"Right": 1080,
"Layer": 1
}
]
}