Cloud Recording
  • Introduction
  • Release notes
  • Getting started
  • Guides
    • Single-stream recording
    • Mixed-stream recording
    • Set mixed-stream layout
    • Multi-sector recording (Whiteboard)
    • Capture screenshots
  • API reference
    • API overview
    • Accessing Server APIs
    • Start recording
    • Stop recording
    • Update the mixed-stream layout
    • Update the whiteboard
    • Query recording status
    • Query the recording task list
    • Pause recording
    • Resume recording
    • Callback of the recording status
    • API testing with Postman
  • Error codes
  • Documentation
  • Cloud Recording
  • Common Features
  • Set the Mixed Stream Layout

Set mixed-stream layout

Last updated:2022-03-31 18:40

When there are multiple users sending audio, video, and whiteboard streams in a room, our Cloud Recording service mixes the streams of all the users into one stream. This document describes how to set the video layout in the mixed-stream recording mode.

We currently support the following video layouts:

  • Customized layout: You can customize the video layout as needed, and this canvas can display up to 25 users.
  • Grid layout: The number of columns and rows and the grid size vary according to the number of users in the room. This layout supports up to 25 users.
  • Horizontal layout: One large region is displayed on the bottom edge of the canvas, and several small regions are displayed along the top edge of the canvas. The space on the top supports up to two rows of small regions with eight regions per row. And when the small regions are more than two rows, the left regions are displayed on the right edge of the canvas. This layout supports up to 25 users.
  • Vertical layout: One large region is displayed on the left edge of the canvas, and several small regions are displayed along the right edge of the canvas. The space on the right supports up to two columns of small regions with eight regions per column. And when the small regions are more than two columns, the left regions are displayed on the bottom edge of the canvas. This layout supports up to 25 users.
  • Floating layout: The first user in the room occupies the full canvas, the other users occupy the small regions on the top of the canvas, starting from the bottom right corner (every small region is the same size). This layout supports up to 25 users.

For an audio-only stream, it still occupies a small region.

Implementation

To set video layout, set the MixMode parameter when calling the StartRecord method.

During the mixed-stream recording, you can change the layout by calling the UpdateLayout method.

The following figures describe several layouts in detail. The numbers in the diagram represent the order of users joining the room.

Grid layout

  • As shown in the diagrams, the region that is not occupied by any user shows the background color.
  • If you have not set the fill-in parameter: The video region will be replaced by the next user when there is a user leaving the room. If you set the fill-in parameter: The video region will be kept and the image displays the last video image of the user when he leaves the room.
  • If whiteboard recording is enabled, the whiteboard region will always occupy region 1.
  • If the aspect ratio of a user's video does not match that of the user's region, the video is cropped to fit the region.

The following diagrams show the actual layouts with different number of users in the room:

In grid layout, video regions are layout by column preferentially.

Horizontal layout

To specify the large region in the horizontal layout, set the MixMaxResolutionStreamId parameter when calling the calling the StartRecord method and UpdateLayout methods.

  • If you don't specify a user (by setting the MixMaxResolutionStreamId), the large region shows the video of the user who joins the room first.
  • If you have specified a user and the specified user doesn't publish a stream (by setting the MixMaxResolutionStreamId), the large region shows the background color.
  • The region that is not occupied by any user shows the background color.
  • If you have not set the fill-in parameter: The video region will be replaced by the next user when there is a user leaving the room. If you set the fill-in parameter: The video region will be kept and the image displays the last video image of the user when he leaves the room.
  • If whiteboard recording is enabled, the whiteboard region will always occupy the large region when the MixMaxResolutionStreamId parameter is invalid.
  • If the aspect ratio of the specified user's video does not match that of the large region, the video is scaled to fit the region, ensuring the completeness of the video. If the aspect ratio of a user's video does not match that of the user's region, the video is cropped to fit the region.

The following diagrams show the actual layouts with different number of users in the room:

Vertical layout

To specify the large region in the vertical layout, set the MixMaxResolutionStreamId parameter when calling the calling the StartRecord method and UpdateLayout methods.

For detailed notes, refer to the Horizontal layout.

The following diagrams show the actual layouts with different number of users in the room:

Floating layout

To specify the large region in the floating layout, set the MixMaxResolutionStreamId parameter when calling the calling the StartRecord method and UpdateLayout methods.

For detailed notes, refer to the Horizontal layout.

The following diagrams show the actual layouts with different number of users in the room:

Customize the video layout

To set the size and position of each user's region on the canvas, set the MixInputList parameter when calling the StartRecord method and UpdateLayout methods.

The MixInputList is an array in JSON format that consists of the configurations of the user regions. You can configure up to 25 regions.

The parameters are defined as follows:

Parameter Type Required Description
StreamId String No A string that contains the stream ID of the user you want to display in the region. If this parameter is not specified, the configurations apply in the order of the users joining the room.
ViewType Int No The type of content displayed on canvas.
  • 1: Audio and video (default)
  • 2: Whiteboard
Type 2 is only valid when whiteboard recording is enabled. And the whiteboard can only occupy one region; otherwise, an error code will be returned.
Top Int Yes The relative vertical position of the top-left corner of the region. The value range is [0, 1920].
Left Int Yes The relative vertical position of the top-left corner of the region. The value range is [0, 1920].
Bottom Int Yes The relative height of the region. The value range is [0, 1920].
Right Int Yes The relative width of the region. The value range is [0, 1920].
Layer Int Yes The layer property, determines the layer level. The larger the value, the higher the layer level.
FillMode Int No The video display mode when the actual video aspect ratio is inconsistent with the canvas.
  • 1: (Default) Cropped mode. In this mode, filling the canvas with the video is preferred. After the original video is scaled to fit the canvas, the video image that exceeds the canvas will be cropped.
  • 2: Scaled to fit mode. In this mode, the completeness of the video is preferred. After the original video is scaled to fit the canvas, a black edge will be filled around the video image.
  • If you have specified a user and the specified user doesn't publish a stream (by setting the StreamId), the region shows the background color.
  • The region that is not occupied by any user shows the background color.
  • If you have not set the fill-in parameter and StreamId: The video region will be replaced by the next user when there is a user leaving the room.
  • If you have not set the fill-in parameter, but specified the StreamId: The video region displays the background color.
  • If you set the fill-in parameter: The video region will be kept and the image displays the last video image of the user when he leaves the room.
  • If whiteboard recording is enabled, the whiteboard occupies the region that with the ViewType is "2".

As shown in the following coordinate system figure:

Sample request

You can update the video layout during a recording.

Content-type: application/json; charset=utf-8

The request example of UpdateLayout is as follows:

{
    "TaskId": "xxxx",
    "MixMode": 1,
    "MixInputList": [
        {
            "StreamId": "xxxx", 
            "ViewType": 1, 
            "Top": 0, 
            "Left": 0, 
            "Bottom": 720, 
            "Right": 540, 
            "Layer": 1
        },
        {
            "StreamId": "", 
            "ViewType": 1, 
            "Top": 0, 
            "Left": 540, 
            "Bottom": 360, 
            "Right": 1080, 
            "Layer": 1
        },
        {
            "StreamId": "", 
            "ViewType": 1, 
            "Top": 360, 
            "Left": 540, 
            "Bottom": 720, 
            "Right": 1080, 
            "Layer": 1
        }
    ]
}
Page Directory