MSC3291: Muting in VoIP calls (#3291)
* MSC: Muting in VoIP calls Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Update MSC number Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Missing : Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Fix MSC number in prefix Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Fix unstable prefix table Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Reword pottential issues Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Add an Alternatives section Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Remove trailing comma Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> * Fix a typo Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> * Fix missing word Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Link to MSC for holding Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Update unstable prefixes Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Simplify things Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Be explicit about deps Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Be clearer about how things work Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Update proposals/3291-muting.md Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> * `disabled` -> `enabled` Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Fix client mention Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Improve explaination Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Be more precise Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> * Fix negation Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> * Don't refer to something that doesn't exist Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --------- Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>travis/msc/strict-user-id-grammar
Šimon Brandner
10 months ago
committed by
Travis Ralston
parent
f7b3903e3b
commit
7ad992abe5
@ -0,0 +1,130 @@
|
||||
# MSC3291: Muting in VoIP calls
|
||||
|
||||
During VoIP calls, it is common for a user to mute their microphone/camera.
|
||||
Ideally, the other side should be able to see that the opponent's camera is
|
||||
muted, so that it could reflect this in the UI (e.g. show the user's avatar
|
||||
instead of their camera feed). We would also want the changes in the mutes state
|
||||
to be quick.
|
||||
|
||||
Using pure WebRTC there are two ways to do muting and both have their issues:
|
||||
|
||||
+ Disabling the corresponding track
|
||||
+ Setting the corresponding track as `recvonly`/`inactive`
|
||||
|
||||
The Alternatives section describes the issues with using these alone.
|
||||
|
||||
## Proposal
|
||||
|
||||
This MSC proposes extending the `sdp_stream_metadata` object (see
|
||||
[MSC3077](https://github.com/matrix-org/matrix-doc/pull/3077)) to allow
|
||||
indicating the mute state to the other side using the following fields:
|
||||
|
||||
+ `audio_muted` - a boolean indicating the current audio mute state
|
||||
+ `video_muted` - a boolean indicating the current video mute state
|
||||
|
||||
This MSC also adds a new call event `m.call.sdp_stream_metadata_changed`, which
|
||||
has the common VoIP fields as specified in
|
||||
[MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746) (`version`,
|
||||
`call_id`, `party_id`) and a `sdp_stream_metadata` object which is the same
|
||||
thing as `sdp_stream_metadata` in `m.call.negotiate`, `m.call.invite` and
|
||||
`m.call.answer`. The client sends this event when the `sdp_stream_metadata` has
|
||||
changed but no negotiation is required (e.g. the user mutes their
|
||||
camera/microphone).
|
||||
|
||||
All tracks should be assumed unmuted unless specified otherwise.
|
||||
|
||||
Clients are recommended to not mute the audio of WebRTC tracks locally when a
|
||||
incoming stream has the `audio_muted` field set to `true`. This is because when the
|
||||
other user unmutes themselves, there may be a slight delay between their client
|
||||
sending audio and the `m.call.sdp_stream_metadata_changed` event arriving. If
|
||||
`enabled` is set to `false`, then any audio sent in between those two events
|
||||
will not be heard. The other user will still stop transmitting audio once they
|
||||
mute on their side, so no audio is sent without the user's knowledge.
|
||||
|
||||
The same suggestion does not apply to `video_muted` - there clients _should_
|
||||
mute video locally, so that the receiving side doesn't see black video.
|
||||
|
||||
### Example
|
||||
|
||||
```JSON
|
||||
{
|
||||
"type": "m.call.sdp_stream_metadata_changed",
|
||||
"room_id": "!roomId",
|
||||
"content": {
|
||||
"version": "1",
|
||||
"call_id": "1414213562373095",
|
||||
"party_id": "1732050807568877",
|
||||
"sdp_stream_metadata": {
|
||||
"2311546231": {
|
||||
"purpose": "m.usermedia",
|
||||
"audio_muted:": true,
|
||||
"video_muted": true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This event indicates that both audio and video are muted. It is suggested the
|
||||
video track of stream `2311546231` should be hidden in the UI (probably replaced
|
||||
by an avatar). It also suggests the UI should show an indication that the audio
|
||||
track is muted but the client should not mute the audio on the receiving side.
|
||||
|
||||
## Potential issues
|
||||
|
||||
When the user mutes their camera, some browsers may keep sending meaningless data
|
||||
which will waste bandwidth.
|
||||
|
||||
## Alternatives
|
||||
|
||||
### Only disabling the corresponding track
|
||||
|
||||
This is the solution that some clients (e.g. Element Android) use at the moment.
|
||||
While this is almost instantaneous, it doesn't allow the other side to know the
|
||||
opponent's mute state. This leads to the opponent showing a black screen for a
|
||||
muted video track and not doing anything for a muted audio track which is bad
|
||||
for UX.
|
||||
|
||||
### Setting the corresponding track as `recvonly`/`inactive`
|
||||
|
||||
While this would be beneficial for low bandwidth connections, it takes time. The
|
||||
delay might be acceptable for video but isn't for audio (with which you would
|
||||
assume an instantaneous mute state change). This is also problematic since there
|
||||
could be a confusion with holding (as defined in
|
||||
[MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746)).
|
||||
|
||||
### Using a separate event for muting
|
||||
|
||||
While this might feel clearer initially, it doesn't have much real benefit. The
|
||||
mute state is in fact a meta information about the stream and using
|
||||
`sdp_stream_metadata` is also more flexible for cases where the user joins a
|
||||
call already muted. It is also more flexible in general and would be useful if
|
||||
we ever decided to do what is described in the next section.
|
||||
|
||||
### A combination of disabling tracks, `sdp_stream_metadata` and SDP
|
||||
|
||||
An option would be using the current method in combination with setting the
|
||||
corresponding track as `recvonly`/`inactive`. Along with this clients would need
|
||||
to set the mute state in `sdp_stream_metadata` to avoid conflicts with holding
|
||||
(as defined in [MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746)).
|
||||
While this solution might be the most flexible solution as it would allow
|
||||
clients to choose between bandwidth and a mute state change delay for each
|
||||
track, it would be harder to implement and feels generally disjointed.
|
||||
|
||||
## Security considerations
|
||||
|
||||
None that I can think of.
|
||||
|
||||
## Dependencies
|
||||
|
||||
+ [MSC3077](https://github.com/matrix-org/matrix-doc/pull/3077)
|
||||
|
||||
## Unstable prefix
|
||||
|
||||
|Release |Development |
|
||||
|------------------------------------|---------------------------------------------|
|
||||
|`m.call.sdp_stream_metadata_changed`|`org.matrix.call.sdp_stream_metadata_changed`|
|
||||
|`sdp_stream_metadata` |`org.matrix.msc3077.sdp_stream_metadata` |
|
||||
|
||||
We use an unstable prefix for `sdp_stream_metadata` to match
|
||||
[MSC3077](https://github.com/matrix-org/matrix-doc/pull/3077).
|
||||
Loading…
Reference in New Issue