MSC3291: Muting in VoIP calls (#3291)
* MSC: Muting in VoIP calls Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Update MSC number Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Missing : Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Fix MSC number in prefix Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Fix unstable prefix table Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Reword pottential issues Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Add an Alternatives section Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Remove trailing comma Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> * Fix a typo Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> * Fix missing word Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Link to MSC for holding Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Update unstable prefixes Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Simplify things Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Be explicit about deps Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Be clearer about how things work Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Update proposals/3291-muting.md Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> * `disabled` -> `enabled` Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Fix client mention Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Improve explaination Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Be more precise Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> * Fix negation Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> * Don't refer to something that doesn't exist Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --------- Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>travis/msc/strict-user-id-grammar
parent
f7b3903e3b
commit
7ad992abe5
@ -0,0 +1,130 @@
|
|||||||
|
# MSC3291: Muting in VoIP calls
|
||||||
|
|
||||||
|
During VoIP calls, it is common for a user to mute their microphone/camera.
|
||||||
|
Ideally, the other side should be able to see that the opponent's camera is
|
||||||
|
muted, so that it could reflect this in the UI (e.g. show the user's avatar
|
||||||
|
instead of their camera feed). We would also want the changes in the mutes state
|
||||||
|
to be quick.
|
||||||
|
|
||||||
|
Using pure WebRTC there are two ways to do muting and both have their issues:
|
||||||
|
|
||||||
|
+ Disabling the corresponding track
|
||||||
|
+ Setting the corresponding track as `recvonly`/`inactive`
|
||||||
|
|
||||||
|
The Alternatives section describes the issues with using these alone.
|
||||||
|
|
||||||
|
## Proposal
|
||||||
|
|
||||||
|
This MSC proposes extending the `sdp_stream_metadata` object (see
|
||||||
|
[MSC3077](https://github.com/matrix-org/matrix-doc/pull/3077)) to allow
|
||||||
|
indicating the mute state to the other side using the following fields:
|
||||||
|
|
||||||
|
+ `audio_muted` - a boolean indicating the current audio mute state
|
||||||
|
+ `video_muted` - a boolean indicating the current video mute state
|
||||||
|
|
||||||
|
This MSC also adds a new call event `m.call.sdp_stream_metadata_changed`, which
|
||||||
|
has the common VoIP fields as specified in
|
||||||
|
[MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746) (`version`,
|
||||||
|
`call_id`, `party_id`) and a `sdp_stream_metadata` object which is the same
|
||||||
|
thing as `sdp_stream_metadata` in `m.call.negotiate`, `m.call.invite` and
|
||||||
|
`m.call.answer`. The client sends this event when the `sdp_stream_metadata` has
|
||||||
|
changed but no negotiation is required (e.g. the user mutes their
|
||||||
|
camera/microphone).
|
||||||
|
|
||||||
|
All tracks should be assumed unmuted unless specified otherwise.
|
||||||
|
|
||||||
|
Clients are recommended to not mute the audio of WebRTC tracks locally when a
|
||||||
|
incoming stream has the `audio_muted` field set to `true`. This is because when the
|
||||||
|
other user unmutes themselves, there may be a slight delay between their client
|
||||||
|
sending audio and the `m.call.sdp_stream_metadata_changed` event arriving. If
|
||||||
|
`enabled` is set to `false`, then any audio sent in between those two events
|
||||||
|
will not be heard. The other user will still stop transmitting audio once they
|
||||||
|
mute on their side, so no audio is sent without the user's knowledge.
|
||||||
|
|
||||||
|
The same suggestion does not apply to `video_muted` - there clients _should_
|
||||||
|
mute video locally, so that the receiving side doesn't see black video.
|
||||||
|
|
||||||
|
### Example
|
||||||
|
|
||||||
|
```JSON
|
||||||
|
{
|
||||||
|
"type": "m.call.sdp_stream_metadata_changed",
|
||||||
|
"room_id": "!roomId",
|
||||||
|
"content": {
|
||||||
|
"version": "1",
|
||||||
|
"call_id": "1414213562373095",
|
||||||
|
"party_id": "1732050807568877",
|
||||||
|
"sdp_stream_metadata": {
|
||||||
|
"2311546231": {
|
||||||
|
"purpose": "m.usermedia",
|
||||||
|
"audio_muted:": true,
|
||||||
|
"video_muted": true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
This event indicates that both audio and video are muted. It is suggested the
|
||||||
|
video track of stream `2311546231` should be hidden in the UI (probably replaced
|
||||||
|
by an avatar). It also suggests the UI should show an indication that the audio
|
||||||
|
track is muted but the client should not mute the audio on the receiving side.
|
||||||
|
|
||||||
|
## Potential issues
|
||||||
|
|
||||||
|
When the user mutes their camera, some browsers may keep sending meaningless data
|
||||||
|
which will waste bandwidth.
|
||||||
|
|
||||||
|
## Alternatives
|
||||||
|
|
||||||
|
### Only disabling the corresponding track
|
||||||
|
|
||||||
|
This is the solution that some clients (e.g. Element Android) use at the moment.
|
||||||
|
While this is almost instantaneous, it doesn't allow the other side to know the
|
||||||
|
opponent's mute state. This leads to the opponent showing a black screen for a
|
||||||
|
muted video track and not doing anything for a muted audio track which is bad
|
||||||
|
for UX.
|
||||||
|
|
||||||
|
### Setting the corresponding track as `recvonly`/`inactive`
|
||||||
|
|
||||||
|
While this would be beneficial for low bandwidth connections, it takes time. The
|
||||||
|
delay might be acceptable for video but isn't for audio (with which you would
|
||||||
|
assume an instantaneous mute state change). This is also problematic since there
|
||||||
|
could be a confusion with holding (as defined in
|
||||||
|
[MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746)).
|
||||||
|
|
||||||
|
### Using a separate event for muting
|
||||||
|
|
||||||
|
While this might feel clearer initially, it doesn't have much real benefit. The
|
||||||
|
mute state is in fact a meta information about the stream and using
|
||||||
|
`sdp_stream_metadata` is also more flexible for cases where the user joins a
|
||||||
|
call already muted. It is also more flexible in general and would be useful if
|
||||||
|
we ever decided to do what is described in the next section.
|
||||||
|
|
||||||
|
### A combination of disabling tracks, `sdp_stream_metadata` and SDP
|
||||||
|
|
||||||
|
An option would be using the current method in combination with setting the
|
||||||
|
corresponding track as `recvonly`/`inactive`. Along with this clients would need
|
||||||
|
to set the mute state in `sdp_stream_metadata` to avoid conflicts with holding
|
||||||
|
(as defined in [MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746)).
|
||||||
|
While this solution might be the most flexible solution as it would allow
|
||||||
|
clients to choose between bandwidth and a mute state change delay for each
|
||||||
|
track, it would be harder to implement and feels generally disjointed.
|
||||||
|
|
||||||
|
## Security considerations
|
||||||
|
|
||||||
|
None that I can think of.
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
+ [MSC3077](https://github.com/matrix-org/matrix-doc/pull/3077)
|
||||||
|
|
||||||
|
## Unstable prefix
|
||||||
|
|
||||||
|
|Release |Development |
|
||||||
|
|------------------------------------|---------------------------------------------|
|
||||||
|
|`m.call.sdp_stream_metadata_changed`|`org.matrix.call.sdp_stream_metadata_changed`|
|
||||||
|
|`sdp_stream_metadata` |`org.matrix.msc3077.sdp_stream_metadata` |
|
||||||
|
|
||||||
|
We use an unstable prefix for `sdp_stream_metadata` to match
|
||||||
|
[MSC3077](https://github.com/matrix-org/matrix-doc/pull/3077).
|
Loading…
Reference in New Issue