MSC3077: Support for multi-stream VoIP (#3077)
* Draft of multi-stream MSC Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Remove unnecessary article and don't use CamelCase Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Fix naming and MSC number Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Make it prefixy Co-authored-by: Tulir Asokan <tulir@maunium.net> * Be more descriptive about keys Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Add more info about usermedia and screenshare Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Simplifie backwards compatibility I would be tempted to skip the capability advertisement for this and just say that the absence of stream metadata means clients just take the first or whatever MSC2746 says. I can't think of anything the capability advertisement caters for specifically? - Dave Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Reword parts of backwards compatibility section Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Improve explanation of backwards compatibility Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Add missing spaces to unstable perifix table Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Remove support for stream-replacement Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Apply suggestions from code review Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> * Fix concerns Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> * Link to specific spec version Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> * Be more readable Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> * Clarify Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> * Don't ref non-existing thing Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> * Remove confusing words --------- Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com> Co-authored-by: Tulir Asokan <tulir@maunium.net> Co-authored-by: David Baker <dbkr@users.noreply.github.com> Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>pull/4038/head
parent
08b3b62e03
commit
641faf2e5d
@ -0,0 +1,111 @@
|
|||||||
|
# MSC3077: Support for multi-stream VoIP
|
||||||
|
|
||||||
|
This MSC proposes a method for differentiating WebRTC streams from each other.
|
||||||
|
|
||||||
|
[MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746) has improved VoIP
|
||||||
|
immeasurably. Yet, there is still no clear way to handle things such as
|
||||||
|
screen-sharing.
|
||||||
|
|
||||||
|
Simple VoIP calls only ever feature one stream, though often clients will want
|
||||||
|
to send multiple - usermedia, screensharing and possibly more. In a situation
|
||||||
|
with more streams, it can be very helpful to provide the other side with
|
||||||
|
metadata about the content of the streams.
|
||||||
|
|
||||||
|
## Proposal
|
||||||
|
|
||||||
|
This MSC proposes adding an `sdp_stream_metadata` field to the events containing
|
||||||
|
a session description i.e.:
|
||||||
|
|
||||||
|
+ [`m.call.invite`](https://spec.matrix.org/v1.7/client-server-api/#mcallinvite)
|
||||||
|
+ [`m.call.answer`](https://spec.matrix.org/v1.7/client-server-api/#mcallanswer)
|
||||||
|
+ [`m.call.negotiate`](https://spec.matrix.org/v1.7/client-server-api/#mcallnegotiate)
|
||||||
|
|
||||||
|
The `sdp_stream_metadata` field is an object in which each key is one stream
|
||||||
|
`id` in the session description. The values are objects with the
|
||||||
|
following fields:
|
||||||
|
|
||||||
|
+ `purpose` - a string indicating the purpose of the stream. For compatibility
|
||||||
|
between client the following values are defined:
|
||||||
|
+ `m.usermedia` - stream that contains the webcam and/or microphone tracks
|
||||||
|
+ `m.screenshare` - stream with the screen-sharing tracks
|
||||||
|
|
||||||
|
### Example
|
||||||
|
|
||||||
|
```JSON
|
||||||
|
{
|
||||||
|
"type": "m.call.invite",
|
||||||
|
"room_id": "!roomId",
|
||||||
|
"content": {
|
||||||
|
"call_id": "1414213562373095",
|
||||||
|
"invitee": "@bob:matrix.org",
|
||||||
|
"party_id": "1732050807568877",
|
||||||
|
"lifetime": "60000",
|
||||||
|
"capabilities": {
|
||||||
|
"m.call.transferee": true,
|
||||||
|
},
|
||||||
|
"offer": {
|
||||||
|
"sdp": "...",
|
||||||
|
"type": "offer",
|
||||||
|
},
|
||||||
|
"sdp_stream_metadata": {
|
||||||
|
"271828182845": {
|
||||||
|
"purpose": "m.screenshare",
|
||||||
|
},
|
||||||
|
"314159265358": {
|
||||||
|
"purpose": "m.usermedia",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"version": "1",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Edge cases
|
||||||
|
|
||||||
|
+ If an incoming stream is not described in `sdp_stream_metadata` and
|
||||||
|
`sdp_stream_metadata` is present, the stream should be ignored.
|
||||||
|
+ If a stream has a `purpose` of an unknown type (i.e. not `m.usermedia` or
|
||||||
|
`m.screenshare`), it should be ignored.
|
||||||
|
|
||||||
|
### Backwards compatibility
|
||||||
|
|
||||||
|
During the initial invite and answer exchange clients find out if the field
|
||||||
|
`sdp_stream_metadata` is missing. If it is not present in the event sent by the
|
||||||
|
opponent, the client should ignore any new incoming streams (i.e. it should use
|
||||||
|
the first one) and it shouldn't send more than one stream (i.e. clients cannot send a video feed and a screenshare at the same time, as is the case in current clients).
|
||||||
|
|
||||||
|
## Alternatives
|
||||||
|
|
||||||
|
Setting the stream `id`s to custom values had been considered. Though this is
|
||||||
|
possible on some platforms, it is not in browsers. That is because the `id`
|
||||||
|
property of `MediaStream` is _read-only_ as the [MDN Documentation
|
||||||
|
states](https://developer.mozilla.org/en-US/docs/Web/API/MediaStream/id).
|
||||||
|
Similar is true for SDP attributes.
|
||||||
|
|
||||||
|
This proposal is also more practical for cases where more complex metadata is
|
||||||
|
needed. For conferencing, a `user_id` field could be added to
|
||||||
|
the objects in `sdp_stream_metadata`; for differentiating between the front and rear camera of a
|
||||||
|
phone, a `camera_type` field could be added.
|
||||||
|
|
||||||
|
Previously, it has been thought that the `purpose` field has to be unique (or
|
||||||
|
another unique field has to be added), though this could only ever be important
|
||||||
|
if we wanted to replace a stream with another one in-place. It was deemed as a
|
||||||
|
rather uncommon thing for which there doesn't seem to be any use-case, so
|
||||||
|
uniqueness is not required.
|
||||||
|
|
||||||
|
## Unstable prefix
|
||||||
|
|
||||||
|
During development, the following fields should be used:
|
||||||
|
|
||||||
|
|Release |Development |
|
||||||
|
|----------------------------|-----------------------------------------------|
|
||||||
|
|`sdp_stream_metadata` |`org.matrix.msc3077.sdp_stream_metadata` |
|
||||||
|
|`m.call.sdp_stream_metadata`|`org.matrix.msc3077.call.sdp_stream_metadata` |
|
||||||
|
|
||||||
|
## Potential issues
|
||||||
|
|
||||||
|
None that I can think of.
|
||||||
|
|
||||||
|
## Security considerations
|
||||||
|
|
||||||
|
None that I can think of.
|
Loading…
Reference in New Issue