Timo K
1 month ago
parent
fe587a7d33
commit
9cbe4481b7
@ -0,0 +1,150 @@
|
||||
# MSC4143: MatrixRTC
|
||||
|
||||
This MSC defines the modules with which the matrix real time system is build with.
|
||||
|
||||
The MatrixRTC specification is separated into different modules.
|
||||
|
||||
- The MatrixRTC room state that defines the state of the real time application.\
|
||||
It is the source of truth for:
|
||||
- Who is part of a session
|
||||
- Who is connected via what technology/backend
|
||||
- Metadata per device used by other participants to decide whether the streams
|
||||
from this source are of interest / need to be subscribed.
|
||||
- The RTC backend.
|
||||
- It defines how to connect the participating peers.
|
||||
- Livekit is the standard for this as of writing.
|
||||
- Defines how to connect to a server/other peers, how to update the connection,
|
||||
how to subscribe to different streams...
|
||||
- Another planned backend is a full mesh implementation based on MSC3401.
|
||||
- The RTCSession types (application) have their own per application spec.
|
||||
- Calls can be done with an application of type `m.call` see (TODO: link call msc)
|
||||
- The application defines all the details of the RTC experience:
|
||||
- How to interpret the metadata of the member events.
|
||||
- What streams to connect to.
|
||||
- What data in which format to sent over the RTC channels.
|
||||
|
||||
This MSC will focus on the matrix room state which can be seen as the most high
|
||||
level signalling of a call:
|
||||
|
||||
## Proposal
|
||||
|
||||
Each RTC session is made out of a collection of `m.rtc.member` events.
|
||||
Each `m.rtc.member` event defines the application type: `application`
|
||||
and a `call_id`. And is stored in a state event of type `m.rtc.member`.
|
||||
The first element of the state key is the `userId` and the second the `deviceId`.
|
||||
(see [this proposal for state keys](https://github.com/matrix-org/matrix-spec-proposals/pull/3757#issuecomment-2099010555)
|
||||
for context about second/first state key.)
|
||||
|
||||
### The MatrixRTC room state
|
||||
|
||||
Everything required for working MatrixRTC
|
||||
(current session, sessions history, join/leave events, ...) only
|
||||
require one event type.
|
||||
|
||||
A complete `m.rtc.member` state event looks like this:
|
||||
|
||||
```json
|
||||
// event type: m.rtc.member
|
||||
// event key: ["@user:matrix.domain", "DEVICEID"]
|
||||
{
|
||||
"m.application": "m.my_session_type",
|
||||
"m.call_id": "",
|
||||
"focus_active": {...FOCUS_A},
|
||||
"foci_preferred": [
|
||||
{...FOCUS_1},
|
||||
{...FOCUS_2}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
giving us the information, that user: `@user:matrix.domain` with device `DEVICEID`
|
||||
is part of an RTCSession of type `m.call` in the scope/sub-session `""` (empty
|
||||
string as call id) connected over `FOCUS_A`. This is all information that is needed
|
||||
for another room member to detect the running session and join it.
|
||||
|
||||
There is **no event** to represent a session. This event would include shared
|
||||
information where it is not trivial to decide who has authority over it.
|
||||
Instead the session is a computed value based on `m.rtc.member` events.
|
||||
The list of events with the same `m.application` and `m.call_id` represent one session.
|
||||
This array allows to compute fields like participant count, start time ...
|
||||
|
||||
Sending an empty `m.rtc.member` event represents a leave action.
|
||||
Sending a well formatted `m.rtc.member` represents a join action.
|
||||
|
||||
Based on the value of `m.application`, the event might include additional parameters
|
||||
required to provide additional session parameters.
|
||||
|
||||
> A thirdRoom like experience could include the information of an approximate position
|
||||
on the map, so that clients can omit connecting to participants that are not in their
|
||||
area of interest.
|
||||
|
||||
#### Historic sessions
|
||||
|
||||
Since there is no singe entry for a historic session (because of the owner ship discussion),
|
||||
historic sessions need to be computed and most likely cached on the client.
|
||||
|
||||
Each state event can either mark a join or leave:
|
||||
|
||||
- join: `prev_state.m.application != current_state.m.application` &&
|
||||
`prev_state.m.call_id != current_state.m.call_id` &&
|
||||
`current_state.m.application != undefined`
|
||||
(where an empty `m.rtc.member` event would imply `state.m.application == undefined`)
|
||||
- leave: `prev_state.m.application != current_state.m.application` &&
|
||||
`prev_state.m.call_id != current_state.m.call_id` &&
|
||||
`current_state.m.application == undefined`
|
||||
|
||||
Based on this one can find user sessions. (The range between a join and a leave
|
||||
event) of specific times.
|
||||
The collection of all overlapping user sessions with the same `call_id` and
|
||||
`application` define one MatrixRTC history event.
|
||||
|
||||
### The RTC backend
|
||||
|
||||
`foci_active` and `foci_preferred` are used to communicate
|
||||
|
||||
- how a user is connected to the session (`foci_active`)
|
||||
- what connection method this user knows about would like to connect with.
|
||||
|
||||
The only enforced parameter of a `foci_preferred` or `foci_active` is `type`.
|
||||
Based on the focus type a different amount of parameters might be needed to,
|
||||
communicate how to connect to other users.
|
||||
`foci_preferred` and `foci_active` can have different parameters so that it is,
|
||||
possible to use a combination of the two to figure our that everyone is connected
|
||||
with each other.
|
||||
|
||||
Only users with the same type can connect in one session. If a frontend does
|
||||
not support the used type they cannot connect.
|
||||
|
||||
Each focus type will get its own MSC in which the detailed procedure to get from
|
||||
the foci information to working webRTC connections to the streams of all the
|
||||
participants is explained.
|
||||
|
||||
- [`livekit`](www.example.com) TODO: create `livekit` focus MSC and add link here.
|
||||
- [`full_mesh`](https://github.com/matrix-org/matrix-spec-proposals/pull/3401)
|
||||
TODO: create `full-mesh` focus MSC based on[MSC3401](https://github.com/matrix-org/matrix-spec-proposals/pull/3401)
|
||||
and add link here.
|
||||
|
||||
### The RTCSession types (application)
|
||||
|
||||
Each session type might have its own specification in how the different streams
|
||||
are interpreted and even what focus type to use. This makes this proposal extremely
|
||||
flexible. A Jitsi conference could be added by introducing a new `m.application`
|
||||
and a new focus type and would be MatrixRTC compatible. It would not be compatible
|
||||
with applications that do not use the Jitsi focus but clients would know that there
|
||||
is an ongoing session of unknown type and unknown focus and could display/represent
|
||||
this in the user interface.
|
||||
|
||||
To make it easy for clients to support different RTC session types, the recommended
|
||||
approach is to provide a matrix widget for each session type, so that client developers
|
||||
can use the widget as the first implementation if they want to support this RTC
|
||||
session type.
|
||||
|
||||
Each application should get its own MSC in which the all the additional
|
||||
fields are explained and how the communication with the possible foci is
|
||||
defined:
|
||||
|
||||
- [`m.call`](www.example.com) TODO: create `m.call` MSC and add link here.
|
||||
|
||||
## Potential issues
|
||||
## Alternatives
|
||||
## Security considerations
|
||||
Loading…
Reference in New Issue