matrix-spec-proposals/proposals/3440-threading-via-relations.md

# MSC3440 Threading via `m.thread` relation

## Problem

Threading allows users to branch out a new conversation from the main timeline of a room
to each other. This is particularly useful in high traffic rooms where multiple
conversations can happen in parallel or when a single discussion might stretch
over a very long period of time.

The main goal of implementing threads is to facilitate conversations that are easier
to follow and smoother to read.
Threading is very clearly a core requirement for any modern messaging
solution, and Matrix uptake is suffering due to the lack of progress.

## Proposal

### Event format

A new relation type (see [MSC2674](https://github.com/matrix-org/matrix-doc/pull/2674))
`m.thread` expresses that an event belongs to a thread.

```json
"m.relates_to": {
  "rel_type": "m.thread",
  "event_id": "$thread_root"
}
```
Where $thread_root is the event ID of the root message in the thread.

When a thread root is aggregated (as in MSC2675), it returns a summary of the thread:
the latest message, a list of participants and the total count of messages.
I.e. in places which include bundled relations (per
[MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675)), the thread root
would include additional information in the `unsigned` field:

```jsonc
{
  "event_id": "$root_event",
  "unsigned": {
    "m.relations": {
      "m.thread": {
        "latest_event": {
          "event_id": "$thread_event",
          // ...
        },
        "count": 7,
        "current_user_participated": true
      }
    }
  }
}
```

* `latest_event`: The most recent event (topologically) which relates to this event, with
  `rel_type` of `m.thread`. Events sent by [ignored users](https://spec.matrix.org/v1.3/client-server-api/#ignoring-users)
  are not considered for the latest event.

  The latest event should be serialised in the same form as the event itself;
  this includes adding any bundled aggregations for the event (and applying edits).[^1]
* `count`: An integer counting the number of `m.thread` events, excluding events sent
  by [ignored users](https://spec.matrix.org/v1.3/client-server-api/#ignoring-users).
* `current_user_participated`: A boolean flag, which is set to `true` if the
  current logged in user has participated in the thread. The user has participated if:
  * They created the current event.
  * They created an event with a `m.thread` relation targeting the current event.

#### Rich replies in a thread

Rich replies are still handled via the `m.in_reply_to` field of `m.relates_to`.
However clients should specify that this is not a thread fallback by setting
the `is_falling_back` property to `false`.

```json
"m.relates_to": {
    "rel_type": "m.thread",
    "event_id": "$thread_root",
    "is_falling_back": false,
    "m.in_reply_to": {
        "event_id": "$event_target"
    }
}
```

It is possible that an `m.in_reply_to` event targets an event that is outside the
related thread. Clients should always do their utmost to display the rich reply
and when clicked, the event should be displayed and highlighted in its original context.

A rich reply without `rel_type: m.thread` targeting a thread relation must be
rendered in the main timeline. This will allow users to advertise threaded messages
in the room.

### Backwards compatibility

A thread will be displayed as a chain of replies on clients unaware of threads.

Thread-ready clients should always include an `m.in_reply_to` property when sending
a threaded event. Unless the user is explicitly replying to another event (see "Rich replies in a thread", above),
the `m.in_reply_to` property should reference the latest message-like event in the
thread, and clients should also specify that `m.in_reply_to`
is a fallback mechanism (rather than a genuine reply) by setting the `is_falling_back` property to `true`.

(If omitted, `is_falling_back` defaults to `false`, and receiving clients will treat the
`m.in_reply_to` part of the event as a genuine reply.)

```jsonc
"m.relates_to": {
    "rel_type": "m.thread",
    "event_id": "ev1",
    "is_falling_back": true,
    "m.in_reply_to": {
        "event_id": "last_event_id_in_thread",
    }
  }
```

Historically replies have been limited to text messages due to the legacy fallback
prepended to `formatted_body`. This MSC is dependant on
[MSC3676](https://github.com/matrix-org/matrix-doc/pull/3676) which strips that
requirement to unlock use of any event type in this context.

### Fetch all relations to a thread root

To fetch an entire thread, the `/relations` API can be used as defined in
[MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675)

```
GET /_matrix/client/unstable/rooms/!room_id:domain/relations/$thread_root/m.thread
```

Where `$thread_root` is the event ID of the root message in the thread.

> Any API which receives events should bundle relations (apart from non-gappy
incremental syncs), for instance: initial sync, gappy incremental sync,
/messages and /context.

### Fetch all threads in a room

[Event filters](https://spec.matrix.org/v1.2/client-server-api/#filtering) (as
used by endpoints including `/messages`, `/sync` and `/context`) are extended
with new options to allow filtering events by their relating events:

* `related_by_rel_types`: A list of relation types to include. An event `A` is included
in the filter only if there exists another event `B` which relates to `A` with a
`rel_type` which is defined in the list
* `related_by_senders`: A list of senders to include. An event `A` is included in
the filter only if there exists another event `B` which relates to `A`, and
which has a `sender` which is in the list.

This can also be combined with the `sender` field to search for threads which a
user has participated in (or not participated in).

```
GET /_matrix/client/v3/rooms/!room_id:domain/messages?filter=...
```

The filter string includes the new fields, above. In this example, the URL
encoded JSON is presented unencoded and formatted for legibility:

```jsonc
{
  "types": ["m.room.message"],
  "related_by_senders": [
    // ...
  ],
  "related_by_rel_types": ["m.thread"]
}
```

Note that the newly added filtering parameters return events based on information
in related events. Consider the following events in a room:

* `A`: a `m.room.message` event sent by `alice`
* `B`: a `m.room.message` event sent by `bob` which relates to `A` with type `m.thread`

Using a filter of `"related_by_rel_types": ["m.thread"]` would return event `A` as it
has another event which relates to it via `m.thread`.

Similarly, using a filter of `"related_by_senders": ["bob"]` would return event `A`
as it has another event which relates to it sent by `bob`.

### Server capabilities

Threads might have sporadic support across servers, to simplify feature
detections for clients, a homeserver must advertise unstable support for threads
as part of the `/versions` API:

```jsonc
{
  "unstable_features": {
    "org.matrix.msc3440": true,
    // ...
  }
}
```

### Limitations

#### Read receipts

Read receipts and read markers assume a single chronological timeline. Threading
changes that assumption making the current API not very practical.

Clients can synthesize read receipts but it is possible that some notifications get
lost on a fresh start where the clients have to start off the `m.read`
information received from the homeserver.

Synchronising the synthesized notification count across devices is out of scope and deferred to a later MSC.

#### Single-layer event aggregation

This MSC does not include support for nested threads.

Nested threading is out of scope for this proposal and would be the subject of
a different MSC.
A `m.thread` event can only reference events that do not have a `rel_type`

```jsonc
[
  {
    "event_id": "ev1",
    // ...
  },
  {
    "event_id": "ev2",
    // ...
    "m.relates_to": {
      "rel_type": "m.thread",
      "event_id": "ev1",
      "is_falling_back": true,
      "m.in_reply_to": {
          "event_id": "ev1"
      }
    }
  },
  {
    "event_id": "ev3",
    // ...
    "m.relates_to": {
      "rel_type": "m.annotation",
      "event_id": "ev1",
      "key": "✅"
    }
  }
]
```

Given the above list of events, only `ev1` would be a valid target for an `m.thread`
relation event.

Servers should reject attempts to send events with invalid thread relations via the
Client-Server API with an HTTP `400` status code and a
`M_UNKNOWN` error code.
Events received over federation should always be accepted without checking
the validity of the relations as it would break the extensibility of this proposal
in a future MSC.

This means that events with invalid thread relations can make their way into the
network, either due by malicious activity or buggy implementation. If a client
receives such events, it should hide them as soon as it can determine for certain
that the associated event is not a valid target.

Servers are expected to not filter out invalid `m.thread` relations from the results when
serving endpoints that deal with message relations. Clients that call those
endpoints should be aware that they may return events with invalid relations,
and deal with them appropriately.

### Client considerations

#### Sending `m.thread` before fully implementing threads

There will be clients that will not or can't support threads. Whether this is a
deliberate choice or because the system bridges to a platform that does not support
threads, there are a number of steps developer of those systems can take to ensure
continuity of conversation in the ecosystem.

Clients that do not offer a threading UI should behave as follows when replying, for
best interaction with those that do.
They should set the `m.in_reply_to` part as usual, and then add on
`"rel_type": "m.thread"` and `"event_id": "$thread_root"`, copying `$thread_root`
from the replied-to event.

If the `m.thread` relation type is not present in an incoming event, it should
be treated as not being part of the thread.  For example, if a client has a
separate area for displaying threads, clients can render the event in the main
room timeline as a rich reply that will open and highlight the event in the
thread context when clicked.

When replying to the following event, a client that does not support threads should
copy in `rel_type` and `event_id` properties in their reply mixin.

```jsonc
{
  // ...
  "m.relates_to": {
    "rel_type": "m.thread",
    "event_id": "ev1",
    "is_falling_back": false,
    "m.in_reply_to": {
      "event_id": "$event_target"
    }
  }
}
```

## Alternatives

"Threading as rooms", building on `m.in_reply_to`, and [MSC2836](https://github.com/matrix-org/matrix-doc/pull/2836) are the main alternatives here.

It is also worth noting that relations in this MSC could be expressed using the
scalable relation format described in [MSC3051](https://github.com/matrix-org/matrix-doc/pull/3051).

### Threads as rooms

Threads as rooms could provide full server-side APIs for navigating trees of events,
and could be considered an extension of this MSC for scenarios which require that
capability

"Threads as rooms" is the idea that each thread could just get its own Matrix room.

Advantages to "Threads as rooms" include:
 * May be simpler for client implementations
 * Restricting events visibility as the room creator
 * Ability to create read-only threads

Disadvantages include:
 * Access control, membership, history visibility, room versions etc needs to be
 synced between the thread-room and the parent room
 * Harder to control lifetime of threads in the context of the parent room if
 they're completely split off
 * Clients which aren't aware of them are going to fill up with a lot of rooms.
 * Bridging to non-threaded chat systems is trickier as you may have to splice
 together rooms

### Threads via m.in_reply_to

The rationale for using a new relation type instead of building on `m.in_reply_to`
is to re-use the event relationship APIs provided by
[MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675). The MSC3267 definition
of `m.reference` relationships could be updated to mention threads (perhaps by
using the key field from [MSC2677](https://github.com/matrix-org/matrix-doc/pull/2677)
as the thread ID), but it is clearer to define a new relation type. It is unclear
what impact this would have on [MSC3267](https://github.com/matrix-org/matrix-doc/pull/3267),
but that is unimplemented by clients.

A big advantage of relations over rich replies is that they can be server-side
aggregated. It means that a client is not bound to download the entire history of
a room to have a comprehensive list of events being part of a thread.

### Threads via serverside traversal of relationships MSC2836

Advantages include:
 * Fits other use cases than instant messaging
 * Simple possible API shape to implement threading in a useful way

Disadvantages include:
 * Relationships are queried using `/event_relationships` which is outside the
 bounds of the `/sync` API so lacks the nice things /sync gives you (live updates).
 That being said, the event will come down `/sync`, you just may not have the
 context required to see parents/siblings/children.
 * Threads can be of arbitrary width (unlimited direct replies to a single message)
 and depth (unlimited chain of replies) which complicates UI design when you just
 want "simple" threading.
 * Does not consider use cases like editing or reactions

## Security considerations

None

## Unstable prefix

Clients and servers should use list of unstable prefixes listed below while this
MSC has not been included in a spec release.

  * `io.element.thread` should be used in place of `m.thread` as relation type
  * `io.element.thread` should be used in place of `m.thread` as a capability entry
  * `io.element.relation_senders` should be used in place of `related_by_senders`
  in the `RoomEventFilter`
  * `io.element.relation_types` should be used in place of `related_by_rel_types`
  in the `RoomEventFilter`
  * `io.element.show_reply` should be used in place of `is_falling_back`

While this MSC is considered stable, but not in a released version of the specification,
clients should look for `org.matrix.msc3440.stable` as an unstable feature flag as part of
the `/versions` API response to determine server support for the stable identifiers.

**Note from the SCT**: This MSC was previously slated for v1.3, however ultimately it
was decided to move it to a later release.

## Dependencies

This MSC builds on [MSC2674](https://github.com/matrix-org/matrix-doc/pull/2674),
[MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675),
[MSC3567](https://github.com/matrix-org/matrix-doc/pull/3567) and,
[MSC3676](https://github.com/matrix-org/matrix-doc/pull/3676) (which at the
time of writing have not yet been accepted into the spec).

[^1]: It might seem like this could cause a loop where each latest event then
includes another bundled aggregation with latest events in it, but this not
possible since [nested threading is not supported](#single-layer-event-aggregation).
This MSC does not limit the allowed other relations to bundle; but, at the time of
writing, the only known relations are:
`m.replace` (from [MSC2676](https://github.com/matrix-org/matrix-doc/pull/2676)),
`m.annotation` (from [MSC2677](https://github.com/matrix-org/matrix-doc/pull/2677)), and
`m.reference` (from [MSC3267](https://github.com/matrix-org/matrix-doc/pull/3267)).