diff --git a/changelogs/application_service/newsfragments/3658.clarification b/changelogs/application_service/newsfragments/3658.clarification new file mode 100644 index 00000000..de0f5534 --- /dev/null +++ b/changelogs/application_service/newsfragments/3658.clarification @@ -0,0 +1 @@ +Distinguish between "federation" event format as exchanged by the Federation API, and the "client" event formats as used in the client-server and AS APIs. diff --git a/changelogs/client_server/newsfragments/3658.clarification b/changelogs/client_server/newsfragments/3658.clarification new file mode 100644 index 00000000..de0f5534 --- /dev/null +++ b/changelogs/client_server/newsfragments/3658.clarification @@ -0,0 +1 @@ +Distinguish between "federation" event format as exchanged by the Federation API, and the "client" event formats as used in the client-server and AS APIs. diff --git a/content/client-server-api/_index.md b/content/client-server-api/_index.md index 31742754..4056373e 100644 --- a/content/client-server-api/_index.md +++ b/content/client-server-api/_index.md @@ -1440,32 +1440,6 @@ any given point in time: [E0]->[E1]->[E2]->[E3]->[E4]->[E5] -{{% boxes/warning %}} -The format of events can change depending on room version. Check the -[room version specification](/rooms) for specific -details on what to expect for event formats. Examples contained within -the client-server specification are expected to be compatible with all -specified room versions, however some differences may still apply. - -For this version of the specification, clients only need to worry about -the event ID format being different depending on room version. Clients -should not be parsing the event ID, and instead be treating it as an -opaque string. No changes should be required to support the currently -available room versions. -{{% /boxes/warning %}} - -{{% boxes/warning %}} -Event bodies are considered untrusted data. This means that any application using -Matrix must validate that the event body is of the expected shape/schema -before using the contents verbatim. - -**It is not safe to assume that an event body will have all the expected -fields of the expected types.** - -See [MSC2801](https://github.com/matrix-org/matrix-doc/pull/2801) for more -detail on why this assumption is unsafe. -{{% /boxes/warning %}} - ### Types of room events Room events are split into two categories: @@ -1496,25 +1470,31 @@ sent by clients and other clients would receive it through Matrix, assuming the client has access to the `com.example` namespace. {{% /boxes/note %}} -Note that the structure of these events may be different than those in -the server-server API. - -#### Event fields +### Room event format -{{% event-fields event_type="event" %}} +The "federation" format of a room event, which is used internally by homeservers +and between homeservers via the Server-Server API, depends on the ["room +version"](/rooms) in use by the room. See, for example, the definitions +in [room version 1](/rooms/v1#event-format) and [room version +3](/rooms/v3#event-format). -#### Room event fields +However, it is unusual that a Matrix client would encounter this event +format. Instead, homeservers are responsible for converting events into the +format shown below so that they can be easily parsed by clients. -{{% event-fields event_type="room_event" %}} +{{% boxes/warning %}} +Event bodies are considered untrusted data. This means that any application using +Matrix must validate that the event body is of the expected shape/schema +before using the contents verbatim. -#### State event fields +**It is not safe to assume that an event body will have all the expected +fields of the expected types.** -In addition to the fields of a Room Event, State Events have the -following field: +See [MSC2801](https://github.com/matrix-org/matrix-doc/pull/2801) for more +detail on why this assumption is unsafe. +{{% /boxes/warning %}} -| Key | Type | Description | -|--------------|--------------|--------------------------------------------------------------------------------------------------------------| -| state_key | string | **Required.** A unique key which defines the overwriting semantics for this piece of room state. This value is often a zero-length string. The presence of this key makes this event a State Event. State keys starting with an `@` are reserved for referencing user IDs, such as room members. With the exception of a few events, state events set with a given user's ID as the state key MUST only be set by that user. | +{{% definition path="api/client-server/definitions/client_event" %}} ### Stripped state @@ -1580,8 +1560,7 @@ updates not being sent. ### Size limits The complete event MUST NOT be larger than 65536 bytes, when formatted -as a [PDU for the Server-Server -protocol](/server-server-api/#pdus), including any +with the [federation event format](#room-event-format), including any signatures, and encoded as [Canonical JSON](/appendices#canonical-json). diff --git a/data/api/application-service/transactions.yaml b/data/api/application-service/transactions.yaml index e4b62e01..de082904 100644 --- a/data/api/application-service/transactions.yaml +++ b/data/api/application-service/transactions.yaml @@ -65,8 +65,7 @@ paths: description: |- A list of events, formatted as per the Client-Server API. items: - type: object - title: Event + $ref: "../client-server/definitions/client_event.yaml" required: ["events"] responses: 200: diff --git a/data/api/client-server/definitions/client_event.yaml b/data/api/client-server/definitions/client_event.yaml new file mode 100644 index 00000000..14aa9e6e --- /dev/null +++ b/data/api/client-server/definitions/client_event.yaml @@ -0,0 +1,47 @@ +# Copyright 2022 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +title: ClientEvent +description: |- + The format used for events when they are returned from a homeserver to a client + via the Client-Server API, or sent to an Application Service via the Application Services API. +type: object +allOf: + - $ref: "client_event_without_room_id.yaml" + - properties: + room_id: + description: The ID of the room associated with this event. + type: string + example: '!jEsUZKDJdhlrceRyVU:example.org' + + unsigned: + properties: + redacted_because: + title: ClientEvent + example: { + "type": "m.room.redaction", + "sender": "@moderator:example.org", + "content": { + "reason": "spam" + }, + "redacts": "$26RqwJMLw-yds1GAH_QxjHRC1Da9oasK0e5VLnck_45", + "event_id": "$Nhl3rsgHMjk-DjMJANawr9HHAhLg4GcoTYrSiYYGqEE", + "origin_server_ts": 1632491098485, + "room_id": '!jEsUZKDJdhlrceRyVU:example.org', + "unsigned": { + "age": 1257, + } + } + required: + - room_id diff --git a/data/api/client-server/definitions/client_event_without_room_id.yaml b/data/api/client-server/definitions/client_event_without_room_id.yaml new file mode 100644 index 00000000..1b6d6073 --- /dev/null +++ b/data/api/client-server/definitions/client_event_without_room_id.yaml @@ -0,0 +1,110 @@ +# Copyright 2022 The Matrix.org Foundation C.I.C. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +title: ClientEventWithoutRoomID +description: |- + The format used for events when they are returned from + API endpoints such as `/sync`, where the `room_id` is implied elsewhere + in the response. +type: object +required: + - event_id + - type + - sender + - origin_server_ts + - content +properties: + event_id: + description: The globally unique identifier for this event. + type: string + example: '$26RqwJMLw-yds1GAH_QxjHRC1Da9oasK0e5VLnck_45' + type: + description: The type of the event. + type: string + example: 'm.room.member' + state_key: + description: |- + Present if, and only if, this event is a *state* event. The key making + this piece of state unique in the room. Note that it is often an empty + string. + type: string + example: '@user:example.org' + sender: + description: Contains the fully-qualified ID of the user who sent this event. + type: string + example: "@example:example.org" + origin_server_ts: + description: |- + Timestamp (in milliseconds since the unix epoch) on originating homeserver + when this event was sent. + type: integer + format: int64 + example: 1632489532305 + content: + description: |- + The body of this event, as created by the client which sent it. + type: object + example: { + "membership": "join" + } + unsigned: + title: UnsignedData + type: object + description: Contains optional extra information about the event. + properties: + age: + description: The time in milliseconds that has elapsed since the event was + sent. This field is generated by the local homeserver, and may be incorrect + if the local time on at least one of the two servers is out of sync, which can + cause the age to either be negative or greater than it actually is. + type: integer + format: int64 + example: 1567437 + redacted_because: + description: The event that redacted this event, if any. + type: object + title: ClientEventWithoutRoomID + example: { + "type": "m.room.redaction", + "sender": "@moderator:example.org", + "content": { + "reason": "spam" + }, + "redacts": "$26RqwJMLw-yds1GAH_QxjHRC1Da9oasK0e5VLnck_45", + "event_id": "$Nhl3rsgHMjk-DjMJANawr9HHAhLg4GcoTYrSiYYGqEE", + "origin_server_ts": 1632491098485, + "unsigned": { + "age": 1257, + } + } + transaction_id: + description: | + The client-supplied transaction ID, for example, provided via + `PUT /_matrix/client/v3/rooms/{roomId}/send/{eventType}/{txnId}`, + if the client being given the event is the same one which sent it. + type: string + prev_content: + description: | + The previous `content` for this event. This field is generated + by the local homeserver, and is only returned if the event is a state event, + and the client has permission to see the previous content. + x-changedInMatrixVersion: + 1.2: | + Previously, this field was specified at the top level of returned + events rather than in `unsigned` (with the exception of the [`GET + .../notifications`](/client-server-api/#get_matrixclientv3notifications) + endpoint), though in practice no known server implementations honoured + this. + title: EventContent + type: object diff --git a/data/api/client-server/definitions/room_event_batch.yaml b/data/api/client-server/definitions/room_event_batch.yaml deleted file mode 100644 index 4a7dab19..00000000 --- a/data/api/client-server/definitions/room_event_batch.yaml +++ /dev/null @@ -1,28 +0,0 @@ -# Copyright 2018 New Vector Ltd -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -properties: - events: - description: List of events. - items: - allOf: - - $ref: ../../../event-schemas/schema/core-event-schema/sync_room_event.yaml - type: object - required: - - event_id - #- room_id - Not in /sync - - sender - - origin_server_ts - type: array -type: object -title: RoomEventBatch diff --git a/data/api/client-server/definitions/state_event_batch.yaml b/data/api/client-server/definitions/state_event_batch.yaml index f3966dcd..d452d348 100644 --- a/data/api/client-server/definitions/state_event_batch.yaml +++ b/data/api/client-server/definitions/state_event_batch.yaml @@ -15,15 +15,7 @@ properties: events: description: List of events. items: - allOf: - - $ref: ../../../event-schemas/schema/core-event-schema/sync_state_event.yaml - type: object - required: - - event_id - #- room_id - Not in /sync - - sender - - origin_server_ts - - state_key + $ref: client_event_without_room_id.yaml type: array type: object title: StateEventBatch diff --git a/data/api/client-server/definitions/timeline_batch.yaml b/data/api/client-server/definitions/timeline_batch.yaml index 23f175fb..1c4ba40e 100644 --- a/data/api/client-server/definitions/timeline_batch.yaml +++ b/data/api/client-server/definitions/timeline_batch.yaml @@ -12,8 +12,6 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -allOf: -- $ref: room_event_batch.yaml properties: limited: description: True if the number of events returned was limited by the `limit` @@ -27,5 +25,12 @@ properties: If no earlier events are available, this property may be omitted from the response. type: string + events: + description: List of events. + type: array + items: + $ref: "client_event_without_room_id.yaml" type: object title: TimelineBatch +required: + - events diff --git a/data/api/client-server/event_context.yaml b/data/api/client-server/event_context.yaml index 689e998c..cf3476b0 100644 --- a/data/api/client-server/event_context.yaml +++ b/data/api/client-server/event_context.yaml @@ -92,28 +92,25 @@ paths: A list of room events that happened just before the requested event, in reverse-chronological order. items: - allOf: - - "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml" + $ref: "definitions/client_event.yaml" event: description: |- Details of the requested event. allOf: - - "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml" + - $ref: "definitions/client_event.yaml" events_after: type: array description: |- A list of room events that happened just after the requested event, in chronological order. items: - allOf: - - "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml" + $ref: "definitions/client_event.yaml" state: type: array description: |- The state of the room at the last event returned. items: - allOf: - - "$ref": "../../event-schemas/schema/core-event-schema/state_event.yaml" + $ref: "definitions/client_event.yaml" examples: application/json: { "end": "t29-57_2_0_2", diff --git a/data/api/client-server/message_pagination.yaml b/data/api/client-server/message_pagination.yaml index f13c4063..b63f9e17 100644 --- a/data/api/client-server/message_pagination.yaml +++ b/data/api/client-server/message_pagination.yaml @@ -125,7 +125,7 @@ paths: are available. Clients should continue to paginate until no `end` property is returned. items: - "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml" + "$ref": "definitions/client_event.yaml" state: type: array description: |- @@ -138,7 +138,7 @@ paths: sent to the client in prior calls to this endpoint, assuming the membership of those members has not changed. items: - $ref: "../../event-schemas/schema/core-event-schema/state_event.yaml" + $ref: "definitions/client_event.yaml" required: [start, chunk] examples: application/json: { diff --git a/data/api/client-server/notifications.yaml b/data/api/client-server/notifications.yaml index 6390bce2..6d141ff8 100644 --- a/data/api/client-server/notifications.yaml +++ b/data/api/client-server/notifications.yaml @@ -112,8 +112,7 @@ paths: type: object title: Event description: The Event object for the event that triggered the notification. - allOf: - - $ref: ../../event-schemas/schema/core-event-schema/sync_room_event.yaml + "$ref": "definitions/client_event_without_room_id.yaml" profile_tag: type: string description: The profile tag of the rule that matched this event. diff --git a/data/api/client-server/old_sync.yaml b/data/api/client-server/old_sync.yaml index bfbbe840..8813d10f 100644 --- a/data/api/client-server/old_sync.yaml +++ b/data/api/client-server/old_sync.yaml @@ -84,10 +84,7 @@ paths: type: array description: "An array of events." items: - type: object - title: Event - allOf: - - "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml" + $ref: "definitions/client_event.yaml" 400: description: "Bad pagination `from` parameter." tags: @@ -206,10 +203,7 @@ paths: type: array description: A list of presence events. items: - type: object - title: Event - allOf: - - "$ref": "../../event-schemas/schema/core-event-schema/event.yaml" + $ref: "definitions/client_event.yaml" rooms: type: array items: @@ -227,8 +221,7 @@ paths: type: object title: "InviteEvent" description: "The invite event if `membership` is `invite`" - allOf: - - "$ref": "../../event-schemas/schema/m.room.member.yaml" + $ref: "definitions/client_event.yaml" messages: type: object title: PaginationChunk @@ -260,10 +253,7 @@ paths: messages that preceded them leaving. This array will consist of at most `limit` elements. items: - type: object - title: RoomEvent - allOf: - - "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml" + $ref: "definitions/client_event.yaml" required: ["end", "chunk"] state: type: array @@ -273,10 +263,7 @@ paths: user has left the room this will be the state of the room when they left it. items: - title: StateEvent - type: object - allOf: - - "$ref": "../../event-schemas/schema/core-event-schema/state_event.yaml" + $ref: "definitions/client_event.yaml" visibility: type: string enum: ["private", "public"] @@ -289,10 +276,7 @@ paths: The private data that this user has attached to this room. items: - title: Event - type: object - allOf: - - "$ref": "../../event-schemas/schema/core-event-schema/event.yaml" + $ref: "definitions/client_event.yaml" required: ["room_id", "membership"] account_data: type: array @@ -336,8 +320,7 @@ paths: examples: application/json: {"$ref": "../../event-schemas/examples/m.room.message$m.text.yaml"} schema: - allOf: - - "$ref": "../../event-schemas/schema/core-event-schema/event.yaml" + $ref: "definitions/client_event.yaml" 404: description: The event was not found or you do not have permission to read this event. tags: diff --git a/data/api/client-server/peeking_events.yaml b/data/api/client-server/peeking_events.yaml index 8f87a28a..4b8a6a96 100644 --- a/data/api/client-server/peeking_events.yaml +++ b/data/api/client-server/peeking_events.yaml @@ -102,7 +102,7 @@ paths: type: object title: Event allOf: - - "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml" + - "$ref": "definitions/client_event.yaml" 400: description: "Bad pagination `from` parameter." tags: diff --git a/data/api/client-server/room_initial_sync.yaml b/data/api/client-server/room_initial_sync.yaml index df212066..64e7f4f2 100644 --- a/data/api/client-server/room_initial_sync.yaml +++ b/data/api/client-server/room_initial_sync.yaml @@ -119,10 +119,7 @@ paths: messages that preceded them leaving. This array will consist of at most `limit` elements. items: - type: object - title: RoomEvent - allOf: - - "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml" + "$ref": "definitions/client_event.yaml" required: ["end", "chunk"] state: type: array @@ -132,10 +129,7 @@ paths: user has left the room this will be the state of the room when they left it. items: - title: StateEvent - type: object - allOf: - - "$ref": "../../event-schemas/schema/core-event-schema/state_event.yaml" + "$ref": "definitions/client_event.yaml" visibility: type: string enum: ["private", "public"] diff --git a/data/api/client-server/rooms.yaml b/data/api/client-server/rooms.yaml index 64079b89..343b6c49 100644 --- a/data/api/client-server/rooms.yaml +++ b/data/api/client-server/rooms.yaml @@ -58,8 +58,7 @@ paths: "$ref": "../../event-schemas/examples/m.room.message$m.text.yaml" } schema: - allOf: - - "$ref": "../../event-schemas/schema/core-event-schema/event.yaml" + "$ref": "definitions/client_event.yaml" 404: description: The event was not found or you do not have permission to read this event. examples: @@ -165,10 +164,7 @@ paths: has left the room then this will be the state of the room when they left as a list of events. items: - title: StateEvent - type: object - allOf: - - "$ref": "../../event-schemas/schema/core-event-schema/state_event.yaml" + $ref: "definitions/client_event.yaml" 403: description: > You aren't a member of the room and weren't previously a @@ -251,10 +247,7 @@ paths: chunk: type: array items: - title: MemberEvent - type: object - allOf: - - "$ref": "../../event-schemas/schema/m.room.member.yaml" + $ref: "definitions/client_event.yaml" 403: description: > You aren't a member of the room and weren't previously a diff --git a/data/api/client-server/search.yaml b/data/api/client-server/search.yaml index b1f6359d..5454f0a1 100644 --- a/data/api/client-server/search.yaml +++ b/data/api/client-server/search.yaml @@ -204,7 +204,7 @@ paths: type: object title: Event description: The event that matched. - "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml" + "$ref": "definitions/client_event.yaml" context: type: object title: Event Context @@ -247,7 +247,7 @@ paths: items: title: Event type: object - "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml" + "$ref": "definitions/client_event.yaml" events_after: type: array title: Events After @@ -255,7 +255,7 @@ paths: items: title: Event type: object - "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml" + "$ref": "definitions/client_event.yaml" state: type: object title: Current state @@ -271,7 +271,7 @@ paths: title: Room State items: type: object - "$ref": "../../event-schemas/schema/core-event-schema/state_event.yaml" + "$ref": "definitions/client_event.yaml" groups: type: object title: Groups diff --git a/data/event-schemas/schema/core-event-schema/unsigned_prop.yaml b/data/event-schemas/schema/core-event-schema/unsigned_prop.yaml index dc7348d5..3ca19c41 100644 --- a/data/event-schemas/schema/core-event-schema/unsigned_prop.yaml +++ b/data/event-schemas/schema/core-event-schema/unsigned_prop.yaml @@ -18,31 +18,5 @@ description: Contains optional extra information about the event. properties: age: description: The time in milliseconds that has elapsed since the event was - sent. This field is generated by the local homeserver, and may be incorrect - if the local time on at least one of the two servers is out of sync, which can - cause the age to either be negative or greater than it actually is. + sent. type: integer - redacted_because: - description: The event that redacted this event, if any. - title: Event - type: object - transaction_id: - description: | - The client-supplied transaction ID, for example, provided via - `PUT /_matrix/client/v3/rooms/{roomId}/send/{eventType}/{txnId}`, - if the client being given the event is the same one which sent it. - type: string - prev_content: - description: | - The previous `content` for this event. This field is generated - by the local homeserver, and is only returned if the event is a state event, - and the client has permission to see the previous content. - x-changedInMatrixVersion: - 1.2: | - Previously, this field was specified at the top level of returned - events rather than in `unsigned` (with the exception of the [`GET - .../notifications`](/client-server-api/#get_matrixclientv3notifications) - endpoint), though in practice no known server implementations honoured - this. - title: EventContent - type: object