Reorganize event structure in c2s spec and clarify event capabilities

Fixes https://github.com/matrix-org/matrix-doc/issues/1166
Fixes https://github.com/matrix-org/matrix-doc/issues/1527
Fixes https://github.com/matrix-org/matrix-doc/issues/1827

Note: In order to fix the "state events have the following fields: [no words]" bug (1827) we need to resolve references on common event types. When doing this we ultimately end up with more fields than may be required to explain the section, however this commit alters the section descriptions to just say "these fields" instead of "these additional fields".

This is also preferable over trying to get the inheritance reversed in the common event types, as the `/sync` endpoint has a high amount of reliance on partial events definitions.

changelogs/client_server/newsfragments/2087.clarification

@@ -0,0 +1 @@
+Reorganize information about events into a common section.

event-schemas/schema/core-event-schema/room_event.yaml

@@ -1,7 +1,6 @@
 allOf:
 - $ref: sync_room_event.yaml
-description: In addition to the Event fields, Room Events have the following additional
+description: Room Events have the following fields.
-  fields.
 properties:
   room_id:
     description: |-

event-schemas/schema/core-event-schema/state_event.yaml

@@ -1,7 +1,6 @@
 allOf:
 - $ref: room_event.yaml
 - $ref: sync_state_event.yaml
-description: In addition to the Room Event fields, State Events have the following
+description: State Events have the following fields.
-  additional fields.
 title: State Event
 type: object

scripts/templating/matrix_templates/units.py

@@ -748,6 +748,7 @@ class MatrixUnits(Units):
 
             with open(filepath, encoding="utf-8") as f:
                 event_schema = yaml.load(f, OrderedLoader)
+                event_schema = resolve_references(filepath, event_schema)
 
             schema_info = process_data_type(
                 event_schema,

specification/client_server_api.rst

@@ -1326,6 +1326,63 @@ the event ``type`` key SHOULD follow the Java package naming convention,
 e.g. ``com.example.myapp.event``.  This ensures event types are suitably
 namespaced for each application and reduces the risk of clashes.
 
+.. Note::
+  Events are not limited to the types defined in this specification. New or custom
+  event types can be created on a whim using the Java package naming convention.
+  For example, a ``com.example.game.score`` event can be sent by clients and other
+  clients would receive it through Matrix.
+
+Note that the structure of these events may be different than those in the
+server-server API.
+
+{{common_event_fields}}
+
+{{common_room_event_fields}}
+
+{{common_state_event_fields}}
+
+
+Size limits
+~~~~~~~~~~~
+
+The complete event MUST NOT be larger than 65535 bytes, when formatted as a
+`PDU for the Server-Server protocol <../server_server/%SERVER_RELEASE_LABEL%#pdus>`_,
+including any signatures, and encoded as `Canonical JSON`_.
+
+There are additional restrictions on sizes per key:
+
+- ``sender`` MUST NOT exceed 255 bytes (including domain).
+- ``room_id`` MUST NOT exceed 255 bytes.
+- ``state_key`` MUST NOT exceed 255 bytes.
+- ``type`` MUST NOT exceed 255 bytes.
+- ``event_id`` MUST NOT exceed 255 bytes.
+
+Some event types have additional size restrictions which are specified in
+the description of the event. Additional keys have no limit other than that
+implied by the total 65 KB limit on events.
+
+Room Events
+~~~~~~~~~~~
+.. NOTE::
+  This section is a work in progress.
+
+This specification outlines several standard event types, all of which are
+prefixed with ``m.``
+
+{{m_room_aliases_event}}
+
+{{m_room_canonical_alias_event}}
+
+{{m_room_create_event}}
+
+{{m_room_join_rules_event}}
+
+{{m_room_member_event}}
+
+{{m_room_power_levels_event}}
+
+{{m_room_redaction_event}}
+
 
 Syncing
 ~~~~~~~

specification/events.rst

@@ -1,73 +0,0 @@
-.. Copyright 2016 OpenMarket Ltd
-..
-.. Licensed under the Apache License, Version 2.0 (the "License");
-.. you may not use this file except in compliance with the License.
-.. You may obtain a copy of the License at
-..
-..     http://www.apache.org/licenses/LICENSE-2.0
-..
-.. Unless required by applicable law or agreed to in writing, software
-.. distributed under the License is distributed on an "AS IS" BASIS,
-.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-.. See the License for the specific language governing permissions and
-.. limitations under the License.
-
-Event Structure
-===============
-
-All communication in Matrix is expressed in the form of data objects called
-Events. These are the fundamental building blocks common to the client-server,
-server-server and application-service APIs, and are described below.
-
-Note that the structure of these events may be different than those in the
-server-server API.
-
-{{common_event_fields}}
-
-{{common_room_event_fields}}
-
-{{common_state_event_fields}}
-
-
-Size limits
------------
-
-The complete event MUST NOT be larger than 65535 bytes, when formatted as a
-`PDU for the Server-Server protocol <../server_server/%SERVER_RELEASE_LABEL%#pdus>`_,
-including any signatures, and encoded as `Canonical JSON`_.
-
-There are additional restrictions on sizes per key:
-
-- ``sender`` MUST NOT exceed 255 bytes (including domain).
-- ``room_id`` MUST NOT exceed 255 bytes.
-- ``state_key`` MUST NOT exceed 255 bytes.
-- ``type`` MUST NOT exceed 255 bytes.
-- ``event_id`` MUST NOT exceed 255 bytes.
-
-Some event types have additional size restrictions which are specified in
-the description of the event. Additional keys have no limit other than that
-implied by the total 65 KB limit on events.
-
-Room Events
------------
-.. NOTE::
-  This section is a work in progress.
-
-This specification outlines several standard event types, all of which are
-prefixed with ``m.``
-
-{{m_room_aliases_event}}
-
-{{m_room_canonical_alias_event}}
-
-{{m_room_create_event}}
-
-{{m_room_join_rules_event}}
-
-{{m_room_member_event}}
-
-{{m_room_power_levels_event}}
-
-{{m_room_redaction_event}}
-
-.. _`Canonical JSON`: ../appendices.html#canonical-json

specification/index.rst

@@ -348,6 +348,11 @@ pushed over federation to the participating servers in a room, currently using
 full mesh topology. Servers may also request backfill of events over federation
 from the other servers participating in a room.
 
+.. Note::
+  Events are not limited to the types defined in this specification. New or custom
+  event types can be created on a whim using the Java package naming convention.
+  For example, a ``com.example.game.score`` event can be sent by clients and other
+  clients would receive it through Matrix.
 
 Room Aliases
 ++++++++++++

specification/targets.yaml

@@ -5,7 +5,6 @@ targets:
   client_server:
     files:
       - client_server_api.rst
-      - { 1: events.rst }
       - { 1: modules.rst }
       - { 2: feature_profiles.rst }
       - { 2: "group:modules" }  # reference a group of files