diff --git a/changelogs/client_server/newsfragments/1642.clarification b/changelogs/client_server/newsfragments/1642.clarification new file mode 100644 index 00000000..0ccb0f3c --- /dev/null +++ b/changelogs/client_server/newsfragments/1642.clarification @@ -0,0 +1 @@ +Clarify and generalise the language used for describing pagination. diff --git a/specification/client_server_api.rst b/specification/client_server_api.rst index 2f17e4c3..549474af 100644 --- a/specification/client_server_api.rst +++ b/specification/client_server_api.rst @@ -138,9 +138,6 @@ Some requests have unique error codes: :``M_INVALID_ROOM_STATE``: Sent when the intial state given to the ``createRoom`` API is invalid. -:``M_BAD_PAGINATION``: - Encountered when specifying bad pagination query parameters. - :``M_THREEPID_IN_USE``: Sent when a threepid given to an API cannot be used because the same threepid is already in use. @@ -1018,39 +1015,21 @@ Pagination is the process of dividing a dataset into multiple discrete pages. Matrix makes use of pagination to allow clients to view extremely large datasets. These datasets are not limited to events in a room (for example clients may want to paginate a list of rooms in addition to events within those rooms). Regardless -of *what* is being paginated, there is a common underlying API which is used to -to give clients a consistent way of selecting subsets of a potentially changing -dataset. Requests pass in ``from``, ``to``, ``dir`` and ``limit`` parameters -which describe where to read from the stream. ``from`` and ``to`` are opaque -textual 'stream tokens' which describe the current position in the dataset. -The ``dir`` parameter is an enum representing the direction of events to return: -either ``f`` orwards or ``b`` ackwards. The response returns new ``start`` and -``end`` stream token values which can then be passed to subsequent requests to -continue pagination. Not all endpoints will make use of all the parameters -outlined here: see the specific endpoint in question for more information. - -Pagination Request Query Parameters -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Query parameters: - from: - $streamtoken - The opaque token to start streaming from. - to: - $streamtoken - The opaque token to end streaming at. Typically, - clients will not know the item of data to end at, so this will usually be - omitted. - limit: - integer - An integer representing the maximum number of items to - return. - dir: - f|b - The direction to return events in. Typically this is ``b`` to paginate - backwards in time. - -'START' and 'END' are placeholder values used in these examples to describe the -start and end of the dataset respectively. - -Unless specified, the default pagination parameters are ``from=START``, -``to=END``, without a limit set. +of what is being paginated, there is a common approach which is used to give +clients an easy way of selecting subsets of a potentially changing dataset. Each +endpoint that uses pagination may use different parameters. However the theme +among them is that they take a ``from`` and ``to`` token, and occasionally +a ``limit`` and ``dir``. Together, these parameters describe the position in a +data set, where ``from`` and ``to`` are known as "stream tokens" matching the +regular expression ``[a-zA-Z0-9.=_-]+``. If supported, the ``dir`` defines the +direction of events to return: either forwards (``f``) or backwards (``b``). +The response may contain tokens that can be used for retrieving results before +or after the returned set. These tokens may be called `start` or `prev_batch` +for retrieving the previous result set, or `end`, `next_batch` or `next_token` +for retrieving the next result set. + +In the following examples, 'START' and 'END' are placeholders to signify the +start and end of the data sets respectively. For example, if an endpoint had events E1 -> E15. The client wants the last 5 events and doesn't know any previous events:: @@ -1067,8 +1046,8 @@ events and doesn't know any previous events:: Another example: a public room list has rooms R1 -> R17. The client is showing 5 -rooms at a time on screen, and is on page 2. They want to -now show page 3 (rooms R11 -> 15):: +rooms at a time on screen, and is on page 2. They want to now show page 3 (rooms +R11 -> 15):: S E | 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | stream token @@ -1086,20 +1065,17 @@ token from the initial request was '9' which corresponded to R10. When the 2nd request was made, R10 did not appear again, even though from=9 was specified. If you know the token, you already have the data. -Pagination Response -~~~~~~~~~~~~~~~~~~~ - -Responses to pagination requests MUST follow the format:: - - { - "chunk": [ ... , Responses , ... ], - "start" : $streamtoken, - "end" : $streamtoken - } - -Where $streamtoken is an opaque token which can be used in another query to -get the next set of results. The "start" and "end" keys can only be omitted if -the complete dataset is provided in "chunk". +Responses for pagination-capable endpoints SHOULD have a ``chunk`` array alongside +the applicable stream tokens to represent the result set. + +In general, when the end of a result set is reached the applicable stream token +will be excluded from the response. For example, if a user was backwards-paginating +events in a room they'd eventually reach the first event in the room. In this scenario, +the ``prev_batch`` token would be excluded from the response. Some paginated +endpoints are open-ended in one direction, such as endpoints which expose an event +stream for an active room. In this case, it is not possible for the client to reach +the true "end" of the data set and therefore should always be presented with a token +to keep moving forwards. .. _`filter`: