8.7 KiB
MSCXXXX: Inline message attachments
This is an alternative to MSC2881: Message Attachments that stores all attachments directly inside current event, to not post a separate event for each attachment and refer to it. I think this way have less complexity and better for performance.
Proposal
As lite alternative to MSC2881: Message Attachments we can send only one event with direct links to all attached media, instead of sending separate event for each attachment, because it will give less "spam" for room. Eg when user is sending message with 20 attachments - it will send only one event to room, instead of 21 like in option one implementation. But the main problem with this option is fallback.
This can be done together with MSC1767: Extensible events in
Matrix with adding new type
m.attachments, which will contain the group of attached elements.
Each element of m.attachments array has a structure like a message with media
item (m.image, m.video, etc), here is example of the message with this
field:
{
"type": "m.room.message",
"content": {
"msgtype": "m.text",
"body": "Here is my photos and videos from yesterday event",
"m.attachments": [
{
"msgtype": "m.image",
"url": "mxc://example.com/KUAQOesGECkQTgdtedkftISg",
"body": "Image 1.jpg",
"info": {
"mimetype": "image/jpg",
"size": 1153501,
"w": 963,
"h": 734,
}
},
{
"msgtype": "m.video",
"url": "mxc://example.com/KUAQOe1GECk2TgdtedkftISg",
"body": "Video 2.mp4",
"info": {
"mimetype": "video/mp4",
"size": 6615304,
"w": 1280,
"h": 720,
}
}
]
}
}
Fallback
For fallback display of attachments in old Matrix clients, we can attach them
directly to formatted_body of message, here is HTML representation:
<p>Here is my photos and videos from yesterday event</p>
<div class="mx-attachments">
<p>Attachments:</p>
<ul>
<li><a href="https://example.com/_matrix/media/r0/download/example.com/KUAQOesGECkQTgdtedkftISg">Image 1.jpg</a></li>
<li><a href="https://example.com/_matrix/media/r0/download/example.com/0f4f88120bfc9183d122ca8f9f11faacfc93cd18">Video 2.mp4</a></li>
</ul>
</div>
and JSON of content field:
"content": {
"msgtype": "m.text",
"body": "Here is my photos and videos from yesterday event\nAttachments:\nhttps://example.com/_matrix/media/r0/download/example.com//KUAQOesGECkQTgdtedkftISg\nhttps://example.com/_matrix/media/r0/download/example.com/0f4f88120bfc9183d122ca8f9f11faacfc93cd18",
"format": "org.matrix.custom.html",
"formatted_body": "<p>Here is my photos and videos from yesterday event</p>\n<div class=\"mx-attachments\"><p>Attachments:</p>\n<ul>\n<li><a href=\"https://example.com/_matrix/media/r0/download/example.com//KUAQOesGECkQTgdtedkftISg\">Image 1.jpg</a></li>\n<li><a href=\"https://example.com/_matrix/media/r0/download/example.com/0f4f88120bfc9183d122ca8f9f11faacfc93cd18\">Video 2.mp4</a></li>\n</ul></div>"
}
And modern clients, that have support of the attachments feature, will cut the
div.mx-attachment tag and replace it to rich gallery block with thumbnails of
attachments.
If MSC2398: proposal to allow mxc:// in the "a" tag within
messages will be merged
before this, we can replace http urls to direct mxc:// urls, for support
servers, that don't allow downloads without authentication and have other
restrictions.
If we will come up with better fallback display, maybe bring this option as main suggestion?
Client support
Compose recommendations:
In the message composer, on "upload file" or "paste file from clipboard" event, the Matrix client must not instantly upload the file to the server, but the client must show its thumbnail in the special area, with the ability to remove it and to add more media. Alternatively, it can start uploading instantly to improve the speed of the following message sending process, but there is no way to delete media in Matrix API yet (MSC2278: Deleting attachments for expired and redacted messages, so server will store each file, even if it is not attached to the message.
On "message send" action, Matrix client must upload each attached media to
server, get mxc of it, post an event to room, and attach its event_id to
current message contents in m.relates_to array (option one); or collect all
mxc urls in m.attachments array on option two.
If the user uploads only one media and leaves the message text empty, media can
be sent as regular m.image or similar message, like in current implementation.
Editing interface can be represented exactly like the composer interface, where
user have the textarea for input message text, and area with all current
attachments as tiny thumbnails, in which he can rearrange attachments, remove
one of current attachments (that will remove its line from array of
m.relates_to and do the redact action on corresponding event with media in
option one, or remove item from m.attachments in option two; and delete media
file using
MSC2278),
add new attachment (that will upload it as new event with refer to it in edited
message m.relates_to array in option one / added to m.attachments in option
two).
Display recommendations:
On the client side, attachments can be displayed as grid of clickable
thumbnails, like the current m.image events, but with a smaller size, having
fixed height, like a regular image gallery. On click, Matrix client must display
media in full size, and, if possible, as a gallery with "next-previous" buttons.
Also clients can implement collapse/expand action on gallery grid.
If the message contains only one attachment, it can be displayed as full-width
thumbnail in timeline, like current m.image and m.video messages.
Example of composer interface implementation with multiple attachments we can lookup in Slack, VK Messenger, Skype.
For prevent showing of attachments as regular media in timeline before main
aggregating event will be added to room, clients should visually hide media
events, that have "is_attachment": true value, to display them later in
gallery, but can already start downloading of attachments thumbnails, for
speed-up display of them in gallery.
Together with MSC2675: Serverside aggregations of message relationships all attachments will can be even aggregated on server side.
Server support
This MSC does not need any changes on server side.
Potential issues
-
On bad connection to server Matrix client can send attachments as events with
"is_attachment": truebut not send finalm.messageevent, this will lead to posting invisible media to room. This can be solved on client side via caching unsent group of events, and repeat sending when connection will be recovered. -
In option one - individual media event, to which
m.messagerefers, can be deleted (redacted) after. As result,m.messagewill contain relation to redacted event. In this situation Matrix clients can exclude this item from display. -
In option one - there are no restrictions, that message with attachments can refer only to other events, that have
"is_attachment": true, because this is not too easy to control, and in theory user can post message, that can refer to other media, owned by other users, andredactevent will try to delete them. But the API should restrict regular user to redact events of other users (if he isn't moderator), so thoseredactactions should already be successfully ignored by server. -
If client attach too much media to one message, he can got rate limiting problem on server side. This can be solved via splitting and delaying send of attachments, to match server rate limits.
Future considerations
In future, we may extend the m.attachments field with new types to allow
attaching external URL as cards with URL preview, oEmbed entities, and other
events (for example, to forward the list of several events to other room with
the user comment).
Unstable prefix
Clients should use org.matrix.mscXXXX.m.attachments and org-matrix-mscXXXX-mx-attachments strings
instead of proposed, while this MSC has not been included in a spec release.