Document the minimum requirements necessary for chatmail interoperability

, , ,
1

I am currently working on Delta Chat interoperability for bower/notmuch ( GitHub - wangp/bower: A curses terminal client for the Notmuch email system · GitHub , https://notmuchmail.org). My goal is to support enough of the features Delta Chat uses that I can comfortably use my command line email setup to exchange messages with Delta Chat users. Currently, I am also using my command line setup in Multi Device Mode together with the Delta Chat Android app from F-Droid.

A couple of days back, I ran into what turned out to be an unexpected (from my point of view) quirk in Delta Chat’s behavior (see Upgrading a 1:1 conversation to e2ee using a second email client does not work as expected · Issue #7945 · chatmail/core · GitHub ). Therefore it would be valuable for me to know what the minimum requirements are for basic chatmail interoperability. To be clear: I do not want to incorporate chatmail/core into bower/notmuch and I do not want to support every single feature implemented in Delta Chat. For the moment, I only want to support as much of what Delta Chat does that I can use bower/notmuch to read and write messages (including group messages).

As of now, I have added, am in the process of adding, or am planning to add support for the following features:

  • Header Protection (RFC 9788)
  • Content-Type: reaction (RFC 9078) [read-only support]
  • Autocrypt Level 1
  • Autocrypt v2 [once that’s finalized]

On the other hand, I am NOT currently planning to add support for:

An additional comment about the non-standard extensions: I am not opposed to these extensions; and I do like the use of Chat-* headers for user avatars, group list modifications, and the like. However, I am currently fine with not having shortcuts for these actions available in my command line setup, so if I can get by without implementing these non-standard extensions, I will prefer that for now.

One thing that came up in Upgrading a 1:1 conversation to e2ee using a second email client does not work as expected · Issue #7945 · chatmail/core · GitHub (and which I took up in Simplify "Show Classic Emails" setting (Resolved) ) is that the presence of the Chat-Version header can actually influence message deliverability if the other party has “Show Classic Emails” set to “No, chats only”. If the only way to fix this is to add a Chat-Version header, I might consider doing this. On the other hand, I guess that adding a Chat-Version header without implementing any of the other functionality described in core/spec.md at main · chatmail/core · GitHub would pretty much be the exact opposite of what this header is intended to achieve; so I would very strongly prefer not to do this.

In any case, it would be very useful to me if there was a list of minimum requirements that third-party email clients need to support to be considered chatmail-compatible. I am aware of core/standards.md at main · chatmail/core · GitHub , but this seems to be the maximum one could try to support, rather than the minimum.

4 Likes
Is it possible to communicate with non-DC user? (interoperability with email)
2

ftr, development topics do not really fit to “Feature Proposal”, which is meant for the end-user of the Delta Chats App to be understandable. moving the topic to “Uncategorized” therefore; if more such topics pop up, we can think over a separate category, but in the past, they were very few, so that seems fine

3

One recent change is split of large messages into a “pre-message” with text and metadata and a “post-message” with an attachment: feat: pre-messages / next version of download on demand by Simon-Laux · Pull Request #7371 · chatmail/core · GitHub

4

On a side note, notmuch is how I think email should be handled instead of IMAP folders. Feed everything into “notmuch” or similar program, then filter out what you need with local search. Sorting everything on the server is not compatible with encryption and computers can do local search faster than any network request.

1 Like
5

I strongly support having formal interoperability standards.

This would be not core/standards.md at main · chatmail/core · GitHub , but the formal Chatmail spec linked from it, I think, if I have understood what you want correctly.

The Chatmail spec is not to be confused with chatmail/core. On names for these standards:

6

In Reply to @Minim

I strongly support having formal interoperability standards.

Thanks. Glad you like the idea as well :slight_smile:

This would be not core/standards.md at main · chatmail/core · GitHub , but the formal Chatmail spec linked from it, I think, if I have understood what you want correctly.

No, the “Chatmail Specification” you mention is (mostly) an informal description of non-standard email headers used by Delta Chat. It would therefore be closer to a full list of features implemented in chatmail/core than it would be to a list of minimum requirements for basic interoperability. What I am interested in are these minimum requirements.

In Reply to @link2xt

On a side note, notmuch is how I think email should be handled instead of IMAP folders. […]

I’m a big fan of the unified inbox design as well. I also noticed that there’s some contention about the deprecation of “Move automatically to Delta Chat folder” – and I personally don’t mind this change at all. I also noticed that “Multi Device Synchronisation” messages are no longer delivered to the Delta Chat folder either and I don’t find this to be an issue. However, I hope that once I can decrypt OpenPGP v6, I will still find a Content-Type parameter that can be used to automatically distinguish these messages from all others :wink:

One recent change is split of large messages into a “pre-message” with text and metadata and a “post-message” with an attachment: feat: pre-messages / next version of download on demand by Simon-Laux · Pull Request #7371 · chatmail/core · GitHub

I noticed this already. I do not currently see a need to make any adjustments for reading such messages. I also do not want to make any adjustments to how I send messages with attachments; but after reading through the github issues, I am no longer so sure if this is possible (and in which exact cases).

I guess my main concern is message deliverability. If I receive messages that I can’t read – or where I don’t get all of the nuances (e.g. I don’t see the Chat-Group-Avatar) – that’s still okay-ish. If messages I send are silently dropped by Delta Chat, that would be a huge problem. Especially since bower/notmuch users wouldn’t even know that their message did not arrive.

So what I would really like to see is an official statement like this: “As long as a message meets at least the following requirements, we will display it to Delta Chat users as a ‘chat message’ and we will not just silently drop it.” This could even be a living document that you could update as needed. If these minimum requirements were documented somewhere, I would at least know what to expect. And I would also have a basis on which to complain on the bug tracker in case my messages still don’t get delivered properly :wink:

Here’s a list of things that have cropped up so far – both things where I am pretty confident that they are hard requirements and things where I don’t know if (or under which circumstances) they are actually required or just nice-to-have:

  • OpenPGP v4 signature and encryption (conformant with Autocrypt Level 1)
  • Intended Recipient subpackets in the signature (see Upgrading a 1:1 conversation to e2ee using a second email client does not work as expected · Issue #7945 · chatmail/core · GitHub )
  • An Autocrypt header (Autocrypt Level 1)
  • Autocrypt-Gossip headers (Autocrypt Level 1)
  • RFC 9788-style Header Protection (also: What’s the minimum requirement for the Header Confidentiality Policy? – if any there is)
  • Either a Chat-Version header or a reference (e.g. via In-Reply-To) to a previous message that contains a Chat-Version header
  • Max. message size (unless split into pre- and post-message)
  • Possibly certain requirements for the message’s MIME structure. E.g. no nesting of multipart/mixed, only a single inline body part, maybe a whitelist of supported content types for “Content-Disposition: inline” (e.g. [text/plain, text/html]), etc.
1 Like
7

I believe the Chatmail standard was originally intended as a list of requirements for client interoperability. It seems it’s pretty out-of-date. I think it counts as a bug if the Chatmail/core backend no longer meets the Chatmail standard. :slight_smile:

Being based on open standards is a really major promotional point of Deltachat. A software-defined standard is not intended, and – as you’ve described – causes problems with decentralization, reliability, and interoperability.

8

For sending it is fine to send large messages as is, this only means receiver will download the message immediately because there is no way to show a preview. Older versions of Delta Chat showed preview for such messages, but it was a preview that just said “a message of size …”.

9

For sending it is fine to send large messages as is […]

Well, that’s a relief.

Still, if you find the time to do a more complete writeup of minimum requirements at some point (maybe starting from the list I included previously, if you like), that would be quite useful. For the time being, I’ll just keep going with trial-and-error – which already got me pretty far, I guess :wink:

10

Shall I try writing up what you’ve done in a wiki post on the forum here? Even if you don’t subsequently add anything, or just add bare links for others to expand on, it would be a start.

A statement about when the specialized chat headers should and should not be used, i.e. what they mean, would also make sense.

1 Like
11

On a side note, notmuch is how I think email should be handled instead of IMAP folders. […]

I’m a big fan of the unified inbox design as well. I also noticed that there’s some contention about the deprecation of “Move automatically to Delta Chat folder” – and I personally don’t mind this change at all. I also noticed that “Multi Device Synchronisation” messages are no longer delivered to the Delta Chat folder either and I don’t find this to be an issue. However, I hope that once I can decrypt OpenPGP v6, I will still find a Content-Type parameter that can be used to automatically distinguish these messages from all others :wink:

One recent change is split of large messages into a “pre-message” with text and metadata and a “post-message” with an attachment: feat: pre-messages / next version of download on demand by Simon-Laux · Pull Request #7371 · chatmail/core · GitHub

I noticed this already. I do not currently see a need to make any adjustments for reading such messages. I also do not want to make any adjustments to how I send messages with attachments; but after reading through the github issues, I am no longer so sure if this is possible (and in which exact cases).

I guess my main concern is message deliverability. If I receive messages that I can’t read – or where I don’t get all of the nuances (e.g. I don’t see the Chat-Group-Avatar) – that’s still okay-ish. If messages I send are silently dropped by Delta Chat, that would be a huge problem. Especially since bower/notmuch users wouldn’t even know that their message did not arrive.

So what I would really like to see is an official statement like this: “As long as a message meets at least the following requirements, we will display it to Delta Chat users as a ‘chat message’ and we will not just silently drop it.” This could even be a living document that you could update as needed. But if these requirements were documented somewhere, I would at least know what to expect. And I would also have a basis on which to complain on the bug tracker in case my messages still don’t get delivered properly :wink:

Here’s a list of things that have cropped up so far – both things where I am pretty confident that they are hard requirements and things where I don’t know if (or under which circumstances) they are actually required or just nice-to-have:

  • OpenPGP v4 signature and encryption (conformant with Autocrypt Level 1)
  • Intended Recipient subpackets in the signature (see Upgrading a 1:1 conversation to e2ee using a second email client does not work as expected · Issue #7945 · chatmail/core · GitHub)
  • An Autocrypt header (Autocrypt Level 1)
  • Autocrypt-Gossip headers (Autocrypt Level 1)
  • RFC 9788-style Header Protection (also: What’s the minimum requirement for the Header Confidentiality Policy? – if any there is)
  • Either a Chat-Version header or a reference (e.g. via In-Reply-To) to a previous message that contains a Chat-Version header
  • Max. message size (unless split into pre- and post-message)
  • Possibly certain requirements for the message’s MIME structure. E.g. no nesting of multipart/mixed, only a single inline body part, maybe a whitelist of supported content types for “Content-Disposition: inline” (e.g. [text/plain, text/html]), etc.
1 Like
12

As a first step, would it be possible to make clients display, in some form, any non-SEIPD message from a non-blocked key that would be accepted by a Chatmail/relay server?

Then the sender gets a message if it does not go through, and otherwise the recipient gets a message even if it is mangled. No messages vanish silently unless they are blocked.