UX regression when onboarding new users

Note: I had previously opened this issue as https://github.com/deltachat/deltachat-android/issues/3902. I hereby open it as a feature request in the forum, suggesting that the UX described below be improved.

tl;dr

I onboarded a friend to Delta Chat a couple of days back and have two major grievances:

  1. Autocrypt didn’t kick in. (Not sure if this is intentional or a bug, though.)
  2. After performing SecureJoin, instead of upgrading our existing conversation to verified end-to-end encrypted, Delta Chat gave both FRIEND and MYSELF two versions of each other. One encrypted and one non-encrypted. I strongly dislike that UX. We just went through the trouble of performing out-of-band verification of our respective credentials; none of us wants to have Delta Chat send unencrypted messages by accidentally selecting the wrong version of two identical names in our chat lists.

The full story

A couple of days back, I onboarded a friend to Delta Chat and found the experience a lot less comfortable than I had anticipated. Some background: FRIEND and MYSELF have been in contact via email for some time. FRIEND is a non-technical user but strongly interested in using end-to-end encryption. I recommended Delta Chat (which I myself have also been using for some time now), expecting that Delta Chat would make end-to-end encryption easy for her. Unfortunately, this was not what I experienced.

Expectation:

  1. FRIEND installs Delta Chat and enters her email credentials.
  2. End-to-end encryption between FRIEND and MYSELF just works.

Reality:

  1. FRIEND installs Delta Chat and enters her email credentials. (Some trial and error here, but in the end we get it working; not much Delta Chat can do about that, obviously.)
  2. We do the QR scan to set up verified end-to-end encryption. I suddenly have two instances of FRIEND in my chat list.
  3. We delete the chat from both apps in order to start again.

  1. We have another look at FRIEND’s settings to ensure that the display name is set to the same string she used for sending regular email.
  2. We do the QR scan again to set up verified end-to-end encryption. Again I have two instances of FRIEND in my chat list.
  3. We delete the chat from both apps in order to start again.

  1. I use Delta Chat to send FRIEND a regular email, hoping that her Delta Chat will pick up my Autocrypt header.
  2. FRIEND replies using Delta Chat, but the reply is not end-to-end encrypted.
  3. I MYSELF reply using Delta Chat, but this reply is not end-to-end encrypted either. (This might have been caused by a missing settings migration strategy, see below.)

  1. We do the QR scan again to set up verified end-to-end encryption. Now we both have two instances of each other’s contact in our respective chat lists. Note that both versions of FRIEND I now have in my chat list have the exact same display name and email address. The only visible cue about their difference is the profile picture, which is an envelope in one case and a colored letter in the other. The same is true about the chat list in FRIEND’s Delta Chat.
  2. I now need to explain the concept of encrypted vs. unencrypted chats in Delta Chat to FRIEND.
  3. I start to wonder if it would have been easier to teach FRIEND how to use gpg from the command line and pipe her emails into sendmail.

What I would have hoped for would have been this:

  1. FRIEND installs Delta Chat and enters her email credentials.
  2. As we continue writing emails to each other, our conversation is automatically upgraded to end-to-end encryption by the power of Autocrypt.
  3. Once we do the QR scan, our conversation is automatically upgraded to verified end-to-end encryption by the power of SecureJoin.

Note that step 1 worked well enough, but steps 2 and 3 did not work nearly as well as I had hoped for. I feel that this is a huge usability issue and arguably a huge regression when it comes to UX. I do not have that much experience with onboarding new users; but I vaguely remember that it was easier once. I also remember that conversations were once automatically upgraded to end-to-end encryption by the power of Autocrypt. This was slightly unexpected for me at the time, but in retrospect that was the nudge I needed to finally enable end-to-end encryption on my laptop as well.

Key takeaways

I believe that the subpar experience I had with onboarding FRIEND is a very serious usability issue. As I already mentioned, I even started to wonder if I can keep recommending Delta Chat to my peers. I believe that Delta Chat needs to figure out what it wants to be. Does it want to be an email client that looks like XYZ but does regular email behind the scenes? Or does it want to be an email client that looks like XYZ but does (opportunistically) end-to-end encrypted email behind the scenes? Currently, it tries to do both; and therefore achieves neither. Having two versions of the same contact in the chat list simply transfers the complexity of whether or not to use end-to-end encryption to the user. If you ask me, this is not acceptable. Delta Chat, in the spirit of Autocrypt, used to remove complexity from end-to-end encryption by making good choices for the user. (Which key format? Which hash algorithm? Which curve? Sign with the primary key or use a dedicated signing subkey? How to handle multiple user ids?) With the UI switch from the padlock to the envelope, Delta Chat apparently started adding complexity. And what is more: UI complexity of the sort that to me seems outstandingly inconsequential. If I need to explain the difference between RSA and cv25519 to someone, at least they learn something useful in the process. If I need to explain the difference between envelope-emoji-me and colored-initial-me in some chat list, this is simply a waste of time.

I strongly hope that this issue will be addressed. I also believe that being opinionated about enabling end-to-end encryption is the way to go. Delta Chat already mandates end-to-end encryption for users on chatmail servers. I would argue that it should take a similar stance for other users. If people decide to use Delta Chat for Android with their regular email account, that is already a strong indication that they wish to use end-to-end encryption. (Keep in mind that installing a third-party email client on your phone is hardly a commonplace thing to do in today’s tech landscape.) I therefore believe that Delta Chat should at the very least automatically enable end-to-end encryption for those 1:1 conversations where it is easily possible. Such as those where both chat partners use Autocrypt-capable email clients or where they even did a QR scan. I do not believe that Delta Chat should cater for the use case of: I did a QR scan with PEER, but I still want the ability to turn end-to-end encryption on or off on a per-message basis.

Sorry about the strong feelings. In general, I am deeply grateful that Delta Chat exists and I am strongly aligned with what you guys are trying to achieve. Also: I want to see you succeed.

Additional things that may be of interest

I have a feeling that my Autocrypt prefer-encrypt setting might be broken. I had once adjusted my settings to prefer unencrypted email; but it has been a while that I can not find that toggle in Delta Chat’s settings anymore. However, inspecting the mails generated while onboarding FRIEND, I realize that FRIEND sends Autocrypt headers like this:

Autocrypt: addr=[FRIEND's address]; prefer-encrypt=mutual; keydata=[[ base64
<tab><space>encoded public key, spanning multiple lines ]]

while I MYSELF send Autocrypt headers like this:

Autocrypt: addr=[MY address]; keydata=[[ base64
<tab><space>encoded public key, spanning multiple lines ]]

What’s the policy here? Did I simply fail to figure out where the prefer-encrypt setting is located? Was prefer-encrypt made non-configurable for Delta Chat? And if so, why does my client continue to use the deprecated setting – that I can’t even change anymore – instead of migrating my settings to match the new way of doing things (i.e. to always send prefer-encrypt=mutual)? Did someone simply forget to add a database migration rule for that case?

That header does seem like a problem. Autocrypt should automatically upgrade to encryption if you and your corresponding FRIEND both ask it to, but you aren’t asking. You could manually go into the database and look at the prefer_encrypted field in the acpeerstates table (DB Browser for SQLite is a decent GUI if you want one, and sorta works on mobile Linux; no idea on Android). Actually, I have some of those set to 1 and some to zero, even just among contact Chatmail accounts on the testrun server (so it’s all encrypted anyway). I have no idea why. This may be a bug.

The forum has many suggestions for changing the UI, some of which you might well agree with. Mockups are welcome.

I agree that assuming that everyone prefers encryption where automatically possible is reasonable, though it is not impossible to think of edge cases.

Looking more closely, I have prefer_encrypted set to FALSE only for sending messages to myself, which is fine.

Seifferth, detailed descriptions of what happened are useful, and the point-form helps. But it is difficult to respond to very long posts. Where you can manage to be more concise, and add subtopics, details, opinions etc. in subsequent discussion, it helps.

Autocrypt should automatically upgrade to encryption if you and your corresponding FRIEND both ask it to

I agree that this should work. Unfortunately, this is currently broken in two ways:

  1. I have no way of getting Delta Chat to add “prefer-encrypt=mutual” to my autocrypt header. Except maybe for manually editing the database as you suggest – but I don’t think that would be consistent with the general UX concept Delta Chat is aiming for. This particular issue has been reopened as "Prefer encrypt" setting was removed without resetting it to the default value · Issue #3911 · deltachat/deltachat-android · GitHub and I hope this will be solved as soon as a new version containing Remove e2ee_enabled preference by link2xt · Pull Request #7172 · chatmail/core · GitHub becomes available.

  2. Even if the prefer-encrypt property is set correctly, autocrypt will not kick in in conversations between Delta Chat users. As link2xt explained: “Autocrypt support is not removed. […] What is missing is the upgrade path for two Delta Chat clients” (see Support for non-chatmail email - #2 by link2xt).

Seifferth, detailed descriptions of what happened are useful, and the point-form helps. But it is difficult to respond to very long posts.

For the key issues I want to address, please refer to the “tl;dr” section. Everything else is just additional context in case that section alone should be too concise. To paraphrase these two key points:

  1. Autocrypt does not work properly any more.
  2. Performing SecureJoin duplicates existing 1:1 conversations. I would very much prefer it if SecureJoin would upgrade them instead.

I suppose these issues could be solved by introducing an upgrade path from non-e2ee-enabled conversations to e2ee-enabled conversations. This upgrade path is something link2xt mentioned in Support for non-chatmail email - #2 by link2xt (already cited above). I have posted some thoughts on what this upgrade path could look like here: Prompt to switch to encrypted chat when it is available - #6 by seifferth. I do realize that that post is not particularly short either.

1 Like