Separate collected contacts from the "contact list"

Feature Proposal
1

Delta Chat currently has a single contact list. Contacts are added there from all groups you participate in. While incoming chat is in the “contact request” state, its members are not added, but as soon as you accept it or send a message there (e.g. from another device), all members are added to your contact list. Similarly, joining a group or channel via QR code or invite link adds the inviter to your contact list.

Users have many times requested that Delta Chat does not add group members to the contact list:

I have recently proposed to stop adding contacts to the contact list in a simple scenario when the group contact request is accepted: Accepting group contact request adds group members to contact list · Issue #7592 · chatmail/core · GitHub
This was not implemented because it breaks the other scenario.
If a family or a small group of friends starts using Delta Chat and creates a group, they can start 1:1 chats with each other easily by clicking the “+” button and selecting the display name there or searching for it. If we do not add contacts to the contact list, users will have to go into the group member list and select the contact from there or click on the avatar next to the message sent by this member to start a 1:1 chat.

So we want to solve two problems:

  1. Have a contact list that can be trusted to only have contacts that the user actually knows. Others should not be able to add contacts to your contact list, e.g. by inviting someone to a group that you participate in.
  2. Make it easy to start conversations with contacts without manually adding them to the contact list first. This should be as easy as clicking the “+” floating button on the chatlist activity, searching for the display name and selecting the contact there.

We also have another constraint: searching in any list should not show results that you cannot find by scrolling through the list. See Increase minimal origin for contact list by link2xt · Pull Request #2411 · chatmail/core · GitHub for the proposal that implemented filtering the contact list without search, but showing more results when searching for specific contact, this was not accepted for this reason.

However, when searching, we can show multiple sections, like “contacts”, “group members” etc.
This is what Telegram does: screenshot.
Signal also has sections in the search accessible from the does something similar, see below.

Testing

I have tested XMPP and Signal because I also use them and can relatively easily check how this is implemented there.

XMPP

XMPP has a roster, your contact list stored on the server. I have tested Dino and Conversations, they both offer very similar options. You add contacts to your contact list manually, so it does not get littered with contacts you don’t know. However, there is no easy way to start a conversation with someone from the group, you have to go into this particular group member list. Basically XMPP clients I tested have a clean contact list but fail the second goal of making it easy to start 1:1 chats.

Dino

On Desktop I have tested Dino 0.5.1. There is a “+” button in the titlebar, tooltip for it says “Start Conversation”. Clicking on the “Start Conversation” button opens “Start Conversation” dialog with a short list of contacts that does not include everyone from any group ever joined. There is a search field, entering something into it filters this contact list but does not allow to find contacts from group chats. “Start Conversation” dialog has its own “+” button which opens “Add Contact” dialog. “Add Contact” dialog has an “Account” field (allowing to select one of my XMPP accounts),
“JID” field (where I can enter contact address manually) and “alias” field (I assume it is the display name).

I did not find a way to search all contacts that appear in any group by display name. Starting a 1:1 chat with group member requires going into the group chat, opening Members list, selecting one of the members and clicking “Start private conversation”.

Conversations

On Android I have tested Conversations version 2.19.7+free installed from F-Droid. When I open the app, I have a “Start chat” button. Clicking on it opens “New chat” activity with two tabs: “Contacts” and “Group chats”. “Contacts” is the same short list as the “Start Conversation” in Dino. “Group chats” tab is some list of group chats, not interesting for “contact list” testing.

On the “new chat” activity there is a floating “+” button, it offers options to “Add contact”, “Create private group chat”, “Join public channel”, “Create public channel”, “Discover channels”. Clicking on “Add contact” offers a small dialog with “Your account” and “XMPP address” fields.

Signal

Signal has clean contact list and makes it easy to start 1:1 conversations by searching for all known contacts from the main screen on Android. However, the search result sections are different on Android and Desktop, and documentation is unclear about what “contact list” means, so we cannot just copy what they do.

I have tested Signal (actually Molly). I do not share my phone address book to Signal. When I click on then “pencil” floating button on my Android phone, “New message” activity appears. There I see my contact list and it does not include everyone from the group. If I start typing there into the search bar, the contact list gets filtered, but no new contacts appear, e.g. I cannot find anyone who is not in my contact list.

However, if I go to the chatlist screen and search there for some display name of someone who is in the group, search results include two sections:

  1. Group members: this includes all groups that have a member with this display name.
  2. Contacts: this includes members who are in the group but are not in my “contact list”, I cannot find them in the “New message” activity but can click on them here.

Signal has an answer to question “Who appears in the contact list?”. It says:

  1. Other Signal users who are saved to your phone’s contact list.
  2. Other Signal users who are not saved to your phone and you share a group with,
    have previously messaged or have explicitly approved.
    However, I do not see “Signal users … you share a group with” in the contact list, so it is not clear what they call the “contact list”.

Signal Desktop also does not show “Group members” section, this only happens on Android to me, so the behavior is inconsistent.

4 Likes
2

How about a directory structure for contacts? People I only know from a group could go into a directory named for that group (maybe optionally, with exempted groups). Sets like “everyone I have 1-to-1chats with” could also be directories.

3

Maybe we can start with smth simple like adding separators between contacts having different “origins” because now (starting from v2.36, not released yet) we sort contacts by “origin” in the first place (more precisely, contacts from groups are sorted to the bottom). Separators could be named “Group contacts”, “Contact requests” etc. (“Contact list” would be the first section w/o any named separator to save screen space). A downside is that if you want to see “Group contacts”, you should scroll through the full (or filtered) contact list, but i have no idea why one may want to see “Group contacts” in the first place – maybe only if the user forgot which group the contact is a member of and searches through all contacts because of that.

There’s also one trick that in a sense simulates a manual “address book” which is missing in Delta Chat: you can create a group with only self and share vCards there, also adding any custom text, then you can search in this chat (but currently search doesn’t look into the vCard displayname, i think this should be fixed), edit/delete messages (i.e. contact entries), forward messages to the bottom to “prioritize” contacts, etc. This is of course not very convenient currently, but can be used as a basis for the “address book” feature. I think it’s better to avoid adding new windows to UIs if possible which users would have to learn. E.g. i like Emacs’s “everything is a buffer” approach and in Delta Chat it could be “(almost) everything is a chat”.

1 Like
4

Contact origins

For those who read this and don’t know what “origin” is: Delta Chat stores for each contact its “origin”. The origin can be a QR code scan, outgoing message recipient, incoming message recipient, address book etc. Origins have corresponding numbers, e.g. AddressBook origin is 0x80000, OutgoingTo is 0x4000, IncomingTo is 0x400. Lower origin may be overwritten with a higher one, e.g. if the contact was initially added because it appeared in the incoming message To field, but the user later responded to the group and has put this contact into the outgoing To field. Higher numbers roughly correspond to higher trust so it is possible to sort by “origin” and have separators between them. The list is unnecessarily long and distinguishes between contacts that appeared in the To and Cc fields (and even Bcc field which only exists in draft), for example. “Origin” existed since earliest versions of Delta Chat and is not well designed. All contacts quickly reach at least “OutgoingTo” level simply because you replied to some group where the contact is a member.

I suggest we forget that contact origins exist and look at concrete scenarios and decide whether we want the contact to end up in the “contact list” and whether we want the contact to show up in search. These scenarios can then be turned into tests.

When contacts are added to the “contact list”

I still do not know how contacts end up in the contact list in Signal. The documentation is not very good and it is difficult to test because I cannot create new profiles for testing. It seems my contact list includes at least the contacts from my phone address book found by the phone number and contacts I have added by scanning the username QR code. I don’t know if starting 1:1 chat with some group member will result in adding the contact to my contact list and if accepting a contact request for 1:1 chat results in adding the contact into my contact list.

We definitely want to add contacts to the contact list when we actively scan the 1:1 QR code (not group join QR code).

We do not want group members to end up in the “contact list” just because we joined the group (either by accepting contact request or scanning invite QR) and messaged there.

Where contact list is used

Contact list is important for the “who can call me” setting:

If we introduce it with “My contacts” as the default, compared to Telegram we will accept much more contacts, basically anyone from the groups we participate in can call us.

5

To get this closer to concrete proposal, I have looked at all the screens that allow to search for contacts that I can find.

I looked at Delta Chat for Android, Delta Chat Desktop, Signal Desktop and Molly (basically Signal for Android). I additionally looked at Conversations in one place.
In the UI, I see these places where we use contact list and can search:

  1. Global search invoked by starting to type into the search field at the top of chatlist.
    Note that this is not a chatlist search, typing there does not just filter the chatlist,
    but replaces it with search results that has sections for chats, contacts and messages.
    I have tested Delta Chat for Android, Delta Chat Desktop and Signal Desktop,
    they all have the same 3 sections for chats, contacts and messages here.

    On Signal Desktop “contacts” section displays collected contacts, not the “contact list”.
    Just clicking on the contact in these search results created a 1:1 chat with the contact and adds it to my contact list and the top of the chatlist.
    Deleting 1:1 chats resulted in also deleting the contacts from the “contact list”.
    In Molly clicking on the contact does not create 1:1 chat and does not add the contact to my “contact list”.

    Molly results are not consistent with Signal Desktop,
    it has at least “Group members”, “Contacts” and “Messages”.
    “Contacts” is still the section with collected contacts, but there is also “Group members” section which lists chats that have members matching the search string.

  2. “New chat”, invoked with “+” floating icon in Delta Chat Android and Delta Chat Desktop and with “pencil”.

    This one displays “contact list”. Typing into the search field filters the contact list, but does not add new results.

    In Signal Desktop typing into the serach field there however finds contacts that are not in my contact list (just some group members) and also adds “Groups” section.
    This looks like global search without the “chats” section, I don’t like it. I did not find a way to search the contact list in Signal Desktop.
    In Molly searching the contact list correctly filters the contact list, but still adds “Groups” section so I can find existing groups when creating a chat.

  3. “Add members” when creating a group chat.

    In Delta Chat, both Android and Desktop, this shows the contact list.

    In Signal Desktop this shows the contact list, but when I start searching, even typing a single letter results in not the filtered contact list, but a search with a single “Contacts” section that strangers from groups.
    In Molly this correctly filters the contact list, but there is no way to add someone not from the contact list. I think this is still better than on Signal Desktop.
    I also looked at Conversations, it shows your contact list and allows to filter it, no global search here.

Since existing UIs even in Signal are inconsistent anyway, there is no solution we can just copy. It still would be nice if someone with access to WhatsApp and Telegram documents how these three screens (“global”, “new chat”, and “add members”) work there so we can learn from them. We have one Telegram screenshot at Increase minimal origin for contact list by link2xt · Pull Request #2411 · chatmail/core · GitHub but I have no idea what screen it is.

Proposal

As a principle, search should not result in finding more contacts. If additional results are added, they should be added below the filtered results after the separator.

For a concrete proposal of what to do, I suggest that once we do the following:

  1. Keep “global search” invoked from the chatlist as is, showing three sections for chats, contacts and messages. The first section “chats” is the filtered chatlist, next section “contacts” searches all collected contacts (not just the “contact list”), then “messages”
  2. “New chat”, invoked with a “+” floating icon, should show the “contact list”. Searching here should just filter this list, not suggest you strangers from the collected contacts. No need to complicate this with sections, if someone really wants to start 1:1 chat with some group member, they can click on the avatar next to the message, use group member list or global search.
  3. “Add members” should only suggest you members from your contact list. Searching here should filter the list. Again, for the first iteration I would not add sections here.

The biggest problem that I see is that if “Add members” only sugggest you contacts from the contact list, you may fail to create the group you want because some member is not in your “contact list” but only a member of some other group chat.
We might want to add “Global search” or “Collected contacts” section right below the filtered results in the “Add members”, but can also skip it for the first iteration because creating groups is a rare action and you can still modify the group while it is in unpromoted state, so you can add members to your contact list and get back to the group to finish the member list.

As for how we decide which contacts should end up in the contact list and which should not, we should at least test the following scenarios:

  1. Scanning an invite QR code for 1:1 chat should create a 1:1 chat and a contact in the contact list at the same time.
  2. Scanning a group invite QR code should not result in creating any contacts in the contact list.
  3. Scanning a channel invite QR code should not result in creating any contacts in the contact list.
  4. Sending a message to the group should not result in creating any contact in the contact list. It should be possible to get added to the group (via QR code or vCard), accept it, chat there, and still have empty contact list.
  5. Accepting a 1:1 chat contact request should create a contact in the contact list.
  6. Creating a 1:1 chat with the contact (e.g. by selecting the profile from the group and clicking “Send message”) should add contact to the contact list.

We only need to test how this works for new profiles. For existing profiles it does not matter if the contact list becomes shorter after upgrade or remains full of collected contacts, in any case it is not worse than it is today.

Implementation

For the implementation we need a new API to get only the contacts from “contact list”.
If we never display “contact list” next to “collected contacts”,
this could be a flag for JSON-RPC call get_contacts and CFFI call dc_get_contacts.
If we later decide to have collected contacts below the contact list in the “add members” screen, we will need another API that returns these two lists with a single database transaction, one more reason not to do this for the first iteration.

Internally separation of the “contact list” and “collected contacts” may be based on the contact “origin”, but this is an implementation detail and should not be tested explicitly and UIs should not know anything about the origins.

1 Like