Spec Proposal: Super Groups

Feature Proposal
1

Delta Chat Spec Proposal: Super Groups

Terminology

semi-public group: A group that is intended to be used for more or less public interactions, its invitation link can be shared in public spaces like social media or websites. It is “semi-public” and not “public” because it is up to the members to distribute its invitation link, they could keep it private and only invite certain people or share it only in certain spaces, that is: there is no public discovery of the group inside Delta Chat.

The problem

There is a need / “market” for semi-public groups by some communities, for example influencer fan group, open source software projects’s support rooms, small communities or collectives that want to have some public invitation link in social media and/or their pages to join a room for support, feedback or interaction with the community. Existing solutions like Matrix and XMPP have several disadvantages (server and client side) and there is a need/demand for better solutions.

It is happening already that individuals and collectives are sharing invitation links to their Delta Chat groups in social media and other public spaces, even if this is officially not recommended by the Delta Chat developers. This is problematic, the existing groups feature in Delta Chat is intended for private groups usage only, this causes bad experiences due to the problems listed below and deteriorates the reputation of Delta Chat.

Existing groups don’t scale well for semi-public usage, they are not suited for more than 100-500 members, the more members the harder and slower it gets to send a message because computational overhead of encrypting for every group member’s public key.

Joining such a huge group with potential strangers also mean they can suddenly start sending you spam in private or adding you to new groups against your will.

When joining a group, it is not possible to see any chat history.

Lack of basic group administration is also a problem, anyone can modify group name and picture, add new people against their will, remove members / destroy the group, etc.

Properties of a Super Group

  1. Scalable: It is a symmetrically encrypted decentralized mailing list like the existing broadcast channels feature but everyone knows the member list and can write messages instead of having a single person allowed to send.
  2. No Spam: People don’t share their public key when joining, but participate using the group key without revealing their public key so no spammer can write to them in 1:1 or add them to other groups, if you leave the group, you forget the key and never see any message again, no one can keep spamming you just because you joined a group once
  3. Invitation only: Since it uses a symmetric key, people can’t add other people manually, only via invite links like with the existing Channels, that avoids getting added to super groups against your will
  4. [Optional] Chat History: Since the group uses a symmetric key, then it is not problem at all for the inviter to resend the chat history when adding a new member, with the existing private groups this is not possible because other people’s messages are encrypted only for the list of members at the time of sending, so if you re-send them as-is the new member can’t decrypt them.
  5. [Optional] Moderation: It could have an admin with something like what “Owned Group” solution proposed at New feature: Group Types - #37 by adbenitez then only admin can edit the group settings and invite or remove other members. This is necessary for the public usage to avoid the mentioned problems of lack of moderation, but could be rolled out in a next step, just with the properties of the previous points alone it is already much more useful than the existing private groups that have a big spam problem via “clone chat” etc.

Annex I: Implementing a super-group prototype today

A basic limited prototype of a pseudo-supergroup as proof of concept can be implemented with the existing Delta Chat apps and tools as follow:

  1. Create a bot that creates a channel and give you the channel invitation link
  2. When someone joins the channel:
    2.1. the bot re-sends the last N messages as chat history (using existing re-send API)
    2.2. the bot sends a welcome message in the 1:1 with the user explaining that any message they send there will be published in the channel (and to leave they just have to leave the channel or send /stop)
  3. Users can use “reply privately” in the channel messages to quote-reply a message in the channel
  4. Distribute the channel’s invite link with users so they can join the “super-group” prototype

The obvious problems of this limited prototype are:

  1. It depends on a central bot
  2. It is slower because messages flow is User -> Bot -> User instead of just User -> User
  3. It is too complicated to use with the separation between the channel and the bot’s 1:1 chat
  4. A webxdc in the channel could be used with the realtime feature, but then UI needs to be created inside the webxdc re-inventing all the features of the parent chat app and it depends on people having the app open.
6 Likes
2

Hi,
I’m exactly in this kind of situation where I want to create a public group and I don’t want to start it on Telegram/Whatsapp/Signal because I found them less secure compared to the end-to-end encryption we have on DeltaChat. This has been a major disadvantage for me to recommend DeltaChat more to friends. I don’t know if Deltachat devs realize the beauty of what they created, but there is a need to add the missing features.

SimpleX chat enable large groups, the only downside is SimpleX is not working well on multi-devices.

2 Likes
3

I personally am not interested in a group mechanism that degrades security. I think there’s a good point to be made where many semi-public groups may still want to have access to slow group key rotation, and so on. On Matrix, which I don’t like using due to how unreliable it is, I have a semi-public group where I don’t have sharing the group history with newly joining people enabled. I don’t find it important.

(I’d go as far as saying, if semi-public groups had worse encryption than the normal groups on DeltaChat, I probably wouldn’t use them but rather stay with Matrix for that.)

I don’t know the technical internals well, but I’m guessing moderation based on owned channels should in theory be possible with the currently existing groups, by having the members refuse any moderation actions and removed member updates that weren’t sent by the owner.

Past tests with 400 people suggested it works, but I don’t know if this has since changed.

TL;DR: I think group ownership is a good idea. Weakening the security, not so much.

4

As a side note that just occurred to me, didn’t DeltaChat already have a problem with missing identity key swap mechanisms, e.g. when a user lost a device but is quite confident the key hasn’t been extracted yet, and wants to change the identity key? (Not sure if that’s fixed?)

Based on that, designs adding new types of unswappable keys seems like a risky route to me.

But I might totally be missing something here, my apologies.

5

I don’t understand what you mean by “weakening the security” this is not weakening anything because the security of the existing private groups is not changed, if you like that use that, this new type of groups has the same security as for the existing broadcast channels, in any case, the usage is public use cases where potentially encryption is not even needed (most public groups and channels are not even encrypted at all in most platforms)

NOTE: I can smell in the air that you will turn the thread in the direction of asking for feature requests for existing private groups (adding moderation/admin for private groups) this is out of the scope of this thread, and the existing private groups can’t solve the problems of the use case described here for the very lack of the property you call “weakness” so please don’t dive into that other feature request here, better if you create a dedicated thread for that other feature request

3 Likes
6

To highlight the need to trust the group managers or hosts in this type of group, maybe we can call this “Managed Group” or “Hosted Group”.

Actually this kind of group that disabling non host/manager to contact other group members is better than Whatsapp/Telegram which only protect the member direct messages at certain size/settings.

7

I have a working proof-of-concept implementing something very similar: a relay bot using a new BlindGroup chat type.

Key properties:

  • Members see a regular group (single visible member: the bot) and chat normally
  • Bot relays messages, preserving quotes and aggregating reactions
  • Members don’t see each other directly
  • Relayed messages are encrypted separately per recipient (but sent in a single message to avoid rate limiting)
  • Works with unmodified clients, only the bot needs core changes

Here is a link to the experimental implementation (which would need more testing before being used). The core changes are minimal, mainly adding the BlindGroup chat type that behaves like OutBroadcast but allows receiving messages from members. Feedback on the architecture and implementation would be much appreciated.

2 Likes
8

nice, I also had similar bot in the past, the problem is that it is much slower since the bot has to distribute (and encrypt) the message to every member individually, when the group size grows, which is the whole point of having a super group, for big groups, it starts to become slow to chat there, messages will arrive minutes late etc.

1 Like
9

I didn’t know broadcast channel had this weaker type of security. I simply think they’re not suitable for a many-to-many chat then, even if semi-public. I think people would expect a moderated chat on DeltaChat to have the same security level as the usual unmoderated ones. (It seems like Signal also doesn’t have shared group chat history, so I’ve heard today. That seems more reasonable to me.)

I don’t mean to derail, I’m just giving honest feedback. Sorry if it came across as anything different.

10

Let me give a more practical example:

This might be the type of users DeltaChat perhaps is catering to. (Not trying to comment on their activities, just what’s the typical DC audience.)

Would they have likely picked a moderated group at that size of movement?

:check_box_with_check: I would say it’s very likely.

Would it have been unproblematic for them to use a shared permanent symmetric key?

:cross_mark: In my opinion no, since their recorded encrypted data streams will likely be looked at now, if there were recorded before.

Would it have been essential to them to have shared history on join?

:cross_mark: In my opinion no, maybe it would have been convenient, but likely they were manually observing new joiners to constantly re-evaluate trust where no shared history seems safer.

IMO, therefore a DC group with such properties isn’t a good fit for DC.

Edit: especially since I think many of the features are in theory doable basing this on private groups, too. E.g. by the core auto dropping messages from strangers if they got introduced via groups marked in some way, making the core not accept being added to some rooms based on a marker, etc. And 400+ people seem to work, too, since we tested this once.

11

So i guess that’s why other big platforms aren’t that secure, because we need to squeeze the performance from existing technology.

I still think DC can highlight that it is more secure than other big platforms though, because people can only see other users interacting in the group, not able to directly messaging

..and because the messages aren’t re-signed, there is plausible deniability that the messages aren’t actually from that other users. [edit: i just realized i may be wrong here, if the messages aren’t resigned, then people can’t deny it is from them. I don’t know why i thought otherwise]

We can then put warnings like what is being shown in browser incognito mode (your ISP can still track you, etc.)

12

Maybe some prototype like this can work:

  • each participant has a channel. They can send messages on this channel.
  • the admins share a channel. This channel is only used for participation, not distribution messages. This channel has a link, this is what identifies the group
  • participants subscribe to this channel. When someone joins/leaves the super group, the list of current people is sent on that channel. Every participant subscribes to everyone else (and removes those that are not in the central list)
  • when a participant sends a message, they send it on their personal channel, every other participant receives it

It’s still very manual though

13

if you need top security, you use a private group, if you need a huge chat with +1k users, the security of a super group is more than good, in fact at that point it doesn’t even matter it is encrypted, and theoretical principles only get on the way of such group’s scalability,

I think we have a clear cut here: private groups and public groups, each can be good at their specialized use cases, while will fail if you want to make them a “jack of all trades”

14

I did answer this in chat earlier, but let me share here:

I disagree, I think the cut is 1. up to 10 close friends (unmoderated, high security desirable), 2. semi-public groups with public links for niche topics where you don’t want trolls (moderated, high security desirable), 3. then somewhere far off in the thousands of users (moderated, high security impractical).

I think if you introduce 3 first, it’ll be misused for 2 and security will falter. I think that’s bad. I’ve also seen nearly nobody ask for use case 3, while 2 is asked for almost every month.

15

false, these people are not using it for 2, but for 3, what you call “semi-public” groups are in fact groups with random people that don’t know each other, and often even let trolls in and destroy the group, as soon as there is public link the “high security desired” is pointless:

so it is clear that the real cut is:

  • very secure, with trusted people, no one can join and spy
  • public link, anyone can infiltrate, members don’t know/trust each other, so security is weak, encryption can’t help

the cut is not about “somewhere far off in the thousands of users”

now as I already asked earlier, please open other threads to propose 2, 1, or whatever this thread is about 3 (super groups for public usage)

16

I linked above where the distinction matters: Kash Patel says the FBI is investigating Signal chats of Minnesotans tracking ICE This seems to be DC’s intended type of audience. So I just don’t see how you come to that conclusion.

Sorry, I don’t mean to derail your feature. Rather suggest I think adding it as-is would cause problems so I would suggest not to.

17

I don’t have time to read that article, I disagree with you that this feature would cause any problem, in fact it will solve a lot of problems, it is anyways needed if you want any group with ~1k users, which any other platform support, if Delta Chat doesn’t support it this would mean they will just not pick Delta Chat

there are need for such public groups as stated in the very beginning of this post in the “the problem” section, currently that people are using public rooms in IRC, Matrix, etc. without encryption or it doesn’t even matter because anyone can join and see history or spy, it is a public room for support etc.

2 Likes
18

Apparently Matrix with sliding sync can now do strong encryption relatively well with sliding sync for the 50 to 500 users range. (Haven’t tested it enough myself, but I’ve heard that reported on the 39C3 too.)

I don’t think I can explain my concerns more than how I already did with conrete examples.

19

I actually going to use this semi public group for strangers. As discussed, we already able to do that for the private group, even if moderation is not there.

I think bringing moderation to private group should be separate topic (such as making it possible to coordinate the election & moderation so no centralized power in the bots existing in the group).

That’s why i don’t really care if intel (government spy in Indonesian lingo) is in the semi public group. As the managers/hosts of the semi public group, i will then selectively invite the people i can take a risk to be in the more secret group. Not that i am doing that, i am just a normal citizen pro government :grinning_face_with_smiling_eyes:

Just to make it more obvious, we can call this semi public group type “Public Managed Group” and give huge big warning message on the beginning of the group creation or even a paragraph in the pinned message.

Sure, I can use IRC, Matrix, XMPP or so on, but it will be nice if DeltaChat has it all included, even if it is the less secured version than the ideals.

If this proposal doesn’t proceed, then my obvious route will be having something like Fredium/Beeper, where a big app containing everything every kind of protocol/network needed to cover all use cases. Delta Chat will just be one of them.

But again, DeltaChat is nice, because it is email upgraded. I really want to see how far this can go. I hope i will have enough skills and free times in this year.

20

I have created separate topic in case so we can stay on track: https://support.delta.chat/t/coordination-moderation-election-without-bots/

1 Like