How to install own chatmail and notification proxy?

Mutual Help
1

Hi there :wave:

So I was planning to upgrade my server which is currently running dovecot+postfix on Arch Linux to chatmail so that iOS devices get instant notifications.

However, after reading docs, FAQs and the forum, Iā€™ve came to the conclusion that I need to deploy the chatmail server. But now I have some doubts regarding thisā€¦

  1. Can I try to install chatmail on Arch Linux or will it fail? I would check and execute the code to see what it does but Iā€™m afraid it could break my current setup :grimacing:
  2. If I installed it successfully, would in need to install the notification proxy too or how would I register(?) my own chatmail with the central proxy? Iā€™m not sure I completely understand how it works, TBHā€¦

In any case, I just need some hints from someone with experience to know if what Iā€™m trying to do may be doable or if it is likely doomed, not the precise steps to make it work.

Also, if I get this to work, I may try to contribute a PKGBUILD to Arch Linux AUR so that other people may benefit from it.

Thanks in advance :blush:

2

Currenty chatmail setup expects Debian system as it installs packages using apt, e.g. here: chatmail/cmdeploy/src/cmdeploy/__init__.py at 610637da8009f13a3f2aa5c7a0af1cb51953e1b1 Ā· deltachat/chatmail Ā· GitHub
You can install chatmaild components manually without cmdeploy tool, but integrating this into dovecot+postfix setup manually is not documented. If you really want to do it, probably better setup a VM with Debian first to see how it works and migrate the configs from there.

Instant notifications rely on the metadata server that is integrated into dovecot via config options and Lua script.

There is a central notification proxy running GitHub - deltachat/notifiers: Notify devices on https://notifications.delta.chat/
There is no need to install your own and it is not really possible to do so as it requires tokens from Apple and Google linked to the Delta Chat application. You only need to install your own if you want to fork Delta Chat clients.
There is no need to register your chatmail server anywhere, chatmaild services will talk to notifications.delta.chat to send notifications but this does not need any special token, in the end phone operating system obtains a device token and it is passed back to Apple/Google servers when a new message arrives. How notifications work is described in FAQ - Delta Chat, you may also want to read the blog post Instant Onboarding and Instant Message Delivery - Delta Chat

1 Like
3

Got it. Thx :slight_smile:

Thatā€™s a good starting point I think. Depending on the time I have I may try to tweak the scripts to make them work in Arch or start with a VM as you said.

Maybe dockerizing it would be a good idea, thereā€™s another thread about that here ā†’ Chatmail server docker deployment

Iā€™ll report what I find.

4

Hi again :wave:

Iā€™ve managed to setup a Debian VM with version 1.0.3 of cmdeploy. Now I will investigate the config and replicate it in my Arch system.

If all goes well Iā€™ll try to document what I did. Maybe it can be added to the FAQ page or something like thatā€¦

However, I have one doubt regarding DNS: which of the requested entries are really necessary and for what? I ask this because Iā€™m not sure which entries are needed for autoconfiguration of clients, which for the push service, which for the SSH keys of the web server, and so onā€¦

In my previous setup it was enough with one DNS entry. :thinking:

The cmdeploy script have asked for all these keys at different moments of execution:

mydomain                     A [...]
mydomain                     MX [...]
_submission._tcp.mydomain    SRV [...]
_submissions._tcp.mydomain   SRV [...]
_imap._tcp.mydomain          SRV [...]
_imaps._tcp.mydomain         SRV [...]
mydomain                     CAA 128 issue "letsencrypt.org; [...]
mydomain                     TXT "v=spf1 [...]
_dmarc.mydomain              TXT "v=DMARC1;p=reject;adkim=s;aspf=s"
_mta-sts.mydomain            TXT "v=STSv1; id=202407122012"
mta-sts.mydomain             CNAME mydomain
www.mydomain                 CNAME mydomain
opendkim._domainkey.mydomain TXT "v=DKIM1;k=rsa; [...]
_adsp._domainkey.mydomain    TXT "dkim=discardable"
5

A is for resolution of domain name to IP address.
MX is not really needed but it is a good practice to say that mails sent to your domain should be delivered to the host specified in MX record. This normally defaults to the same domain anyway.

SRV records are not really needed, Delta Chat does not use them as they are not secure.

CAA record prevents someone who gets control over your port 80 but not the keys (e.g. via a vulnerability in nginx after dropping privileges or MITM at your hoster) from issuing a new TLS certificate via Letā€™s Encrypt or other CA. It is not strictly necessary, but adds some security.

SPF TXT record is for compatibility with receiving hosts that donā€™t support DKIM. It is a legacy email authentication mechanism.

DMARC is basically useless, but some providers like Google care that it exists, no matter the contents. Could be ā€œv=DMARC1;p=noneā€ and they are still happy, this is what they use themselves, you can check _dmarc.gmail.com. and similar providers. This is just to maybe increase deliverability to such providers.

MTA-STS records are to say that your host uses TLS for message delivery and other hosts should use TLS too. It actually increases security, better have it.

www domain is just for redirect to primary domain in case someone types www in front of domain, not really needed. When we did not have it, we got some reports of people adding www to URL and thinking the server is down.

opendkim record is a ā€œDKIM selectorā€, chatmail servers use ā€œopendkimā€ in DKIM signatures to point to this key. DNS record contains the public key that chatmail servers sign messages with. DKIM is absolutely necessary if you want your mails to be accepted by chatmail servers, it is the only authentacation mechanism chatmail cares about.

_adsp is a historic standard RFC 5617: DomainKeys Identified Mail (DKIM) Author Domain Signing Practices (ADSP), this record says that any mails from us not signed with DKIM should go to trash. Probably nobody looks into this record. Officially this standard was superseded by DMARC. This is likely never resolved by anyone.

1 Like
6

First of all, thanks very much for the explanation :slight_smile: .

So, after reading it, in my case, where Iā€™m only hosting a private server for friends a family I assume Iā€™m OK with just two entries:

  • The A record for my domain.
  • The opendkim record so that delta chat clients use DKIM (which IINM is needed for push notifications at mobile phones).

Iā€™m going to try with those two and the configuration I got from the script and see what happens :crossed_fingers:

Will come back with more newsā€¦

1 Like
7

So Iā€™ve retaken this task and have successfully run cmdeploy in my debian virtual box. Now Iā€™m replicating the configuration and installation in my Arch Linux.

I have chatmail-metadata service running and dovecot has been configured to pass metadata through on every new message.

It seems to be running though I think Iā€™m missing some libraries and configuration because currently Iā€™m seeing this warnings in chatmail-metadata daemonā€™s logs:

oct 30 17:37:35 mybox chatmail-metadata[69295]: WARNING:root:lookup ignored: ['shared/68a6d42f77b61f57f177000067c2ffbe/vendor/vendor.dovecot/pvt/server/vendor/deltachat/irohrelay', 'xxx@my.domain.com']

Iā€™ll continue my investigation in the following days, just wanted to document what Iā€™ve been able to achieve for now.

BTW, Iā€™ve installed chatmail like this:

  1. Clone GitHub - deltachat/chatmail: chatmail service deployment scripts and docs in /usr/local/lib/chatmail
  2. Run scripts/initenv.sh to create a Python virtual env in /usr/local/lib/chatmail/venv
  3. Create an /etc/systemd/system/chatmail-metadata.service based on the contents of chatmail/cmdeploy/src/cmdeploy/service/chatmail-metadata.service.f at 1.4.1 Ā· deltachat/chatmail Ā· GitHub
  4. Create a configuration file in /etc/chatmail/chatmail.ini based on the contents of chatmail/chatmaild/src/chatmaild/ini/chatmail.ini.f at 1.4.1 Ā· deltachat/chatmail Ā· GitHub
  5. Edit the .service file to point the executable to /usr/local/lib/chatmail/venv and the configuration file path to /etc/chatmail/chatmail.ini
2 Likes
8

So apparently now I need to set an iroh relay and, by default, chatmaild will connect to it using iroh.my.domain.com which is a DNS name cmdeploy asks for and suggests to configure as a CNAME to the real host name.

I will try to configure it and, possibly, use /etc/hosts instead of DNS since the connection will be internal. Will that be a problem?

I would also like to know if I can set iroh_relay configuration property to localhost. That would be even easier.

If anyone knows the answers to my two previous doubts Iā€™d be grateful to hear them. If not, I will make the experiments to answer them in the following daysā€¦

Thx :slight_smile:

9

You can completely ignore setting up iroh-relay, in this case Delta Chat will use default iroh relays.

I am still updating iroh-relay setup so you will not need a subdomain if you want to setup one:

Pointing to localhost will not work, it is the address that is sent to clients.

1 Like
10

Got it.

BTW, I forgot to say that Iā€™m testing latest stable version (1.4.1). Should I use the development branch instead? :thinking:

Regarding the iroh_relay I thought it was the cause of failure and of the log message, but seeing the code in more detail now I think thatā€™s not the case, so I will do more tests.

Just to understand this better. When the FAQ says:

A Delta Chat app obtains a ā€œdevice tokenā€ locally and stores it on the chatmail server.

When does this happen? On startup?

And how does DC know which metadata server to contact? Using the DNS entry?

I havenā€™t setup the DNS yet so that may be the cause why this is failing the lookup since no DC client will register any token if my hypothesis is true.

To finish with, I see two kind of petitions in the metadata server lookup here (priv and shared) ā†’ chatmail/chatmaild/src/chatmaild/metadata.py at 1.4.1 Ā· deltachat/chatmail Ā· GitHub, whatā€™s the difference?

11

it uses the IMAP METADATA extension. so it happens over the imap connection.

1 Like
12

Device token is obtained on startup via Android APIs if you use Google Play version, in F-Droid it is disabled. Then when you connect to chatmail server this is discovered via IMAP capability string XDELTAPUSH and stored using IMAP METADATA extension on the server. When a new message arrives, if the token is stored, it is relayed to notifications.delta.chat service and it notifies the device that Delta Chat got new messages. Delta Chat then checks all accounts on the device (because notifications.delta.chat and Delta Chat application do not know anything about email accounts).

2 Likes
13

There is a ā€œprivateā€ IMAP metadata such as the token stored per-user and ā€œsharedā€ metadata e.g. admin email address.

1 Like
14

This is a good reason why I didnā€™t see anything :smiley:

15

Not really. At least not in version 1.4.1 if you donā€™t set iroh_relay to anything in the config file.

However, I followed the link you mentioned and set iroh_relay to euw1-1.relay.iroh.network., then added some traces to the chatmail server to see if lookups worked and they seem to be working fine.

I havenā€™t confirmed that the push notifications are already working yet, but Iā€™m almost there I think :slight_smile:.

16

Setting iroh_relay to empty string should work. If it doesnā€™t, itā€™s a bug.

17

Maybe itā€™s taking the default value from the config defaults. I simply omitted the key completely (didnā€™t set it to an empty string). Iā€™ll check if an empty string worksā€¦

18

Yep, it works. You need to set iroh_relay to "", simply omitting it in the config file doesnā€™t work.

19

It is documented in the template config that default is different from setting to empty string:

Default is deploying local relay while empty string does not deploy anything and makes client use the default relay.

Good that it works as documented :slight_smile:

1 Like
20

So Iā€™ve made it work :tada: . With iOS and (I think) Android. I say ā€œI think Androidā€ because Android is using IMAP push too so I donā€™t know if it is instant due to notifications or due to the push :man_shrugging:

The funny thing is that I migrated my DNS domain from Duck DNS to ClouDNS because I thought the DKIM DNS entry we discussed earlier in the thread was necessary but in the end I have not created more than the canonical address for the IMAP host.

So now I wonder if the DKIM entry is really necessaryā€¦ Everything seems to be working correctlyā€¦ :thinking:

I will recap all Iā€™ve done in the next week(s) and try to automate it for Arch Linux or at least document it somewhere so that people can setup their own chatmail-metadata server.

Thanks for the quick and useful support @link2xt :heart: