Fastest server-side implementation to self-host Delta chat?

I can’t measure the speed precisely but it’s about half a second at its best. Sometimes it was so bad that a screenshot of my messages on delta.chat, was received by my friend on Telegram (using a VPN), sooner than the messages themselves.
Unfortunately the latency is so high (like I said half a second or more) that each and every messenger is faster.

I am hoping this latency is due to my server-side implementation, If you think that half a second is normal and other people are experiencing the same kind of delay on other mail server software, then maybe I should look for solutions other than delta.chat
On my other VPS I ran a Matrix server and it doesn’t have the same kind of latency, but for many reasons I prefer delta.chat and would rather use it.