I was not able to reproduce this on a fresh account until I exported a backup for the fresh account. Then it immediately stopped working, as Lothar explained.
I exported a backup for the profile in the past but I did not add a second device. It seems that exporting a backup changes the setting to delete messages from the server after download to “automatic”. After changing this setting back to delete after download there is the expected behavior, messages are not sent to the server.