Use a ratchet protocol like Signal / Scatter

You can see how this similar app called Scatter implements the Signal protocol over email.

I know that this app uses Golang, so the code might not make sense to port over, but I think this is a great step to improve security.