Process old emails once again

Hi everyone!
Is there an option to make Delta Chat process old emails once again?
My Situation: I started using and testing Delta Chat for Android some weeks ago. Now I’m running the first group that I use productively plus one private chat. The update to Delta Chat 1.2.1 via F-Droid mysteriously ate all my data so that I had to start over from scratch. I have no backup, but all the chat emails are still on the server. Can I somehow tell Delta Chat to process them again? They should contain all the needed information.
Very shortly before this accident I had installed a second Delta Chat instance and synced both. This allowed me to transfer my keys back to the main instance on my phone. So I guess I should even be able to decrypt my messages in the private chat (the group is not encrypted) – if I find a way to feed them to Delta Chat again, that is.
What if this approach doesn’t work? Will I see the group and any new messages again as soon as any member posts something? I mean, I should still be a member, because if I understood correctly, no other client can know about my Delta Chat 1.1.2 having gone to Nirvana.

maybe you can create a empty backup and modify it so that deltachat think it was made before your messages, mark your messages as unread on the imap server and then import that modified backup.

Unfortunately there isn’t an official method for this yet.

Will I see the group and any new messages again as soon as any member posts something?

Yes it should work that way.

It is possible only by resetting imap.mailbox.INBOX and imap.mailbox.DeltaChat settings in the config table of the database, but it is hard to access it on Android. It should be possible to open it with SQLite browser on Desktop, though.

Interesting! What value should I put there? Tried empty string for both keys without success.

when you receive a message from that groups it will pop up in your Delta Chat, as you said you are still a member, if you want to get old messages, maybe moving them to inbox causes the app to think they are “fresh” messages and they get shown as new messages in your app and moved back to the DeltaChat folder?

Thank you for your input! Unfortunately, that doesn’t seem to work either. My settings were not to move any messages to the DeltaChat folder (since it’s a dedicated email account anyway) and to just watch the inbox. But copying everything to the DeltaChat folder and having the app watch this folder as well or instead of the inbox folder didn’t bring any messages back. Had marked all messages unread.

@Hans the number in there is an unix timestamp and the uid of the last seen message on imap. So empty string is invalid.

the source code comment that explains it:

the entry has the format
`imap.mailbox.<folder>=<uidvalidity>:<lastseenuid>`

In the backup of my second Delta Chat instance, I see these values:
imap.mailbox.INBOX: 1583621360:62
imap.mailbox.DeltaChat: 1583621361:0

I was meanwhile suspecting a UNIX timestamp. But that gives 2020-03-07 for uidvalidity. I have used both instances way later. So what exactly does uidvalidity mean, and, even more interesting, what value should I try?
And the lastseenuid? I thought this part of the entry was the message count, since my DeltaChat folder was empty, and I have 62 messages in my inbox. Makes sense to me, but has nothing to do at all with a user ID.
I’m a bit confused…

I don’t know the details of IMAP, @link2xt should know more.

Now it worked. This is what I did. Or perhaps more what I think I did, because I was experimenting away. :wink: So please don’t take the following as a serious HOWTO and feel free to add corrections or clarifications.

  1. Do a backup, set the uidvalidity time stamps (imap.mailbox.xxx) to some time in the past and the lastseenuid to 0. (Maybe only one of these changes is needed?)
  2. Re-import the backup.
  3. Let Delta Chat watch an empty folder (I used DeltaChat, but my guess is inbox would work as well.)
  4. Copy all messages into this folder.

The result is not perfect but very close from what I’ve seen so far. The group which has 9 members shows 11 members now, because deleted mail addresses reappeared. I hope this can be healed by either having another member send a group message or by deleting the old addresses one more time.

I would call this concrete issue solved. Perhaps a feature request to implement a proper and easy way to re-process old emails from inside the app would make sense?
Thanks a lot for your help!

UID validity is a value associated with a folder on the server. It seems to be equal to the folder creation timestamp in your case, but can be anything the server decides to use. The second value is a message sequesce number, resetting it to 0 is enough. But also if you change UID validity, Delta Chat should think that folder was recreated and also set last seen UID to 0, so this is an alternative method.

Makes sense. If you open it, also link to https://github.com/deltachat/deltachat-core-rust/issues/1338 please which seems to be related.