New messages only after restart; not receiving "read" notification

DC 1.20.5
Android 10
Samsung S9
email-Provider GMX

  • I only receive new messages on restarting DC, while I do see their arrival in my email-prog.
  • I rarely receive read notification, while I do receive transmission notifications.

What does “restart” mean? Switching away from the app and switching back, or do you have to force stop the app every time?

Did you try the “Reliable background connection” option (permanent notification)?

Did you check the guide at Samsung | Don’t kill my app! ?

Restart means “stop app” at the phone. After this, the app is restarted automaticely or if I open it.

Did you try the “Reliable background connection” option (permanent notification)?

I didn’t try the “Reliable background connection” option as I didn’t know, it exists. I activated it now. What does it do and why isn’t it activated by default?

Did you check the guide at Samsung | Don’t kill my app! ?

No I didn’t check that. Is it required to receive messages even with DC running in the foreground?
The case is I don’t receive new messages, even if I look at the specific conservation in DC.
If that is required, I can exclude DC from enery saving in my phone, but I didn’t know that is required to make DC work properly.

If DC does not receive messages even in the foreground, could be some incompatibility with IMAP protocol implementation either on GMX server or in DC, maybe GMX changed something in their server setup.

A log from Delta Chat at the moment when it got stuck could be helpful.

My chat-partner also uses GMX. On his side it works. So probably there is no incompatibility between DC and GMX.

Regarding energy saving: Might it be possible that putting DC to sleep leaves it in a state, that it is not open to receive the messages, which were collected in the past.
Something like: Moving the message to the DeltaChat wasn’t done at receive and wasn’t fetched later? I don’t know, if some of these processes are time critical in that they cannot be done later.

Do I need to exclude DC from energy saving on a phone?

I include the header of the log-file, if this helps ??

This log may contain sensitive information. If you want to post it publicly you may examine and edit it beforehand.

device=samsung SM-G960F (starltexx)
android=10 (G960FXXUFFUE1, QP1A.190711.020.G960FXXUFFUE1)
sdk=29
memory=26M (68.33% free, 512M max)
memoryClass=256
host=SWDH2822
applicationId=chat.delta
app=Delta Chat 1.20.5-gplay
installer=com.google.android.packageinstaller
ignoreBatteryOptimizations=true
notifications=true
reliableService=true
lang=de_DE
rtl=false

arch=64
bcc_self=1
blobdir=/data/user/0/chat.delta/files/messenger.db-blobs
bot=0
configured_mvbox_folder=DeltaChat
configured_sentbox_folder=Gesendet
database_dir=/data/user/0/chat.delta/files/messenger.db
database_version=76
delete_device_after=0
delete_server_after=0
deltachat_core_version=v1.56.0
display_name=xxxxxxxxxxxx
e2ee_enabled=0
entered_account_settings=xxxxxx imap:xxxxxx::imap.gmx.net:993:cert_automatic smtp:xxxxx::mail.gmx.net:587:cert_automatic 0
fetch_existing_msgs=1
fingerprint=XXXXXX
folders_configured=3
inbox_watch=0
is_configured=1
journal_mode=wal
key_gen_type=0
last_housekeeping=1623420780
level=awesome
mdns_enabled=1
media_quality=0
messages_in_contact_requests=2322
mvbox_move=1
mvbox_watch=1
num_cpus=8
number_of_chat_messages=126
number_of_chats=3
number_of_contacts=1030
private_key_count=2
public_key_count=3
scan_all_folders_debounce_secs=60
selfavatar=
sentbox_move=0
sentbox_watch=1
show_emails=2
sqlite_version=3.35.4
uptime=0h 1m 19s
used_account_settings=xxxxxx imap:xxxxxx::imap.gmx.net:993:cert_automatic smtp:xxxxx::mail.gmx.net:587:cert_automatic AUTH_NORMAL 0x4
webrtc_instance=

You probably need to enable “Watch Inbox folder” in “Advanced"→"IMAP folder handling”.

You probably need to enable “Watch Inbox folder” in “Advanced"→"IMAP folder handling”.

Is it required to always watch the In-folder, why can I deactivate its observation?
My imagination was, that moving messages to the DC folder would make the observations of the In-folder superfluous.

One strange observation right now: After more than one day delay, I now received the read-confirmation, while my chat-partner had already answered my message yesterday. What is this?

Yes, you still need to watch the Inbox. By default on most servers mails fall into Inbox unless you configure some filters server-side. “Move to DeltaChat” means messages detected in other folders are moved to DeltaChat, but first you need to detect them there. If you disable watching, DC still occasionally scans all folders (e.g. on app restart, as you observed), but much less frequently.

Disabling inbox watching is only for rare use cases like when you run your own server, or Mailcow-based server, and have a server rule to detect chat messages and move them into DeltaChat for you. This does not really save much traffic or anything, I recommend to just leave the “Watch Inbox folder” enabled all the time.

@Hocuri did some work to eventually remove all these options completely. DC already scans all folders from time to time, this helped a lot with cases when messages fall into Spam because they are now extracted from there, even though with some delay. Next step is to make DC learn which folders to watch automatically (e.g. if the server moves messages to DeltaChat folder and there is never a message in the Inbox during the scan, then there is no need to watch Inbox) and remove all the settings.

OK, I now have activated watching the In-Folder.
(Indeed these potentially misleading options should be either explained or hidden). May be it was a good idea, to generally put the DC-folder to be preferably watched, as it is DC’s own folder.

For me there is still some question:

  • What does the *“Reliable background connection” option change?
  • Is it required to exclude DC from battery management?

If you want to receive messages in background, then yes.

“Battery management” is just a fancy name for killing/freezing the app forcefully at random times, and it prevents Delta Chat from maintaining IMAP connection in the background. This does not save the battery unless the app has a bug and it uses your CPU all the time. Phone vendors have to do it because there are many poorly written apps in Google Play and similar stores, and users who installed them will blame phone vendors for low battery time otherwise.

Centralized apps usually don’t mind because they use Google hub for push notifications, and Google services are never “battery managed”. E.g. in Signal, WhatsApp, Skype etc. all your incoming messages go through Google, unless you remove Google services, in which case Signal maintains its own connection and Skype simply stops receiving messages in the background completely. Decentralized apps like Delta Chat and XMPP clients like Conversations maintain their own connections and when they are killed by “battery manager”, they can no longer receive messages.

This option simply adds a permanent notification in the notification area, like when you record a track in OsmAnd~, for example. It sometimes prevents the app from being killed on phones that are too aggressive in killing background tasks. Normally it is not needed, e.g. on LineageOS or AOSP or official Google phones, but many vendors do unreasonable things to squeeze as much time as possible out of their phones in battery benchmarks: https://dontkillmyapp.com/

If you frequently miss notifications while the app is in background, this option might help. At the very least it allows you to see if the app is still alive, because otherwise permanent notification disappears.

If this permanent notification is too annoying and you don’t have problems with receiving messages in the background, you can disable it.

OK, so it’s suggested to

  • exclude DC from battery management.

  • And I can deactivate the reliable background connection (as I feel it irritating, as it seems to indicate a new message.)

  • Yes I know these hassles with the battery management. I need to deactivate it for tracking, as otherwise GPS data storage is stopped. And I need to deactivate it with Threema.

  • But I do not need to deactivate it with email-Apps. They do work also without it.

What makes the difference between email and messenger app?

And btw do You know something about how DC recognizes which email it has to collect? As it does not show messages in the In-folder or the DC-folder from the past. Is the difference, that it somehow exclusively looks for timestamps after DC’s last time of check, which might have been stored? Or does may be does it some kind of data base?

There is no real difference between Delta Chat and email apps.

If your app works with “battery management” enabled, then you can enable it, but there is no real point in doing it, it can only stop/pause your app and if it stops exactly before you receive the message, you will miss the notification or have it delayed until the app is started again. Normally DC should never be stopped.

Some email apps, like K-9 Mail when it had IMAP IDLE disabled for several versions, don’t watch the folder, but occasionally wake up, reconnect and check the mail. They probably don’t care about being “battery management”, but will not notify you immediately, only when they wake up.

Delta Chat simply remembers the last seen message UID for each folder (unique sequence number which only increases with each new message added to the folder) and asks the server for UIDs from last seen and upwards when it fetches new messages. This saves a lot of traffic because DC never receives anything about old messages and only downloads each message once.

There is one time when DC downloads some old messages, that is right after setup of the new account. This is to populate your address book with addresses from your inbox, so you immediately have some contacts you can write to.

Hi @link2xt
Thank You for explanations, which shed some light on the processes.

So it is like I expected, that the smartphone’s battery management should not affect operational capabilities of DC by any means. Messages and Read confirmations should be displayed, as they all arrive in terms of traditional emails.

  • I’m not shure about how the “delivered” notification (single tick) is transmitted?
  • Probably battery management will stop the smart phone capability to announce a message has arrived if DC is in sleep state.
  • As concerns the

UID for each folder (unique sequence number which only increases with each new message added to the folder)

who is setting this number? Is it a number given by the email-provider for each incoming message and stored in the email header?

IMAP server assigns this number whenever a message is delivered. Usually this happens when SMTP server (such as Postifx) receives an email and delivers it to IMAP server (such as Dovecot) over LMTP. It is not stored in the email header, usually in the database or in the name of the file storing the message.

@link2xt
Ah, interesting!

Do You also know something about that?

Single tick is set when message is sent via your SMTP server. It does not confirm that the message is delivered to destination. DC does not use successful delivery notifications, but will process delivery failure notification if it arrives later.

@link2xt
I.e. the SMTP server at my email-provider is submitting a confirmation on forwarding my message, which initiates the tick notification of my DC client?

Yes, one tick simply means your SMTP server accepted the message for delivery. It may later fail to deliver it to some recipients, maybe even more than a day later, if the connection between SMTP servers is broken, recipient server is offline, recipient mailbox is full etc.