Screen Sharing & Backgrounds, Filters, Avatars for Desktop Calls...worth adding?

Hi everyone. I’ve really been enjoying Delta Chat, and i appriciate the sovereignty and privacy that other chats dont have. I’m a beginner/novice coder, and i took it upon myself to add new features to the desktop app that i thought would be useful. I had AI help me with the code, and i’ve never submitted anything for review before so please bear with me.
This is what i have built:

  • Screen Sharing in calls (a whole screen or a single window, with your camera in the corner)
  • Virtual Backgrounds (choose from your picture library) Blur, nand colour filters.
  • Face tracked avatars that can be 2d “photo puppet” and 3d VRM models. There are a few preset VRM models but i found it extremly difficult to find good quality ones but it is pretty neat how they can follow your movement, blink, and speak.

It’s all sender-side, so the person you call needs no changes. I’ve been running it for a few days and tested calling stock Delta Chat on Android and stock DeltaTouch on Ubuntu Touch — the effects show up on their end with nothing installed there.
Right now it’s a runtime patch I have to re-apply after every update, which is why I’m hoping it could be discussed or implemented officially — so updates don’t keep breaking it.
Code’s all here if anyone wants to look: `github.com/Rabs9/deltachat-desktop-effects`
Happy to explain what I learned about how it works and its limits (it’s desktop-only — phones can’t capture the screen from a web view).
If anyone wants to try it, improve it, or suggest changes, please do — it’s open source (MIT), and I’d genuinely welcome help making it cleaner or more robust. I’m learning as I go, so corrections and tweaks are very welcome. Ideas and discussion here are great; anyone who wants to get hands-on can open an issue or PR on the repo.

Expected behavior

Actual behavior

Example Images

That’s cool, but I doubt all this should be integrated.

This is useful, but I would personally like to integrate an existing solution at this point, as I suggested here How to implement group calls: integrate Jitsi or something. Maybe can be considered if my idea doesn’t get support.

Not really a job for Delta Chat, at least not when there is other more important stuff to do.
And probably can be handled by third-party software that wedges itself between the camera and Delta Chat, at least on Desktop.

The problem is that updates would still keep breaking it, but we would be forced to immediately deal with it.

One suggestion I have is to add a screenshot or a video to your repo.

I think this is great. Can it be implemented with webxdc? That way, there’s no need to apply patches after updates.

Yeah, I agree, group calls with screen sharing would really be the ideal. I did try to figure out group calls, but I couldn’t find any way around the 1:1 limitation. the calls are peer-to-peer with signaling over email, so a group would need either a mesh (which doesn’t scale past a few people) or an SFU server, and that’s core level work rather than something a patch can do. So I stuck to what was actually reachable — 1:1 screen sharing and the camera effects. (And I added those screenshots to the repo, good call.)

Thanks! I looked into webxdc for this, and unfortunately it can’t work webxdc apps are sandboxed with no camera or screen capture access, and WebRTC is blocked in their CSP, so they can’t touch the call’s video at all. That’s the whole reason it had to patch the calls window, since that’s the only part with camera/media permission.

https://obsproject.com/ can output to a virtual camera, this is build in, may need v4l2loopback kernel module on Linux, but worked without any extra dependencies for me on macOS. I haven’t tested it on Windows, but I assume it will work out of the box there too.

So the only thing DC desktop calls would need is a camera switcher, which is also useful for switching between back and front cameras on (Linux) phones and also between multiple cameras on professional streaming/studio setups.