Other than writing an entry in my docker-compose.yml that was all the configuration required. The rest is in the GUI and it's super simple.

get the certificates from Let’s Encrypt manually

https://httpd.apache.org/docs/2.4/mod/mod_md.html just add MDomain myapp.example.org to your config and it will generate Let' Encrypt certs automatically

it’s kind of a pain in the ass every time I add something new.

You will have to do some reverse proxy configuration every time you add a new app, regardless of the method (RP management GUIs are just fancy GUIs on top of the config file, "auto-discovery" solutions link traefik/caddy require you to add your RP config as docker labels). The way I deal with it, is having a basic RP config template for new applications [1]. Most of the time ProxyPass/ProxyPassReverse is enough, unless the app documentation says otherwise.