I use Authelia. I found it pretty easy to set up. They even provide guides with examples on how to integrate with other applications. For instance, I use Traefik for my reverse proxy: Traefik | Integration | Authelia
You can use a simple YAML file for your IdP or LDAP if you need more than a handful of users.