That first page says exposing it to the Internet is "not recommended". Putting a reverse proxy in front of it does not meaningfully change the security posture. A malicious request to http://jellyfin.homelab.com/exploitable-page will be sent to jellyfin in effectively the same way, whether through a reverse proxy or not. You would need a WAF set up specifically to look for relevant exploit attempts.

https://github.com/jellyfin/jellyfin/issues/5415

Those are some outstanding known vulnerabilities, most of them unfixed. They are not particularly severe, but it shows that thorough security is not a priority for the jellyfin devs.