Regular websites can do that too using service workers - Lemmy's webapp uses this to show an error when an instance is unreachable
What we call a PWA is usually just a webpage with a webmanifest, and a service worker script to manage loading those cached resources you mentioned