It still knows your IP and browser fingerprinting still works, they still know who you are and what sites you visit. If you change your VPN server you're a little closer, and of course there's Tor assuming you don't get a malicious exit/entry node set, but private browsing isn't as private as people seem to expect.