It is slow, and a massive memory hog. Its not exactly javas fault (i mean it is to a point). Its just minecraft's code is trash. Mods are the only thing that keeps people on using java.

It's not like the Java server is slow tho, it only becomes a problem when mods are added

Lol, the java server is dogshit slow, it just doesn't show until you have a big enough map if you're not running mods. I played on a semi-public server and was a mod back in the day, and wed regularly have to do world resets because it started slowing down when people got far enough away from the spawn. Hell, even with mods it doesn't start to slow down immediately, it waits until you've done a lot in your world before it starts chugging like dad after work.