Joy-Con joysticks use a potentiometer to read the voltage at a wiper that slides across a strip of resistive material. That material wears down over time, or plastic and dust can dirty the sensors.
Stick drift is a huge problem with other Switch models. One survey found that 40% of Switch owners had problems with their Joy-Cons drifting, and things didn’t get any better with the Lite or OLED editions. After a bunch of lawsuits, Nintendo’s president even admitted it and apologized, setting up a free repair program for customers in some parts of the world.
PS/Xbox controllers have more internal space, so their joystick modules are much, much larger than what goes in the joycon. That means they can have more material in the potentiometers, meaning less susceptible to wear and dust/dirt intrusions.