Once you go down to logic gates. A XOR gate for example makes 1+1 = 0

The other replies are simplifying too much. Just adding or subtracting in a loop would be far too slow.

A multiplier will find the partial products by using AND gates, and then sum them, which is very similar to long multiplication as they teach you in school. This article explains it pretty well.

Division is more complicated. It's sort of done like long division, but apparently that is slow and there's some magic with two's complements that can make it faster. Honestly I don't fully understand it yet.

It recursively subtracts until the number goes at or below zero. The iterations is the output and the reminder is how much it went below zero.