If your encryption algorithm is secure, you have no use for automatic lock-out.
This isn't true. You need your algorithm and your key to be secure. If the key needs to be remembered or entered often it probably can't be secure. So brute force protection becomes very important.
If it’s not, automatic lockout won’t do much against an attacker with physical access to the device.
This isn't true. Yes, with enough time and effort it is possible to extract any data from any device. But in practice physical HSMs do an excellent job at raising the cost of key extraction. I would much rather have an attacker steal my Yubikey than a USB with my GPG key lying on it.