While Google isn't generally good for privacy GBoard actually does this. IIRC they actually completely removed the sync service and your typing history is only kept on-device and Android backup.

However it is a bit of a privacy nightmare otherwise as many of the other features phone home. But last I checked (~4 years ago, worth checking again) the core typing functionality is actually fully offline and private.

So yes, it is possible.