Despite what OP and most of the comments here would have you believe, that is actually the crux of what was in OpenAI’s recent paper. They observed that most benchmarks and loss functions used for LLMs had a lower penalty overall for guessing than for admitting ignorance, and called for this to change across the industry.