And by making their user's initial impression one that might be wrong, they effectively poison people's thought processes as they sift through the rest of the actual results. Is that a better experience for the users?
And just to be absolutely clear, my understanding is that giving your users the best experience possible is what's going to result in retaining the most market share. Search is free, users will happily use a different search engine if a particular one doesn't work or annoys them.
So here's my conclusion: this method of using AI in search is not only frustrating for users, but also bad for Google. For the sake of their users, they should stop. For their own sake, they should stop.