There are actually a bunch of these. Adding glue to pizza sauce (scraped from an old reddit post), replacing Blinker Fluid every two years, etc.

https://arstechnica.com/information-technology/2024/05/googles-ai-overview-can-give-false-misleading-and-dangerous-answers/