I agree. You're spot on. This isn't a technical problem. We need to think less like engineers and more like lawyers on these things.

You don't require AI companies to have their models refuse to give out legal advice. As you note, that's a fool's errand. Instead, you make it a felony to use any LLM to write legal advice. Currently lawyers are already facing court sanctions for using AI to write their briefs. If you really want to get serious, you make it a felony. Apply the same penalty to a doctor using an LLM to diagnose their patients as you would if I decided to open a clinic in my house and start practicing medicine without a license. Charge people operating AI therapists with practicing psychology without a license.

This isn't a technical problem, it's a legal one. You don't try to regulate the LLMs themselves. You just make it a severe felony to use that technology in certain highly sensitive applications and professions.

And sure, one could say that we don't have a reliable way of screening AI from human made work. But again, that would be thinking like an engineer, not a lawyer. You don't enforce these laws by screening thousands of legal briefs or medical records with an AI detection filter. You offer huge cash rewards to whistle blowers, so that a disgruntled employee can rat out their bastard boss that's forcing everyone to illegally use AI. Apply massive fines and give half of whatever is collected to the person who blows the whistle.

Are you a structural engineering company owner forcing your employees to use AI to engineer buildings? Now you have to worry about one of those employees, an employee who probably really dislikes using AI, ratting you out. And their whistleblower reward will be so large, they'll end up owning the company. They won't have to worry about being blacklisted from future employers, as they'll be getting enough money to retire.

Don't look for technical solutions for the problem of AI abuse. Look for human solutions.