@joe @mobyduck648 [citation needed]

@joe if it's a tool for doctors to use then it will simply increase the costs of providing care even if there is an increase in the quality of that care. That means adoption of that particular type of "AI" will be limited. If it's possible to replace doctors with a chatbot, that will decrease costs even if the quality of care also decreases.

Which one do you think is more likely to be widely used in a profit-driven, anti-human system?

@mobyduck648

They're not using LLMs to analyse MRI scans. There are image processing algorithms for that, but they're a whole other thing to what this entire thread is about.

Also, are you a fucking bot? I've read all your comments so far in this thread and uhh yeah. Please feel free to jog on now.

How is "helping the medical field" not compatible with better healthcare quality? If the end user isn't experiencing benefits, then how can you claim that benefits have been provided? Who is getting these "benefits" if it's not the end user? If someone IS getting benefits, yet the end users still haven't gotten a better experience... Then can you agree that AI is only helping intermediaries? Like... The corporations who provide the garbage experience?

Do you see your logical fallacy here?

@joe Please cite your sources. What improvements? How were these improvements tested, studied, and documented? Where were the results published?

I think discussions like this are important.

I find that writing out and defending my position helps me refine it. Sometimes, my perspective on things shifts just by trying to explain it. Discussions are the whetstone by which arguments are sharpened.

I'm not a native English speaker so some of this is a bit hard to put into words for me.

That makes two of us :D
Sometimes, these discussions help expand my Engish skills too.

So I do think it is a valid point along the other ones - AI does create value.

Yesn't. It's a text generator that predicts a likely series of words, based on the language patterns it learned from its training material. It doesn't so much create value as aggregate the value of other people's work into a weighted reproduction.

However, those weights are biased by quantity, not quality. It has no way to assess which responses are good, only which ones are likely. The average value it (re-)produces is an average of the value of the training material. Models trained on a wide variety of content (such as ChatGPT) will inevitably include a lot of material of little value to specialised topics.

Hence my argument: It can get lucky and predict a high-value response, but the problem is that it isn't guaranteed to do so. A layperson doesn't have the expertise to tell the difference. That not only dilutes the value, it invites false confidence in the results. In cases where accuracy is critical, this may actually produce negative value if people consult an unreliable model rather than a specialist.

And that's the critical difference to human advice: human experience is shaped by a number of factors beyond just language. We create semantic connections to abstract concepts and attach specific meaning to certain words and patterns.

LLMs don't have that abstraction. They can predict sequences of text that sound plausible and might coincide with something describing reality, but they can't tell whether it's accurate.

Advice has to be grounded in reality to be useful, and that's what LLMs are missing.

They're perfectly suitable for tasks where correlation is enough, but if the task requires actual understanding, LLMs aren't equipped for it.

Otherwise there would be no hype.

Hype doesn't always need a solid reason. In this case, a lot of hype stems from the hope that we may one day have the type of Artificial General Intelligence that SciFi has long dreamed of, the illusion that they may be able to do our work for us and the promise to company managers that they may be able to save money by replacing human employees with AI.

The various executives of the big AI vendors are obviously capitalising on that, stoking the hype with grand and utopic visions because they want to sell their product.

It's not that it's useless. It's that the utility is far less than the lofty promises made by people milking the hype for all it's worth. And that little utility comes at a terrible social, economic and ecologic price.

We might have different perspectives on this btw because I live in the EU. People don't lose their home or water supply here because of AI companies.

I'm in the EU too, I just read a lot of US news. I don't think we should dismiss the consequences our use of US infrastructure has. Worse yet, I don't think we should dismiss the political dependencies that creates.

We need to be specific about what the problem is: the problem is not AI, the problem is that some people are getting super rich on the expense of almost everyone else.

I think that's only a part of the problem. The second part is the lack of understanding you mentioned, and the resulting mis- and overuse. I've seen people trying to argue with experts because "ChatGPT said" because they genuinely do not understand that ChatGPT is a parrot, not an expert.

And the third part, again, is the disastrous effect on our world.

You can't fight technological and scientific progress [...]
I'd say we might actually in an "industrial revolution" kind of situation.

I'm not fighting it. I'm trying to pull it out of the pit that the current obsession with imitation has dug. When college students, the next generation or scientists, trades their scientific understanding for the convenience of high-tech parrots, that is the opposite of progress. It's stagnation, fostered by those few people that are happily trading our future for their present profits.

That's the mirage of this "industrial revolution" analogy: They've built something in the shape of a steam engine, promised the functionality of Spinning Jenny, sold fabric factory owners on the idea who fired their workers to buy these machines. The people they fired were promised that they'd have to work less, but weren't told that they would be paid less too.

Now these machines turn out to not actually provide the smooth, spinning motion required for spinning thread. Factories are facing expensive production outages, compounding the expenses of getting these machines installed and can't afford to hire all the workers back.

The machine shops built to produce these Sloppy Jennies or their parts will eventually find it harder to sell their iventory. Once they go under, their workers will also become economic casualties.

We're at the point where we need to reinforce worker protections. We need to push for a system where our livelihood isn't contingent on the amount of work we do. Only then can that utopia even manifest. And to actually make progress:

We'll need all the social sciences now that we've cut funding for over the last decades

Yes.

We'll need to understand the social dynamics of such technology to better prevent the disastrous side-effects. We'll need to compare historical developments with present circumstances and plans to account for future developments. We'll need to study the psychological effects of interacting with human-like machines to be able to correct course where needed.

Whatever control mechanism is supposed to prevent machines from producing harmful output will need heuristics based on those disciplines to assess the dangers of that output.

I'd include philosophy too. We don't need neural networks that are a lesser version of human ones, nor just scaled-up variants that also replicate all the human inefficiencies, errors and biases. Those are the pure MINT approach to the problem, and it's clearly coming up short.

We need to find a rigorous mechanism for representing semantic knowledge in digital systems that don't just imitate, but surpass human cognition. We need a workable philosophical grounding for logical and mathematical approaches to modelling knowledge of concepts rather than just language.

We need to get over the error that LLMs are "thinking" or "just like humans". They're not, but as long as we're stuck on the idea, we can't fix it.

Also, we really, really should spend less time thinking about whether we could and more about whether we should.