I remember watching Battlestar Galactica, where they talked about how they refused to wire the ships systems together so at to prevent AI from hacking them. And the one time they had to manually connect the different systems for plot reasons, the AI hacked into them in comically fast real time. I thought it was hokey at the time, but this sounds surprisingly similar. (although less murderous)

Maxxesota

Interesting look into an area I was completely unaware of.

Yeah, it's in AI companies interest to hype how all powerful their product is. I'm not necessarily talking about that.

When I say potentially, I'm not a doomer, so not trying to say there's no hope, I do think we can make the world better than it is.

That said, one example of dystopian future: one of my clients makes networked security cameras. I saw a presentation where they were showing off their AI search capability. They had a static shot of an expressway, with hours of footage, and they said "I want to find a person in a bus wearing a brown shirt" and just like that pulled up the relevant moment. Not something you want an authoritarian to have. They're going to get it anyways. But would love to have similar power available to the people.

Likewise when you think about accumulation of capital, which is shorthand for accumulation of power. AI will allow elites to accelerate that accumulation and have less use for people. "Who cares if you go on strike? I don't need you."

Having open source AI can turn the tables: "Who cares if you raised the price? I'll make it myself."

I don't think you're engaging with what I'm saying sincerely, and I'm trying to do that with you.

You're right, we don't "need" most modern conveniences. But they save us time and provide an improved quality of life. Why do we need power tools? Why motors? Why electricity, running water, artificial light? I joke, but AI is one of a long chain of tools designed to save us time, or extend our capabilities and reach. Why are you particularly opposed to this one over others? I return to my argument: AI is concerning because of its potential for change. I don't think we can prevent it from existing, but we can make it accessible to normal people and not solely the domain of the rich.

It doesn't need to be our new cyber god for it to matter. All the software developers I know use AI as a tool. One person I know who heads a department at a publicly traded company you've probably heard of said they're not hiring more junior developers, and is worried about his long term career outlook.

That's concerning right? Capitalists who have a choice between hiring ten people or hiring one person who supervises an AI doing the work of nine are going to choose the cheaper option, and pocket the difference.

To say that LLMs are not useful to me indicates a lack of familiarity. I'm not taking about bullshit "write this email for me" type stuff, I mean like "write this web app for me" "find this type of document in this massive trove of documents" "troubleshoot this technical issue". And it can do it.

LLMs are not replacing humans, but they are reducing labor required for many tasks. Who receives the benefit from that reduced labor? Right now, owners. That's why I support open source AI or preferably democratic-entity-only open source AI.

I don't see it as fear mongering. What do you think of open source software like Linux, Firefox, VLC and so on? Because a similar argument is made there - we need to use these tools day to day, and the only option being to pay a corporation to use them would make us literally poorer, and we also wouldn't have the features and quality we see in Windows, Mac, Chrome etc if there weren't open source competitors.

Is lack of open source AI currently a civilizational existential threat? No. Could it be if current trends continue? Potentially. Things could get pretty dystopian if only the mega rich control tools with ultra-intelligence.

There are some open models right now, but they're mostly ones created by private enterprise that were released to the public. Creating models is more resource intensive than making open source software. So it makes sense to pool resources for the public good.

You're right that global warming is an existential threat. And right now the power grid is contributing to it.

At the same time, we're seeing constant innovations in solar power and battery technology, while the cost of both are going down. Solar is now the cheapest source of energy per unit. There is enough energy from the sun to cover multiples of all current energy use by humans.

AI models started out as extremely energy intensive, but are becoming increasingly more efficient. You can now run models on your phone with abilities which previously required a server running dedicated graphics cards.

You're also right that we shouldn't outsource critical thinking. Still to say AI has "all the worst use cases" is wrong. Yes, it has plenty of bad use cases, by many great ones. I use it in my day to day work to save time.

Something like a law banning the use of AI will never pass, and if it did, other countries wouldn't also pass it. A country which uses AI will outperform one that does not (economically, although probably militarily as well). In that scenario, leaving AI capability in the hands of private enterprise makes us subject to the whims of rich elites. For an analog see the case of Elon Musk turning Starlink on/off in Ukraine. Which again is why I do think there's value in open source AI, and more specifically AI only available to democratically controlled entities - nation states but also things like worker cooperatives to compete with corporations.

A majority of Americans (who voted) voted trump in, that's true. But the other branches of government which could check the president are undemocratic:

Each state gets two two senators, meaning places like Wyoming and North Dakota each get two senators while they have fewer people than individual cities in other states (500k people in Wyoming vs 8.58 million in NYC, 800k in North Dakota vs 1.5 million in Philadelphia)

Congress seats are gerrymandered so that the number of seats awarded is not proportional to the amount of votes to each party.

The supreme court is appointed for life (bad enough to start) and then has one stolen seat (should have been appointed by Obama) and another semi-stolen seat which wouldn't have been appointed if we didn't have the electoral college (Trump first term).

The supreme court has ruled that corporations can spend money promoting politicians just like a person can.

One might argue these are the rules, but we didn't put these rules in place, and the people in power have worked together to prevent reform. Blaming all Americans for the current administration's stance on Canada is counterproductive.

All that said, Canadians have a right to be freaked out, to be mad, to be concerned. I'm mostly saying it's a stretch to call the U.S. democratic.

I bet me and the person I'm responding to have a ton in common. Likewise me and the person who wrote that post. But I don't relate to the position of "all AI should stop."

I think there should be democratically accountable AI development. I'm interested in open source but the problem with it is that private corporations take advantage of it until they get big enough to try to lock out competitors. (See Google with Android) I would love to see an "open source" license which allows use by individuals and democratic entities, but forbids use by private enterprise.

It's not going to. Saying all AI should stop is like saying "all gunpowder should stop". Would everyone be better off if no one was using it? Maybe. But that's not going to happen at this point, and will give you a strategic disadvantage if you unilaterally disarm.

And to be clear I'm not talking about using AI to write a screenplay or something creative, I mean like using it to write software, to optimize industrial processes etc.