> Creativity is connecting ideas from different domains and see if something from one field applies to another.
That's true. The question is whether the produced pattern has any value. LLMs are incapable of determining this, and will still often hallucinate, and make random baseless claims that can convince anyone except human domain experts. And that's still a difficult challenge: a domain expert is still needed to verify the output, which in some fields is very labor intensive, especially if the subject is at the edge of human knowledge.
The second related issue is the lack of reproducibility. The same LLM given the same prompt and context can produce different results. This probability increases with more input and output tokens, and with more obscure subjects.
The tools are certainly improving, but these two issues are still a major hurdle that don't get nearly as much attention as "agents", "skills", and whatever adjacent trend influencers are pushing today.
And can we please stop calling pattern matching and generation "intelligence"? This farce has gone on long enough.
Only amongst the chronically online. There is very much still a big world out there, unmediated by a glowing screen, in which that would still sound ridiculous.
There sure used to be! Funny thing is, it keeps getting smaller, doesn't it? And all the people walking around out in it are on their phones, building and synthesizing their worldviews entirely from what they read on the Internet. Every real-world fact, every observation, is filtered through this placental mass, consumed and cleaned and put away in the language and ideologies of the day. For any other treatment would render them wholly inscrutable.
Weird, that the real world now fits inside the synthetic one. Bourdain lost to Bourdieu.
> never ask a model for confirmation or encouragement; but you can absolutely ask it to critique something, and that's often of value.
What's the difference? The end result is equally unreliable.
In either case, the value is determined by a human domain expert who can judge whether the output is correct or not, in the right direction or not, if it's worth iterating upon or if it's going to be a giant waste of time, and so on. And the human must remain vigilant at every step of the way, since the tool can quickly derail.
People who are using these tools entirely autonomously, and give them access to sensitive data and services, scare the shit out of me. Not because the tool can wipe their database or whatnot, but because this behavior is being popularized, normalized, and even celebrated. It's only a matter of time until some moron lets it loose on highly critical systems and infrastructure, and we read something far worse than an angry tweet.
And yet you have certainly used and enjoyed software published by others free of charge, and your employer, company or favorite service has relied on it. Your career may even be entirely dependent on it.
If you demand remuneration for all your work, then it's only fair for you to also pay for every single piece of software you ever use. If OTOH you're willing to trade some of your time and effort for the time and effort someone else spent on the software you enjoy for free, then you might appreciate that a financial transaction is not required for value to be created in the world. What is required is fair collaboration.
The issue isn't with the amount of guardrails in place to perform an action. Yes, it is obvious that there should be some in place before doing any critical operation, such as deleting a database.
The issue is that the "agent" completely disregarded instructions, which in the age of "skills" and "superpowers" seems like an important issue that should be addressed.
Considering that these tools are given access to increasingly sensitive infrastructure, allowed to make decisions autonomously, and are able to find all sorts of loopholes in order to make "progress", this disaster could happen even with more guardrails in place. Shifting the blame on the human for this incident is sweeping the real issue under the rug, and is itself irresponsible.
There are far scarier scenarios that should concern us all than losing some data.
Well the user chose the tool. The tool is an LLM. LLMs are non deterministic. You can not predict what comes out ouf an LLM for a given input, especially without weights. This should be known.
There is currently no way to prevent this apart from not giving the LLM full control. It will not delete what it can not delete.
Use an LLM to write an ansible playbook or some terraform code if you want, but review it, test it, apply it. Keep backups (3-2-1 rule at minimum).
Letting an LLM have access to everything is just a bad idea and will lead to bad outcomes. You can not replace a person with a mind and experience with an LLM. You can try. But you will probably fail.
> There is currently no way to prevent this apart from not giving the LLM full control. It will not delete what it can not delete.
But deleting something is just one action you might not want it to take.
The recent "agentic" craze is fueled by the narrative pushed by companies and influencers alike that the more access given to an LLM, the more useful it becomes. I think this is ludicrous for the same reasons as you, but it is evident that most people agree with this.
We can blame users for misusing the tools, and suggest that sandboxing is the way to go, but at the end of the day most people will favor convenience over anything else a reasonable person might find important.
So at what point should we start blaming the tools, and forcing "AI" companies to fix them? I certainly hope this is done before something truly catastrophic happens.
I agree that the marketing is crazy. The dangers are not nearly talked enough about.
Still if I cut off my finger with a bandsaw that is usually my fault. I didn't use tool in a safe way. People have to learn how to use their tools in a safe way. You wouldn't give an intern that much power on day one.
An LLM generates plausible text token by token. It is at its core a deterministic function with some randomization and some clever tricks to make it look like an agent dialoguing or reasoning.
Plausible text sometimes is right, sometimes not.
Humans have a world model, a model of what happens. LLMs have a model of what humans would plausibly say.
This is such a motte-and-bailey argument. Whenever people point out LLMs aren't actually intelligent then you're an anti-AI Luddite. But whenever an AI does something catastrophically dumb it's absolved of all responsibility because "it's just predicting the next token".
I think they are not actually intelligent. Fix all random seeds and other sources of randomness, and try the same prompt twice, and check how intelligent that looks, as a first approximation.
On a more technical level very serious people have voiced doubts, for example Richard Sutton in an interview with Dwarkash Patel [1].
> The major hurdle right now is actually pivoting LLMs from just generating code: integrating those tasks into workflows.
Funny, I thought that the major hurdle is improving accuracy and reliability, as it's always been. Engineering is necessary and useful, but it's a much simpler problem, which is why everyone is jumping on it.
As much as that’s true it’s clear a huge amount of people have accepted the current state and are working around it, successfully(in terms of ticking an executive’s checkbox) in a lot of cases. And it’s worth considering we’re seeing strong strides outside of model quality in the tooling and integration
It's ok friend, all I did is put acceptance criteria in a list so I can parse it and quickly track cross-references. The rest is just Elixir/Phoenix and some creative writing.
> your extreme atheists aren't much different from your extreme believers; they both have strong beliefs about things they can't prove, and for some reason want to go off on them.
You have a mistaken understanding of what atheism is. It is not a belief in anything, but an absence of belief in a deity.
> there are a whole lot of things that we all go around everyday "not believing."
Sure, and yet theism is part of 75% of the world population and influences everything from education to politics. It's perfectly reasonable to talk about atheism within appropriate settings.
The word seems to be used both ways, despite what anyone might like: either as a person who doesn't believe in a god, or as a person who believes there is no god. It's a subtle difference.
>You have a mistaken understanding of what atheism is. It is not a belief in anything, but an absence of belief in a deity.
I consider that to also be a wrongly held position, because you'd need proof either way. So atheists are just making a bet.
I think agnostic is the most valid position as far as I am concerned, lacking proof of one or the other. I do not know. We can get into technicalities as well. What exactly do we mean by God? What if some religious God does exist but it's wrongly interpreted by believers? What if there's some highly technologically advanced entity that meets the criteria as far as the more primitive religious perspective is concerned? Do we have proof such thing exists? Do we have proof such entity cannot exist in our universe? I find both perspectives shortsighted.
Having certainty something that can be perceived as God by believers cannot exist in our universe is in the end a belief, with no proof.
> I consider that to also be a wrongly held position, because you'd need proof either way.
Proof that something doesn't exist? Ever heard of Russel's teapot?
The burden of proof is on the claimer.
> What exactly do we mean by God?
Absurd question. Pick up any holy book, or ask any believer. An atheist is simply a person who doesn't hold those beliefs.
A famous Dawkins quote is apt in this discussion:
> We are all atheists about most of the gods that humanity has ever believed in. Some of us just go one god further.
> Having certainty something that can be perceived as God by believers cannot exist in our universe is in the end a belief, with no proof.
Again, you're mistaking what atheism is. It's not being certain that a deity cannot exist—it's not having any reason to think that it does.
People who claim certainty in either direction are equally delusional. The problem is when a belief crosses into realms of reality, defines the identity and culture of people, and influences the rest of society. Based on history and personal experience, theists are far more prone to this than atheists.
Again, I'm more talking about "atheism in effect" than I am "textbook definition."
I'd say, sure, it's reasonable to discuss, but it's often overblown. The older I get, the less significance "atheism" feels like it has just about anywhere.
No shade to anyone, but I have very little interest in "internal beliefs that seem not to affect people much," which atheism seems to fall square into.
From what I understood, any color and material involved in high precision manufacturing requires careful design and thorough testing. They likely prioritize the brown color and material due to branding, so changing this to anything else requires redoing large parts of the pipeline.
You have to redo injection moulds anyway as they have a limited life. And you can do a lot with materials too, some materials simply shrink more than others as they cool down.
That's true. The question is whether the produced pattern has any value. LLMs are incapable of determining this, and will still often hallucinate, and make random baseless claims that can convince anyone except human domain experts. And that's still a difficult challenge: a domain expert is still needed to verify the output, which in some fields is very labor intensive, especially if the subject is at the edge of human knowledge.
The second related issue is the lack of reproducibility. The same LLM given the same prompt and context can produce different results. This probability increases with more input and output tokens, and with more obscure subjects.
The tools are certainly improving, but these two issues are still a major hurdle that don't get nearly as much attention as "agents", "skills", and whatever adjacent trend influencers are pushing today.
And can we please stop calling pattern matching and generation "intelligence"? This farce has gone on long enough.
reply