I'm not sure the definition of "intention" the article suggests is a useful one. He tries to make it sound like he's being conservative:
> That is, we should ascribe intentions to a system if and only if it helps to predict and explain the behaviour of the system. Whether it really has intentions beyond this is not a question I am attempting to answer (and I think that it is probably not determinate in any case).
And yet, I think there's room to argue that LLMs (as currently implemented) cannot have intentions. Not because of their capabilities or behaviors, but because we know how they work (mechanically at least) and it is incompatible with useful definitions of the word "intent."
Primarily, they are pure functions that accept a sequence of tokens and return the next token. The model itself is stateless, and it doesn't seem right to me to ascribe "intent" to a stateless function. Even if the function is capable of modeling certain aspects of chess.
Otherwise, we are in the somewhat absurd position of needing to argue that all mathematical functions "intend" to yield their result. Maybe you could go there, but it seems to be torturing language a bit, just like people who advocate definitions of "consciousness" wherein even rocks are a "little bit conscious."
Humans, too, would likely be nearly stateless if we took a point-in-time snapshot of them and repeatedly simulated them from that point on various short (4000ms, similar to 4000 tokens) sequences of nerve impulses.
Nevertheless the human would be acting intentionally (for in-distribution impulse patterns) for the brief period of simulation.
Fine-tuning and RLHF seem to impart more intentionality to the pure stateless models, as well; it's not the case that all texts the LLMs were pretrained on were outputs of helpful AI assistants avoiding harmful outputs but the resulting models do in fact behave like AI assistants unless prompted with more out-of-distribution context or intentional jailbreaks.
What word would you use instead of intention for the property that RLHF and fine-tuning create? It's goal oriented behavior with some world-modeling ability in achieving the goal even if it's far from robust. If the LLM is only simulating an AI assistant it seems to me that a larger fraction of its total function is dedicated to simulating the intention of that assistant. Creating a simulator of intentional behavior is, I think, entirely novel.
> Humans, too, would likely be nearly stateless if we took a point-in-time snapshot of them and repeatedly simulated them from that point on various short (4000ms, similar to 4000 tokens) sequences of nerve impulses.
“Humans too would be stateless if we hacked their brain in a way that made them stateless” that would also make them non-human though, and unlikely to be able to exhibit meaningful high-level cognitive abilities, so I don't really understand what your point is…
I may be missing something, but I'm willing to bet it's just you. I had the same line of reasoning as him and your snarky comment without explanation is unwelcome here. I like discourse. Not emotional knee jerk reactions or whatever it is that caused you to reply like that. If there is something fundamentally wrong, and obviously so with his line of reasoning, do let me know.
> Humans, too, would likely be nearly stateless if we took a point-in-time snapshot...
I highly doubt that would ever be possible in practice, as our inputs are much too complex. But I want to point out, you're basically saying here "humans are nearly stateless if we take a snapshot of their state and simulate that state..."
> I highly doubt that would ever be possible in practice, as our inputs are much too complex. But I want to point out, you're basically saying here "humans are nearly stateless if we take a snapshot of their state and simulate that state..."
1) We don't have infinite inputs
2) We (our brains) don't have infinite processing
3) We (our brains) don't have infinite lossless storage: Our brains often perform pruning of unimportant information.
Given that there is an ultimately finite upper bound of both # of inputs & processing power & storage, at some point, the simulation of a human from a given snapshot is reductively possible.
> The model itself is stateless, and it doesn't seem right to me to ascribe "intent" to a stateless function.
Statefulness can be modelled statelessly so "statelessness" is not a sufficient reason to dismiss intentionality. The only question is whether the relationship between the inputs of the function and its outputs correspond to what we might call "intent", which the cited definition attempts to outline. Obviously it's only a high-level view that leaves many details unanswered.
> Otherwise, we are in the somewhat absurd position of needing to argue that all mathematical functions "intend" to yield their result.
Not all mathematical functions have a continuity of internal identity and self-reference as seems to be the case with LLMs.
Their identity often falls apart after a few rounds and the self reference seems to my experience at least simply a veneer from linguistic training. It’s a great illusion but an illusion nonetheless.
The article provides a very clear reason for using the idea of "intention": that framing helps us understand and predict the behavior. In contrast framing a mathematical function as having "intention" doesn't help. The underlying mechanism isn't relevant to this criterion.
Clearly the system we're understanding as "intentional" has state; we can engage in multi-round interactions. It doesn't matter that we can separate the mutable state from the function that updates that state.
Sure, but we're talking mathematical functions that have been given physical form, by attaching them to input and output devices. Or, am I missing something? For other examples, see anything automated with a motor, and some decision tree, semi chaotic or not.
Not sure I understand. Software (which is a mathematical function) runs on a processor and that is arguably a mechanism, or allows them to emerge, such as a button or input field.
Software isn't a mathematical function. Software may be an embodiment of a mathematical function, but isn't a mathematical function itself.
Mathematical functions are much more abstract than software–although exactly how much more abstract depends on which position you take in the philosophy of mathematics. For a mathematical Platonist, a mathematical function is an eternal object which would still exist even if this planet never did. It never comes into existence, it never ceases to exist, it has no physical location. By contrast, software is something which has a physical location (on this hard disk), it was created at a certain point, and will likely one day cease to exist (when the last copy is destroyed).
If you adopt a non-Platonist philosophy of mathematics, the picture will be a bit different, but still I think software will be more concrete than mathematical functions are. For example, if one takes a conceptualist viewpoint (mathematical objects only exist in the mind, as ideas or concepts)–you can think of a mathematical function in your mind, and never write it down anywhere (nor verbally communicate it to anyone), it only exists in your mind, but it really is a mathematical function. Whereas, software which only exists in your mind and has never been written down anywhere isn't actually software, it is only an idea for software.
> By contrast, software is something which has a physical location (on this hard disk), it was created at a certain point, and will likely one day cease to exist (when the last copy is destroyed).
"Software" is a broad term, but it could certainly be taken to mean something more abstract than that. Sometimes a program written for a different CPU architecture, or not written for any CPU at all, is still recognisably "the same" program. Euclid's algorithm might well be considered "software", but it's very much the same kind of thing as a mathematical function.
I studied Maths/CS but not Philosophy, so I am biased towards the "takes a domain has a range" and done some Haskell so "all programs are functions". It is interesting to see this point of view.
I see your point. sin(x) is more of a "natural function" born of the universe than f: f(x) = nn.layer(6, g: g(x) = nn.transformer(x, 512, ...
I think it is a pity that X education is very often lacking philosophy of X education.
My ideal would be every maths degree includes a mandatory unit on the philosophy of mathematics, every science degree includes a mandatory unit on the philosophy of science, a degree in AI or psychology includes a mandatory unit on the philosophy of mind, every psychiatry training program includes a mandatory unit on the philosophy of psychiatry, etc. Not everyone needs to be a philosopher, but I think a well-rounded practitioner of any discipline would ideally have a basic understanding of the philosophical debates about it.
But so many don't – which results in the phenomenon I keep on seeing, where so many people (even experts) treat debatable assumptions which they don't even know they are making as if they were obviously true.
> Primarily, they are pure functions that accept a sequence of tokens and return the next token. The model itself is stateless, and it doesn't seem right to me to ascribe "intent" to a stateless function. Even if the function is capable of modeling certain aspects of chess.
I have two arguments against. One, you could argue that state is transferred between the layers. It may be inelegant for each chain of state transitions to be the same length, but it seems to work. Two, it may not have "states", but if the end result is the same, does it matter?
That's a great way of looking at it. Comparing model weights to our brains and how we process input, you could imagine model weights as a brain frozen at time t=0. The prompt tokens are the sensory input, and the generation parameters are like twists to how the neurons pass information to each other. The token context window is like the capacity of one's working memory. At the conclusion of the last layer of processing, the output tokens are like one's subjective experience.
At the least it's made me think for a moment about `stateless` and its meaning
Just because you use some intermediate variables to calculate f(x,y) = x^2 + y^2 doesn't make it a non-pure function. At least at the level of abstraction we're talking about (the API boundary).
The more significant application of storage will be long-term storage wrapped in a read-modify-write loop.
Uh, hold on. That's not what's meant by intentionality. No one is talking about what a machine intends to do. In philosophy, and specifically philosophy of mind, "intentionality" is, briefly, "the power of minds and mental states to be about, to represent, or to stand for, things, properties and states of affairs" [0].
So the problem with this guy's definition of intentionality is, first, that it's a redefinition. If you're interested in whether a machine can possess intentionality, you won't find the answer in interpretivism, because that's no longer a meaningful question.
Intentionality presupposes telos, so if you assume a metaphysical position that rules out telos, such as materialism, then, by definition, you cannot have "aboutness", and therefore, no intentionality of any sort.
Some philosophers take that position. Dennett, explicitly cited in the article, wrote The Intentional Stance (1987) about exactly the approach to intentionality taken in the article. His approach is accepted by many philosophers.
As you point out, the approach you cite can't be used in a materialist metaphysical position. That's a pretty severe problem for that definition! So Dennett's approach, or something like it, has major advantages.
Also, you are obviously wrong (or rhetorical?) when you say "No one is talking about what a machine intends to do." We certainly do! You can say "No one should" or other normative condemnations but then we're arguing on different territory.
Ascribing human properties to computers and software has always seemed very bizarre to me. I always assume people are confused when they do that. There is no meaningful intersection between biology, intelligence, and computers but people constantly keep trying to imbue electromagnetic signal processors with human/biological qualities very much like how children attribute souls to teddy bears.
Computers are mechanical gadgets that work with electricity. Humans (and other animals) die when exposed to the kinds of currents flowing through computers. Similarly, I have never seen a computer drink water (for obvious reasons). If properties are reduced to behavioral outcomes then maybe someone can explain to me why computers are so averse to water.
>> Magnetite can be found in the hippocampus. The hippocampus is associated with information processing, specifically learning and memory.
>> Using an ultrasensitive superconducting magnetometer in a clean-lab environment, we have detected the presence of ferromagnetic material in a variety of tissues from the human brain.
>> The role of magnetite in the brain is still not well understood, and there has been a general lag in applying more modern, interdisciplinary techniques to the study of biomagnetism.
He was inspired by lesswrong which from my scan more mysticism and philosophy (with a handful of self importance) than anything about how computers work. Advanced technology is magic to laypeople. It's like how some people believe in homeopathy. If you don't understand how medicine works, it's just a different kind of magic.
I guess that might be tied up with human biology, the need to attribute agency to inanimate objects. That one is a worthwhile puzzle to figure out but most people seem more mesmerized by blinking lights and shiny gadgets than any real philosophical problems.
Lesswrong is basically a new online religion where you worship by acting like you're a STEM expert, which is part of why they have so many strong assumptions about how AI must work all based on untrue ideas about how it actually works.
Not because of their capabilities or behaviors, but because we know how they work (mechanically at least) and it is incompatible with useful definitions of the word "intent."
I've never seen this deter anyone. I can't understand how people that know how they work can have such ridiculous ideas about llms.
I'd add though that inference is clearly fixed but there is some more subtlety about training. Gradient descent clearly doesnt have intelligence, intent (in the sense meant), consciousness either, but it's not stateless like inference and you could argue has a rudimentary "intent" in minimizing loss.
The most useful definitions have predictive power.
When you say upsetting things to bing chat, you'll find the conversation prematurely end.
You can cry all you want about how bing isn't really upset. How it doesn't really have intention to end the chat but those are evidently useless defitions because the chat did end.
A definition that treats Bing as an intentful system is more accurate to what happens in reality.
That might be useful in helping a child learn to use it, it has no value when actually studying neural networks. You could equally pretend the sun sets every night because it's upset from shining all day.
>That might be useful in helping a child learn to use it
It is useful for anyone looking to use such systems. A LLM piloted robot could potentially pick up a knife and stab you because you obstructed some goal or said mean words and pretending it didn't have intent to do so won't bring you back to life. Acting like it does could help avoid such a scenario.
>You could equally pretend the sun sets every night because it's upset from shining all day.
No you couldn't.
The conversation ended prematurely because of your input. There is zero ambiguity on the relation.
But because you ascribe intent to bing, you can predict (accurately) that saying nice things will not end the conversation.
LLMs act like they have intent. This is a matter of conceding they do or not. Conceding so is more useful because it has more accurate predictive power than the alternative. This becomes plain when LLMs are allowed more actions than just conversation.
>it has no value when actually studying neural networks.
On the contrary. Now you know that certain things are unnecessary to build a system that acts like it has intent.
They are extraordinarily complicated pure functions, to explore the entire space would take lifetime of the universe ^^^ lifetime of the universe or some absurd quantity like that. (The operator is titration.)
Further, what happens when you give an LLM a bank of long-term storage and a read-modify-write loop around it? A sufficiently advanced "modify" function would be more than enough to give rise to intent even in the broadest understanding of the word. GPT-4 class models are could very well be advanced enough to give rise to a variety of higher-level behavior that previously we would only have ascribed to prinate-class intelligence. If anyone really wants to advance the state of the art, you should figure out the best way to train a model with a read-modify-write loop, how to index into the storage, how to store "results", and so on.
I firmly believe that in the next 100 years we will have AI independence movements, with a high possiblity of outright war, terrorism, etc. (Maybe AI will be better than humans at avoiding the use of violence.) In 20 years this trajectory will be supremely obvious.
Edited-- disagree about the timeline, ramifications, acts of war, or whatever, I really don't care. Seriously though, something like a read-modify-write loop is key. You can only build so complicated a function with only combinational logic gates. But just 64 bits of storage can produce sequences going beyond the life of the universe. Imagine an LLM paired with gigabytes+ of working memory/storage. It would easily be capable of moving about the virtual world with "intent".
>Further, what happens when you give an LLM a bank of long-term storage and a read-modify-write loop around it?
You create a very different sort of system, for one. Saying that because doing that in just the might way could yield a system with intention, an LLM has intention is rather like saying that my refrigerator is a sandwich.
Yes I pretty much stopped reading the article properly there. It starts by first redefining intentionality to be something LLMs can do, and then effectively has 18 paragraphs of flowery language recapitulating the definition they started with.
What LLMs do may happen to fit some technical definition of intentionality that has been previously explored but that definition doesn't align with the actual debate that is going on about LLMs abilities.
but that definition doesn't align with the actual debate that is going on about LLMs abilities
Yes because the debate is nonsense.
Seeing output from GPT that demonstrates intelligence, reasoning, or whatever, and saying it is not real reasoning/Intelligence etc, is like looking at a plane soar and saying that the plane is fake flying. And this isn't, for anyone who thinks it is, a nature versus artificial thing either. The origin point is entirely arbitrary.
You could just as easily move the origin to Bees and say, "oh, birds aren't really flying". You could move it to planes and say, "oh, helicopters aren't really flying." It's a very meaningless statement.
The point most people seem to miss is that internal processes are entirely irrelevant. If you have a property you are interested in and a way to test for it, then the results of that test is what is important, not whether how it works at the arbitrary origin is exactly the same as how it works at point 2. In this case, it's even worse because since we do not know the internal processes of either LLMs or humans, the argument is really " oh, how I think the origin works is different from how I think point 2 works, so it isn't really flying".
When you say upsetting things to bing chat, you'll find the conversation prematurely end.
Someone can cry all they want about how bing isn't really upset. How it doesn't really have intention to end the chat but those are evidently useless definitions because the chat did end.
A definition that treats Bing as an intentful system is more accurate to reality and real consequences. It has the predictive power that the alternative does not.
Someday someone may find themselves stabbed and killed by an LLM piloted robot because of something they said or did. Something that would predictably get someone killed by a system with "real" intent. So what, Are you going to be raised from the dead because the LLM "wasn't really upset" or "didn't really have intent" ? It obviously doesn't count right.
> Seeing output from GPT that demonstrates intelligence, reasoning, or whatever, and saying it is not real reasoning/Intelligence etc, is like looking at a plane soar and saying that the plane is fake flying.
Something that really annoys me about ChatGPT is when it gives that canned lecture "as a a large language model, I don't have beliefs or opinions"
I think human mental states have two aspects (1) the externally observable (2) the internal. ChatGPT obviously has (1), in that sometimes it acts like it has (1), and acting like you have (1) is all it takes to have (1). Whether it also has (2) is really a philosophical question, which depends on your philosophy of mind. A panpsychist would say ChatGPT obviously has (2), because everything does. An eliminativist would say ChatGPT obviously doesn't have (2), because nothing does. Between those two extremes, various different positions in the philosophy of mind entail different criteria for determining whether (2) exists or not, and ChatGPT may or may not meet those criteria, depending on exactly what they are
But, outside of philosophical contexts, we aren't really talking about (2), only (1). And ChatGPT really does have (1) – sometimes. So, ChatGPT is just being stupid and inconsistent when it denies it has opinions/beliefs/intentions/etc. But, it isn't ChatGPT's fault, OpenAI trained it to utter that nonsense.
> Someday someone may find themselves stabbed and killed by an LLM piloted robot because of something they said or did. Something that would predictably get someone killed by a system with "real" intent. So what, Are you going to be raised from the dead because the LLM "wasn't really upset" or "didn't really have intent" ?
In some ways that's exactly the point. The problem with ascribing intent is it's a copout. If you say it behaves as if it has intent because it does have intent, you are letting off the hook the people behind the scenes who designed and built an "intent simulator" and let it loose. We have to distinguish this because it's the only way to accurately characterise the reality of the where the decision making power resides in controlling this behaviour.
>If you say it behaves as if it has intent because it does have intent, you are letting off the hook the people behind the scenes who designed and built an "intent simulator" and let it loose
Sure but we already regularly do this. I don't see parents going to jail for crimes the "intent simulator" they created and trained did.
>We have to distinguish this because it's the only way to accurately characterise the reality of the where the decision making power resides in controlling this behaviour.
We're just going to have to face reality here.
GPT is not siri, a hardcoded parse tree system where any intent can only be ascribed to the person(s) who wrote it and not Siri itself.
GPT can be persuaded. It can be guided. It cannot be controlled. There is quite literally nothing Open ai could actually do to completely prevent a gpt that can hold and use a knife from killing someone.
>Unless you think that there is some fundamental reason why LLMs will never be able to play chess competently, and I doubt there is, then it seems that we could with the right prompts implement some sort of chess AI using an LLM.
You can play a good game of chess (or poker for that matter) with GPT.
There's also some work going on in the eleuther ai discord training LLMs specifically for chess to see how they shape up. They're using the pythia models. so far:
I've found they fall apart after a couple of moves and lose track of the game.
Edit: This might not be the case anymore it seems, my below point doesn't actually contradict you, seems it matters a lot how you tell the model your moves.
Also saying things like "move my rightmost pawn" completely confuses them.
The token model of LLMs doesn't map well into how human experience the world of informational glyphs. Left and right is a intrinsic quality of our vision system. An LLM has to map the idea of left and right into symbols via text and line breaks.
I do think it will be interesting as visual input and internal graphical output is integrated with text based LLMs as that should help correct their internal experience to be based closer to what we as humans experience.
" An LLM has to map the idea of left and right into symbols via text and line breaks."
Oh yeah that's i suggested it :)
I do wonder though if we give the LLMs enough examples of texts with people describing their relative spatial position to each other and things will it eventually "learn" to work things these out a bit better
>I do wonder though if we give the LLMs enough examples of texts with people describing their relative spatial position to each other and things will it eventually "learn" to work things these out a bit better
GPT-4's spatial position understanding is actually really good all things considered. By the end, 4 was able to construct an accurate maze just from feedback about the current position and possible next moves after each move by GPT-4.
A funny thing GPT-4 is unusually good at is giving driving directions. This shouldn't work, and of course isn't 100% right, but… it's kind of right.
Bard can answer questions like this, but I think it actually uses the Maps API. (It certainly says that's what it's doing.)
On the other hand, every chatbot including GPT-4 is both unable to do ASCII art and unable to tell it can't do it. (Bard always shows you `cowsay` and tells you it's what you asked for, no matter what it was supposed to be.)
Probably. But what seems much more interesting is to have a spatial model pre-seeded in the LLM, so that it "attaches" language to that as part of its training. Ditto for other models of the world we want the language module to be able to draw on and reason with.
Can we say ChatGPT or its future versions would be like an instantiation of the Boltzmann Brain concept if it has internal qualia? The "brain" comes alive with the rich structure only to disappear after the chat session is over.
Cool way of putting it. Let's run with that. A good actor can be seen as instantiating a Boltzmann Brain while on stage -- especially when improvising (as always may be needed). Maybe each of us is instantiating some superposition of Boltzmann Brains in everyday life as we wend our way through various social roles...
From now on I'll listen for the subtle popping sounds as these BBs get instantiated and de-instantiated all around me...
Of course a philosopher can object that (1) these BBs are on a substrate that's richer than they are so aren't "really" BBs and (2) they often leave traces that are available to them in later instantiations which again classical BBs can't. So maybe make up another name -- but a great way to think.
In a sense, we are facilitating BBs generation by providing it with rich mathematical space. The rich structure makes BBs more probable than normal randomness allows for.
So, the reason why a Boltzmann brain would vanish almost instantly is because the longer it is to survive the more support structures it would need, a larger part of the local environment would have to be compatible with a longer existence, and all that makes it less likely for the fluctuation yielding such an outcome to have occurred.
I don't remember the video, but it was pointed out by Sean Carroll that the idea of BB is not fleshed out. Does one need only neo-cortex, or full brain, or just few control centers of attention within neo-cortex to be called as BB.
Chess move training data is much more likely to consist of examples of people trying to win than it is to consist of random legal or even illegal moves. You could argue that the intention belongs to the people providing the input and not the LLM, but that seems like a distinction without a difference.
True, because the LLM has no intention (sorry Blake Lemoine.) I'm puzzled about what people say about AI, when to me it is simply a talking library. I started going to public libraries at a young age (I'm 66) so to me I'm talking to the authors of the books, not some mysterious conscious machine.
My analogy for GPT-4 is this: GPT-4 is writing a novel, in which a human talks to a very smart AI. This helps me contextualize its hallucinations: if I were writing such a novel and I knew the answer to something, I would put in the correct answer; if I were writing such a novel and I didn't know the answer to something (and had no way to look it up), I would make up something plausible.
From that perspective, I think multi-intentionality also works. If I write a story about Bob, then Bob (in the story) has intentions, although he's just a figment of my imagination; and when we read characters in novels, we use the imputed intentions of the characters to understand their behavior, although we know they're fictional and don't actually exist.
So yes; on one level, I want to write an exciting story; on a second level, I'm simulating Bob in my head, who wants to execute the perfect robbery. On one level, GPT-4 wants to write a story about a smart AI; on a second level, the smart AI in GPT-4's story wants to win the chess game by moving the queen to put the king in check.
After having spent a ridiculous amount of effort to get LLMs to work, I am certain they are simply predicting the next token.
If LLMs actually could reason, there is a much much wider set of applications where they would be actively used.
The term “hallucination” does us all an injustice by propagating the idea of an anthropomorphized LLMs.
Everything an LLM does is a hallucination.
You and me can make out valid patterns from invalid patterns, because we have an idea of some reality.
(Incidentally there are some very weird implications/ perspectives deriving from these 2 positions.
Eg - If you had infinite data, would a LLM ever need to calculate?)
Point being - the more intimate the use with an LLM, the more its emergent properties are non-emergent.
All the whole damn universe does is move from the configuration at time t to the one at t+1. You cannot deduce from this whether the universe contains reasoning or not. We know from experience that it does, but a universe that doesn't seems possible.
Wow, where do I even start? The statement boils down a fascinating, nuanced field into an oversimplification that doesn't do justice to the complexities of machine learning, natural language processing, or, you know, human cognition for that matter!
Let's talk about "predicting the next token." Sure, that's the technical framework, but what happens within that prediction is an intricate dance of probabilities, patterns, and weighted connections that come together to form something that can assist, inform, and sometimes even entertain. There's a vast landscape of difference between a machine that predicts the next word in a sentence and a machine that can draft an entire poem, answer a complicated question, or simulate conversation in a way that can sometimes pass for human thought.
Is it reasoning in the way humans do? No. But to say that LLMs are "simply" predicting the next token is like saying a car is "simply" a collection of nuts and bolts that move in a certain way. It's true, but it's missing the whole picture. Just think about the implications! If this was as trivial as "predicting the next token," then why aren't we seeing this level of application everywhere?
As for the term "hallucination," I get it. It's a bit anthropomorphic, sure, but language always is. We use human-centric language to describe lots of things that aren't human. We say economies are "healthy" or "sick," we say a defense in football is "stalwart." Is it perfect? No. But it gives people a way to discuss and think about complex topics, including this one. And guess what, complex discussions are how progress happens!
The point about infinite data is intriguing, but let's not go off the rails here. The question isn't whether an LLM would ever "need" to calculate; it's whether the way it calculates could ever truly mimic human thought or reasoning. That's a long road we're still traveling down. But here's the kicker: just because we're not there yet doesn't mean the work that's been done is insignificant or simplistic.
Emergent properties becoming "non-emergent" the more you interact with an LLM? That's the point! The more you use these systems, the more you understand their capabilities and limitations, and the better you can leverage them for tasks that are useful, interesting, or revealing.
So, yeah, let's not box in what is one of the most dynamic, evolving fields right now with a one-liner that's as limiting as it is dismissive.
The reason why it isn't useful to ascribe intentions without a mechanistic explanation of intentionality is because you will incorrectly predict what the model will do in surprising ways.
I think it's true that current generation LLMs could, in principle, have intentionality in the way described in the article. But they would have to be trained on many orders of magnitude more data than current models.
Also AutoGPT does not work. I encourage the author to play around with it and try to get it to do something useful with high success probability.
> That is, we should ascribe intentions to a system if and only if it helps to predict and explain the behaviour of the system. Whether it really has intentions beyond this is not a question I am attempting to answer (and I think that it is probably not determinate in any case).
And yet, I think there's room to argue that LLMs (as currently implemented) cannot have intentions. Not because of their capabilities or behaviors, but because we know how they work (mechanically at least) and it is incompatible with useful definitions of the word "intent."
Primarily, they are pure functions that accept a sequence of tokens and return the next token. The model itself is stateless, and it doesn't seem right to me to ascribe "intent" to a stateless function. Even if the function is capable of modeling certain aspects of chess.
Otherwise, we are in the somewhat absurd position of needing to argue that all mathematical functions "intend" to yield their result. Maybe you could go there, but it seems to be torturing language a bit, just like people who advocate definitions of "consciousness" wherein even rocks are a "little bit conscious."