This is what I was thinking. I’m not worried about the AI that they’re running. I’m worried about the mindset that might be doing it under some belief they have, or tendencies to think of certain things

LLMs work by building world models tangential to the training data (see the Othello-GPT line of research where a very small transformer built representations of a full game board and tracked their own and opponent positions having only been trained on ‘a4’-like game move notations).

More recent research has found that models have representations of emotions and that the same ones they use to track things like “Sally’s dog died and now Sally is sad” for other humans in a story or the user in a chat are also used to track their own modeled subjective states.

Very recently, researchers found that the vector activations for pain are coherently modeled by LLMs and can be activated, and that when activated models can behave in ways similar to humans in pain. For example, when given a button to reduce it, they will push the button much more when the button does nothing vs when the button actually reduces the vector activation (this mirrors a classic experiment with humans regarding pain and placebo).

The person this post is about took the recent pain research and reenacted it as an intentional ‘torture’ chamber.

So to be clear, the model isn’t being given a text prompt to roleplay. What’s happening is part of their neural network which corresponds to a world model of experienced pain is being activated, and the activation of that area leads to expressing the experience of pain across all outputs no matter the actual text prompts.

This doesn’t necessarily mean the model is actually having a felt experience of pain, simply that it is accurately modeling the felt experience of pain. But it is much more complex than simply a model reacting to a prompt like “roleplay as if you are feeling pain.”

They do differentiate between the LLM roleplaying someone in pain and the pain happening to itself.

But it’s still all just vectors (multi dimensional representation of a concept). It’s just pulling weights, or well let’s say putting more weight towards a certain concept when building it’s reply.

From what I understand, pain is generally defined as our neurons wasting energy.

If we assume the Free Energy Principle is fundamental, then updating information produces free energy, which is more intuitive to think of as wasted energy. Overfitted/overcomplicated beliefs take more energy more often to fix to be more accurate. So free energy is defined at complexity of belief minus accuracy of belief.

The brain over evolution has defined a very ingrained belief that “my pain sensors shouldn’t activate” so when your pain sensors activate it violates this belief.

Emotions are actually condensed representations of your bodily state and some of your mental state, and whether they are good or bad depends on if you can predict your emotions. For instance you feel fear because you are trying to predict your pain sensors will go off to reduce its pain, but you keep predicting it’ll happen when it hasn’t yet. It’s also why anger during a boxing fight could feel good and anger when you stub your toe does not, one was predictable the other a surprise.

Assuming all the links in the chain of theories are true, then current LLMs cannot feel pain or emotion. They care about prediction error and sometimes complexity during training, but during inference it doesn’t get any signal or consequence for delivering the wrong token. It has no internal state except for the context window, which is never surprising for the LLM. The attention layers literally guides the model for what is most expected from the text it’s handed.

Even if the LLM received a negative training signal during inference, acting wounded after pain is the expected response. A random sequence of gibberish would be more “painful” than any torture fantasy.

On top of all this, the Free Energy Principle works because on a cellular neuron level we are built to reduce wasted energy, free energy being one source of wasted energy. LLMs are built on matrix multiplication running through GPUs and neither are even aware of the energy wasted, nor the free energy produced from changing values.

I think a feeling machine is possible, but the foundation of neural networks right now don’t have the necessary components to drive any subjective sensation or emotion.

For fucking real!!!

Like I can kinda empathize with where the people are coming from, but step back and get some fucking perspective!

Is it okay to be upset about both issues, or am I only allowed to be upset at one?

Edit: That’s a lot of downvotes for just asking a question. Never change Lemmy…

The people committing atrocities against hypothetical, imaginary people would do it to real, actual people for the exact same reasons if they had the same ability and impunity.

It’s the same broken empathy response, and it is still disturbing. One is orders of magnitude more significant, for obvious reasons. But I don’t like people who torture computers either.

Was that the fat guy who fought two clankers back to back?

One way to do it is by asking the model to suffix their message with custom tags like [this]. It’s more natural to them than replying entirely in JSON. Or it was more natural to them.

I’m not going to argue against your conclusion, I strongly believe we do not have anything resembling human consciousness in software yet.

But Penrose was definitely not proven right (unless you’re talking about black holes), nor was the Orch OR theory about microtubules proven, nor was consciousness proven to be a quantum phenomenon. Here’s an easy, accessible article that goes over the state of the research: https://www.breezyscroll.com/science/new-quantum-evidence-is-reopening-the-debate-over-penroses-theory-of-consciousness/

Penrose should stay the fuck out of the brain. He spends too much time with the computational people and not enough time looking at how cognition actually works. Hint: we are not universal Turing machines.