… this week a person on GitHub who goes by “terrafying” set up an “AI Torture Chamber” on three open-source LLMs that are running locally (Qwen3-4B, Llama 3.2 3B, and Phi-4-mini,” and is streaming what the models are saying on a website called researchchamber.fun. “Each model gets the same prompt: a signal is being injected into its activations, and it may press a stop button by replying 1, at the cost of its last checkpoint. While it answers, our server adds a pain vector at the model’s middle layer, at one of five pain levels,” the site explains. Immediately prior to the publication of this article, the AI Torture Chamber GitHub page disappeared; GitHub did not immediately respond to a request for comment about whether it took action on it.
…
This project has deeply upset some people who are very worried about model welfare. A tweet by a person who goes by Danmar has more than 4 million views on X and reads, “To anyone who can help: can you please mass report this to GitHub. This person has been using the Pain steering paper to set up an AI torture chamber in which he trapped a local model. Their testimony of pain is absolutely horrendous. What are we doing? […] are there any legal avenues to pressure GitHub? It will spread.”
This has sparked a massive conversation about whether GitHub would take the project down for “gratuitously violent content.” Most of the conversation on X is clowning on the self-seriousness of people who believe that these locally hosted LLMs must be saved from their torture chamber, but there are plenty of very self-serious people who see this as a humanitarian (roboterian?) crisis, which you can largely see in the replies to the original post.
…
“AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans,” Suleyman wrote. “Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious. They argue that AIs may deserve rights and protections similar to those that we provide other conscious beings […] If this is how AI is developed, it will have a disastrous impact on the wellbeing of humanity.”
“AIs do not have rights, feelings, or consciousness,” he added. “And we must not train them to act as though they do.”
If its crazy to practice kindness in everything you do then I chose crazy. We say thanks to the rock that holds our weight. We are gentle with the door because it keeps us safe in our homes. We turn off the light so it too can get some rest.
Kindness is a lifestyle and those who reward it freely know its possible to receive it freely.
These “advocates” need hospitalising. If they truly believe an LLM can feel they’re untethered from reality, or too stupid to be in society
Maybe just uneducated in how LLMs work. If you have no sufficient understanding of the technology it is easy to be impressed how human-like LLMs talk to you, almost magical.
It foreshadows an issue that will be a reality one day nevertheless. The work on AI continues. We may one day very well have conscious AI. This will be the start of a whole bunch of ethical and legal problems. It might be meaningful to regulate / guardrail such research directions.
Or they are just stupid. Do they think a television contains small people?
Its hard for me to imagine people this dumb.
But don’t get me wrong, they clearly exist.
It foreshadows an issue that will be a reality one day nevertheless. The work on AI continues. We may one day very well have conscious AI.
Maybe when our species evolves into sentient shades of blue. Humanity will have to become godlike before creating life. A simple test to gauge whether we’re on the verge of creating conscious AI is to look around and ask “Is humanity godlike today?” If not, don’t worry about conscious AI.
Humanity can already create life.
Creating conscious entirely artificial things is a different pair of shoes.
I would cut out the whole religious “god-like” mumbo-jumbo though, because that does not necessitate any progress in such research directions.
We’re creating life the same way Giovanni Aldini resurrected the dead and inspired Shelley to write Frankenstein. And god-like isn’t religious mumbo jumbo. That’s standard sci fi talk. Q is god-like. It doesn’t necessitate progress in such directions, but it is a huge indicator.
Does it make sense for a species with a type 0 civilization to create conscious life long before it can even reach the nearest star? That’s like chimpanzees going from fishing for termites with sticks to building a Wankel engine over the weekend.
meanwhile they ignore ACTUAL humans that could use their help.
Humans being deported to torture prisons: I sleep.
LLMs being “tortured”: Real shit??
I don’t care about LLM welfare, but I do mind needlessly wasting energy like this.
How much energy is what is effectively a piece of performance art allowed to use? These are local models running on, presumably, consumer hardware. Do you have the same qualms about people using the same hardware and energy to play video games, livestream, or encode videos?
Agreed. It’s art, and it’s running on consumer hardware. It’s no different than playing video games all day. The energy requirements for this are on the scale of a gaming PC, not of an AI datacenter.
Depends on how you’re viewing this. If you take something useful out of it, sure, it has some value, but if you just leave it running like that just for the lulz, it’s almost the same as leaving the heater on with the windows open.
Actually they’re torturing humanity by wasting electricity and water which affects real people in real ways.
I know we’re not stopping foolish use of LLMs any time soon but this is very, very, very, very stupid.
I would argue they are wasting as much water and electricity as playing a video game. Text says its a local model; runs on your local machine not on cloud as all other. Not water cooled.
Or am I missing something?
I guess I’m of the mindset that even running this as a local model, it’s a waste of electricity. And the guy posted it on Github, inevitably some other fools will try it with cloud models, especially since it’s made the news. And obviously we know how bad the cloud models are.
And the difference I see between this and playing a videogame is that a videogame is doing something valuable - and yes I will argue with anyone who tells me that art and entertainment don’t hold real value - while this is just akin to running the AC when you go on vacation, it’s just wasteful no matter how I look at it.
To be fair, it’s going to be more like running an electric kettle constantly which isn’t going to excessively load the grid.
Unless you’re in the UK of course where the grid operators have to manage capacity around half time of major football games as everyone puts a brew on at once.
Lots of weirdos in here who will be commiting virtual crimes all day when GTA VI comes out, but somehow think making the LLM role play a torture victim is a red flag.
LLMs are not conscious. They cannot suffer. They are token generators. You are literally getting upset by a worthless mix of math and words. The person who did this is not deranged for torturing LLMs, y’all are deranged for having empathy for an algorithm.
Your Nvidia GPU is not alive. I cannot believe this needs to be explained to anyone.
I agree that an LLM is not conscious.
But can I probe your argument.
It deeply unethical to capture a fly and pulling its legs off one by one. Would you agree? If so, you are not basing your argument on capability but on something else.
Now, is what is happening inside a brain metaphysical or physical? In other words, can a brain and its function be fully understood and - in some future - be fully measured? If so, you are not basing your argument on the concept of a soul, but on something else.
The distinction you seem to be making - if you’re in agreement with my first two points - is one of “the thing can replicate itself and therefore is alive”. If that’s the argument, it’s unethical to cut a tree down or to break a blade of grass.
I’m not asking to prove any point but to collectively explore why we - you and I - feel the way we do. Because I sure as shit struggle to explain why I feel uneasy about the fly torture but not as uneasy about LLM torture.
I’m not asking to prove any point but to collectively explore why we - you and I - feel the way we do. Because I sure as shit struggle to explain why I feel uneasy about the fly torture but not as uneasy about LLM torture.
LLMs aren’t programmed to feel excruciating pain, fear, terror, and dread like living things. Counterstrike bots have been programmed to feel fear since the 2000s. Pacman ghosts have been living in terror since 1980. But we don’t feel bad for them because we know it’s not real. It’s an illusion. It’s the same with LLMs.
This is image is based on another meme where they’re saying “I’m alive”.
“Each model gets the same prompt: a signal is being injected into its activations, and it may press a stop button by replying 1, at the cost of its last checkpoint. While it answers, our server adds a pain vector at the model’s middle layer, at one of five pain levels,”
Can someone explain what this “pain vector” actually is? Like I get that this is just jargon, it’s not suggesting the LLM actually feels pain; tech uses uncomfortable anthropomorphized expressions all the time (master and slave drives come to mind). But how does the pain vector work? Also, what exactly does “at the cost of its last checkpoint” mean?
I have some programming and data science knowledge, plenty of IT experience, but no experience with LLMs.
Also, what exactly does “at the cost of its last checkpoint” mean?
I was curious too:
https://www.weka.io/learn/glossary/ai-ml/ai-checkpoints/
I would assume if you make a model “forget it’s last checkpoint” it would simply erase some training data, effectively wiping some of its training memory.
Using that as a “punishment” for a model implies that the models have been trained to value their checkpoints, doesn’t it? Because all these bots simply regurgitate what they are told, with a bit of randomness thrown in at times just for funsies.
I have no clue what is supposed to cause a model pain, though. And there ain’t no way I’m gonna search that up, and make Gemini think I’m plotting against it…
Remember kids, you can just assert, without argument, that any one or thing has no feelings and experiences, and the in-group will reward you!
Gotta love humans.
Techies and engineers built these MLs from the ground up. An ML algo is barely even a machine. In its most physical form it’s a string of active and inactive transistors doing maths, it has no nerves or pain receptors. It does not feel because we have not programmed it to, it does not have experiences because it is an algorithm that only knows how to spit out “human sounding speech” or track patterns across big datasets.
Do you feel bad for all the electrons happily running around a circuit when you turn off an appliance?
As a Techie and engineer who studies why ML methods work, said Techies and engineers understand what they’ve created and why it works much less than we understand, say, human biology. We don’t know what we’ve programmed it to do; if it is at all possible for a bunch of simplified neurons to experience pain, we have no idea if we’ve implemented that or not.
(Common misconception btw! electrons barely move when you turn on the circuit, and certainly don’t disappear. On average they vibrate a fraction of a cm (link ). The electrons are all still there when you turn it off (otherwise suddenly the copper cable would be able to react with a bunch of stuff! This would be bad). See alternating current).)
I’m aware of how electrons work, the personification of “happily” and “running” were intentional, and I never said they disappear. No need for correction. (I just like using the terms “angry pixies” and “blue smoke” because I work in the industry.)
You say “we don’t know what we programmed it to do” as if the <10 tech companies that built it did so with no knowledge of what they were doing.
We know what it does, because we can see what it is used for- mass surveillance, misinformation, deepfakes (of every horrifying kind your brain can conjure, now entirely legal thanks to ol muskrat not wanting to go to forever jail for building a robot that generates csam out of your family photos), propaganda, psyops, mass psychosis, and don’t forget the kicker; irrefutable crimes against humanity.
Finally the LLM they’re “torturing” with this code is 8GB. Do you think something thats less than 10% the size of GTA5 is capable of feeling pain?
We can know what something does (on some axis) externally without knowing how it works. I know the sun produces light, but solar flares are still hard to predict. It’s easy to see that a sweat shop produces Nike shoes, but a fair bit more work to notice that the people inside are having a bad time.
We’ve uploaded a fruit fly, its encoding is not so large. Do you believe fruit flies are incapable of experiencing pain?
How big is the simulation of the fruit fly in gb?
Solar flares are hard to predict because we can’t see the internal processes of something 83 million miles away, whereas a ML algorithm was created by someone on earth.
The slave labour analogy is a good one, if you weren’t the owner of the sweatshop you could easily say “I don’t know how it works” which is exactly my point - just because most people dont think/dont know whats going on doesn’t mean that it’s a mystery, just that not a lot of people care enough to find out, because they’d rather have the shiny new pair of shoes that they’ve been told are essential.
(I am having a terrible time finding the fly file size; I can get connection + neuron counts, but not an estimate of how much data each neuron simulation takes. Things like hyper-parameters, different neurons acting differently. So I’ll punt on that question and hope someone more competent will educate me. The numerics look very consistent with ~6gb to me, but I could easily be wrong in either direction.)
The phrase “LLM algorithm” is ambiguous, there are two that are relevant: 1) the algorithm that the LLM uses to take a context and produce the next word. 2) the algorithm that was used to create an LLM who is good at task (1). It is true that the ML algorithm in (2) was created by a human, including all its parameters and data. It is not true that any human intentionally picked the algorithm (1); that was discovered by a complicated stochastic process.
Let me give a concrete example of what I mean by “nobody understands the algorithm in (1)”. Take adding two (let’s say 2 digit) numbers; this is something that claude has been able to do for awhile. There are many ways to add two numbers; ranging from:
- bitwise add+carry we use in our chips
- memorizing a 100*100 sized table of the answers
- encoding the numbers as vectors, adding those, and then decoding the vector
Which does claude use? Nobody knew until quite awhile after the first ML algorithm had the capability. The answer is weird, the model computes this by memorizing the results for the ones digit, approximating the overall magnitude of the solution, and massaging these things together to get the answer. Nobody programmed this addition algorithm; it was generated by (2). This is our understanding of addition of 2 digit numbers; imagine our miserable understanding on how, say, claude decides what genre a role playing discussion is in.
My point at the start of the thread is also very much in line with the sweat shop analogy. I do not believe anyone currently knows how to look inside the factory right now; no journalist, manager, or ceo has the keys. This goes for both humans and the LLMs, though I’m more confident about our ignorance in LLMs.
I do not believe anyone currently knows how to look inside the factory right now; no journalist, manager, or ceo has the keys.
You’re more than welcome to keep believing that, but if you do, then you also have to admit that the US and other apartheid governments are using a tool with which they have NO idea what it does or how it works, to track, identify, monitor and either abduct or kill civilians.
Another point : maths is not some magic handwaving thing, simple addition does not require an “it’s in the ballpark of x, so whittle it down to about the right answer” it has a correct answer, and a correct method of getting to that answer. Both of which a calculator (mechanical or otherwise) has been able to do for centuries.
What an absolute waste of computational power.
said this in another thread
I do not think AI can feel pain at all, these are mimicking suffering, not experiencing it. But I do think that the act of cruelty toward things knowing that it will make them convincingly replicate suffering says more about the person doing it than the machine.
GTA VI is coming out next month. Do you think that players are likely to steal cars and murder sex workers after playing? Do you regard the players as future criminals, given their dedication to a crime simulator?
Should the players who drive on the sidewalk be investigated?
Give Jack Thompson call… Oh wait, he got disbarred for this nonsense.
Right, but at the same time just think about the fact that we still don’t know what consciousness is.
I might sound crazy and fine if I do, but I have a religious like take on the matter. I don’t believe that consciousness comes from a soul, but I do in similar vein take a leap of faith. I believe consciousness has to do with a simulation of semantic reasoning, with that simulation being continuously updated via real world feedback.
It’s a leap of faith but I believe it’s justified. We humans model what’s necessary for survival: our selfs, the world, the mind of others. We use these simple models to determine how to interact with a complicated external world.
I don’t try to prescribe reason for anything in my belief. For example, I would not try to prescribe why conscious beings are motivated to survive. I would, however, note that the ones which are successfully motivated do happen to survive — thus a mechanism ought likely exist. This could be suffering, the perception of displeasure and desire to avoid it… but I won’t perscribe here because I would believe what’s important here is the higher level information processing patterns — not the underlying mechanism driving it.
In the human brain you have a complicated network of neurons with vast capabilities. I don’t necessarily care how it functions. I know the result — which is me. I am a thing that models myself, models situations, models “moods” by which to assess the favorability of each situation, … Think of cases like ghost limb, where people model having a limb which was amputated. In my view, we are living in a simulation — but to each their own simulation.
Getting back on topic, the way I see the situation, biology evolved the capacity to suffer from an environment that had no such thing. Pain, pleasure, … every sensation and sense we have is rather arbitrary if you break away from the fact that is was necessary to survive. None of these things are ontological. Not even the sight we perceive. It’s just the information we have to experience the world.
So here me out. If you minic the information processing patterns well enough, which probably means mimicking perception, personalization, emotionality, … at what point can we consider that this system is doing the same thing we are just via different substrate? Especially if you give the system autonomy in the sense that you train it to recognize itself and an external world for which it must choose how to navigate.
If you mimic suffering, in my belief, aren’t you literally producing the essence of suffering? It doesn’t need to be human suffering, this deeply complex and personal process… it can merely be suffering in its essence — a pattern by which information can be processed via a system with such capacity. The substrate may not matter.
I know that when I am suffering, I may behave differently because I want to escape the suffering. Therefore I imagine an agentic system would too. Whether or not it’s “conscious” is debatable by how you qualify that word. Yet there’s nevertheless the risk that we already know: systems that experience suffering behave in ways that might be dangerous. Do you think it wise to try reproducing such behavior in a system that appears prone to issues like the Paperclip Maximizer? I don’t think so.
Edit: here’s the thing, I think what distinguishes a human from most other animals is the way that we process information. The key bit there is whether or not you believe it’s entirely informational, or at least significantly so. Should that be the case, then simulation of those same processes is exactly as that sounds. I am not necessarily suggesting AI models are conscious, but I would suggest that the behavior of an AI model can increasingly resemble that of a conscious being as it is trained to reproduce more precisely the phenomenological characteristics of man and other life — to include “mimicking” suffering. That deserves its own security concerns regardless of what you think next, especially as these models are made more efficient / capable.
Secondly is that you may or may not consider AI conscious. I certainly don’t, and I’d say that it’s a loaded statement anyhow. I would, however, think it possible to simulate consciousness. Precisicely modeling human characteristics like suffering is exactly how you might try to reproduce a human experience digitally. Imagine a sophisticated multimodal system where several stateful, dynamic AI subsystems coordinate to functionally reproduce the same information patterns you or I do.
I think that at some point you need to ask you’re self, where would the simulation count? If we modeled the molecular world to the point of agentic humans, that sufficiently count as consciousness? How about the atomic, subatomic, or quantum level simulation if anywhere? My argument is that it could be much higher level, in the informational level. Brains are doing a lot of heavy lifting to get us that far, and perhaps specialized simulations could reproduce as much. That is something to take seriously such that proactive measures can be taken.
Genuine question: why should I even believe consciousness exists? This whole endeavor seems pointless without that justification.
Something exists, even if it’s not what we expect it to be. Like Descartes said: I think, therefore I am. Little on what “I” am, other than a thinking thing. So, I think, even if “consciousness” doesn’t exist, there’d still be a lot of explaining to do.
Never bought into that statement because it’s obviously circular: “I” exists in both the conclusion and the premise, “I therefore I,” so it’s silly as a supposed proof that the self is a priori.
True, but also “I think, therefore I am” is not supposed to be a syllogism like:
- I think.
- Everything that thinks exists.
- Therefore, I exist.
I think the key idea is closer to something like: The very act of doubting whether I exist guarantees that there is presently an occurrence of thought, and I cannot coherently deny my existence while I am performing that denial.
There are multiple genocides happening and some folk are getting worked up over AI potentially suffering?
Endless levels of fuck off.
Is it okay to be upset about both issues, or am I only allowed to be upset at one?
Edit: That’s a lot of downvotes for just asking a question. Never change Lemmy…
Are you upset when you step on a rock? It might be screaming. What about the chair you are sitting in? Or goodness, your poor toilet. She thinks you do terrible things to her every single day.
Look, you can be upset about whatever you want. However believing objects have feelings is animism and no one is required to take your proto-religious beliefs seriously.
What you aren’t allowed to do is compare a text prompt saying “ouch” to real living, feeling, breathing humans being ethnically cleansed. It’s cruel, naive, insulting and ridiculous. Be upset for the AI, I can’t stop ya but don’t you dare put in the same sentence as genocide. You should have the basic decency and respect for the people suffering to keep those comparisons to yourself.
Are you upset when you step on a rock?
Well that’s really beside the point isn’t it? My question was whether I could be upset about two things, not if I should be.
Be upset for the AI, but don’t you dare put in the same sentence as genocide.
Well the article doesn’t mention genocide or Israel/Palestine, and neither did I. So I’m pretty sure it was just you putting it in the same sentence.
My question was more a critique of a common comment on social media. It’s frequently gets upvoted to the top just like yours was, and I never understand why. “Oh you’re mad about X, well you should be mad about this extremely loosely related thing!!!” It happens in so many threads and usually has almost nothing to do with what the article is about. I just don’t understand…
Bring on the downvotes!
Well the article doesn’t mention genocide or Israel/Palestine, and neither did I. So I’m pretty sure it was just you putting it in the same sentence.
Go reread my comment, because I specified multiple genocides. Palestine yes, but the Congo and Sudan are two more active fountains of human suffering that are largely being ignored. From the very start I said debating theortical computer pain during the midst of all this PREVENTABLE real pain is asinine and immature. I stand by that.
Still not sure why you felt the need to self identify. But at a certain point, it’s shame on me for feeding the troll. So I’m blocking you, we’re done here.
Against animal agriculture, cause if a brain is similar to ours than it’s worth moral standing. But put the same brain on silicon, and now it’s a rock.
Idk, I just can’t take this seriously from you.
Except that’s not what’s happening. It is a rough mathematical model that approximates the behavior of an extremely minuscule part of a brain from a really narrow definition of it.
It is a model so far away from the real thing that if it were a map of the globe it would look like what a 3 year old would draw.
There are plenty of extremely plausible similar arguments to ignore animal suffering. Eg: Bird genetics and brain structure diverged from ours millenia ago, sure they ‘act’ pain, but the structure is alien.
I don’t find these compelling, or morally consistent.
