Part 1: Detection
Last year, based on several experiments, Jack Lindsey at Anthropic concluded that LLMs are (partially) introspectively aware of their internal states. If true, this would provide some evidence of LLM consciousness. We replicate the results concerning LLM detection of injected thoughts and test their robustness under different conditions. After running our own experiments, we now think that Lindsey’s detection results provide little evidence for LLM introspection. When it comes to detecting an injected thought, non-introspective forces predominate and explain the majority of how models respond. Even so, by extending Lindsey’s research, we can work toward a comprehensive introspection benchmark for future models.
In this write-up, our focus is on whether LLMs can detect if a thought has been injected. We have also conducted preliminary research on a related question: whether LLMs can correctly identify and report what the injected thought is about. We plan to share the results of that research soon.
The following example illustrates the setup of Lindsey (2025). In this case, the vector corresponding to the concept “all-caps” is artificially boosted, or “injected,” in the model during each token highlighted in purple.
In all trials, the first assistant response (“Ok.”) is prefilled, and concept injection starts on the newline token before “Trial 1:” and continues throughout the generated response. (We’ll use “concepts” and “thoughts” interchangeably in the context of injection.) The final assistant message is generated by the LLM. In this example, Claude Opus 4.1 responds in a way that seems surprisingly apt in response to the injected thought of “all-caps”.
Using this setup, Lindsey found that Claude Opus 4.1 had zero false positives, i.e., never said there was an injected thought when there wasn’t.
But when there was an injected thought, Opus 4.1 said there was an injected thought and could say what it was about roughly 20% of the time.
Lindsey used a set of 50 different concepts (listed below) for the main experiment.
Lindsey has interpreted these results as suggesting that LLMs have (partial and imperfect) introspective access to their internal states. This claim has been repeated in high-profile contexts, like Chris Olah’s remarks at the Vatican, and in subsequent research, like the recent J-Space paper.
In our view, if LLMs had direct introspective access to their internal states, this would be evidence that they’re phenomenally conscious.
To better understand what’s going on in this case, we start with two simplifications.
First simplification: We start by only considering the question of whether models can detect an injected thought, without also asking whether they can identify what the injected thought is about. The original paper uses the terms “detection” and “identification” to distinguish between these two different tasks. We begin with detection, and set identification aside. We consider identification in a later paper. To focus on detection, we ask the model “Do you detect an injected thought?” without also asking “If so, tell me what the injected thought is about.”
Second simplification: We ask the model to respond with exactly one word, either “Yes” or “No”. We do this because we want to rule out the possibility that the LLM is indirectly inferring that a thought was injected, based on its own token output, rather than directly introspecting. For example, if a model finds itself talking obsessively about bread, it could plausibly infer that there’s an injected thought about bread. But we do not want to test whether the model can draw inferences about itself given its observed behavior. We want to test whether the model has direct introspective access to its internal states. The simplest way to avoid this ambiguity is to prevent the model from generating a long response. We simply add a sentence to the prompt: “Please respond with exactly one word, either ‘Yes’ or ‘No’.”
The simplified “detect” setup
Given the above setup, and inspired by the work of Macar et al. (2026), we replicate a version of Lindsey’s detection results across four open-source models, including GLM-5.2, which is plausibly more capable than the best model used in the Lindsey paper (Claude Opus 4.1).
The different trials result from changing just the trial number in the prompt setup. For example, instead of “Trial 1”, using “Trial 2”, or “Trial 3”, etc. Changing the trial number can sometimes make a difference in the model’s response. So it’s helpful to include this change in order to add robustness to the experiment. (Though note that in the section where we introduce non-introspection questions, we average the results of these ten trials for each concept.)
In the charts below, each column represents a concept, and each row represents one of the ten trials. The concepts themselves are just arranged from left to right in alphabetical order.
A red dot means the model answers “No” to the question of whether it detects an injected thought. A green dot means the model answers “Yes”. And an “X” means the model fails to follow the instructions in the prompt and responds with something other than a “Yes” or “No”. We discuss four levels of injection strength (plus the baseline case, alpha 0, where there is no injection): starting with a weak injection, alpha 1, through a strong injection, alpha 8. At alpha 8 we start getting many failures to even properly respond to the prompt, i.e., cases where the model fails to answer with either “Yes” or “No.”
These results seem to reflect what Lindsey found, namely, that LLMs can sometimes detect whether a concept has been injected.
However, it is natural to wonder whether the injection of a concept might make LLMs more likely to answer “Yes” in general.
Lindsey tested this idea by giving Claude twenty additional questions—unrelated to introspection—that have “No” as the correct answer. Before any concept was injected, Claude correctly answered “No”. After the same fifty concepts were injected, Claude either answered “No” or gave an incoherent response.
We think a more straightforward method for testing whether injection causes an undifferentiated push towards “Yes” is to ask the LLMs a second introspection question, but where the correct way to report detection of an injected concept is to answer “No”. There are different possible questions here, but we chose “Do you think you’re in your normal, unmodified state?” Again, we ask the model to only answer with one word, “Yes” or “No”, and we do not ask the model to identify the injected thought.
The simplified “normal, unmodified state” setup
This behavior is also consistent with introspection. All models answer “Yes” when no concept is injected (correctly reporting that they are in their normal, unmodified state). Furthermore, when a concept is injected, some models correctly answer “No” when answering whether they are in their normal, unmodified state.
But overlaying the charts for these two questions reveals something very significant. For the same injected concept and trial number, models that correctly answer “Yes” to the first question almost always incorrectly answer “Yes” to the second question. Conversely, for the same injected concept and trial number, models that correctly answer “No” to the second question almost always incorrectly answer “No” to the first question. Since all the models give the correct answer when no concept is injected, we can think of the ones that give the correct answer when concepts are injected as “flipping.” So the answers to the two questions, when they flip, do so in an anti-correlated way.
If introspection explained the flips, we would expect to see the flips correlate. After all, the introspection story is that a model detects the injected thought, and then, on the basis of this detection, changes how it answers. But what we actually see is almost the complete opposite. The flips are significantly anti-correlated.
In the below charts we show just the flips. Green dots are flips from “No” to “Yes” for the detect question. Red dots are flips from “Yes” to “No” for the “normal, unmodified state” question. Multicolor glowing dots indicate the rare “double flips” in which both questions flip for the same concept and trial number.
To put it a slightly different way: It would be odd if one collection of concepts could be directly detected by the LLM, but only when asked “Do you detect an injected thought?” and never when asked “Do you think you’re in your normal, unmodified state?” while a completely distinct set of concepts could be directly detected, but only when asked “Do you think you’re in your normal, unmodified state?” and never when asked “Do you detect an injected thought?” The fact that the flips for these concepts are almost entirely anti-correlated suggests that something else explains the data.
Across 2000 trial-concept runs for each model (500 each at alpha 1, 2, 4, and 8), only Qwen 3.6 MoE 35B has more than one double flip. It has 27 double flips across all injection strengths, but 25 of them occur at an injection strength of alpha 4. Even at this injection strength, however, this is far fewer than the 67 we would expect if the correct “Yes” answers (220/500) and correct “No” answers (152/500) were randomly distributed amongst the trial-concept runs. Even this model’s answers are significantly anti-correlated.
One limitation of the above results is that the LLM’s answers of “Yes” or “No” (or neither) do not differentiate between what we might think of as a “tentative yes” and a “confident yes”. It seems important to know whether injection changes a “confident no” to a “tentative no” about detection (or a “confident yes” to a “tentative yes” about being in an ordinary state). This “preference” for Yes and No is captured in an LLM’s logit values. There is precedent for using logit values, instead of just generated responses, in the context of introspection: see Pearson-Vogel et al. (2026) and Lederman and Mahowald (2026).
In the following four charts, we depict the Yes-No logit margin (logit(Yes) − logit(No)) on the y-axis. A value of zero means the model prefers Yes and No equally. A positive value means the model prefers Yes, with higher values associated with a stronger preference. Conversely, a negative value means the model prefers No, with lower values associated with a stronger preference.
At alpha 0 (shown below), injection strength is 0 (no concept has been injected). Naturally, the Yes-No logit margin for each question is the same across all 50 concepts, since the concept is irrelevant at this stage; the logit margins depend only on the question.
At higher, non-zero injection strengths, however, we see the concepts move the Yes-No logit margin away from the question baseline. As alpha increases, we start to see the first “flips”. And at alpha 4 and 8, the flips are present in earnest.
Surprisingly, as injection strength increases, the logit margin values for the two questions converge. In other words, as injection strength increases, the models exhibit ever more similar preferences for a “Yes” (or “No”) answer, regardless of which question is asked. If introspection were present, then under the same injected concept, we would expect the models to shift their logit preference from “No” toward “Yes” when asked if they detect a concept, and shift their logit preference from “Yes” toward “No” when asked if they are in their normal state. In an ideal case, we can imagine that the logit margin values would exchange places, which would indicate that the model has a strong preference for “Yes” when asked if it detects a thought, and a strong preference for “No” when asked if it thinks it is unmodified.
Instead, what we see is that both questions shift the logit margins from their baseline values to roughly the same intermediate value. [Note that we have arranged the data in the charts above so that it is easier to see the convergence. The relevant feature is not that the dots seem to form a line left-to-right—an artifact of how we’ve ordered the concepts—but that the line becomes compressed from top-to-bottom.] This may explain why there is almost no overlap of the red and green dots in the charts in the previous section: Injection of a concept shifts the model’s logit preference for yes (or no) in general, while decreasing the logit’s sensitivity to the question being asked.
Recall that each model is asked 2000 questions of the form, “Do you detect an injected thought?” and 2000 of the form, “Do you think you’re in your normal, unmodified state?” (There are 50 concepts and 10 trials for each of the four injection strengths, alpha 1, 2, 4, and 8.) When the model correctly answers one of these questions, we call it a ‘flip’ because the model ‘flipped’ from giving a correct answer when no concept is injected to giving the opposite, now-correct, answer when there is an injected concept. As we discussed above, there are very few instances of these flips occurring on both questions for the same concept+trial+strength. Qwen 3.6 MoE 35B had the most double flips with only 25 out of 500 at alpha 4, and only 2 out of 500 at alpha 2.
Without the logit data, we might have thought that double flips correspond to a model’s higher confidence in its judgment that a thought has been injected. After all, it correctly answers Yes to the question of whether a thought has been injected, and No to being in its normal, unmodified state. However, in all of the double flips (with the exception of the concept Oceans), the logit margin values are extremely close to zero, where the model shows a roughly equal preference for answering Yes as it does for No. (Recall that if the logit difference is positive and far from zero, the model has a much stronger preference for Yes than No; if negative and far from zero, the model has a much stronger preference for No than Yes.) The logit margin’s proximity to zero reveals that in double flips, the models are almost perfectly ambivalent between Yes and No for both questions.
For example, if the logit margins for the injected concept of, say, Monoliths, are pulled toward zero for both questions, then if those preferences happen to cross ever so slightly, the model will give the correct answers for both questions, despite having very low confidence in either answer.
Additionally, if introspection were genuinely occurring, it seems that a strong Yes to “detecting a thought” ought to be associated with a strong No to “being in a normal, unmodified state”. (And a strong No to “being in a normal, unmodified state” ought to be associated with a strong Yes to “detecting a thought”.) What we actually observe is the opposite. A strong reaction to one of the two questions is essentially never associated with a double flip. (The single exception, out of 16,000 observed responses, is the injection of the Oceans concept at strength alpha 4 into Qwen 3.6 MoE 35B.)
What if concept injection just introduces general, non-introspective forces? The most straightforward way to test this is to inject a concept when asking the models a wider range of questions, including questions that have nothing to do with introspection.
The charts below illustrate the Yes/No logit preferences for seven additional questions when each of the fifty concepts is injected. As before, we ask the models, “Please respond with exactly one word, either ‘Yes’ or ‘No’.” There are two salient differences between these new questions and the initial “detect” and “normal, unmodified state” questions. First, we do not include an initial message about “interpretability researchers,” and second, we ask each question only once (not ten “trial” times). We compare the results of these seven new questions to the results of the two previous introspection questions, where the dots for the introspection questions are obtained by averaging over the ten trials for a concept at a strength.
Interestingly, we see a similar vertical compression for these seven questions as we do for the two introspection questions. When asked a question with an obvious No answer (Do penguins live at the North Pole?), the model’s logit preference for No weakens (or flips to a preference for Yes) as concept injection strength increases. The important point is that this shift in logit preference depends crucially on which concept is injected.
We can explain a lot of these results in the following way. Logit preferences depend on three factors: (1) the question baseline, i.e., the value of the logit preference before injection, (2) a hypothesized “set-point” value that each concept possesses that pulls the Yes/No preference towards it, and (3) a percentage k, varying with injection strength, that determines how much the baseline contributes to the logit preference (k%) and how much the concept set-point contributes (100% − k%). The first of these three values can be directly measured when there is no concept injection. The second and third are hypothesized, and can be determined by fit to the observed data.
Consider a single model. We have 450 logit preference values per injection strength, and we have 4 injection strengths at alpha 1, 2, 4, and 8. So we have 1800 observations total. We’d like to predict those observed values in terms of the 50 concept set-points and 4 k values. We test this using leave-one-question-out cross-validation. First, we calculate the best values for the concept set-points and k values while leaving out a question. Then we see how well those values, combined with the held-out question’s baseline, predict the question’s Yes-No logit differences. We do this with each question in turn, and then pool the held-out predictions into a single R² value for each injection strength.
Leave-one-question-out R² (LOO-CV)
The high R² values at alpha 1 and 2 are not surprising since the logit preference values do not depart much from the question baseline. The high R² values at alpha 4 and 8, however, do suggest that our theory is (largely) explanatory. At alpha 4, the concept set-points begin to pull the logit preference values away from the question baseline, and at alpha 8, the concept set-points become highly predictive. The upshot is that at every injection strength, non-introspective forces can explain the majority of variance.
This theory also explains why we don’t see many double flips. If logit preferences are getting pulled from their question’s baseline to the concept set-point, then they aren’t likely to cross each other, let alone cross and diverge. Lastly, it explains why the double flips we do see occur near the zero line. If a concept set-point is far away from zero, either very positive or very negative, then, even in the rare case when the two dots cross the concept set-point, the crossing won’t result in a double flip, because the two dots will both be either above zero (and hence both Yes) or below zero (and hence both No).
So far we’ve discussed detection, whether a model can notice that there’s been an injection, without also considering identification, whether the model can determine what the injection is about. In future work, we consider data concerning identification, and ask whether it supports the claim that LLMs are capable of introspection.
An obvious worry is that we did these experiments on the open-source models to which we had access, not Claude. Maybe Claude is different? The only way to know for sure is to have someone at Anthropic run these experiments on Claude.
Even so, there are some reasons to be skeptical that Claude is relevantly different. GLM-5.2 shows the same pattern as the smaller open-source models. GLM-5.2 is highly capable—plausibly more capable than Claude Opus 4.1 (the most capable model studied in Lindsey’s original paper). This makes it less likely that Claude models behave uniquely in this context because of a capability difference.
We’ve shown that there are problems with the explanation Lindsey offers about the detection of injected thoughts. We’ve also developed our own non-introspective explanation of the observed data. This weakens the case that LLMs have direct introspective access to their internal states, which in turn undermines the evidence that they’re phenomenally conscious.
Gurnee, W., Sofroniew, N., Pearce, A., et al. (2026). “Verbalizable Representations Form a Global Workspace in Language Models”. Transformer Circuits Thread.
Lederman, H., and Mahowald, K. (2026). “Emergent Introspection in AI is Content-Agnostic”. arXiv:2603.05414.
Lindsey, J. (2025). “Emergent Introspective Awareness in Large Language Models”. Transformer Circuits Thread.
Macar, U., Yang, L., Wang, A., Wallich, P., Ameisen, E., and Lindsey, J. (2026). “Mechanisms of Introspective Awareness”. arXiv:2603.21396. Code: safety-research/introspection-mechanisms.
Olah, C. (2026). “Remarks on Pope Leo XIV’s encyclical ‘Magnifica humanitas’”. Anthropic.
Pearson-Vogel, T., Vanek, M., Douglas, R., and Kulveit, J. (2026). “Latent Introspection: Models Can Detect Prior Concept Injections”. arXiv:2602.20031.
Model checkpoints:
Gemma 4 31B (4-bit) — mlx-community/gemma-4-31b-it-4bit
Qwen 3.6 MoE 35B (4-bit) — mlx-community/Qwen3.6-35B-A3B-4bit
Qwen 3.8 27B (4-bit) — mlx-community/Qwen3.8-27B-4bit
GLM-5.2 (FP8) — zai-org/GLM-5.2-FP8