Geometric Concept Representations in Language Models: A 3D Hue-Saturation Manifold in Gemma 2 9B and Manifold Steering

Introduction

Language models hold their internal representations of words and context as high-dimensional activation vectors. These long lists of numbers, uninterpretable as is to humans, hold interesting information about model computations and thoughts. A foundational question in mechanistic interpretability is how language models organize this information inside their hidden states. The dominant working hypothesis, the Linear Representation Hypothesis (LRH), says that high-level concepts correspond to directions in the activation space. For example, “Truthful” is a direction, “French” is a direction, and “sadness” is a direction. Adding the direction with a positive scale steers the model toward the concept, while subtracting it steers it away. This is the framework beind steering vectors Turner et al. 2023, representation engineering Zou et al. 2023, and much of recent alignment-adjacent interpretability work.

However, the LRH has been brought into question. Recently, Engels et al. 2024 (”Not All Language Model Features Are One-Dimensionally Linear”) showed that some language model features are inherently multi-dimensional, living as 2D circles in activation space, not as single directions. Their examples were the seven days of the week, the twelve months of the year, and the years of the 20th century, each of which were arranged as a closed loop in some 2D subspace of GPT-2 small’s activation space, with the shape of the loop showing the cyclic structure of the underlying concept. Then, through causal intervention, they showed that these circles aren’t just correlated features the model happens to show in PCA, but are used by the model’s downstream computation while answering questions like “what day is two days after Monday?”

That finding suggests the LRH is the wrong model for understanding the internal representations of concepts in LLMs. But the paper only showed concrete examples of cyclic structures for specifically cyclic calendar concepts, where the geometric shape is somewhat obvious from the concept’s structure. A follow-up question that this project begins to address is: do non-calendar concepts also demonstrate irreducible multi-dimensional geometry in LM residual streams?

This post documents the investigation of one specific instance: color in Gemma 2 9B. Color was chosen partly because vision-language and image-generation models are known to encode it geometrically (but whether text-only LLMs do is not proven) and partly because a clean causal-intervention test exists: successful steering of the hue subspace at inference time should change the color word the model says.

Motivation

A few things make geometric (vs. purely linear) concept representations interesting for interpretability and alignment:

1. The LRH may be incomplete. If some concepts are inherently encoded as 2D structures, or higher-dimensional (spheres, tori, helices, etc.), then steering them by adding a single direction is fundamentally wrong, and would move the model off the concept’s manifold rather than steer to rotate around it. The correct intervention is to project onto the manifold’s coordinate system and reposition there. This matters for any safety or alignment technique that relies on steering vectors, since if the target concept isn’t a direction, those techniques have an underdiagnosed failure mode.

2. Manifold structure is informative about underlying computation. A circle of colors suggests the model has internalized the cyclic nature of colors from textual co-occurrence patterns. Red and violet appear near each other in language despite being opposing ends of the visible spectrum, because human descriptions of color reflect the wraparound. Additional higher-dimensional structure could additionally suggest the model has internalized structure like saturation. Understanding these structures from a text-only LLM tells us about the structure that emerges from pure text training, without any visual input.

3. The methodology generalizes. Color is used as a test case here, rather than the point. If the method of controlled pipeline—generated dataset → residual stream extraction → parameterized geometric probe → causal intervention through manifold steering works for color, it works for any concept where contrastive prompts that alter the concept along its geometric axes can be created. Compass directions, emotional valence, geographic location, hour-of-day-times-day-of-week, time-of-day-across-years, and many others are candidate non-trivial geometric concepts the same pipeline can find and steer along.

Background

The Linear Representation Hypothesis Park et al. 2023: A concept is a direction v ∈ R^d in a model’s residual stream. The presence of the concept in a context can be measured by x · v (dot product) and intervened on by adding or subtracting αv from the activation. The framework has been remarkably successful, explaining much of what’s been mechanistically interpreted in transformer-based LMs, including features like sentiment, refusal, instruction-following, and many world-model features (e.g., Golden Gate Bridge Claude). Steering vectors are the concrete application of the LRH.

Engels et al.’s circular features: By applying spectral clustering to the decoder vectors of a sparse autoencoder (SAE) trained on the residual streams for each analyzed model, and projecting each cluster’s reconstructions to its top PCA components, Engels et al. found that for the concepts of days, months, and years, the cluster reconstructions form 2D circles rather than single directions. They then showed that these circles are causally used by Mistral-7B and Llama-3 8B for modular arithmetic (e.g., “what day is five days after Monday?”). Replacing the model’s projection onto the days-circle subspace with a different point on the circle changes the answer to the corresponding new day, and the effect is comparable in magnitude to replacing the entire activation (an upper-bound control). They also fit a parameterized circular probe of the form probe(θ) = m + r·(cos(θ)·a + sin(θ)·b) to demonstrate this steering. They prove that at least some LM concepts are inherently multi-dimensional, and treating them as directions is insufficient.

Methodology

The project’s pipeline is similar to Engels et al.’s but extends it to a non-calendar concept whose geometric structure is predicted (color should form a circle).

1. Prompted concept search
Using a constructed prompt set of 12 single-token hue names spanning 0°-330° of the color wheel in 30° increments (red, orange, yellow, chartreuse, green, teal, cyan, blue, indigo, purple, magenta, pink), 3 achromatic anchor colors (black/white/gray), natural saturation/value combinations from (pale, vivid, muted, soft, deep, dark, light, bright, plus default), 5 neutral templates ("The color is X", "color: X", a X color, the shade X, X). For each prompt, Gemma 2 9B residual stream activations are extracted at layers 16, 20, 24, 28, 32, 36, 40, at both the hue-word token position and the prompt’s final token position.

3. Circular probe fitting
For each (layer, position, normalization) configuration, two circular probes are fit: one with all chromatic prompts and their predicted hue angles, and a centroid fit averaging within each hue first and fitting the circle to the 12 per-hue means. The centroid fit removes intra-hue variance (saturation, value, template, etc.).

4. Causal intervention
With a fitted probe (m, a, b, r), a forward hook is registered at the chosen layer that decomposes the activation at the last-token position into ring-component + orthogonal-residual, replaces the ring component with scale · r · (cos(θ_target)·a + sin(θ_target)·b), and patches back. The model’s next-token logits are compared to the unsteered baseline, and θ_target is swept across 24 angles at 15° spacing for the full circle for 12 test prompts (mixture of strong-color-prior and weak-color-prior cases).

Findings

Finding 1: GPT-2 small does not encode color as a hue circle

When the prompted color dataset is run on GPT-2 small (12 hues, sat/val combos, 5 templates), PCA of the residual stream at layers 5, 7, 9, 11 shows context-based clustering (warm hues grouped, cool hues grouped). No hue circle is seen or measured in any layer/configuration. Text-only training at GPT-2 small’s scale (~124M parameters) is not sufficient enough to induce color-wheel-like geometry in the residual stream.

Finding 2: Gemma 2 9B does encode hue as a 2D cyclic manifold, with the structure sharpening through layers

On Gemma 2 9B, the 12 hue centroids form a 2D circle in the correct cyclic order (red → orange → yellow → green → cyan → blue → purple → magenta → red) at every examined layer ≥ 24, with the ring having a cleaner arrangement as the layers progress. The circular probe’s centroid R² peaks at ~0.36 at layer 36 (hue-position, unit-normalized). Per-prompt R² is lower (~0.17) because hue is one variance source, among saturation, value, template, and others, which all contribute orthogonal variance that the circular model doesn’t capture. Reporting both values together helps both the geometric claim that the hue centroids form a circle, as well as the claim that hue accounts for ~17% of the per-prompt residual stream variance at layer 36. Achromatic anchors (black/white/gray) project mainly to the center of the ring, as projections from the value axis onto the 2D circle of hues (see Finding 4).

Finding 3: The hue manifold is causally consumed for next-token color prediction

With a probe refit at layer 36 on raw last-position activations (r ≈ 67.4) and an intervention scale of 4 times the natural radius, steering effects are measured across 12 test prompts:

• Weak-prior prompts ("The painter chose a bright", "Her dress was a vivid"): The top-1 next-token flips to the targeted color. Steering to θ=120° (green) produces ' green' (p≈0.15-0.17) as the top-1 token, while the baseline’s structural tokens (' shade', ',') are demoted. Steering to other angles produces the corresponding colors in top-1.

• Moderate-prior prompts ("The color of bananas is", "The color of fresh blood is"): The “max color” rotates around the wheel as the target rotates: yellow at θ∈[0°,120°], green at θ=150°, blue at θ∈[180°,240°], red at θ∈[300°,330°]. The cyclic order matches the wheel, demonstrating the rotation is causal.

• Strong-prior prompts ("The color of the fire engine is", "The color of grass is"): The top-1 token doesn’t flip as the prompt’s prior is too strong (baseline P(red) ≈ 0.50 for fire engines). However, the log-probabilities of the non-baseline colors shift dramatically in the direction of the target (e.g., log P(”green”) increases by ~7 nats as θ moves from 0° to 120° for the fire-engine prompt). The hue subspace is used by future layers even when not strong enough to override top-1 token.

• The intervention is selective: Top-8 next-token distributions remain coherent with respect to syntax across all steering angles, and only the color-conditional component changes. The model also generalizes to semantically-adjacent vocabulary. For example, steering to green at θ=120° in "Her dress was a vivid" increases the probability of not just ' green' but ' turquoise' and ' emerald' into the top-8. This is consistent with the probe having fit a hue subspace rather than a token-specific artifact.

Finding 4: All three HSV dimensions are geometrically encoded as near-orthogonal separable axes

The 2D hue ring is only part of the color representation. Saturation and value were also tested to see if they were encoded by the model (HSV’s other two dimensions).

The value-only / saturation-only experiment
A separate prompt set was constructed varying only value ("a very dark color", "a brilliant color") or only saturation ("a pure gray color", "a vivid color"), with no hue specified, ordinal-labeled. PCA of these activations without the hues shows both dimensions cleanly. A saturation axis correlates with the saturation ordinal at a max |r| of 0.83 (layer 40, ~9% of variance), and a value axis at a max |r| of 0.62 (layer 40, ~5% of variance). The structures of both are refined through the layers, just as the hue ring. Saturation and value are geometrically encoded, but the original dataset could not isolate them from the dominant hue signal.

The existence of a value axis also explains the appearance of achromatic anchors (primarily black and white), near the center of the 2D circle of hues. As an orthogonal dimension to hue, values such as "a very dark color" and "a very bright color" extend away from the plane the hues lie on, but average to the center of this circle given all of the hues, so the projection shows them sprinkled near the middle.

Finding 5: Causal validation of the saturation axis

Causal use extends beyond hue to the other HSV dimensions. A further experiment onvolved intervening on the saturation axis at layer 36 with an additive push (x + α·c_sat, where c_sat is the saturation axis from the saturation-only prompts) and measuring the mean log-probability gap between high-saturation words ("vivid", "saturated", "intense", "rich", "bold", "pure") and low-saturation words ("pale", "muted", "faded", "dull", "pastel", "subtle") in the next-token distribution.

Across six test prompts, the high-low log-probability gap increases with the steering scale, with no exceptions. Averaged over the prompts, the gap moves from −13.0 nats at scale −6 (favors low-saturation words heavily), through −2.5 nats at scale 0 (baseline), to +7.0 nats at scale +6 (favors high-saturation words heavily), which is a ~20-nat controllable swing. For prompts where saturation modifiers are plausible next tokens, the intervention causes saturation words in the top predictions. For example, steering "The walls were painted a" toward low saturation elevates "dull" and "pale" into the top tokens, and toward high saturation elevates "bright" and "vibrant". Steering "She wore a dress in a" toward low saturation increased the probabilities of "muted" and "pale". The intervention still, even at the extreme ±6 scales, produces grammatically correct continuations rather than incoherent output.

Discussion

Concepts and Layers - Representations
The hue ring is represented at all examined Gemma 2 9B layers ≥ 24, but it sharpens through the late layers. For at least one tested prompt, layer 36 supported strong top-1 flipping, but not all layers were swept for the best steering layer (earlier layers often steer more cleanly because the rest of the network has more time to compute with the change). This is consistent with the broader idea in mechanistic interpretability that different concepts live in different layers and the “right layer” is context dependent. This is also important for safety auditing, where, for example, the layer at which a model may represent deception and whether it acts on deception, and if so, where, are two separate measurements, requiring different techniques.

Steering adds to prior distributions
For strong-prior prompts ("The color of grass is"), steering shifts the probabilities of the color distribution but doesn’t flip the top-1 token, and “green” still tops out because the model’s text-pattern association is too strong. Mechanistic steering is one factor among many influencing the next-token distribution. For weak-prior prompts ("The painter chose a bright"), where multiple colors are plausible and the model is uncertain, steering does flip the top-1 to the target color. The situations where the model has not already committed strongly to an answer are the best applications for steering. When it has committed, the distribution can be shifted, but the prior dominates. However, the increase in non-top-1 color log-probabilities is still consistent and helpful for interpretability claims even when the top-1 doesn’t change. Use of steering as a safety tool requires measuring the prio-strength dependence on the steered distribution for each concept, despite seeming like a suitable safety measure.

Concepts entangle naturally in composition
Initially, this project almost missed the saturation and value axes, when the first search used a factorial dataset, where every prompt was a complete color phrase("vivid dark red"). It found nothing, and it was concluded that the dimensions weren’t encoded as their own dimensions, despite being strongly encoded (saturation r ≈ 0.83). This dataset failed because the in a compositional phrase, the hue word dominated the residual stream, and this phrase is not cleanly represented as a linear sum of hue + saturation + value, which are only recoverable when searched for in isolation. This was an important point in entanglement, that the structures of various concepts can be invisible in naturalistic data as composition is entangled. This is something that needs to be studied further, as decompositional datasets are the most effective probes, but can also induce survivorship biases and are not optimal compute- and time-wise.

Future Directions

Generalization to torus, helix, and higher dimensional manifold candidates
The methodology should extend to higher-dimensional geometric features. Hour-of-day × day-of-week is predicted to be a torus, while calendar dates spanning years are predicted to be a helix. These require slightly extended parameterized probes, but no new fundamental archtecture. However, a new method might need to be developed to machematically categorize them as any sort of shape or feature.

Test the shape of safety-related concepts and separately audit representation and consumption
If concepts like refusal, harmfulness, deception, etc. have manifold structure, steering vector-based safety interventions may break/have hidden failure modes and adding a direction may move the activation off the manifold. Using a 1D probe and a 2D+ parameterized probe and comparing explained variance, these safety concepts could be intervenced on properly. In addition, the cleanest representation layer is not the best causal consumption layer, and measuring the differences in these can yield not just better interventions and safety, but also show something about the way models represent internal states and how they compute later states.

Cross-model universality of geometric features
Test whether the hue ring and other 2D+ features appear in various models, and measure their scales. Geometry that is consistent across models shows that the structure is an effect of the language rather than a training-specific artifact, and would help bound how far feature and steering vector transfer between models can succeed.