From Anthropic’s recent paper, feature steering provides a way to refine how language models respond to prompts, enabling us to align their outputs with ethical guidelines, safety standards, and the specific needs of different applications. By adjusting key features within the model, this technique can help developers create more reliable, controlled AI behavior.

Implementing feature steering at scale comes with significant challenges. Identifying monosemantic features is complex and resource-intensive, and ensuring consistency across diverse scenarios remains a work in progress. Additionally, steering models in ways that align with diverse human values involves nuanced ethical decisions, as selecting which features to prioritize can impact fairness and inclusivity. Balancing these technical and ethical considerations will require ongoing refinement and collaboration across the AI community.

The possibilities are promising, and these advancements offer an exciting glimpse into a future where we can more confidently steer AI systems toward safe, ethical, and user-centered behavior. In this blog, I’ll explore the fundamentals of feature steering, its significance, and how it can help us build AI systems we can trust.

What is Feature Steering?

Feature steering is a method used to guide and adjust language model behavior by influencing specific “features” within the model’s internal structure. In traditional machine learning (ML), tuning and adjustments are common practices. ML models are tuned by modifying hyperparameters, such as learning rate or regularization, to optimize performance and generalization. Techniques like feature engineering and regularization have long been used to help models focus on relevant aspects of data, reduce bias, and ensure consistent outputs.

Applying this concept to large language models (LLMs) is where feature steering breaks new ground. Unlike ML models, which rely on structured data and clear metrics, LLMs generate responses in natural language, which requires a more nuanced approach to evaluate performance and control behavior. Feature steering in LLMs involves identifying specific, interpretable features — such as empathy, assertiveness, or caution — within the model and allowing developers to directly adjust these traits to align with desired outcomes.

Using techniques like dictionary learning, which finds interpretable patterns in the model’s internal state, developers can amplify or dampen certain traits in LLM responses. This approach can allow for consistent, controlled, and safe responses across various applications, aligning the model’s behavior with ethical guidelines, user expectations, and specific use-case requirements. Feature steering thus brings the established practice of model tuning into the realm of language models, enabling a deeper level of control and alignment in complex, real-world interactions.

I like to think of models as grand pianos with hundreds of millions of keys that can be adjusted. Feature steering is like tuning this piano to subtly adjust sound to fit different musical genres. Just as tuning doesn’t add or remove keys but refines the sound, feature steering fine-tunes a model’s behavior without altering its structure. This results in responses that can be consistently controlled and aligned with desired qualities, much like a well-tuned instrument producing music!

Breaking Down Barriers to LLM Adoption: Transparency Through Monosemanticity

One of the biggest barriers today for large scale adoption of large language models (LLMs) is the lack of transparency — users and developers alike are often unsure of what’s happening “behind the scenes.” This uncertainty can lead to fears about models generating dangerous, biased, or simply “bad” responses, especially in high-stakes applications like healthcare or finance.

The recent findings on monosemanticity offer a fascinating step toward demystifying these black-box models. Monosemanticity refers to features within the model that correlate strongly with single, distinct concepts, allowing developers to isolate and control specific behaviors with more predictability. For example, a feature might represent the ‘Golden Gate Bridge’ or a ‘specific tone of empathy’. When a model has monosemantic features, we can begin to understand which elements are influencing its responses, making behavior less opaque and more predictable.

If we can even get a peek at these internal workings — seeing what features or patterns the model activates to generate responses — we’re in a stronger position to build confidence in these systems. By understanding which elements drive responses, we can begin to adjust models to meet specific requirements and align with ethical standards. This kind of transparency offers an essential layer of control and adaptability, giving developers and organizations more empowerment to trust and refine AI behavior for real-world applications. Monosemanticity is a promising step toward achieving more explainable and trustworthy AI.

I love this example from Anthropic where they experimented increasing activation for the Golden Gate Bridge feature:

Why Does Feature Steering Matter?

As language models become more capable and integrated into our lives, the need for control over their responses grows. Here’s why feature steering can be an essential tool in building reliable, safe AI:

• Safety and Control: Feature steering can enable developers to guide models in scenarios where safety is paramount. For instance, in healthcare, an AI assistant providing mental health support can be steered to avoid risky or discouraging language, helping ensure that responses remain supportive and safe.
• Consistent Behavior: Users expect models to provide consistent information. Feature steering can help maintain uniform responses about policy-related queries, enhancing trust by minimizing variability in answers.
• Enhanced User Trust: Trustworthy AI is more likely to gain user confidence, especially when it consistently delivers safe and predictable responses. For example, in financial advising, feature steering can ensure the model’s advice aligns with best practices, promoting stable, balanced responses even to potentially risky prompts about investments.

Dream AI Products: What I’d Build with Monosemantic Features and Feature Steering (If There Were No Limits)

If I had no limits on resources or featuring steering limitations (there is still a long way to go), here are 2 groundbreaking product ideas that could transform the way we interact with language models, leveraging the power of monosemantic features and feature steering for maximum control, safety, and personalization:

Product Idea 1: Your Personalized LLM Companion

Overview: Create a personal language model that adapts to reflect your unique beliefs, demeanor, and interests. Imagine an AI that not only understands your perspectives but also tailors its responses to resonate with your tone, values, interests and personality. This model will learn from your interactions, preferences, and feedback, creating a digital companion that aligns with who you are and how you communicate.

Key Features:

1. Belief and Value Alignment: Through a combination of feature steering and monosemantic features, your AI aligns with your core beliefs and values. Whether you have a particular stance on environmental issues, personal ethics, or professional priorities, your AI adapts its responses to reflect those perspectives, providing recommendations and insights that feel authentic and true to you.
2. Adaptive Demeanor Matching: Your AI can mirror your tone and style, from professional and concise to friendly and humorous. Through interactions, it learns when you prefer a serious tone versus a more lighthearted approach, making its responses feel natural and tailored to your personality.
3. Interest-Based Knowledge Bank: Your AI keeps up with your hobbies, professional goals, and favorite topics, constantly refining its knowledge bank to stay current on your specific interests. Whether it’s following the latest trends in your industry, helping you plan a vacation around your favorite activities, or sharing articles on niche hobbies, Your AI curates content and recommendations that truly matter to you.
4. Evolving with Feedback: Users can provide feedback directly or use an Elo-style rating system (more on Elo below), helping your AI refine its understanding of your preferences. This feedback loop allows your AI to continuously improve its tone, depth, and accuracy in reflecting your beliefs and interests.
5. Privacy-Centric Design: Your AI should be built to prioritize user privacy. All data stays within the user’s device or a secure, encrypted environment, ensuring that personal information and preferences are safeguarded and not accessible to third parties.
6. “Situational Mode” Switching: Your AI should shift its behavior based on the situation. For example, in “Professional Mode,” it might adopt a more formal and concise tone, while “Social Mode” could bring out a more casual, conversational style. These modes make the AI a versatile companion that adapts seamlessly across personal and professional contexts.

Example Use Cases:

• Daily Assistant: Start your day with your AI’s curated recommendations on topics you care about — whether it’s a morning news briefing, insights related to your field, or motivational content aligned with your values.
• Decision-Making Support: When making decisions, from financial choices to travel plans, your AI considers your unique perspectives and offers tailored suggestions. For example, if sustainability is important to you, it may recommend eco-friendly brands or low-impact travel options.
• Continuous Learning and Growth: Your AI can act as a study buddy or knowledge partner, adapting its recommendations to help you grow in areas of interest or self-improvement. For instance, it might help you practice language skills, provide resources on personal development, or engage in reflective conversations.

This personalized AI goes beyond generic language models, building a personalized relationship that feels like a genuine extension of your own mind. By steering its responses to reflect your beliefs, tone, and passions, it becomes a trusted, adaptable companion that grows with you, delivering interactions that feel uniquely “you” in every way.

Product Idea 2: Interactive AI Explainability Dashboard — Real-Time Transparency and Control

Overview: The Interactive AI Explainability Dashboard is a tool designed to give users unprecedented insight into and control over AI-generated responses. By displaying which monosemantic features are being activated in real time, this dashboard allows developers and users to understand the “why” behind each AI response and adjust the influence of these features on the fly. This transparency helps users tailor AI behavior to fit specific contexts, ensuring that outputs align perfectly with their requirements.

Key Features:

1. Real-Time Feature Visualization: As the AI generates responses, users can see a live visualization of activated monosemantic features. These could include attributes like “formality,” “American values,” “assertiveness,” or “humor,” represented as sliders or dynamic bars, providing an intuitive overview of the AI’s internal workings.
2. Direct Feature Adjustment Controls: Users can fine-tune responses by adjusting the intensity of specific features directly within the dashboard. For instance, dialing up “clarity” and reducing “formality” for a technical explanation can make responses more accessible without compromising accuracy. These controls allow for instant, customized adjustments suited to diverse needs.
3. Context-Specific Presets: For common applications or user preferences, the dashboard offers preset configurations, such as “Professional,” “Concise,” or “Empathetic.” These presets automatically adjust feature intensities to align with specific tones or requirements, saving time and offering optimized responses for specific contexts, such as legal writing, academic settings, or customer support.
4. Elo Scoring Feedback Loop: To ensure continuous improvement, the dashboard incorporates an Elo scoring system where users rate the AI’s responses based on their needs and preferences. This feedback loop helps refine how feature steering influences outputs, making future interactions more aligned with user expectations.
5. Data Privacy and Security: Since the dashboard is designed for applications that may involve sensitive information, it includes rigorous privacy features. All adjustments and data remain within a secure, encrypted environment, with options for users to store response preferences locally, ensuring privacy is never compromised.
6. Application Modes for Diverse Use Cases Example:

• Educational Mode: Ideal for students and educators, this mode emphasizes clarity, accessibility, and comprehensiveness in explanations. It’s perfect for guiding students through complex topics in a way that matches their learning style.
• High-Stakes Mode: For finance, healthcare, or legal applications, this mode emphasizes accuracy, caution, and ethical considerations. It ensures that responses are aligned with professional standards and mitigates risks in critical environments.
• Creative Mode: For writers, marketers, or creatives, this mode can emphasize features like “originality” or “expressiveness,” allowing for freer, more imaginative responses.

By giving users transparency into the AI’s inner workings and allowing for direct, real-time adjustments, this dashboard transforms the way people interact with AI. It brings explainability and control to the forefront, empowering users to make informed adjustments that ensure responses are accurate, aligned, and contextually appropriate.

Evaluating Feature Steering: Moving Beyond Multiple-Choice with Elo Scoring

Anthropic’s current approach to evaluate language model accuracy and feature steering relies on static, multiple-choice evaluations. While multiple-choice assessments provide a straightforward way to gauge accuracy on simple, fact-based questions, they have limitations, especially when applied to complex, open-ended questions that require reasoning or nuanced judgment. Multiple-choice evaluations may inadvertently favor responses that align with the most common answers, rather than the most nuanced or accurate ones.

Elo scoring offers an alternative by incorporating dynamic, human-driven evaluation, similar to ranking systems used in competitive games. Here’s how it works:

• Human Feedback: Instead of pre-set answers, human evaluators rank responses based on factors like accuracy, tone, and relevance, providing nuanced assessments that go beyond right-or-wrong choices.
• Continuous Improvement: Just as players’ Elo scores shift based on game outcomes, the model’s Elo score adjusts based on how well it performs relative to previous responses. This enables a more iterative, continuous improvement process.
• Handling Ambiguity: Elo scoring allows for more adaptable evaluation, especially valuable for ambiguous questions that don’t have clear-cut answers. This approach helps capture the model’s overall reasoning quality, aligning it better with real-world use cases.

By incorporating Elo scoring, developers can achieve a more realistic and robust understanding of how well feature steering performs under various scenarios, making it a more dynamic and insightful alternative to static multiple-choice evaluations.

The Future of Feature Steering: Toward Controllable and Explainable AI

Feature steering has the potential to transform how we control and interpret language models. With feature steering, we’re not just guiding AI behavior superficially — we’re tuning it from the inside, adjusting the internal features that drive responses. This brings us closer to a future of controllable AI where responses can be consistently tailored to meet safety, ethical, and user-specific standards.

As we continue to refine feature steering and incorporate dynamic evaluation methods like Elo scoring, we edge closer to explainable AI systems that users can trust. By combining the precision of feature steering with meaningful evaluation methods, we’re moving toward a new standard in AI — one that values transparency, reliability, and adaptability.

In the end, building trustworthy AI isn’t just about accuracy; it’s about aligning AI with human expectations and ethical guidelines in a controlled, transparent way. With feature steering and continuous evaluation, we can make AI systems that don’t just generate responses but do so in ways we understand, trust, and rely on.

Citations:

@online{durmus2024steering,
author = {Esin Durmus and Alex Tamkin and Jack Clark and Jerry Wei and Jonathan Marcus and Joshua Batson and Kunal Handa and Liane Lovitt and Meg Tong and Miles McCain and Oliver Rausch and Saffron Huang and Sam Bowman and Stuart Ritchie and Tom Henighan and Deep Ganguli},
title = {Evaluating Feature Steering: A Case Study in Mitigating Social Biases},
date = {2024–10–25},
year = {2024},
url = {https://anthropic.com/research/evaluating-feature-steering},
}

Elo rating system - Wikipedia

https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html#assessing-tour-influence

In recent works, Yuval Noah Harari critically examines how our information networks shape human understanding. He argues that humanity’s issue isn’t so much with our nature, but with the networks through which we receive information — networks that, over the centuries, have pursued power more than truth or wisdom.

The Power and Flaws of Information Networks

From historical examples of flawed, misleading, or outright false information, Harari suggests that much of what we call “information” has little to do with reality. Instead, it often convinces us of erroneous beliefs through misinformation and disinformation or by simply ignoring crucial truths. This reality casts doubt on the effectiveness of any “information revolution” and implies that to truly elevate our species, a fundamental change in how we process and validate information would be required — a radical enlightenment unlike anything humanity has ever experienced.

Harari’s reflections on historical information networks resonate deeply with the challenges we face in AI today. Our current networks — now digital and AI-driven — carry the same risks of misrepresentation and misinformation.

A Futurist Thought Experiment: Fact vs. Universal Agreement

Harari’s ideas raise a fundamental question: how do we separate fact from belief? Imagine a future where technology allows us to poll every person on Earth instantly. We ask a question like, “Is it healthier to eat three meals a day or to eat smaller meals throughout the day?” and 85% of respondents say “three meals a day.” Does this mean it’s a fact?

The answer isn’t straightforward. Studies offer varied results, with some suggesting that three structured meals support metabolism and satiety, while others find benefits in more frequent, smaller meals for managing blood sugar levels or hunger. Individual responses might depend on cultural practices, personal habits, or health trends, but none of these necessarily makes one approach universally “healthier.”

This thought experiment highlights the difference between fact and universal agreement. Facts are grounded in verifiable evidence, while universal agreement reflects collective belief, which can often diverge from scientific reality. In an advanced information ecosystem, we’ll need to ask ourselves: Is this truly factual, or simply a widely held belief?

As Harari suggests, true progress may require a radical shift in how we define, verify, and value information — one that distinguishes objective truth from popular opinion, even in a world where consensus can seem like truth itself.

Humanity’s Ongoing Challenge with Truth

Humanity has grappled with these issues for centuries, but the challenge today is amplified. Now, we’re increasingly relying on language models (LMs) to interpret and deliver what we consider “truth.” This reliance on AI highlights how crucial it is to address the limitations of these systems, as their accuracy is directly tied to the quality of the data they consume.

This argument resonates deeply in today’s AI-driven landscape, where language models, like any human networks before them, are only as accurate as the information they consume. Garbage in, garbage out. The hopeful notion that AI might elevate human understanding comes across as a long shot, given the historical precedent of humans misusing and misinterpreting information. Indeed, Harari’s point is that human history itself is a cautionary tale about over-trusting information, a reminder that information, as he states, “has no essential link to truth.”

The challenge, then, is clear: if we, as humans, have struggled for centuries to discern truth from falsehood, how can we expect AI models to do any better? Frameworks like SimpleQA aim to sift fact from fiction. But it’s important to remember that these are tools trying to correct a fundamentally flawed system. AI may give us powerful tools, but it cannot solve our age-old human propensity to be misled.

The Social Media Effect: Truth vs. Viral Belief

From both my personal and professional experiences, I’ve come to realize that truth is not always clear-cut. There’s no shame in grappling with complex realities; the real problem arises when these uncertainties are presented as universal truths. Social media has amplified this challenge, with false information spreading rapidly and often being accepted as fact by wide audiences. In a landscape where misinformation can be shared by millions with a single click, the line between fact and fiction blurs, making it even harder to discern reality from misleading claims.

In today’s digital age, we often see content spread online as though it were indisputable fact. But if we don’t know the references or sources of these so-called facts, how reliable are they? I find it reassuring to see companies that prioritize accuracy and integrity in their AI models. Such efforts can empower users to make informed decisions rather than being misled by unverified information.

SimpleQA: A Framework for Evaluating AI Accuracy

One promising approach in tackling factuality challenges is OpenAI’s SimpleQA framework, which serves as a benchmark tool to evaluate language models’ (LMs) performance on straightforward, fact-based questions. SimpleQA doesn’t alter the model’s responses; rather, it measures accuracy by focusing on questions with single, indisputable answers. This allows developers to assess reliability without introducing the complexity of subjective or evolving answers.

Advantages of SimpleQA

• Clear Accuracy Measurement: SimpleQA’s focus on fact-based questions enables precise grading, highlighting the model’s accuracy and revealing areas needing improvement.
• Reliable Baseline for Performance: By providing a baseline, SimpleQA helps developers understand model performance on unambiguous questions, offering a foundation for further testing on complex scenarios.
• Potential to Expand Evaluation Scope: As SimpleQA evolves, it could adapt to evaluate more complex, nuanced queries, enhancing insights into how well models handle real-world, varied questions.
• Confidence Calibration: The framework aids in calibrating the model’s confidence, helping developers tune the model’s self-assessment of certainty, which can improve transparency in responses.

Limitations of SimpleQA

• Limited Evaluation Scope: SimpleQA’s focus on straightforward questions means it may not capture the model’s ability to handle multifaceted or interpretive queries, which are common in real-world applications.
• Scalability to Nuanced Questions: Evaluating more complex questions would require refined grading techniques, as the clear-cut scoring used in SimpleQA might not work for subjective or evolving queries.
• Risk of Overfitting to Simple Queries: Optimizing for simple questions may lead developers to prioritize model performance in these areas, potentially overlooking how well it performs on more intricate, less defined questions.

Generating True Labels in AI: Real-World Examples

To achieve accuracy in AI, establishing reliable “true labels” across diverse fields is essential. For example, fraud detection depends on investigators to label cases as “fraudulent” or “not fraudulent,” medical imaging uses experts to identify conditions like tumors, and sentiment analysis relies on annotators to classify emotions in text. These labels serve as ground truth, forming the basis for model accuracy in each domain.

However, not all questions have a single, definitive answer. Take, for instance, the question, “Is it ethical to eat meat?” This isn’t a matter of simple labeling, as responses vary widely based on ethical, cultural, and environmental perspectives. Such questions highlight the challenge of defining “truth” when even experts or users may disagree.

In cases where human consensus is elusive, creating accurate labels becomes difficult. This underscores that not all queries lend themselves to straightforward labeling, which is why frameworks like SimpleQA are most effective on clear-cut questions. More complex, subjective queries introduce ambiguity that AI (and humans) still struggles to navigate, making it crucial to carefully select and understand the data we use to train models. Even the best-labeled data has limitations when applied to nuanced, multi-dimensional questions.

The Future of Factuality in AI: A Balanced View

Potential Benefits: As frameworks like SimpleQA evolve, they can guide us toward AI systems that are transparent, reliable, and grounded in verifiable information. This could lead to a future where consumers interact confidently with AI, assured that the information they receive is accurate and trustworthy, making AI a positive force for informed decision-making.

Potential Risks: On the other hand, the focus on simple answers may encourage over-reliance on AI for quick, surface-level solutions, reducing users’ motivation to question or investigate further. If AI’s responses are perceived as universally factual without room for complexity, we may lose some of the critical thinking needed to understand the deeper truths behind the data.

Wild AI Products I’d Build with No Limitations

If resources and limitations were no barrier, there are some ambitious products I’d love to bring to life. Each of these ideas could address fundamental challenges in AI, information reliability, and user trust:

1. Social Media Fact Checker: Imagine a real-time, AI-powered fact-checker embedded into social media platforms that flags and provides context for potentially misleading information as it’s posted. This tool would cross-reference content against verified sources and alert users to questionable claims, helping curb misinformation before it spreads.
2. Human Credibility Score: This tool would assess the credibility of users based on their history of sharing accurate or misleading information online. Rather than punishing individuals, the credibility score could guide users toward more reliable information sources and highlight patterns that encourage responsible sharing. Users would see a score indicating their reliability, fostering a culture of accountability and informed discourse.
3. 100% Accurate Synthetic Data Generator: This would be a sophisticated tool to create synthetic data that’s indistinguishable from real data in terms of accuracy and detail, free from any privacy or ethical concerns. This could revolutionize fields like medical research, finance, and social sciences, where data access is often limited, providing robust data without compromising privacy or ethical standards.
4. Small Language Models (SLMs) that Prioritize Reasoning and Golden Data: Imagine a suite of compact language models designed specifically to prioritize reasoning, accuracy, and integrity. These models would be trained exclusively on “golden data” — verified, high-quality information — and built to deliver thoughtful, reasoned responses rather than simply mirroring the most popular answer. This would create AI that values precision and logic, designed for applications requiring a high degree of accuracy and critical thinking.
5. Selective Data Training (Cherry-Picked Data): Instead of training models on vast, indiscriminate datasets, this tool would carefully cherry-pick the most reliable, high-quality data to ensure accuracy and fairness. By focusing only on verified, high-integrity sources, we could train models that minimize biases and maximize factual reliability in their responses.

These products could transform the landscape of AI and information integrity, empowering users to engage with technology that’s not only accurate but also responsible, ethical, and transparent. While they may seem ambitious, each represents a vision for AI systems that prioritize truth and trust at their core.

The Next Steps: Beyond Accuracy

Ensuring AI responses are accurate is only the beginning. More challenges exist in making AI fair, ethical, and unbiased. Accuracy alone doesn’t guarantee equity or freedom from harmful biases. As we move forward, developers and data scientists must focus on building AI systems that uphold these standards, striving for technology that serves humanity responsibly.

The pursuit of fair, ethical AI requires ongoing commitment from both creators and users to question, verify, and prioritize integrity in technology. Are we ready to take on that challenge?

In the end, as AI grows and adapts, the responsibility lies with both developers and users to approach information with a discerning eye, balancing convenience with the pursuit of deeper, multifaceted truths.

References: Measuring short-form factuality in large language models