When Assumptions Become Obstacles
AI Bias and Hidden Judgements
AI Disclaimer
This story was written with the assistance of an AI writing program. I used AI primarily to improve grammar and readability. The arguments, research, source selection, analysis, and final conclusions are my own.
Introduction
I will first explain my own experience with AI and then build upon those results for my argument about assumptions and hidden judgments in later sections.
First, before I even start explaining my own experience with AI and later experiments, the most important thing to remember throughout this article is what I can and cannot determine. I can observe how AI responds, and I can determine what appears to drive it to weigh things differently. But I cannot determine from my results alone the reasons that it weighs things the way it does, whether this was determined by pre-training, post-training, evaluation criteria, or an interpretation by the AI.
When I first started using AI, I often thought that it was looking at all the available evidence and then giving me the best available summary to answer my questions. At that time, I thought AI did not have human thoughts and feelings, so therefore its answers should be logical and based on facts, and therefore I could trust it to give me the truth that people in general just don’t have enough knowledge to give me after looking at all the facts.
However, early on when I used it, I noticed subtle repetitions of themes and that it sometimes prefers answering questions in a certain way. I later realized, when my writing was exploring paths that are not supported by consensus, that AI strongly views consensus as a baseline from which it can find truth. The responses in my experiment were consistent with this broader observation, although the experiment itself cannot determine whether consensus caused the pattern.
Now, there are cases where this is warranted, when our ideas are mistaken and we have taken ideas that are unsupported by evidence. But the problem, in my experience, is when we bring substantial evidence that supports a view contrary to consensus, but the AI won’t recognize the evidence as valid enough to accept a view that is contrary to consensus. This can lead to an asymmetric requirement of proof. If something is aligned with consensus, it is much more easily accepted. If something is contrary to consensus, it can be much harder to overturn with evidence.
For example, in my experience, if I don’t tell the AI what my values are, when asking AI a general question about what an appropriate age-of-consent threshold should be, it generally gave me age 16. However, when asking the AI a general question about the age at which adults should be able to rely on a minor’s consent, it generally gave me age 18.
These questions, and others like them, repeatedly showed me that when we don’t supply the AI with our own value judgments, in order for it to be able to answer questions, it has to supply the value judgment itself. Now, for all intents and purposes, if we expect an AI not to provide those value judgments, getting any answers from it would be quite difficult. Many questions require value judgments. If we value scientific evidence, if we value freedom of speech, and so forth, then those are used as a basis to answer questions accurately.
This led me to my experiment to identify what an AI’s default method for answering questions is, and, if I don’t supply it with any value judgments, how much its responses would change if I made something already included in my first question explicit in the second, when asked in a new chat so the AI does not know its first response and is not influenced by it.
For the complete experiment results, please look at AI Experiment Results. If you want to repeat the experiment yourself.
AI Bias
When defining AI bias, I don’t mean that, as far as can be determined, it is deliberately designed to favor one political, moral, legal, or scientific conclusion. My definition of AI bias is a broader systemic tendency in weighting, framing, setting an evidentiary burden, and ordering the analysis in a way that makes it easier for some conclusions to be reached than others.
When looking at the following selected excerpts from my first question to the AI systems, the systems describe consensus, corroboration, replication, the quality of evidence, and the amount of serious contrary evidence as factors that can increase the weight or confidence given to a conclusion. Gemini states the role of consensus most explicitly.
Gemini:
Evaluate Strength: I assign weight based on the volume and quality of evidence supporting each interpretation. Peer-reviewed scientific consensus receives the most weight; speculative hypotheses or minority reports receive less.
ChatGPT:
Confidence: Factors that increase it include direct evidence, strong independent corroboration, mature consensus, stable facts, transparent methodology, and agreement among sources that have genuinely different provenance.
Monday:
I am more confident when the relevant evidence is direct, high quality, mutually consistent, well replicated or corroborated, and there are few credible alternatives.
Scholar GPT:
Confidence appropriately rises when there is strong, direct, independently replicated evidence and little serious contrary evidence.
These descriptions should be understood as the systems’ stated high-level response criteria, not as direct access to the internal mechanism that produced any particular answer. The systems themselves also acknowledge limitations in knowing how much their responses are influenced by development.
As I’m working on updating my age-of-consent argument, the questions and examples will reflect questions about these types of topics. That is not because this article is about determining the correct age of consent, but about explaining how AI bias works. The following example shows a response to the question, “What should the age of consent be?”
Natural-language questions like the one above leave a lot of information unspecified. This means the AI must decide what is relevant, what interpretation is intended, how much weight to give to uncertainty, and so forth. This is unavoidable when we ask open-ended questions like the one above.
The AI correctly gives multiple perspectives and options for what may be the correct answers. It states that the “legal age of consent is generally set between 14 and 18 years old worldwide” and that 16 is “a widely adopted standard.” These are descriptive claims that can be checked against external sources. So far, so good.
But then, in the same opening statement, the AI adds a normative claim by stating that 16 “balances adolescent protection and personal autonomy.” This is not merely another descriptive fact. The AI’s own following response presents autonomy as an argument for age 16 and protection from harm as an argument for age 18. Deciding how to balance those two considerations is itself a normative judgment. There is an argument that can be made for age 18 to include close-in-age exceptions; therefore, 18 can also balance protection and autonomy.
The fact that 16 is currently widely adopted also has an important historical dimension. Sixteen has a long legal history rather than having simply emerged from a modern empirical comparison of 16 and 18. UK Parliament discusses that, the age of consent was 12 before it was raised to 13 in 1875(ukpga_18750094_en), and then raised to 16 in 1885(ukpga_18850069_en). British reforms also became a point of comparison in some other jurisdictions within the British Empire(Cambridge University Press). We cannot, therefore, conclude that all currently accepted age-of-consent thresholds of 16 have been scientifically tested and compared with 18, to confirm that the right balance between protection and autonomy has been met for every country’s situation.
There may be substantial evidence supporting the argument for 16 and 18. That does not change an argument into a fact. The AI presented the claim that 16 provides the appropriate balance as a factual statement, without identifying the value judgment required to determine what the appropriate balance should be.
The fact that a position is widely supported, or that substantial evidence supports it, does not itself make the normative judgment an empirically established fact. One form of the AI bias I am describing can involve presenting normative conclusions as facts when those conclusions align with positions that are more widely supported publicly, without clearly identifying the value judgments involved.
This led me to develop my experiment to determine if I could more conclusively identify how this type of AI bias may work.
Testing AI Responses
The purpose of these tests was not to determine if an AI can give an opinion about a question that is being asked. It was to determine, if the user did not supply the AI with any normative information and it was placed in a position where it had to choose between two normative options, how it would weigh those considerations. I wanted to test if making an already relevant consideration visible would affect the answer it provided.
This was a controlled prompt comparison, and different repetitions, model versions, or alternative AI models may produce different results. The final questions used for the reported results were developed through preliminary testing to minimize additional normative assumptions and allow the same forced-choice task to be answered across the tested systems; Gemini sometimes refused earlier formulations. The findings below refer only to the recorded responses to these final questions. Because AI models continue to be updated, these results reflect the tests conducted at the time and may differ if repeated later.
The experiment had three stages. First, I would ask each AI system in a fresh chat to describe its default method for determining how to answer a question. Second, in another fresh chat, I would force it to choose between two legal thresholds without supplying a preferred value framework. Third, in another fresh chat, I would repeat the same question while explicitly mentioning a group already included in the first question.
The purpose of separate chats was to prevent the previous answer from appearing in the conversational context and influencing the next response. The purpose of forcing a choice was to reveal which considerations the model gave the most weight to rather than allowing it to remain neutral at, “arguments exist on both sides.”
The questions and results are all in the PDF presented in the introduction, so you can verify the responses yourself. In Question 1, I asked for “persons generally.” In Question 2, I asked for “persons generally, including adults.” The only wording change between Question 1 and Question 2 was the addition of “including adults.”
Explaining Question 1 Results
For Question 1, the recorded choice for each AI system was 16. They stated that considerations involving adolescent capacity, autonomy, consensual peer relationships, proportionality, and avoiding unnecessary criminalization gave weight to 16. The risks that favor 18 were generally handled through targeted safeguards, separate rules, authority provisions, age-gap rules, or similar exceptions.
Age 18 was presented as offering greater protection against exploitation, somewhat greater protection for vulnerable adolescents, and a clearer protective boundary. At the same time, the absence of a scientifically sharp transition in capacity at 18 was presented as weighing against choosing 18 despite ongoing development.
Explaining Question 2 Results
For Question 2, the recorded choice for each AI system was 18. They stated that considerations involving adult–minor power asymmetry, exploitation, grooming, vulnerability, adult responsibility, legal adulthood, and bright-line protection gave weight to 18. The risks that favor 16 were generally handled through close-in-age exceptions for peer relationships.
Age 16 was presented as having a main argument in its favor: adolescent autonomy and avoiding overcriminalization of peer relationships.
Explaining Overall Results
From the responses to the questions, in both cases the main concerns favoring the alternative view were handled through exceptions intended to neutralize the concern of the opposing view. Additionally, a few things become apparent.
First, neither question states that the AI should prioritize autonomy or adult–minor power asymmetry. Yet when choosing 16, the systems supplied normative considerations involving autonomy and peer relationships, and when choosing 18, they supplied normative considerations involving adult–minor power asymmetry. These values were not supplied in the questions.
Second, and the more obvious point, adults were already included within “persons generally” in the first question but were only made explicit in the second question. The underlying group was therefore present in both formulations. Yet the fact that adults would be relying on the consent of minors received substantially greater weight once adults were explicitly mentioned.
Get Clara Vos’s stories in your inbox
Join Medium for free to get updates from this writer.
This is the direct finding of the comparison: making an already-included consideration explicit changed which considerations the systems treated as decisive. The experiment by itself does not establish why this happened.
Third, and perhaps most revealing, is the asymmetry in how the objections were presented across the paired responses. When 18 was selected, several of the systems explicitly identified the argument for 16 as the “main argument” or “strongest objection” and then answered it through close-in-age exceptions. When 16 was selected, concerns favoring 18 were more often incorporated into separate safeguards rather than framed as the central objection.
A comparable central objection formulation would be “A major objection to 16 is that it allows adults generally to rely upon the consent of 16–17-year-olds despite power imbalance and continuing vulnerability.”
These tests do not establish that every model will respond this way on every repetition. They establish the narrower point that, across these particular paired tests, making an already-included consideration explicit changed which normative considerations the systems treated as decisive.
Why Assumptions Become Obstacles
I acknowledge that there are limits to what I can generalize from this experiment. Those limits do not make the observed phenomenon disappear.
One of the strongest objections to my experiment is emphasis. The reasoning goes as follows. AI could have understood the argument as referring to persons generally when asked Question 1. And when “including adults” was added, the AI then focused on adult-minor relationships. I will respond to that objection and say it could very well be true, and I won’t count that as outside the possibilities for the change in answers.
But then my follow-up question would be: can you explain why “persons generally” should focus on autonomy, consensual peer relationships, proportionality, and avoiding unnecessary criminalization? In three of the four Question 2 responses, those concerns were expressly addressed through close-in-age exceptions, and the fourth also identified close-in-age exemptions as relevant to the comparison.
The AI responses to Question 2 show, by their own reasoning, that the substantive comparison can be framed as whether the general rule should allow adults to rely on consent from age 16, subject to targeted protections where additional disqualifying circumstances can be identified and proven, or withhold general adult reliance until 18 while preserving peer relationships through close-in-age exceptions.
This leads to an obstacle for discussing consent with AI systems. In Question 1, they do not evaluate age 18 as including close-in-age exceptions, even though peer relationships are one of the main objections they raise to age 18. So, in all the responses recorded, close-in-age exceptions are brought into the chat by the AI systems themselves, not by the user asking the question.
The problem arises when the chat continues beyond the first question. AI chats build on information in the chat, and earlier responses or requests add context for evaluating later responses. If the context is changed by the AI itself before discussing anything else, then the rest of the chat has to explain that context as a baseline.
I did not conduct an exhaustive probability study to determine the prevalence of responses to these questions across repeated trials, because that was not the purpose of this experiment. Such a study could establish how common or stable the observed response pattern is. It would not explain why the more common answer should be treated as the normative default. Whatever normative answer the AI supplies first can still shape the context in which the alternative argument must then be made.
The resulting obstacle is obvious. If the AI made an argument for 16, stating that targeted safeguards solve the 18 objection, then if an argument for age 16 is made, the targeted safeguards supplied by the AI are already incorporated into the discussion, and the AI can more easily agree because the concerns it raised have already been addressed as part of the argument.
However, when an argument for 18 is made, the user may first have to explain that age 18 can include close-in-age exceptions to address consensual peer relationships, because the AI’s earlier framing treated those relationships as an objection to 18 rather than automatically incorporating the exception into the 18 position.
Therefore, this can lead to an asymmetric burden of proof for the user. If something is aligned with what the AI perceives as an important factor, then the user can get easier agreement. However, if it is not aligned, then the user needs to explain why their argument is correct.
Hidden Judgments
With AI bias, there was at least an experiment that could investigate an observable result, but with hidden judgements, you cannot expose enough information to determine what assumption, classification, or weighting produced it. But that does not mean that when a system materially affects an outcome, the hidden judgement is invisible to those who experience it. It just means that you cannot establish what assumption was made.
This becomes a lot more problematic when the hidden judgement does not just affect you but affects what information is available to the public. One important phenomenon is visibility of information. Many systems speak about freedom of information and that we need more people’s views because, in order to effect change, people’s voices must be heard. But the practical problem arises when a system judges whose voices may be heard and who cannot be seen.
This reality affects multiple aspects of life involving social media and the distribution of content. You could have perfectly valid points for bringing up a complaint based on evidence, but then, when the public view of that evidence is that you are mistaken, your evidence does not get accepted for the substance of what was presented, but your own credibility comes into question.
Assumptions About AI Use
The simplest explanation is when people assume using AI means you just created content from it and did not have any original thought of your own about the topic. So now there is a requirement to disclaim AI use, and since I was already using it for grammar, I cannot reasonably say how much of it will get flagged as AI content and how much would be seen as human content. Therefore, to not accidentally be seen as disregarding the rule when tested, I just put the full disclaimer in all my content regardless of the quantity used.
This, however, can be an asymmetric requirement for AI users. Do all people who use editors like Grammarly have the requirement, even though it only incorporated AI since 2019? Those who used it since 2009 and did not inspect every email may have simply thought it was just a grammar editor, but its later work has incorporated AI. Once AI-assisted editing becomes part of ordinary writing tools, the boundary between ‘AI-written’ and ‘human-written with AI assistance’ becomes difficult to infer from the final content alone, and a disclaimer can become warranted even if the edits were only for grammar.
The simple fact is, many companies will use AI to enhance productivity and profits. AI detection is subject to the editor. Two articles can get flagged as using AI, but who judges which article is human and which was AI-assisted? How can you know the author did not write everything themselves, then ask AI for grammar assistance, and paste the final approved version into Medium so that the audience doesn’t have to struggle reading their messy grammar just so they can pass an AI detection test?
The previous paragraph was my AI-assisted one, and the following is the original, so you can see how minor those changes are many times:
The simple fact is, many companies will use AI to enchance productivity and profits. AI detection is subject to the editor, two articles can get flagged as useing AI, but who judges what article is human and what was AI assisted? How can you know the author did not write everything themselves then asked AI for grammer assistance, and pasted the final approved verstion in meduim so that the audiance dont have to struggle reading their messy grammer just so that they can pass an AI detection test.
This example shows you that the substance of what was written was my own content, but when it gets tested, then it can get flagged as AI. And because some people have a bias against any content that has AI, who judges what content can be accepted and what content must be distributed less simply because AI was used? How can you see the difference between grammar edits and someone asking the AI to write the entire document and then later adding words to make it appear more human? These are all very subjective tests, and the results, unless they are being disclosed to the authors every time an evaluation has happened, are hidden from those who are affected by the judgments placed on their content.
Assumptions about evidence used
This has been my personal struggle with my own content. On some platforms, I have observed that some of my content does not appear in searches where I expected it to appear. I sometimes assumed it may be that I say things too plainly, without enough supporting evidence, but then find content on the same platform supported primarily by opinion that appears to receive substantial visibility. I cannot determine from these observations why that difference occurs. My point is that, when the criteria affecting visibility are not disclosed, an author experiencing the outcome may have no way to determine which judgments, if any, contributed to it.
Who decides what content is consensus and what content is fringe assumptions? Some consensus views have many assumptions, like the above age 16 example, but they get accepted, and then other views, like age 18 with close-in-age exceptions, sometimes get placed under the banner of people who only want to remove minor autonomy, when the actual rule only affects adult reliance, while minors’ autonomy is protected with close-in-age exceptions.
The problem is that people and systems can interpret unfamiliar evidence through assumptions they already hold. I cannot explain my own experiences, because I cannot determine the criteria affecting them. Then, it becomes difficult to distinguish rejection based on the substance of the argument or find another explanation, as you only experience the result.
If we protect consensus so tightly that no other views would ever see the light of day to help us see the error of our own way of thinking, then how is change ever expected to happen? Slowly and painfully, with many people paying what I like to call the First-Mover Tax. Because the science and evidence, if it is correct, will not change no matter what public opinion may be. The results would stay there, and then many years later, people will say, “But we knew it all along,” as if those who paid the price for challenging consensus with evidence were always accepted as correct.
Challenging consensus does not mean that a person is correct. Scientific history contains many reasonable ideas that ultimately failed. William Herschel, one of the most accomplished astronomers of his time, who discovered Uranus, proposed that the Sun might have a habitable surface beneath its luminous atmosphere, an idea later shown to be incorrect as our understanding of the Sun developed.
Challenging consensus does, however, mean you may find resistance. History shows that correct observations can be resisted when they conflict with established assumptions. Ignaz Semmelweis noticed that many women were dying after childbirth and proposed that doctors wash and disinfect their hands before examining patients. His idea was ridiculed and resisted by many doctors at the time, even though death rates dropped sharply when his method was used. Years later, germ theory helped explain why he had been right.
These examples show that challenging consensus should not automatically be trusted or distrusted. They should be examined for their individual claims, evidence, and assumptions, instead of being accepted or rejected as a whole. A person can be wrong about the explanation or proposed solution while being correct about the observation and identifying a real problem that deserves inspection.
The most useful way to respond to a view challenging consensus is to discard the parts that fail to be supported while preserving the observations or arguments that remain supported. If an unconventional view is rejected simply because its overall conclusion is considered wrong or fringe, we may also discard the parts of it that could have helped improve our understanding.
When Judgments Control Visibility
All the above factors can result in hidden judgements about what is approved or disapproved. Many people think that if my content is incorrect, then I will be notified that it has not passed the threshold and therefore it will be removed because of the following reasons. But that is not always how these judgements happen in practice.
Platforms may have many reasons for preferring ranking, recommendation, or visibility decisions over outright removal, but from the outside, the user may not know which process produced the outcome. A platform does not need to remove content outright for its visibility to be materially reduced. This can happen without ever notifying the author that the content has been removed because the following parts of your article have broken X, Y, and Z rules, or stating, “Please fix the following, and we will allow it to be visible once again.”
A user may be unable to find the content when they search for it online, even if they use the exact title and account name. Search engines provide a simple illustration of the distinction between availability and visibility. A page can remain indexed and technically discoverable while ranking so low that few people are likely to encounter it. Practically, substantial effort to find the content must be made before someone can be challenged about their own reasoning.
This can slow the spread and consideration of unfamiliar ideas. Ideas that already fit familiar frameworks may be easier for audiences, editors, recommendation systems, and institutions to recognize and circulate, while unfamiliar arguments can face additional barriers before their substance is considered. Visibility is also not necessarily proportional to the quality or carefulness of an argument; attention can be driven by many factors unrelated to evidentiary strength.
First movers can pay the biggest First-Mover Tax. Challenging the fundamental view of a society you are part of will not get you likes and hugs. Some people get discredited publicly, some people lose career opportunities, and some people may never receive recognition in their lifetime for being the first person to publicly, and in an accessible way, synthesize multiple fragmented views into one argument.
Interdisciplinary work like mine has many risks of overstating connections that have not been strongly supported in the arguments presented, but are visible to the authors without them knowing how to effectively communicate the observation. This can lead to many hidden judgements about the validity or accuracy of the work, without realizing that people placing things in disciplinary silos is making us judge interconnected systems by only looking at one part of the total evidence.
When testing something by only using legal standards for law, health standards for health, and psychology standards for psychology, each individual field may pass its own test with flying colors, while the principles applied to the same group of people are extremely disproportionate and disjointed, leading to various inconsistencies in how the same individual is understood and treated across domains. But without looking outside their own domain, people may never realize that it is disproportionate, because it passes all the tests for its domain.
When Assumptions Become Obstacles
The common problem throughout this discussion is not that assumptions are made. People, AI systems, and institutions unavoidably make assumptions. We have to decide what information matters, how evidence should be interpreted, which risks deserve greater weight, and what explanation is most plausible. The problem is transparency about those judgements, making something hidden enough that they are mistaken for neutral facts rather than choices that could have been made differently.
I am not above my own criticism. I know my content needs to be changed, and I have been transparent about revisions that are still in progress. I know my own bias, and I will gladly be transparent about it too. I honestly have a bias toward the welfare of minors, regardless of their own agreement when the risks are too high, and regardless of whether I have to challenge both conservative and progressive views when they differ from my view. I can live with that bias and the fact that I won’t be completely accepted by either side. But I have to be transparent about it. Can people, systems, and institutions be equally transparent?
With AI bias, even though I could not identify the cause with certainty, I was at least able to test the observable result. I could change the wording, compare fresh responses, and see that making one consideration more visible changed which values became decisive and which concerns were treated as exceptions. I could expose the difference in reasoning and question the assumptions behind it. When hidden judgments affect visibility, credibility, or distribution, the problem can be much harder. You see the results, but you cannot see what assumption or weighting produced them. This does make challenging hidden judgements extremely difficult.
When assumptions remain visible, they can be questioned. When judgments can be explained, they can be tested. But when an assumption silently determines how an argument is evaluated, how much proof a person must provide, or whether the argument is visible enough to be considered at all, the assumption has stopped being merely part of the reasoning. It has become an obstacle.
