Auditing the Wikipedia Android App for Accessibility
A reflection on testing what hadn’t been tested, and what I learned in the process.
Client: Wikimedia
The comment that stayed with me
Wikimedia requested our team to audit the Explore and Discover features in Wikipedia Android app. After weeks of testing, we presented our findings to the team, walked them through our personas, what we tested, what failed, and what we thought they should fix based on 3 tiers (Tier 1, Tier 2 & Tier 3).
When we opened it up for feedback, one of the things they kept was that they had never tested the Android app with a keyboard before. We had done that because of the persona I focused on, and the issues we found were clearly new to them. They said they were going to start including keyboard testing going forward.
All those weeks of testing and documenting, and the thing that actually stood out was not how many issues we found. It was that we tested something nobody at Wikimedia had thought to test before.
Here is how it all came together.
Why Wikipedia is the hard case
Wikipedia is one of those products that almost everyone has used at some point. There are a wide variety of audience, students, researchers, people fact checking something during dinner, people using a screen reader. It is everyone. Which means if accessibility breaks down anywhere in the app, it breaks down for a real person trying to learn something.
Our team was asked to audit the Wikipedia Android app for the Wikimedia client for their Explore and Discover experiences. That is everything a user does before they search for a specific article. The Explore home feed, Places, Saved articles and reading lists, History, and the Activity tab. Multiple screens, lots of interaction patterns, and a lot of edge cases that only surface when you start poking at them with assistive technology turned on.
Starting with people, not screens
Before we audited the app, we spent time researching. Our team split four ways across the major disability categories, hearing, vision, cognitive, and motor. I researched cognitive disability. We pulled from W3C WAI guidelines, Android accessibility documentation, WebAIM, and condition specific research on Usher Syndrome, Muscular Dystrophy, and Autism Spectrum Disorder. The goal was to understand how the disabilities interact when combined rather than looking at them in isolation. Real users rarely fit neatly into one category.
That research turned into two personas.
I focused on Ryan.
The setting that surprised me
A lot of apps work fine when you turn on one accessibility setting at a time. But start to fall apart when you turn on three.
We ran the whole audit on an Android simulator, with each of us turning on the actual stack of tools our persona would use. For Ryan that meant TalkBack and a Bluetooth keyboard and Voice Access at the same time. Stacking them revealed a lot of navigation issues within the app. A focus indicator that was technically present became invisible to Ryan. A tooltip that worked with one tool was unreachable with another. A gesture that was fine with touch became impossible without it.
This surfaced the keyboard problems we eventually showed Wikimedia. This made me reflect that single setting testing, the kind most teams follow, misses the real experience of real users.
Two methods
We used two methodologies side by side.
A cognitive walkthrough is a task based inspection where we walked through five feature flows from the persona’s point of view, asking the standard questions at each step. Will the user understand what things are? Will they be able to find what they need? Can they avoid or recover from mistakes? It is slow work, but it forces you to stop assuming.
A design evaluation is where we checked every screen against the relevant WCAG 2.1 and 2.2 success criteria at Level A and AA, alongside the Material 3 accessibility guidelines. We focused on the levels that legally and practically matter most. Our recommendations took both WCAG and M3 into account.
For Martha, the audit covered 19 WCAG criteria across hearing and vision categories. For Ryan, we worked through 19 criteria split across motor and cognitive, since his condition combination requires both lenses. I personally evaluated the 10 cognitive criteria, and contributed to the Cognitive WCAG and Material 3 evaluation focused on predictable structure, gesture alternatives, and accessible labels.
Get Shreesa Shrestha’s stories in your inbox
Join Medium for free to get updates from this writer.
Here is how the cognitive side broke down.
Six passed, four failed. Two of the four failures are Level A, which is the most fundamental conformance level. Cognitive accessibility is harder to measure compared to other disabilities. But every one of these failures showed up as a concrete, reproducible barrier when we walked through real tasks.
How we prioritized what we found
The issues and recommendations were organized into three tiers as follows
Having this framework made our recommendations feel actionable rather than just a list of complaints. It also helped us have a clearer conversation with the Wikimedia team about where to start.
The findings that stood out to me
A lot of small issues turned up in the audit. I want to call out a few that stood out to me.
The History feature lives in the wrong place
During auditing I realised how much literal language matters. The app has History inside the Search tab. If you are Ryan, and you want to find an article you read yesterday, your first instinct is to look in Activity. Because Activity is where things you did live. Search is for finding new things. The label and the content do not match, and there is no sign anywhere in the app telling you where History actually is.
To add to it, recently viewed articles are then split off into Activity, at the very bottom of a long scroll past reading stats, contributions, highlights, and a WikiGames card. So one type of history is in Search and another type of history is in Activity, and the user has to remember which is which.
This fails 3.2.3 Consistent Navigation. The fix is straightforward, put all history related content in one predictable place, ideally inside Activity, and use literal labels like “Reading History” and “Recently viewed or edited pages.”
Our recommendation was to consolidate all history related content into Activity, which is already the more intuitive home for it, and to use literal, descriptive labels like “Reading History” and “Recently viewed pages.” No metaphors, just words that say exactly what the thing is. Small change, big difference for someone like Ryan.
The screens missing title
When Ryan opens the Places screen, there is no title at the top telling him where he is. Same on Explore. TalkBack announces something internally, but a user looking at the screen sees only a search bar and a Map / List toggle. For someone who reads literally and benefits from predictable orientation cues, this is a low effort fix with a high impact.
This fails 2.4.2 Page Titled, a Level A requirement and M3 predictable structure. Adding a visible, descriptive page title at the top of each screen, that matches what the screen reader announces, gives every user a clear answer to “where am I right now?”
Our recommendation was straightforward. Add a visible, descriptive page title at the top of each screen that also matches what the screen reader announces. It is one of the lower effort fixes in the audit, and it answers one of the most basic questions a user can have when navigating an app: where am I right now?
Icon only controls with no labels
The filter and delete icons in History sit right next to each other in the top right corner. Neither has a visible text label. Neither has a tooltip on long press. Ryan, who reads labels literally to predict what will happen, cannot tell what either icon does until he taps one.
Across the rest of the app the three dot overflow menu rotates between vertical and horizontal versions, and uses different tooltip text in different places. “More” in some screens, “More options” in others, nothing at all on Explore. For someone whose comfort comes from consistency, that kind of small variation adds up fast. This fails both 3.3.2 Labels or Instructions and 3.2.4 Consistent Identification.
We recommended adding a persistent tooltip to every icon only control, and standardising both the icon and its accessible name across the entire app. The same action should always look the same and say the same thing, wherever it appears.
Why the keyboard mattered
Keyboard access is foundational. If a user cannot reach an element with Tab, they cannot reach it with Switch Access or Voice Access either, since both rely on the same underlying focus model. So when the Places map pins are not keyboard focusable, that is not just a problem for keyboard users. It is a problem for everyone who relies on any non-touch input method.
The Wikimedia team told us they planned to incorporate keyboard testing as a standard part of their future process. That commitment, more than any single recommendation in our report, was the outcome that made the work feel worth it. Not because we found a lot of issues, but because the way we tested surfaced a whole category of barriers that had been invisible to the people who build the app.
What I took away
- Stacking accessibility settings is not optional. Ryan would have TalkBack, an external keyboard, remove animations, and bold text on at the same time, and most of the failures we found only showed up under that combination. One setting at a time is not how people actually use these tools.
- Cognitive accessibility also deserves the same rigour as the more measurable categories. The gaps are not always obvious until you walk through real tasks with a literal reader in mind. A label that is technically present but inconsistent, an icon that is tappable but unpredictable. That is everyday friction, not an edge case.
- The last thing surprised me a little. The teams building these products usually want to do the right thing. The gap is not intent, it is testing coverage. And the best feedback we got from Wikimedia was not about our findings. It was that the way we tested was going to change how they tested next.
