You have just graduated and are now on the hunt for entry-level jobs. In the process, you stumble upon an internship opportunity. What would you do? Would you take the internship?

I was in the exact position not long ago — in two minds on whether or not to apply for internships. I took a leap of faith, applied and this experience was just the perfect eye opener for me. In retrospect, this is one of the best decisions I have ever made and will always make without thinking twice. This is why.

I had the opportunity to intern as a User Experience Designer at Wikimedia Foundation for 9 months. Not knowing what to expect, I walked in with an open mind. Everything felt right from the start. I even remember thinking to myself, “I am happy here. It really does feel like home”. True to their values, my experience at Wikimedia really was a depiction of working in an environment where one feels a sense of belonging ― a family.

Julieta Fernandez, my fellow intern and I, were warmly welcomed and introduced to the larger team. We even got the opportunity to meet every member at an individual level. What better way could there be to connect and learn more about your team members! My key take-aways from the fun and little chats with the individual team members were:

• Most people are nice: Team members are more than willing to support and watch you grow, don’t shy away from asking for help. Communicate, reach out as much as possible to reap the most out of the benefits of teamwork.
• It’s okay to be nervous: Give yourself an opportunity to push beyond your comfort zone, that’s how you grow and become better.

During the nine months, I supported 2 teams: Editing and Growth. I worked closely with Product Managers, Researchers and fellow Designers to improve the editing experience of Wikipedia. I worked alongside mentors who guided me all through the way, growing my knowledge.

While at Wikimedia, I worked on two major projects: The Personalized Praise project and the design of the Reply Tool New Comment Indicator. The weekly catch-up with my Mentor, Principal Designer and Product Manager helped align the design goals and expectations. During these check-ins I received feedback, and directions; making working on the projects seamless, enjoyable and fun! My manager was always there to support and ensure my well being throughout. This gave me a peace of mind, and a calm environment to work in.

Here’s a quick summary of the projects:

Project 1: The personalized praise design

For this project, the team asked: How might we encourage newcomers who have visited our homepage and tried our features to keep editing and build on their momentum?

To approach this problem, I used a user centered design thinking approach:

1. Empathize: Gathering high-quality user understanding

In this phase, I reviewed current plans, designs and user research materials. I also conducted further discussions with mentors, subject matter experts, and the Growth team to deeply understand user needs, barriers, attitudes and aspirations and the context of this feature’s existence.

2. Define: Formulating the challenge into a clear articulation

In this phase, I gathered all the insights I had collected from the empathize phase, and began to make sense of the landscape of solutions I was exploring by answering the following questions:

a) What do we want to do?

1. What is the ultimate goal/vision for this project?

Raise retention. Increase the number of newcomers in the Growth experiment group who have activated (made 1 edit) to become active contributors through revising existing and new positive reinforcement mechanisms that motivate them to keep editing.

2. What is the short term goal/need for this project?

Provide encouragement by improving the existing positive reinforcement mechanisms available on wiki.

b) What don’t we want to do?

1. Set up incentives that encourage reckless editing that prizes quantity over quality.
2. Disrupt or subvert community philosophies on how editors should be incentivized.
3. Burden or affect work on Scaling, Structured tasks, or the Mentor dashboard.
4. Build too much infrastructure or service that requires high effort.
5. Make changes that affect the analysis of the overall newcomer experiments
6. People feel unduly scrutinized for their contributions and feel as though their privacy is impacted.

3. Ideate and collaborate

In this phase, the key goal was to create an environment where divergent and provocative options are embraced and assessed, with the eventual goal of converging on a few strongest pathways to pursue.

This came to a reality with the help of the Community Ambassadors. Community Ambassadors are experienced Wikipedia editors that teach, mentor, and lead others, helping marginalized groups (such as speakers of a foreign language) to contribute to, and benefit from, the Wikimedia projects. They took a major role in helping us brainstorm ideas that would require minimum effort but are highly impactful.

After doing a S.W.O.T. (Strengths, weaknesses, opportunities, threats) assessment, we built confidence in the strategic viability of the pathway: Integrating a Send Appreciation module into the existing Mentor Dashboard.

4. Prototype: Transforming ideas into tangible “artifacts”

In this phase, proposed solutions were improved, redesigned and rejected through a series of reviews and critiques from the broader team.

This rapid iterative process allowed me to be imperfect and detach from my work in a healthy way by embracing collaboration by “piggybacking” ideas from different sources. What a fun experience!

The evolution of the send appreciation feature:- A dedicated module on the mentor dashboard that surfaces appreciation-worthy mentees.

Try it yourself :)

5. Test: Getting feedback from the community

In this phase, we were keen on asking open-ended and solution-focused questions to avoid answers that shut down iterative improvement. We aimed to encourage the experienced Wikipedia editors/mentors that would be using this feature to give personalized praise to new editors to build on the concept or solution in productive ways.

Our goal is to get fast and productive feedback from the real world, insert our new learning back into the process, and optimize the concept until we have solved the problem.

This project is currently under development :) Getting out there is what is next; yay!

Impact: What I have learnt

The rapid iterative process allowed me to be imperfect and detach from my work in a healthy way by embracing collaboration by “piggybacking” ideas from different sources.

Project 2: The Reply Tool New Comment Indicator

With a global community of contributors on Wikipedia, the Editing team was focused on making participating productively on talk pages, administration pages where editors can discuss improvements to articles or other Wikipedia pages easier and more intuitive for our Junior contributors, registered users who have made <100 cumulative contributions to that particular Wikipedia project and Senior contributors, registered users who have made >500 cumulative contributions to that particular Wikipedia project.

This new contributor workflow for replying to specific comments is intended to make participating productively on talk pages easier and more intuitive for Junior and Senior Contributors.

The Challenge, Phabricator task: T300560

The alert should be presented in such a way that people: Are aware of its presence, but are not distracted by it. Understand they have the option to engage with the alert, but do not feel obligated to.

User experience

The moment a qualifying change has been made, show an alert in/around the Reply Tool informing people that the page has changed from the version they are looking at.

The alert should allow people to:

• Show the new comment(s) that have been posted
• Ignore or dismiss the alert

Secondary research

In this phase, I did understand the context of the feature’s existence. This is where I also stopped and thought about the users I am designing for, gathering insights about their goal and key aspects.

Competitive audit

I audited a few social media platforms like Instragam, LinkedIn and Facebook. This provided direction and opportunities to address with the feature.

The Impact

A new experience is live for the new comment indicator that appears in the Reply Tool when someone publishes a comment while you are drafting a reply, improving the contributor workflow for replying to comments on talk pages, across Wikipedia’s 16 talk namespaces. Yayyy!

What I learnt

1. Finding the balance between the organization’s design system constraints and creativity, thinking outside the box!
2. The more familiar users are with something, the more they like it.

The highlight of my internship was being able to SIT BACK, OBSERVE, and LEARN. How do teams collaborate? How are problems approached and solved? How do team members relate with one another on a professional and personal level? I was able to attend many different team meetings such as research readouts, project kickoffs, and many more. I made sure to bring a notebook to every one of them cause oh, dear! You can never run out of fascinating concepts!

To sum it all up, I achieved immense personal growth. I also developed a diverse set of skills such as illustration, presentation, and giving and soliciting feedback in a team setting. My newfound interest in design systems and their component libraries was just the icing on the cake.

“Hey! You are here to learn, get hands-on experience and meet wonderful people that you can always look up to and get inspired by. Just make the best of it. We are here to support you, mentor you and watch you grow” those were the words I heard when I needed them the most and so I’ll pass them on to you. At the end of the day, that’s what internships are about.

A huge thank you to the entire Wikimedia Product Design, Editing and Growth teams for the opportunity, support, feedback and the unforgettable experience!

Originally published at https://design.wikimedia.org on March 30, 2023.

Wikipedia’s mission is to enable every single person on the planet free access to the sum of all human knowledge in the language of their choice.

But how does one go about finding what they are looking for on Wikipedia? Today most of our readers come across Wikipedia content through Google and other external search engines. But Wikipedia also has its own search that often goes unnoticed by readers. So what is the purpose of having search on Wikipedia when it is so easy to use external search engines to find content on Wikipedia? The Search Platform team has a summary of some reasons why search on Wikipedia is important. Privacy, safety, sharing and preserving knowledge, and caring about all language communities regardless of profitability, are some of the principles that govern search on Wikipedia.

It is not our goal to have Wikipedia replace popular search engines. However, it is important for search on-wiki to keep up with the key features that readers expect from search and serve those who are specifically looking for content on Wikipedia. After all, how difficult could it be to find things in the world’s largest encyclopedia? With the enormous amounts of great content added by our volunteers, not just on Wikipedia but all of its sister projects, improving the search and discovery of content is of vital importance.

Origins of the project

One of the initiatives to improve finding and discovery of content is the Structured Data Across Wikimedia program, which aims to structure existing Wikipedia content so that it is machine-recognizable. Making our content machine recognizable or in other words adding metadata to organize unstructured data & media, makes searching easier and content more discoverable.

The Search Improvements project, with the help of structured content, aims to provide a more inviting and more efficient way to search and find content on the Wikipedias.

This post is a reflection on how we arrived at the improvements we made for search.

Understanding reader behaviour

The design process began with research to better understand how people use search on Wikipedia. We reviewed the analytics, did UX analysis of current search, and ran qualitative interviews with readers. These research inquiries led us to general understanding on how readers are using our search and what might be blocking them from using its full potential.

We learnt that most of the research participants followed a general path to Wikipedia’s content.

1. Most participant’s usual route to knowledge on Wikipedia begins in Google.
2. Once the participant arrived on Wikipedia, a few would use the search bar on Wikipedia to initiate another search.
3. When an article match is found the participants were redirected to that article.
4. When the desired content (in this case, an article) was not found (or not served in the auto suggestions in the search box), participants abandoned their search on Wikipedia.

However, not finding the desired content on Wikipedia is often a discovery problem rather than a content problem.

What influences the current behavior?

The current search box does an excellent job of finding article title matches. e.g. if one types “Nicaragua” the article name will be suggested in the dropdown.

But when one searches for “rainfall in Nicaragua”, because there is no article with that title, there are no suggested articles displayed.

This is because article suggestions in the search bar are not the result of a robust search over the entire text of Wikipedia articles, but are instead a result of a quicker process of matching the search query with only the title of an article.

However, as the typical reader doesn’t know this, their usual reaction is that the information they are looking for does not exist on Wikipedia. And this is when readers are likely to abandon searching on Wikipedia.

Other ways to search on Wikipedia

In such cases, even though an article matching the reader’s search term does not exist, it is likely that the information the reader seeks is hidden but available under sections or articles with different names or in sister projects like Wikimedia commons or Wikivoyage and other sources within our ecosystem. However, those places are not easily discoverable for readers.

If the readers were to click on the link in the search box that says “Search for pages containing rainfall in Nicaragua” it would take them to the Search page, which performs a more thorough search, resulting in other articles that may contain the desired information. The discovery of this page was missed consistently by the participants due to it being a bit hidden away in the UI. And the function of this page was not obvious for those who were shown the page.

Search bar vs Search page

Can the aforementioned more thorough search be made available in the search bar? Certainly. But only to some extent so that we don’t bloat the lightweight search bar with too much information. Including this level of broader search in the search bar would also require significant technological infrastructure improvements to maintain performance and latency. And while we could work on making the search bar more responsive to such queries, that would require a huge effort in both time and resources in building what already exists on the search page.

However, we found that the search page was difficult to use for readers based on our initial research. Improving this page would both encourage more usage and highlight the already existing functionality. Which is why for this project, we decided to focus on addressing the problems on the search page.

What problems did we discover on the Search page?

Problem 1: Search UI reinforced mental models that prevented participants from exploring the search results.

Due to the learnt behaviours on the Search bar, participants had a strong mental model that if the article is not suggested in the drop down it was a failed search — even if the Search page would show a wide variety of relevant results for the same search query. The current messaging that appears at the top of the results (a highly salient location) indicating “article not found”, reinforced this mental model.

We found out that a lot of these messages were customized by their respective communities with the intention of being one of the main ways in which users were taught to start their article creation process. Altering this message would cause more disruption in the short to medium term, so we concentrated on other immediate opportunities for improvement instead.

Problem 2: Lack of proper visual design

There were several visual design issues identified on the search page:

• The page lacked visual content which made scanning and locating information on a text heavy page more difficult.
• The colors on the page were in-appropriately used and drew attention to less important things for e.g. green to highlight metadata, thereby taking the focus away from the content.
• The page column layout was not consistent with other pages on Wikipedia. The ultimate goal for this page would be to use our new design grid system, the benefits of which are explained in this blog post.

Problem 3: Lack of context for a given result.

The snippets that appear on the search page help readers discover content within the article’s sections. However, most of the snippets are partial sentences that do not provide users with enough context of the article’s information. To read more of the snippet, the user would have to click through to the article and find that snippet on the page manually, increasing the amount of time it would take to find the information they are looking for, and leading them away from other potentially relevant results.

Problem 4: Search results from other projects were either not discovered or misinterpreted.

Most participants did not notice or pay attention to the second column of the search results which shows results from other projects that are part of the Wikimedia ecosystem. Participants assumed them to be ads due to the similar location of ads on other search engines. Ads are, in fact, against the principles that govern WMF products, but readers were not aware or conscious of this fact enough during their search to overcome their initial assumptions. We found that due to the general unawareness of these other WMF projects, the value of seeing search results from these projects was not made clear with our current page design.

Fixing some of the above problems and creating new opportunities for discovering content would pave the way for future search enhancements.

Choosing an audience for search improvements

The Search page would especially be beneficial when an article title match is not found in the search bar. This may happen more often while searching for content in emerging language Wikipedias that often have fewer articles. Hence, we saw a great opportunity in improving this page for emerging language Wikipedias. A better broader search would allow searchers on smaller wikis to discover content within articles and in other Wikimedia projects, and not entirely rely on an article title match in the search bar.

By focusing on Portuguese, Russian and Indonesian Wikipedias, we could also build on the existing relationships between those communities and our team’s (Structured Data) community ambassadors in those wikis, allowing for better facilitations in receiving feedback from the community.

User Stories

By this point we had a fairly good idea of what problem and opportunities we were going to go after. Which brings me to our next steps in the process: drafting user stories.

User stories help articulate the user needs and allows us to have a better understanding of why users want a certain functionality. It also ensures that the team remains on the same page when ideating on solutions. It serves as not only a tool for understanding user needs but also a tool for better collaboration.

The following user stories were written to provide focus areas for the search page improvements.

User Story 1

When I am looking at search results on the Special:Search page

I want to be able to easily scan the page,

so that I can quickly locate the information I am looking for.

User Story 2

When I am looking at the results on the Special:Search page

I want to be able to see key information,

so that I can evaluate the relevance of various search results and find the information I was seeking..

User Story 3

When going through search on special:search page

I want to be able to see links to relevant content like linked galleries, wikiquotes etc,

so that I can discover and explore other Wikimedia projects for relevant content.

Proposed design improvements

After a series of team activities, design sessions, and feedback rounds we settled on the following improvements for the Search page.

Article thumbnails

To address the user needs presented in the user story #1, we introduced article thumbnail images in search results. They were added to serve as a visual navigational aid in an otherwise text heavy page. Thumbnails can also assist searchers in finding information quickly: e.g. if one only knew roughly how a yellow daisy looked, thumbnails can easily help that user locate the right name for the flower. Adding visuals proved to be useful to the research participants we showed these designs to.

Design considerations

• A few different explorations were discussed before we settled on a square sized thumbnail. One of our goals was to provide an efficient way to scan information and varying sizes of images would come in the way of this goal. Another design rationale in support of not showing the full size of the images was that these images were supportive of the content users searched and not the content itself. Since this was not an image search we did not think showing the full size of images was serving our core purpose.
• While we thought thumbnails were a good addition for readers, some editors found them to adversely affect their workflow so we added a specific preference to turn it off in the settings.
• We decided to not show thumbnails for mobile devices due to data and bandwidth considerations of a portable device.

We also used this opportunity to fix some of the other visual design problems mentioned earlier in the post. Since we felt that most readers would benefit from all these changes, we released this across all Wikipedias.

Search preview — Desktop

Search Preview was introduced to address the needs presented in user story #2 and #3. Its purpose was to surface key information pertaining to the result and provide necessary context, so that the user could evaluate the relevance of search results without leaving the results page. Previews also help the readers go directly to the section of an article that they are interested in, thereby reducing the amount of the time it would take the user to find information within articles. Search previews also serve as a space to surface relevant content from other Wikimedia projects for users to explore — such as related images from Wikimedia Commons. These designs were well received by readers in user tests and were reviewed with community members.

Design considerations

• Preview design was partly inspired by Wikimedia Commons search previews and Wikipedia page previews. We did not want the search preview to feel like a completely new element introduced to our system and so we tried to follow the existing patterns and visual look while only differing in places where we needed to differ.
• Several solutions for opening a preview were explored before we settled on something simpler where the visual affordance (arrow) for discovering previews would only appear upon hover and the users would be made aware of the feature through a first time tutorial. We tested this approach and once the user tried it, the interaction became intuitive very quickly.
• To decide what content should appear in previews we went back to our user story #2. The following key information would provide users with enough context to decide whether to further explore a particular result or not.

*Lead image of the article

*Expanded snippets

*Sections of the article

• Expanding snippets turned out to be more complicated than expected. A rudimentary solution was suggested here and several ideas were captured here for improving the expansion of snippets in the future.
• In order to address the user story #3 related to the discovery of relevant content in other Wikimedia projects, the following content was also added to search previews.

*Relevant images from Wikimedia Commons

*Links to other relevant pages in other Wikimedia projects

Due to the experimental nature of these solutions we decided to launch in Portuguese, Russian and Indonesian wikis after a few rounds of community consultation. We plan to learn from the usage of these features before launching it to other language wikis.

Search preview — Mobile

The process of defining what search preview on mobile would look like was more nuanced than just taking the desktop preview and designing it to fit the mobile screen.

Design considerations

If we were to take the desktop design as is and put it on mobile it would have resulted in a new page that the user would have to navigate to. This would defeat the purpose of having a quick glance at the contents of the article. If users have to go to a new page to see a preview they may as well go to the article itself.

We had to go back to the drawing board to see what would work for mobile. After several design explorations and feedback rounds, we settled on the following carousel design which when tested worked well with our users.

Challenges and learnings

• The challenge that the product teams face in an open source environment is that many things are customizable. In our project we found there were elements on search pages customized by the editor community for their specific workflows. So ensuring the newer better UI is adopted by all communities is not always guaranteed.
• While we want to design for casual readers, changes can be disruptive for existing editor workflows. The needs of both types of users need to be constantly balanced: e.g. thumbnails in search proved very useful for the readers while some editors found them distracting.
• Engaging with the community early in the process is important. However, sometimes not everyone will discover or notice the proposed changes until it is released. Extra time needs to be budgeted in the project to account for unforeseen feedback coming in after the feature is released.
• There can be different values and perspectives amongst the community — even when working on the same mission — as illustrated in this discussion of whether to allow non-free images as thumbnails or not. This demonstrates the diversity of our community and making constructive compromises is inherent to how we create products at the foundation.

Next steps

A few things we would like to do in the future to continue improving the search experience:

• Find ways in which the Search page can be made more discoverable. Now that we have improved the Search page a bit, we want more people to enjoy making use of its full power! In order to do this, we want to raise its visibility and awareness.
• Use learnings from the improvements on this page to improve search in the top of the search experience funnel: i.e. most searches begin with the search bar, and we believe there is also a lot of potential to improve this feature.
• Do research on the editor needs of search. Our data indicates that 20% of users end up on the Search page on English Wikipedia. However, we do not know if these are casual readers or editors. Since the research we did with readers suggested that they were unaware of the Search page, our assumption was that the vast number of people using this page might be editors. While for this project we focused our efforts on reader experience, as a next step we would like to understand how to improve search for editors.
• Figure out the high impact areas for search improvement. There is a lot we can do to improve search but we cannot do it all given the resources we have. For that reason it would be good to identify what changes will have the most positive impact on the experience and prioritize those.
• Continue to learn about reader behaviour and find opportunities for improvement. We have only started uncovering some of the basic problems with search but there are way more undiscovered opportunities out there.

If you are part of one of the wikis where these features are released ( Portuguese, Russian and Indonesian) I hope you will try it out and provide feedback on the community page.

Thanks to Rita Ho & Matthew Williams for their design mentorship throughout the project; Michael Raish for helping with initial phases of design research; Rita Ho, Carolyn Li-Madeo, & Mike Pham for thoughtful edits and feedback on this post; and not to mention the Structured Data team for building these features & the Search platform team for providing technical support.

Originally published at https://design.wikimedia.org on March 17, 2023.

Improving search on Wikipedia was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

With a design team of 26 people spread out through 5 continents, 14 countries, and 9 time zones, finding ways to make remote work effective is something we have worked hard to achieve through our meetings, interactions, and practices. But with 50% of our current team having never met each other in person due to starting during the Covid-19 pandemic, we knew there were certain collaborative and relationship pieces missing in our puzzle to create a welcoming and impactful team culture.

It’s a unique situation to spend hundreds, if not thousands of hours working closely with others and never spend time in person. But with most travel restrictions loosening in the countries where we all lived, we decided it was time to connect together as a team in person. With the high investment of time and resources to get us all into the same room, we decided to approach planning the offsite like any other design project to help ensure our time together would be well spent. Here was our approach.

Research: What did the team want out of an in-person offsite?

We knew that this offsite would be different than the past few, which were done remotely but we wanted to hear directly from the team. We sent out a short survey to collect opinions on these questions: What do you want to achieve by attending the team offsite, what concerns do you have about attending, and what type of theme should we explore together?

The responses about what we wanted to achieve at the offsite were unanimously centered around building stronger relationships with our colleagues:

“I’d love various opportunities to connect with people on the team and interact with them in ways I usually am not able to remotely”

“I hope we can feel even more connected with everyone on the team through inspiring conversations and learn more about and from my colleagues”.

Other key points we learned from the survey were that the team was interested in being playful and generative together to improve our collaboration as a team:

“I hope we don’t over plan in order to leave time for spontaneous moments together. Something as simple as playing games together could be an effective team-building exercise”

“I think it would be fun and engaging if we did activities that sparked meaningful conversations. I always like to explore things that I have in common (or not, and give room for discussion!”

A theme focused on systems thinking also began to emerge from the survey responses, which mapped closely to our team’s longer-term goals. It was also clear that we should approach this systems-thinking theme from a wide perspective to show the many different intersecting systems at play in our day-to-day lives and connect those to how they might influence our team and design system.

With a theme and direct input from the team on what they wanted to achieve, we were making progress and ready for the next step.

Exploration: Brainstorming to connect our goals and theme

We knew what the team wanted, but how do we get there? Our first step was to produce as many ideas as possible. Creating a range of ideas at the beginning of any project has always yielded a similar path of confusion > excitement > clarity for me, and this was the case even when designing an in-person experience. We brainstormed together as a group of design managers for a few weeks and made lists of possible presentations, activities, and games that could connect to our theme. We also made sure that each of us had dedicated time on our own to explore and refine our ideas to bring back to the group.

A twist: Hybrid meeting strategies

Every project has twists along the way and as we started to make progress and dive into the details we realized that a few of our team members weren’t going to be able to make it in person to the offsite (for various good reasons). We wanted to make sure that the experience felt natural to them and that they weren’t just “watching” others participate for a full workday. With a large range of time zones (14 hours!) separating the folks that couldn’t join in person, we came up with these approaches in an effort to include them in the offsite:

• Asynchronous activities related to what we were doing in-person
• Having a recording of each presentation that could be shared beforehand
• Recreating the location-specific activities to work in other locations, or digitally
• Creating a Slack channel with the folks joining remotely where they could collaborate together. We also made sure to have a couple of people who were attending in person to join to answer any questions or resolve any technical difficulties
• Setting up the schedule so that there was one 2-hour meeting per day that accommodated all time zones, where remote attendees would join the in-person folks by video call
• Multiple cameras and a microphone (including inexpensive solutions, such as using the mobile devices we had at hand) to make sure the remote participants could see and hear everyone

Planning: Bringing it all together

Once we felt like we knew how to support the folks joining remotely and had plenty of ideas, we began to group the ideas and create sub-themes for each day. The offsite was going to be 3 days, so we split the days among the managers on the team to begin the process of turning a rough outline into a detailed view of the day, including how our days would support those joining remotely. We also created one central Notion document where we put each day’s notes and agenda which helped connect the remote and in-person attendees as well as creating a central point for everyone to see what was happening.

Here are a few of the specific ways we connected our goals and theme:

Assigned “Coffee meet-ups”

One of our goals was for as many different individuals to have quality time with as many other people as possible. We did this by assigning everyone to a one-on-one meeting to start the day. Printed conversation starter cards and prompts that related to that day’s activities were given to allow folks to get to know each other while getting in the right mindset for the day’s events.

Day 1: Connecting physical systems with digital systems.

With introductions and icebreakers to kick off the offsite, we all attended an immersive digital art exhibition (at L’ateliers des Lumieres). (The remote attendees did a related activity online). We divided into smaller groups and were tasked with finding a collection of physical objects that were connected in some way to the immersive art exhibit they’d just seen. Each group shared their findings with the larger group and the remote attendants at the end of the day, resulting in a dynamic conversation of varying opinions about the experience we’d all had. We realized that the opportunity to challenge and disagree with each other in a safe environment outside of our direct work really connected us in a deeper way, and was an important skill to bring back to our day-to-day work. That balance of creating space to disagree without a “falling out” is a difficult but worthwhile goal for team building.

Day 2: Seeing systems and defining emergent systems around us

In order to investigate the various systems that are required for a city to operate we split up into smaller groups, each with a specific activity related to systems thinking in a city which included an architecture tour, a sewer museum, a transportation exhibit, and a visit to a city garden and park. The folks participating remotely also went on various field trips in their local areas to map to the in-person activities. We came back together after and rearranged the groups so that there was a representative from each of the previous groups to collaboratively draw a map of the ‘perfect’ city (our remote participants built their own map together using Figma).

In our team meeting, we focused on prompts such as: How can you apply what you learned from your system of focus to improve the quality of life of your designed city? In what ways can connecting systems together improve or detract from their impacts?

This activity allowed everyone to connect in new ways, see the interaction of systems, and understand how those principles could apply to our team and mission.

Day 3: Playing with systems

On the final day, we created another opportunity for small groups to use design thinking to create together. Everyone received the same set of supplies and was tasked with creating a game and its corresponding instructions within a certain time frame. The groups then took turns to play one another’s games and give feedback, allowing the creators an opportunity to iterate and improve. This activity gave everyone a chance to see a different, more playful side of their co-workers while in the act of creation. It also showed that even with all of the groups starting with the same component that systems don’t constrain our possibilities but allow us to explore in a more effective way, mirroring how we could approach our day-to-day work.

Reflection: How to measure the effectiveness of an offsite

Immediately following the conclusion of our offsite, the team Slack channel was filled with photos from the week, notes about how great it was to get to know each other, and ideas and plans for the next time we would be able to meet in person. This showcased how our relationship as a team had really evolved. We also saw the spirit of the “coffee meet-ups” continue as there was a team-initiated Slack channel that assigned folks to meet virtually in a casual manner.

Even with this overwhelming feeling of positivity, we wanted to give attendees a safe space to give feedback and suggestions, we did this by sending a survey where we asked 3 questions to be graded on a scale of 1 through 5 (e.g. I have connected with folks who I’ve never met in order to build healthy working relationships), along with 3 open-ended questions (e.g. Is there anything you think could be improved upon for next time?) and a space for any other comments.

“The offsite was a great opportunity to get to know my colleagues and spend quality time with them. I feel much more connected to this amazing team”.

With positive grades and responses, along with a list of possible ways to improve next time, it was a fitting conclusion to using design thinking to plan an effective offsite.

Originally published at https://design.wikimedia.org on February 17, 2023.

Applying design thinking to planning an offsite was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

When I first started working at the Wikimedia Foundation, I was surprised to discover that although our projects used some basic layouts, there was no underlying layout grid system. As a visual designer, who originally studied graphic design from a print perspective, this felt like an elementary part of our design toolkit was missing. Without a grid system, making decisions about where items should sit on the page, and maintaining visual consistency across multiple projects would be more challenging.

Maybe I shouldn’t have been surprised. Wikipedia launched in 2001 when the web, and certainly modern web design, was in its infancy. Since its launch there have been relatively few major design iterations. Indeed when I started in early 2021, the last release of a new ‘ skin’ — the template which changes the look and feel of a Wikipedia — was Vector (now called Vector Legacy) which was deployed in 2010. Back then, there were limited methods to apply layout grid systems on the web — both CSS Grid and Flexbox were still in the early stages of development and not well supported — so it is not surprising that a grid system was not implemented as part of this release. There were later attempts to introduce a grid system, which was implemented to our Content Translation project, but to retrofit a grid to already existing projects would have been a major undertaking, so that grid was not applied globally.

Luckily, when I joined the Foundation, one of my first projects was working on the visual design for the Desktop Improvements project. This would be our first major redesign of the web interface in twelve years and as such would be a rare opportunity to rethink how we approach layout across the wikis and, importantly, to develop a grid system.

What is a grid system and why do we need one?

A grid is a system for organising content within space — that could be a printed page, screen or the built environment. A grid creates visual rhythm that helps us navigate space and content. To appreciate the importance of grids think about how you read a newspaper or walk around a city. You might not be particularly conscious of the grid but without it your ability to consume the content or find your way around would be much more difficult.

In terms of modern user interface design a grid system is particularly important in allowing us to create flexible but consistent layouts across the huge variety of screen sizes that people may be using. By using a fluid grid we can divide the page width into several equally sized and spaced columns. Page content is then placed in alignment with these columns and when the viewport expands/shrinks horizontally, these fluid grid columns will expand proportionally.

A grid system is also a valuable tool for the design team because it will allow us make design decisions based on a documented structure that we are all using, which in turn will help us to design more efficiently and consistently.

Initial proposal

In my initial grid explorations as part of the Desktop Improvements project I quickly settled on using a 12-column grid for larger screen widths. The advantage of a 12-column grid is that it offers a wide range of column combinations to create both symmetrically balanced layouts and flexible, asymmetrical layouts. The latter is particularly useful for less standard compositions with different types of content, such as the dashboard format of the newcomer homepage.

A 12-column grid, however, is only functional up to a certain screen width. Once the screen gets too narrow then single columns stop being a useful measure. I proposed to use different grids for the different breakpoints and devices (desktop, tablet, mobile) since at smaller screen sizes you need less layout flexibility.

One of the added advantages of this proposed flexible grid is that it opened up the possibility of making the new skin we were working on more responsive to screen sizes.

This proposal was then tested with various layouts used across the Wikis and then passed over to Bárbara and the Design System team to turn it into a workable design tool.

Defining a systematic solution for the grid

By Bárbara Martínez

After drafting the initial proposal, the Design System team and I started working on the systematic solution for this grid in order to create a solution that scales to any current and future Wikimedia web project.

Elements to fit in the grid

A systematic grid solution must include the following elements:

Grid elements

• Grid columns: vertical blocks are used to create and align the screen content. Breakpoints will determine the number of columns displayed in each screen size. The layout will be adapted to the available space on larger screens by using more grid columns. We will therefore use more grid columns on larger screens than on smaller ones.
• Margins: separation between grid columns and screen edges. In each breakpoint, margins might differ, and they could be fixed or scaled. We will use bigger margins on larger screens in order to create more space between the content and the screen edges.
• Gutters: horizontal separation between the grid columns that help separate the content blocks on the page. Gutter width might be different in each breakpoint range and they will be bigger on large screens since they help to create more space between columns.

Breakpoints

Breakpoints are the screen size ranges where the design fits certain characteristics. They are created to adapt the layout as well as possible to each screen size.

Breakpoints are decisive in the creation of the grid system since the grid elements will adapt to each breakpoint range.

Layouts

Layouts define the content structure for each page. Layouts are created by placing blocks of content on the page, in order to create a template that can be reused on other pages. We will use each layout according to the design requirements.

After analyzing all the Wikimedia projects, we categorized the page content into 3 different layouts:

Problems found with the 12-column grid on desktop

Although the 12-column grid is one of the most common grids used for desktop, it didn’t allow us to cover the main use cases of our product. After testing the 12-column grid with the most important Wikimedia projects (Wikipedia, Abstract Wikipedia, Wikimedia Commons, Special Search, Growth…) and layouts, we ran into some problems creating a systematic solution with this desktop grid.

We found a major issue to fix while testing the 12-column grid with the Wikipedia article content and the table of content (ToC). Based on previous research and user testing, we needed to cover some minimum and maximum widths for certain page elements. In order to ensure proper legibility, a max-width of 960px was required for the article content. Also, the ideal min-width for the ToC was 200–245px.

Trying to find the ideal margins and gutters in this 12-column grid for desktop was difficult. We either achieved the article content max-width or the ToC min-width, but never both at the same time. In most cases, the ToC was too narrow.

The ToC was too narrow when placed in the first two grid columns (especially on screens below 1280px). This was something we needed to fix since the ToC is really important in our Wikimedia projects.

We tried to expand the width of the right panel from 2 to 3 grid columns. However, this solution did not meet our needs since the ToC went from being too narrow to too wide.

Due to the wide columns of the 12-column grid, the width difference between a 2-column and 3-column ToC was too great.

We couldn’t find the perfect width for the right panel, either it was too narrow (2 columns) or it was too wide (3 columns).

Solution proposed: 24-column grid for desktop

As the 12-column grid was not working for our Wikimedia desktop use cases we decided to find another solution. After some explorations, we decided to use a 24-column grid for desktop instead.

The 24-column grid offers the following benefits:

• The 24-column grid is just twice as many columns as the 12-column grid.
• The combination of columns is greater and allows us more flexibility when deciding the width of the side panels, as well as the central content.
• By combining 24 columns, we have greater flexibility when creating new layouts or updating existing ones.
• Users won’t notice the difference between a 12 or 24-column grid, since the grid columns are not visible to them in the interface. So a 12 or 24-column grid will be perceived as the same visual solution for them.

Testing the new systematic solution

The 24-column grid was tested with different projects and use cases to make sure it would work for both our current real use cases and future possible use cases.

Tested with article content and ToC

We tested the table of contents and the article content with the 24-column grid in this prototype and they worked perfectly on large, medium, and small desktop sizes.

It was then tested with the most important Wikimedia projects to cover all current use cases.

Tested with Wikimedia projects

It was then tested with the most important Wikimedia projects to cover all current use cases.

Tested with different languages

As we work with more than 300 languages, we also tested the 24-column grid with many languages to check if it worked well with different scripts.

Tested with the base font

Our current base font is 14px for desktop. But we also tested the 24-column grid with a base font of 16px in case it changes in the future.

Final grid proposal

After testing this new solution with our different projects and use cases, we determined that the 24-column grid was the best solution. Our final solution was the following.

In addition to being the most appropriate solution for our desktop projects, the new grid will cover the 3 layouts we are currently using in production. The number of grid columns used varies in each layout and screen size.

On desktop breakpoints (from 1120px screens onwards), side panels will be visible by default, while on tablet and mobile, they will be hidden. As a result, both the content of the page and the navigational menus will be wide enough for any screen size.

Also, this new grid covers an endless number of design solutions on different screen sizes since the combination of columns is highly variable.

Where will this grid be used?

This new grid will be applied to any new Wikimedia web project. So it will be applied in both Vector 2022 (desktop skin) and Minerva (mobile skin). Due to this new grid system’s exclusive use in new projects, it won’t be used in Vector legacy (desktop legacy skin).

Lessons learned

The most important lesson we learned after designing the new Wikimedia grid was to focus on what really worked for us, and not to worry about what is commonly used on other websites. Although a 12-column grid is one of the most common desktop grids, it didn’t suit our Wikimedia use cases, so we needed to evolve our approach.

It is important to adapt the final solution to what you really need for your project, even if your first iterations are based on lessons learned from something commonly used. Following lessons learned from other projects is worthless if this solution is not the right solution for you.

Get in touch

Thanks for reading! You can read more details about the Design Systems Team and all our current and future initiatives.

Thanks to Alex Hollender, Sarai Sanchez and Volker Eckl for your feedback and collaboration creating the new Wikimedia grid system. Thanks to Lucy Blackwell for the awesome review and feedback on this post.

Originally published at https://design.wikimedia.org on January 25, 2023.

Designing a single grid system for all Wikimedia projects was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

My product team, Trust and Safety Tools, has been tasked with building a system for people to safely report harassment on Wikipedia.1 As a woman my instinct was to first ask, how does harassment affect women on the platform? What about other people who are targeted based on their gender identity? Knowing that the internet is rampant with sexist, homophobic, and transphobic rhetoric, what could our team do to make a small difference in this complex space?

Understanding the problem

I’m a designer, and I follow a design process. Therefore, my first question is to ask how well do we understand the problem of gender-based harassment? Who experiences it and in what form is the harassment? What do targets of gendered-harassment do about it? What do these “users” (to use product-speak) need? What are their goals and motivations? If we were to create personas for them, what would they be?

In order to answer these questions we, of course, need to do user research. However, in order for this research to be properly contextualized, it needs to be parsed through a feminist lens. Here we push up against the limitations of the individual designer. To design for complex, sociocultural scenarios, I believe a designer needs to be fluent in sociology and the disciplines it spawns, such as the study of race, gender, class, and religion. You see, when analyzed in a vacuum, gendered-harassment is easily misunderstood as “bad actors misbehaving”. However, if one understands the mechanics of patriarchy, one is able to see how gendered-harassment is the byproduct of a misogynistic culture. I, as the designer, don’t need to be an expert in sociology, but I need to know who to talk to, to appropriately understand the problem space. As I proceed with this project, I will be doing exactly that!

A disproportionate number of editors who experience harassment on Wikipedia are women and/or identify as LGBTQIA+. They are vulnerable to harassment in the form of name calling, discrimination, stalking, trolling/flaming, and threats of violence.2

First, we need to understand and accept that we have a cultural problem on our hands, and cultural problems are not easy to fix. They require intervention and prevention, such as inclusion training, protections for the vulnerable, accountability for those who inflict harm, and strong, egalitarian values embodied by leadership. So, before my team begins this work I would just like to say, no singular product can solve gendered-harassment.

However, solving gendered-harassment can and should be prioritized, and not an afterthought which is why I’m writing this blog post and would like to drive this work forward.

Why this matters

Solving gendered-harassment is not only a moral imperative, it is central to the mission of the Wikimedia Foundation. When editors are harassed they stop contributing, and we will never reach our vision of free knowledge without equitable participation from folks of all genders.

To quote Professor Jessica Wade’s TEDxWomen London 2018 talk, a Wikimedian herself and voice for diversity in science, “…the majority of history has been written by men, about men, for other men … and the majority of content on Wikipedia is written by white men in America.” Professor Wade, and many other community members, are working to recruit more women to edit Wikipedia, but we must also ask, why aren’t more women organically evangelizing or joining Wikipedia? Once they join, what is their experience like? Do they stay? Do they encourage more women to join? Why or why not? Is the editing community an inviting, welcoming, and friendly space for women?

Simply put, it is not enough to acquire women who edit, we must retain them.

The community’s efforts

For many years various Wikipedia community members have asked for improvements to reporting systems, specifically to protect women and LGBTQIA+ editors. Daniellagreen proposed a way to end sexism and intolerance on Wikipedia. Wikimedian-in-residence blueraspberry sketched the basics of a centralized harassment reporting and referral service. User beauxlieux centered ‘ apologies’ in their suggestion for how to make Wikipedia a kinder and more civil place. Editors QEDK and Djembayz teamed up to propose a whole suite of reporting tools and best practices specifically for addressing gender concerns such as “have moderators who are trained to understand the reality of women’s safety needs’’. One of my favorites is by Wikimedian and sociologist radfordj who derived inspiration from the U.S. Women’s Rights movement in the 60s and 70s, and proposed a consciousness raising-style repository to document and elevate the stories of traditionally marginalized groups, in the hope that this data can be used to understand and mitigate marginalization.

Online gender-based harassment is not a new phenomena or a simple problem to fix. A few infamous examples include Harassment via Wikipedia Vandalism — Feminist Frequency, Women and Wikipedia: Perspectives on the gender gap, harassment, and the Gamergate controversy, Quora’s misogyny problem: A cautionary tale | ZDNET, and a woman-volunteer bringing a harassment issue straight to Jimmy Wales.

Demonstrating the use of a feminist lens

In this blog post, I want to demonstrate how to critically examine the existing ecosystem on Wikipedia through a feminist lens.

English Wikipedia has five foundational principles.3 One of these principles is about conduct and the two values that underpin conduct are “civility” and “assume good faith”.

For the sake of analysis let us assume a new, woman-identifying editor is harassed by a veteran male editor and searches for resources on Wikipedia. Below are the first four instructions she’s given. I have highlighted parts of the instructions which I find counterproductive to our shared goal of supporting targets of harassment.

When dealing with incivility:

1. First of all, consider whether you and the other editor may simply have misunderstood each other. Clarify, and ask for clarification.
2. Consider the possibility that something you said or did wrongly provoked a defensive, irritated or fed-up response. Be prepared to apologize for anything which you could/should have done better. (If an awful lot of people seem to be getting frustrated with you, the problem may be with you.)
3. Even if you’re offended, be as calm and reasonable as possible in your response. Until there is clear evidence to the contrary, assume that the offense was unintended.

In a case of ongoing incivility, first decide if anything needs to be done. Confronting someone over a minor incident — particularly if it turns out that you misinterpreted what they meant — may produce more stress and drama than the incident itself. Consider your own behavior, and, if you find you have been uncivil, apologize to them instead. ( civility documentation)

Steps 1–3 undermine the target’s emotions and her judgment. One could go so far as to make the argument that these instructions gaslight the target, by first asking her what she did wrong.4

The second core value is to “assume good faith”.5 Asking victims of harassment to assume good faith, when trust has been broken, places an unfair burden on them. This can lead to inequitable distributions of emotional labor and a higher rate of burnout. Other advice we give is to “first and foremost act calmly (even if difficult)”, “simply ignore it”, and to stay cool when the editing gets hot.

I understand that our current policies are intended to be helpful. Wikipedia is written and run by volunteers with limited time, and these policies attempt to save volunteers time and emotional labor by asking people to resolve conflicts themselves. However, policies that don’t serve victims of gendered harassment ultimately create more work for the responder.

Using the inclusive product development playbook

Changing policies is out of scope for my team, Trust and Safety Tools. We are focused on designing a minimal viable product (MVP) for how someone will report harassment and what the responder will see on the other end. I am advocating that we prioritize solving for gendered-harassment, and reporters who are women or LGBTQIA+. With the rollout of the inclusive product development playbook, we have the tools and support to approach an old, systemic issue in a new way.6

Our team is in the first phase of product development, also known as the STRATEGIZE phase. I’ve read through the playbook and grouped together a set of tasks pertaining to our audience:

Establish a baseline of current users and use cases, and opportunities for equity based on your context. Include existing research (internal and external) of the problem space you want to address.

Be intentional and clear about who you will empower and engage with (have a clear why), and what that engagement will look like (target audience). If your answer is everyone, consider thinking about who you will engage with first for learning and then scale.

Establish “Who are we leaving out?” and be clear if you will include those that are being left out in this iteration or in the future based on your baseline understanding of “established” audiences and “growth” audiences.

When establishing objectives and key results, be specific about who you are serving, what assumptions you are making and how you will validate or challenge those assumptions.

Clearly define what barriers and/or knowledge gaps you are aiming to address as well as what new opportunities you are planning to create for your target audience and why your team is uniquely positioned to do so.

Our challenge is to understand the pain points of women and LGBTQIA+ folks because a Wikipedia that is safer for the most vulnerable, is safer for all. Within our target audience, we should focus on those who are the least well-connected and influential within the community i.e. the newcomers. We will need to conduct user interviews to better understand who we are building for and speak with responders who have helped resolve or witnessed gendered-harassment. Given how complex and expansive this project is, it will be necessary to who we are not designing for as well.

Establish what partners (internal and external) need to be brought into each phase and give them advance notice of the goal you intend to accomplish if not sooner. Advanced notice should occur as soon as you are aware you will need the team’s support, ideally at least a quarter in advance.

Some potential partners include mission aligned organizations such as Geek Feminism Wiki, Black Girls Code, Pollicy, Feminist Internet, Ada Initiative, Fundación Karisma, Cyber Civil Rights Inititiave, GenderIT, APC, The Bachchao Project, CLEAR, and HeartMob. For internal partnerships a few potential partners include the Global DEI team, the Trust and Safety legal department, and the inclusive product development working group.

Anti-harassment work is inclusion work

It is easy to identify the root cause of harassment to individual personalities e.g. “so and so is just a bully”. Harassment can indeed be random and isolated, but it is frequently about power. When women or LGBTQIA+ folks are targeted because of their gender-identity, it can be with the intention of systematically driving them away. This form of harassment is not interpersonal, it is ideological. If we didn’t view harassment in this way, a feminist way, we would be missing a very important insight about the problem space.7 According to Valerie Aurora, ED of Ada Initiative, anti-harassment work is part of a successful diversity initiative.8

If we wish to retain and cultivate gender diversity on Wikipedia, we must design and build a reporting system that addresses the nuances of nonmale needs. This blog post is an analysis, a call to action, and just the beginning.

Get in touch

Please email me at avardhana@wikimedia.org if you’d like to discuss this work further.

Thanks to Selene Yang, Bethany Schowengerdt, Lucy Blackwell, and Carolyn Li-Madeo for their thoughtful edits.

1. Universal Code of Conduct/Revised enforcement guidelines — Meta ↩
2. File:Harassment Survey 2015 — Results Report.pdf — Wikimedia Commons ↩
3. Wikipedia:Five pillars ↩
4. Gaslighting — Wikipedia ↩
5. Wikipedia:Assume good faith ↩
6. Wikimedia Product/Inclusive Product Development — MediaWiki ↩
7. Violence | Feminist Principles of the Internet ↩
8. https://commons.wikimedia.org/w/index.php?title=File:Successful_diversity_initiatives_in_open_tech_and_culture.pdf&page=13 ↩

Originally published at https://design.wikimedia.org on November 2, 2022.

Solving gendered-harassment: A feminist framework for creating safety on Wikipedia was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

Trying to find a job during the peak of the pandemic as a junior UX designer was a grueling process, but after months of applying, I was given the opportunity to start a 9-month internship at the Wikimedia Foundation. There are a limited number of junior design positions out there that allow new designers to gain the experience required to propel their careers forward. This blog post is intended to share insights from my experience, specifically in these areas:

• The application process
• Design at the Wikimedia Foundation
• Summary of the Growth project
• Summary of the Editing project
• Summary of the iOS work
• What’s next?

The application process

I was applying to purpose-driven organizations because my personal values most closely align with workplaces that aim to have a positive social impact. I was interested in working at the Wikimedia Foundation because I have been a supporter of their mission and have learned about their impact through commons, peer-to-peer, and Transition Design literature. Hence, I decided to apply for a UX Design role there, and to my surprise, I was invited to interview for the position a couple of weeks later.

The interview process

The interview flew by and the hiring manager shared that there were many strong candidates that applied for this role. Generally, the Foundation has only one designer per product team so they hire designers with more experience. She mentioned that there might be an internship program later on but currently there was none. I did not get the UX Designer role that I applied for, but weeks later they reached out and offered me an internship role. Due to the pandemic, they needed someone to help out on the Design team. Since this position didn’t exist in the past, the management didn’t have a clear vision of how I would spend my time, but they offered me to work on a couple of teams and participate in the Foundation’s organizational meetings and events. I accepted the role right away.

Internship overview

During the nine months, I rotated through three different teams, which included Growth, Editing, and iOS, where I took over small projects parallel to the team’s main work. Spending time with more than one team allowed me to observe different team processes and work on a variety of deliverables. I was also able to attend all of the design team meetings, product pre-release demos and take part in Wikimedia-wide events.

The Foundation is a fully remote, nonprofit organization that has a very welcoming, engaging, and collaborative environment. The design team is distributed across a number of projects and time zones, but the whole team meets on a weekly basis, which allows the designers to share their work, discuss relevant topics and get valuable feedback from one another.

Work on the Growth team

The first 4 months of the internship I spent on the Growth team working on a mentorship dashboard. The team generally focuses on developing software that can help engage new contributors, especially those in mid-size Wikipedia projects. Engagement over the years has decreased because new contributors need to learn technical and cultural skills to start contributing to Wikipedia, and it is not always straightforward to acquire these skills. The team has previously built a newcomer homepage, which is a primary feature in the team’s core objective to improve newcomer retention. The newcomer homepage has a range of modules, one of which is the mentor module where the newcomer can reach out to an experienced editor with questions or concerns. Currently, the mentors don’t know anything about their mentees until the mentee reaches out to them. I was assigned to improve the mentor experience, which included understanding the mentor’s needs and frustrations and designing a dashboard that would allow them to be more proactive in their mentorship role.

Support

• Throughout the project I had support and guidance from the designer on the Growth team, who met with me on a weekly basis and gave me feedback on my process and designs. These meetings kept me on track and helped me plan out my weeks.
• Additionally, I met with the project manager on a biweekly basis where we discussed timelines and other milestones like presenting research to the team.
• Later in the process, I had weekly meetings with the engineer who worked on the project with me and was a great support throughout my time on the team.

Discovery phase

I began with exploring the problem space by studying mentorship, existing Wikipedia projects, and external mentoring platforms. The research helped me get accustomed to the Wikipedia-scape, but I still knew little about mentorship on the platform. Therefore, I decided to conduct interviews with mentors on wikis to which the Growth features have been deployed and talk with mentors on Russian, Ukrainian, Arabic, Vietnamese, Czech, Hungarian and French wikis.

The research goals for the interviews were to:

1. Understand the general profile of a mentor
2. Understand the mentor’s motivations and attitudes regarding mentoring
3. Understand processes & behaviors

The interviews

After transcribing and consolidating the interviews I learned that there was a continual thread that linked the mentors: their willingness and patience to teach and help newcomers. Almost every mentor I interviewed had a good experience with a mentor figure when they first started editing Wikipedia and many of the mentors were motivated by this. Most importantly I got to understand what these mentors would like to have in their dashboard:

• An overview of their mentee list and mentee contributions.
• Template answers for the most frequently asked newcomer questions
• Better mentor-mentee related notifications
• Better support and training for mentors
• For mentees and mentors to be matched by interest or demographic

Defining and Visual Design Phase

Based on that information I proceeded to design modules that would showcase information that would help mentors communicate with their mentees, motivate them and help them be better in their roles. It was important to consider the existing workflows of experienced editors and ensure that the dashboard was not a hindrance to them. I designed the following six modules for the first version of the dashboard:

1. Mentees overview — a module where mentors can view information about the newcomers that were assigned to them.
2. Settings — dashboard settings where the mentor has an overview of their current status and the welcome message the newcomers see
3. Unanswered questions — a space where mentors have access to questions they haven’t yet answered from their mentees.
4. Replies to FAQs — the mentors own saved answers that they can reuse when answering mentees’ questions.
5. Your impact — this would show the number of contributions their mentees have made, thanks they have received and total questions the mentor has answered.
6. Resources — is a module that holds the most useful links to resources a new mentor might need.

Reviewing the designs

The designs were reviewed by the Growth team and discussed at the design team’s weekly design review. The feedback that followed was incorporated into the mockups. Receiving feedback from the other designers was an excellent way to improve my work and it was one of the main ways that I was able to continuously learn. Additionally, the presentations by other designers were a great source of learning and inspiration.

Development Phase

I worked closely with an engineer to build out the top 3 modules for the first version of the dashboard, which was then tested with the community. The modules included the mentee overview, settings and resources. Once the modules were almost ready I ran a moderated remote usability test with the goal of identifying opportunities for usability improvements related to placement, priority, navigation, and orientation. I also wanted to test whether the V1 of the dashboard had met the mentor’s expectations and see what a V2 version should prioritize. The key findings included:

• Everyone quickly located the dashboard and the participants found the ‘mentee overview’ to be important. However, they didn’t find the module as functional as others. They wanted further ability to interact with the table.
• According to the mentors, the second version of the dashboard should include further modules such as, ‘unanswered questions’, ‘replies to FAQs’ and the ‘impact module’.

Lessons learned

Biggest learnings and takeaways from the Growth team project:

• Experience working on a constantly evolving, iterative project.
• Took part in team discussions about their ongoing projects and processes.
• Learned about ways the product teams deploy projects and disseminate information amongst other teams.
• Learned how to work with the community.

Daily skills and processes:

• Got in the habit of documenting all of my work to ensure that the information can be easily shared among other teams and the Wikimedia community.
• Ran moderated usability testing.
• Learned about design systems and got more comfortable using the component library.

Work on the Editing team

For the second part of the internship, I spent time with the Editing team which works on editing tools across projects and improves the editor’s ability to contribute to Wikimedia projects. My project revolved around researching and exploring a tool that would help new contributors become more successful at publishing edits that adhere to Wikipedia’s policies and guidelines. Currently, many new contributors’ first edits get reverted at Wikipedia because newcomers are not aware of existing policies and guidelines. Having their edits reverted demotivates the newcomers from making further edits and lowers newcomer retention. Hence, the tool could:

• Make contributors aware when their edits defy existing policies or guidelines before they publish their edits
• Help to educate newcomers about policies and guidelines
• Diminish the frequency of the newcomer’s first edits being reverted

Support

• Similarly to the Growth team, I met with the Editing teams designer on a weekly basis
• In this team I had a set of deliverables that were determined by the project manager and designer with whom I met multiple times to discuss and present these deliverables

Discovery phase

The first phase of the project included putting together a research report that would identify the known and unknown areas of the problem space. The research included information such as:

• The newcomer editing experience
• Ways experienced contributors interact with newcomers
• The history of policies and guidelines at Wikipedia

During the initial research, I learned about the evolution of the policy space at Wikipedia. Writing articles for Wikipedia was an iterative, inclusive and collaborative process where editors could co-create content. As the size of Wikipedia grew, one of its strongest criticisms was that it was an unreliable resource. Over time, community members have made quality control one of the wikis primary focuses. In light of making the information more reliable, the number of policies and guidelines has increased over the years. There are now more policies and guidelines than ever before, they have become complex and are cited much more frequently by editors as a means to support their activity on Wikipedia.

The team spoke about choosing one policy and using it as a way to help guide the project, it would serve as the policy to showcase the potential editing tool. After consideration, I decided to focus on verifiability, as it is one of the most important policies for newcomers to be aware of. Since Wikipedia is an online encyclopedia, its content must stem from reliable sources, and therefore defying the verifiability policy while editing will ensure that the editors’ contributions will be reverted or deleted. During the next phase of the project, I did a comparative analysis of Wikipedia projects as well as external platforms and apps whose objectives were to help teach contributors a specific skill set. I looked mainly at:

• Grammar correction apps/writing apps
• Education apps and extensions
• Existing Wikipedia projects
• Gamification methods

This aided in narrowing the ways the editing tool could be displayed.

Defining phase and the visual design phase

This was a difficult phase of the project. With so much valuable information and perhaps more questions than when I had started, I took the first stab at designing the editing tool. The editing tool is a feature that contributors would be able to use to see areas in the text that defied the verifiability policy. For instance, the tool would suggest placing citations in certain areas and it would highlight titles that need to be linked. Additionally, the tool would allow users to quickly access resources about verifiability.

Presenting at Wikimania

Through the editing team, I was able to present the editing tool project at Wikimania 2021. I presented with other volunteers in a talk called ‘ Editing the Wiki Way: software and future of editing ‘. I was able to meet numerous committed community members that are passionate about various projects and discuss the future of editing with them.

Lessons learned

Biggest learnings and takeaways from the Editing teams project:

• Gained experience working on a research-heavy project
• Took part in team discussions about their ongoing projects and understood the processes and how these are similar/different to the Growths team.
• Learned a lot about policies and guidelines on Wikipedia.

Daily skills and processes

• Put together a research report
• Gained practice presenting my process and artifacts to a multitude of people.
• Became more comfortable using Miro diagrams.

Work on the iOS team

The final and shortest stretch of the internship was with the iOS team. They are part of the Wikimedia Apps project together with the Android team. The apps teams are historically considered to be more experimental, in part because they have lower user traffic than the desktop and mobile versions of Wikipedia and are used more by readers rather than power editors, who generally engage differently with existing features. It was very interesting to learn a completely new set of processes and to get accustomed to using the apps style guide and a slightly different product altogether.

Support

In the iOS team I also met with the designer on a weekly basis and had meetings with the project manager every 2–3 weeks.

Tasks

The team was working on a notification center during my time there and I was tasked to complete a set of audits, do research, and do smaller design tasks that would be helpful in the future. I really appreciated that the team prioritizes designing and developing for a variety of accessibility needs. This is made possible by Swift, Apple’s native programming language, which allows one to easily create products that give support to those who might be, for example, visually impaired. Through the variety of deliverables, I was able to engage with different parts of the app and learn more about another pocket of the wiki-verse.

Lessons learned

Biggest learnings and takeaways from the time spent on the iOS team

Daily skills and processes:

• Designing for iPad
• Learned more about Apple’s accessibility support
• Learned about iOS UI and deployment processes

Daily skills and processes:

• Learned about the iOS team’s design & engineering processes
• Became comfortable doing design audits
• Got to know Apple’s Human Interface Guidelines in greater detail

What’s next?

I could not have wished for a better internship experience! The 9 months were incredibly diverse, challenging, and fun. I was supported throughout by amazing designers, project managers, engineers, and numerous different team members from the Growth, Editing, and iOS teams, who worked with me to improve my skill set as a designer and helped me find my footing in the wiki-verse.

Since my internship, I have been offered the opportunity to join the iOS team full-time. The design team is interested in creating more formal internship opportunities for junior designers. This year they have welcomed two new interns to the Design team, and hopefully there will be more internship positions available in the future.

Thank you to Lucy Blackwell, Rita Ho, Jess Klein and Carolyn Li-Madeo for this opportunity, as well as your support and feedback!

Originally published at https://design.wikimedia.org on July 12, 2022.

Entering the Wikimedia universe: starting with a design internship was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

By Robin Schoenbaechler & Jazmin Tanner

Over the past year, the Wikimedia Foundation Android team has focused on improving the communications experience in the Wikipedia Android app. Asynchronous collaboration is a core component of the Wikipedia editing experience, and the lower the entry barrier to editing, the more diverse the content will be for our 14 projects across 300 languages. This diversity of content and collaborators is core to the movement strategy, which focuses on knowledge equity and knowledge as a service. The movement strategy brings us closer to ensuring our projects represent the sum of all knowledge.

As the organization focuses on knowledge equity, the Product and Technology departments have worked on a framework for inclusive product development. To ensure the framework was reflective of the varying scope and team composition of our development teams, seven Wikimedia Foundation teams signed up to test the playbook and provide feedback. The feedback would serve as an input for V2 of the playbook in our framework. With our team’s goal of reducing the barrier to communication and collaboration in the Wikipedia Android app and being very intentional about accessibility, we were a logical test team for the playbook. We also recognized there would be a lot to balance as we worked towards this goal.

Although the original request to improve communications in the apps came from experienced members of our community that largely wrote in English, we wanted to be very intentional about partnering with members of the community who haven’t been historically centered in our projects. Depending on the development team and platform the partner communities could look different. In our context, the identified partner community members consisted of women and non-binary editors on Japanese Wikipedia, French and Arabic editors in Mali, Egypt, Morocco, and DR Congo, Hindi and Indonesian Wikipedia editors, English Wikipedia editors in Nigeria and visually impaired English Wikipedia editors in India. You can read more about our process in selecting these partner communities here.

The egg-laying wool-milk-pig for Android

In German, there’s a popular idiom of the “Eierlegende Wollmilchsau”. It roughly translates to an egg-laying wool-milk-pig. The phrase describes an imaginary animal that combines unique characteristics of some of the most renowned animal species: chicken (lay eggs), sheep (provide wool), cow (give milk), and pig (meat).

This idiom encapsulates what we are trying to achieve; a seemingly unachievable and perfect outcome of a product for diverse user needs: all the benefits, no drawbacks.

This article shares the steps we took to develop a design approach that reflects the ideal the egg-laying wool-milk-pig represents. When you finish reading, we hope you come away with the following lessons:

-The importance of designing with people instead of for them

-Understanding when to focus and when to see the big picture

• User Segmentation: Meeting audiences where they are
• Leveraging Data: Data doesn’t show where to go; it shows where you are
• Focus Mode: Adding capabilities without distraction

-Customization as a means to include: Optimizing the article experience based on different user goals

“We shape our tools, thereafter our tools shape us”

This quote by Father John Culkin, Jr. feels very true for Wikipedia. Here’s why: When we improved searching for articles, we realized that people use Wikipedia in unique ways. Use cases and opinions for those who regularly use the Wikipedia app vary substantially. This difference of perspectives even rang true when learning from our target audiences what their communication preferences were:

Editors see a need for more editing capabilities:

“The only missing feature is providing notifications like on the website”

Readers love to read articles:

“Why would I need Wikipedia notifications? I don’t even have a login!”

Wikipedians long for missing key features:

“Please make it possible to access talk pages and edit histories.”

And others simply prefer the Wikipedia web experience:

“Functionally very similar to the web browser experience, if not less intuitive.”

This is just a tiny sample of the sometimes contradictory feedback we receive from people using our app primarily or in conjunction with other platforms. However, these examples exemplify that qualitative feedback points us in multiple directions, especially when we weigh the maturity of a wiki. So, when user feedback pulls us in multiple directions, how do we decide where to go?

It could be easy to rely on the intuition of the team. However, as we learned from the Inclusive Product Development framework, even with a globally dispersed, fully remote team that aims to live up to the organizational goal of embracing diversity, we must be wary of introducing our biases into the product and strive to be inclusive of these different audience goals.

Data doesn’t show where to go, it shows where you are

As we delve deeper into our quantitative data, there’s evidence of commonalities amongst different cultures, use cases, and needs. Our data scientist, Shay Nowick, was able to provide the insight that 62% of those that opt-in to share anonymized data visit the article view most frequently in the app.

This insight seemed logical considering the Wikipedia Android app was initially created as a “readers-first” tool. It always had editing capabilities, but the editing experience was historically limited, had a high participation barrier and resulted in a low edit count compared to other platforms.

Focus areas to create the egg-laying wool-milk-pig

Balancing reading and editing

In 2019, the “readers first” strategy shifted from full-article editing to explore the concept of micro contributions with the introduction of a feature called suggested edits. Micro contributions aim to lower the barrier to contributing by introducing an alternative to longform writing.

In addition, we identified it as gentle onboarding for new editors, and creating new forms of content. As this capability was rolled out, the visibility and editing calls to action (CTA) were intentionally made prominent on the home page toolbar and in the frequently visited article view, shifting the UI away from the “readers first” experience.

However, this change could ultimately distract someone who just had the goal of reading as you can see in the example below:

Data shows that the edit asks were somewhat effective in encouraging readers to add missing article information. The + ADD ARTICLE DESCRIPTION button alone is the source of almost a fourth of all edits (~23%) made in the Wikipedia Android app. Adding the CTAs was a deliberate strategic move to encourage editing, and it seemed to be effective. Despite the distracting nature, hiding them for all people at all times was not an option.

But to the point someone raised about not always wanting to see editing features when they’re in the mood to read, how could we accommodate them?

In addition to not wanting to distract with too many editing call to actions, a community member left the following feedback on the Google Play store which made us question the article’s bottom toolbar:

“Why have you taken away the ability to go full screen? Why do we have to see the bottom toolbar all the time?”

“I want the option to hide the bottom bar as I scroll down. Thank you.”

Usability tests from three years ago gave us the insight that some users preferred seeing the bottom toolbar at all times or struggled to find features when the toolbar disappeared.. For example one user shared:

“I’m deleting the app because it doesn’t support find in page”

The article most certainly did support Find in page, we call it “Find in article” and it was located in the bottom toolbar that formerly disappeared on scroll.

Reading distraction-free

The insights mentioned above, paired with our desire to introduce more prominent editing features, like a prominent icon for talk pages to make it more discoverable, made our interest in honoring some explicit requests from the community to add a reading focus mode that much more urgent.

We chose to put the reading focus mode toggle into “Theme” because it’s related to existing preferences like font size or family. When reading focus mode is enabled, CTAs disappear, and the article becomes full screen on scroll, meaning the top and bottom toolbars disappear; limiting distractions while reading.

There was a concern within the team that users would have trouble remembering how to access editing features again once reading focus mode was enabled. To ensure we could overcome this obstacle we conducted usability testing with our partner communities.

Instead of using a clickable design prototype, our developers created a working prototype that our participants could download:

Image caption: Not only were people able to successfully disable the feature if they wanted to edit, they provided other positive insights as well:

“It’s better, the full-screen mode makes it easier to read, there’s nothing that bothers me!” — Usability test participant, Arabic Wikipedia reader

“This reading focus mode is really helpful, excludes all unwanted options, it makes the page more comfortable to read” — Usability test participant, Hindi Wikipedia reader

“It’s a good feature as the majority of us are here to read articles. It’s a good function to have to limit distractions.” — Usability test participant, Japanese Wikipedia reader

These insights gave us the confidence to release the feature to production and enhance discoverability. Check out the detailed analysis to dig deeper into our findings.

Reading focus mode was also a success from a quantitative perspective. After its release, 80% of all people discovering and interacting with the feature kept it as their default preference. We did consider making this feature a default experience, but wanted to ensure users knew they could edit, and for that reason we will continue to have reading focus mode disabled by default.

Customizing the experience

Reading focus mode was not the only feature where we aimed to strike the right balance between the needs of readers and editors. As we added more features to the app, our overflow menu grew, and we noticed users asking for features that existed but were hidden in the overflow menu:

“Doesn’t allow you to read the page’s Edit history or Talk page. So practically only useful for reading, not editing.”

“It has good features like a menu option to switch between languages”

“No option to add to Watchlist, I don’t like the interface or look.”

“It’s missing categories.”

“Please add the capability to share an article’s URL.”

Depending on the maturity of the wiki, the audience, and user goals, the preferences for what links were available varied, which made it apparent that the next phase of our work before releasing new communication features needed to take a closer look at shortcut usage in the bottom toolbar and the top menu:

Designing for people’s preferences while keeping the experience focused challenged us, but is becoming more of an industry standard with the launch of Material You. Relatedly, one of our approaches to toolbar customization was inspired by Google Chrome. We explored the idea of adding all items to the top menu.

We asked ourselves if putting all options into one place is considered good or bad information architecture and found the answer in existing research. Hiding interface elements reduces their usage as they are less discoverable. We did not want actions related to the article to be less used, as our audience finds them helpful, so we continued with having shortcuts in the bottom toolbar.

How the-egg-laying wool-milk-pig came to life

Prototyping the first version

We landed on letting people customize their shortcuts in the bottom toolbar, and developed several prototypes to test which approach best achieved our goal.

We learned that people had difficulties discovering the “Customize toolbar” feature within the “Theme” shortcut, trouble discovering where items that were moved out of the toolbar and into the overflow menu were placed and issues dragging and dropping items, a gesture we incorrectly assumed would be intuitive. This implored us to have a closer look and ensure our drag and drop movements met Web Accessibility Guidelines( WCAG).

To address the discoverability issues we added onboarding tooltips, even with this addition, users were struggling to find the feature and effectively scrolled past it making it disappear. You can read our analysis for full details.

Testing, discarding, and iterating

We now faced a real example from the inclusive product development playbook of having to choose equity over speed. After sharing the results with our communities and collaborating with other Wikimedia Foundation teams like the design team, we created two iterations of the original design:

The goal was to determine which of the two versions participants preferred, why they preferred it, if there were differences amongst language preferences and if the participant would use the feature. Both variants featured:

1. A persistent tooltip that only disappears when pressing the newly added ‘GOT IT’ button.
2. In partnership with UX copywriter Jacqui Clydesdale, we created default helper tooltips to explain the drag and drop function.
3. Simplified language (e.g., ‘shortcuts’ instead of ‘links’), to ensure terminology was accessible .

After doing one more round of usability tests, we gathered feedback on the two variants from ten Arabic, Hindi, Japanese, Indonesian, and French-speaking participants. The results were almost evenly split, so we could not take a majority wins approach. It was evident that both variants were drastic improvements when it came to discoverability and usability. With no clear winner between the two variants we took into consideration Google material design guidance, and considered which change would be less jarring for our existing audiences. This led us to move forward with variant B. The reactions to the updates in variant B were:

“It’s awesome, it was easy to understand and find.”

“The design is very good, I haven’t seen this anywhere else.”

“I’d use the feature 2–3 times and think it’d improve the experience.”

“It’s a very good feature for people who use Wikipedia daily.”

“There’s one more shortcut in the bottom toolbar, a lot of people have a habit of clicking the top menu to look for things.”

Although we chose our path forward between the two variants, there were still issues that were exposed in usability testing that we found important to address before a full roll out could occur.

Designing cross-cultural

As we mentioned above, we aimed to partner with very specific audiences. When conducting usability testing it was important to test with our partner community in their native languages. Building real installable app versions in Android’s file format apk made the difference. Due to testing working app versions, we identified various localization issues, and prioritized solving them before release.

Within the Wikimedia Foundation it is customary to use an open-source translation service in partnership with our community called translatewiki.net. The project, made available by Niklas Laxström, is sensitive to the unique vocabulary used on Wikipedia and is able to consider nuances that machine translation doesn’t always capture. However, because it is community sourced, we do have to take our time in waiting for the translations to come through, or even reaching out for support before launching usability testing.

Notably, a third of our readers access Wikipedia in multiple languages. As usability test participants were bilingual too, we noticed that people’s device and app language differed. This is not necessarily bad, as it leads to observing even more diverse forms of use. However, for our purposes we deliberately wanted to test in Arabic, Hindi, Indonesian, Japanese, and French, so we asked people to set their device language at the beginning of the test.

This step was very important because it exposed bugs and unexpected behaviors for right-to-left (RTL) environments earlier than waiting for users to alert us via the app store. Namely, when conducting usability testing, we observed glaring bugs with the direction of typography, icons, and images in RTL that needed to be corrected prior to release.

With the support of our Product and Technology leadership to build for equity and inclusion over speed, conducting design research in regions and languages beyond the English-speaking community will continue to be our team’s default. As will it continue to be our team’s default to prioritize fixing potential localization challenges and improving accessibility across cultures, languages, abilities, interests and backgrounds.

While an egg-laying wool-milk pig does not exist in nature, we have the unique opportunity to design the closest ideal in a digital product. And, by fulfilling one user group’s needs, tremendous advantages will inevitably present itself in serving the needs of others.

We are committed to equitably design with everyone and modify the proverbial table to include as many as possible — and we think you should too.

All the features mentioned in this article are now available in the official Wikipedia for Android app on Google Play. Our team is working on many more exciting features to offer all the advantages and satisfy (most of) our user’s sophisticated and sometimes even contradictory needs in the near future.

Follow @WikipediaMobile on Twitter for the latest news related to Wikipedia apps. If you don’t want to miss the latest design changes to Wikipedia, follow @WikimediaDesign. Please let us know what you think about the updates in the comments below, on MediaWiki or email.

Thanks to Matt Cleinman, Shay Nowick, Matthew Williams, Carolyn Li-Madeo, Johan Jönsson, Lucy Blackwell, Sam Patton, Rita Ho, and Jess Klein for their contributions to this article.

Hey, check out this egg-laying wool-milk-pig: said no one ever was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

By Rita Ho

As hinted at in the name, the purpose of the Wikimedia Foundation’s Growth team is to grow the number of new people who come and make a first edit on Wikipedia, and to keep more of them around (a.k.a. “editor retention”).

There is an element of teaching and persuasion involved, since we need to convey to the majority of people extremely familiar with using Wikipedia one way (reading and consuming information) that in fact they should consider using it quite differently (participating in writing and changing the information), and then guiding them how to use it in this brand new way. It’s like telling someone who has eaten eggs their whole life that actually, the egg shells can be eaten as a calcium supplement too, if properly prepared ( it’s true).

Our approach to this mission has been to work briskly, and try out our ideas of how to provide opportunities to edit as a series of experiments. Instead of releasing a new feature or tool for everyone and iterating on the results, we test them out on smaller groups (typically limited to people on a few smaller Wikipedia languages and rarely on the bigger languages like English), then compare how the experimental feature performs against the control (the existing experience). We then iterate on the designs based on a mixture of usability feedback, community feedback, and finally from interpreting the resulting data from people using the feature (or in some cases not using it!).

For one of the key features on which our team has worked, we decided to run an A/B test. Whilst it is relatively common for the Fundraising department to multivariate test small UI variations (e.g., different button placement, copy differences) on donation banners, it has been less commonly used by the Product Design team to experiment with different user experiences in reading and editing on Wikipedia itself. A major reason we don’t typically run such A/B tests is that there is a high bar for respecting the privacy of people who use Wikipedia, especially those who are anonymous. However, since the Growth features are offered to those who have already created an account, this was a case where A/B testing was a good fit for our purposes.

A/B testing

The inciting incident: A new experiment

We knew from past research that many people who create a Wikipedia account often do so with the intention of editing, but then aren’t able to easily find articles to which they can contribute. This led us to design a feature that appears for newcomers after they join Wikipedia, which encourages them to do some small “suggested edits” (like copyediting or adding links) on articles across a variety of topics they might be interested in.

Our hypothesis underpinning this feature was that if we give people specific, and achievable small editing tasks to do, they will be more likely to try a first edit, and more of these people will do enough to build their expertise and interest to become long term contributors.

The diagram below illustrates the user flow initially designed for the feature. Newcomers would first be provided with a prompt to “opt-in” to seeing these suggestions, and if they choose to proceed, they are then provided with some short onboarding screens to learn what it’s about and how to customise suggestions to their interests before seeing the actual suggested articles.

Experimenting within the controlled experiment: Enter Variant B

We started developing the design for this feature, with the design described above as the new feature that would be implemented in a controlled experiment — comparing editing retention with a control group of new accounts which would not receive these suggested edits at all.

During the course of development, Marshall, our Product Manager had a hypothesis that if we showed people the suggestions sooner (without onboarding), that they would do more edits. I agreed this was a valid point, and wondered how we could balance this consideration with my reasons for the importance of user education in letting people understand why doing one was valuable and relevant to them (going back to my earlier analogy of explaining how one can eat egg shells).

And this is how we decided to run our first simple variant test — a nested test within the controlled experiment. We would split the experiment group so that half the participants received variant A which had the “full” designed flow with opt-in and user education; whilst the other half received variant B where they would see the suggestions immediately in their newcomer homepage.

The results are in and the winner is…

Together with the data analyst and PM, the key metrics were examined and interpreted to figure out what was significant.

As it turned out, the results vindicated the user-education hypothesis! The results varied depending on the wiki and platform, but generally showed that whilst more people did indeed interact with the suggested edits module when it was shown to them immediately in variant B on desktop; in the end, more of those in variant A who had to opt-in to the feature and saw the onboarding made successful edits.

Data-informed iteration

So what did this mean for design improvements moving forward? As we saw it, the three key takeaways from the test were:

1. Featuring suggested edits prominently was key to getting people to try the tool; but at the same time…
2. Onboarding is crucial to driving more people to successfully completing an edit.
3. Mobile and Desktop performed differently so as not to be strictly comparable.

It was based on these findings that the next two new variants C/D were created, which applied the best aspects we interpreted from the results of the first test, roughly translated into the designs in the following ways:

1. Variants C and D centred much more on the suggested edits feature in the homepage, removing a smaller ‘start’ module and making it more prominent in size and the focus of onboarding.
2. Onboarding is available in both variants C & D, though it is optional (and more easily dismissible) in variant C.
3. On mobile, variants C and D explore different ways to give more direct access to the suggested edits feature immediately from the homepage.

Design decisions affirmed by data

And it was great to see our interpretation of data and design decisions payoff! In the form of both C & D variants in the next iteration leading to both more usage and successful edits.

Learnings and tips

Besides the gratification of seeing variant tests make a difference in people’s experience of a design, it’s been a great learning experience to apply this method of design optimisation often seen as hard to utilise effectively with other more qualitative approaches.

In my opinion, these are the key aspects of running A/B tests:

• Set a clear, testable hypothesis. Formulating the rationale for why different designs bring about better outcomes will help in devising an experiment that accurately measures the impact of those design changes. Besides that, it’s also good practice for critical thinking that is helpful for design decision making when there is no time/need to run experiments.
• Determine if the effort is worth the investment. Once the hypothesis is set, ask whether it is worth the considerable time and investment request to run an experiment to get the answer, or whether other tactics (such as user testing) can be used instead to help move the design forward.
• Be bold with the variant differences. If the test ends up being about different button colours, it’s probably a multivariate test that is needed. On the other hand, it’s also not advisable to have too many variables (such as testing complete redesigns) since this would make it hard to pinpoint what changes made the difference.
• Ask the right questions of the data: For our team, we care about how many people finish an edit (saving it), not if they started it.
• Collaborate closely with data-savvy colleagues. It was vital to work closely with the data analyst and product manager to check the hypothesis and test was sound, to ensure the correct (and sufficient) data was being measured, and then properly analysed. Having an data analyst expert is also very important in removing confirmation bias, akin to having a design researcher conduct usability testing. They are the key person in determining the experiment feasibility and timeframe from an audience size perspective to get statistically significant results. For folks without an awesome analyst at hand, there are free tools available (like this one, the experiment calculator) that provide sample sizes and time needed to run robust experiments.
• Involve your engineering team early too. Another important partnership is obviously the engineering team who are responsible for both implementing the different features and instrumentation to measure the experiment. Involving them as early as possible as to the parts of the experience that may vary will help them to build them a flexible framework for A/B testing.

Bonus learning before I go

One final learning is not about the A/B test method itself, but about people we design for that came from these variant tests: there’s nothing wrong with providing guidance. I don’t really like the term “user education” or “onboarding” because it is a little condescending (the implication is that the onus is on people to learn how to get on board with this awesome product). User onboarding is actually an opportunity for us as designers to provide persuasive and clear communication, explaining to our newcomer editors what the value is for them to make contributions to Wikipedia in this way.

Thanks for reading! You can read more details about the Growth team’s variant testing on the project page, or about all of our initiatives for bringing new editors to Wikipedia on our team page.

Thanks Marshall Miller, Morten Warncke-Wang, and Matthew Williams for your feedback, editing help, and additional insights into A/B testing.

[1]Details about “Privacy Conscious A/B testing for anonymous readers” can be read here

[2]Phabricator task with more details of the implementation https://phabricator.wikimedia.org/T246533

[3]https://www.nngroup.com/articles/putting-ab-testing-in-its-place/

Originally published at https://design.wikimedia.org on April 4, 2022.

Designing variant experiments is as simple as A/B. See? was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

What is Wikifunctions

Wikifunctions will be a collaboratively edited multilingual library of functions. It will be the first platform The Wikimedia Foundation launches since Wikidata in 2012, and part of a bigger vision towards an Abstract Wikipedia, which you can read about at Abstract Wikipedia on meta-wiki.

A function and its parts

If you’re not a programmer you might not be familiar with what a function needs to be usable, or to use technical-speak “to run”. A function takes an input (e.g. the population of a city) and provides an output (e.g. the acres of watershed needed to supply residents with water). It can do this calculation over and over again for different cases.

A function has three parts. The function definition (part 1), the implementation (part 2), and a test (part 3) to validate. These three components — the definition, implementation, and tests — are what a function needs to perform and be usable.

A definition defines the functions behavior. What type of inputs does it accept? How many? What will it output? An implementation is the set of instructions that transforms an input into an output. A test is what validates the implementation and ensures it performs correctly.

What can functions do?

For starters, you might use a function to wake up in the morning. Let’s say you want to sleep 8 hours tonight, you could ask a function, “If I go to sleep at 10 P.M. what time should I wake up to get 8 hours of sleep?”

Examples of general questions that could be answered by functions:

• What is the distance from my house in northern Sao Paulo, Brazil to the nearest beach?
• If I took out a $240,000 loan and my loan term is 30-year fixed with a 4% interest rate, what is my monthly mortgage?
• If the omicron variant has a transmission rate of 25% among vaccinated people, how quickly could it spread in a city with 2,059 people per square kilometer, 95% of whom are vaccinated?
• When in the next ten years will my birthday fall on a Saturday if I was born March 28, 1954?

Examples of Wikimedia-related questions that could be answered by functions:

• Given the dates of birth and death of a person, how old did the person get?
• Given a population and an area, what is the population density?
• Given an image on Commons, crop it and run it through a filter
• Given two images on Commons, which one is more red?
• Based on Wikidata, how are Queen Elizabeth II and Barack Obama related to each other?

Functions are ubiquitous in modern life, from the apps we use daily (e.g. maps and search) to the analyses made by scientists, doctors, journalists, and financial advisors, functions enable our activities by efficiently and precisely making meaning through calculations.

Functions will provide the code that translates language-independent articles from Abstract Wikipedia into natural language Wikipedia articles. This will theoretically allow more people to share more knowledge in more languages!

This blog post is about how to create, view, and edit functions in Wikifunctions, and the design process behind making this happen.

Our audience

Similar to other Wikimedia projects, our hope and aim is for Wikifunctions to be inviting, accessible to, and available to all people across the world. This is why we are designing a multilingual platform that is mobile friendly.

There are several roles for people within the Wikifunctions ecosystem. There are “answer seekers”, or people who simply want to learn about functions and use them. A level deeper are those who edit or materially contribute to Wikifunctions in some capacity. They could be function patrollers, programmers who write implementations, translators, admins who approve of proposed implementations and tests, or test writers who QA implementations. These folks will maintain functions and documentation. Among these editors there will be technical and nontechnical folks. Leading up to launch and post-launch, our team aims to increase the agency of the nontechnical folks and “junior” techies by creating pathways for them to contribute meaningfully.

Part of creating these pathways is simplifying a complicated process. Functions are not simple. They are complex in that they require computational thinking and well-written documentation. When designing the creation experience, our objective was to design an editor interface that looks and feels friendly to nontechnical people and radically simplifies a complex process.

Our approach

We explored different mental models for Wikifunctions such as collaborative coding, a structured Q&A forum, a homework helper, infrastructure for Abstract Wikipedia, and a ‘generic library of functions’. We settled on the structured Q&A model, where we conceive of functions as tools which can answer structured questions.

WMF’s mission is to unlock knowledge and help all people share in it, freely. The Abstract Wikipedia team believes that functions can play a key role in unlocking knowledge, and sees Wikifunctions as a crucial player in moving all of us towards this mission.

It is a source of power to know how to write and use functions. It gives the person the power of knowledge creation. Our objective when designing the function creation experience was to make it as friendly, intuitive, and simple as possible so that more people can have access to this power.

Our design principles

1. Conceptual simplicity: Given the complex nature of this project, our team aims to radically simplify the concepts involved so that we might be inclusive of all people, especially those who are less familiar with functions, code, natural language processing, or lexicographic knowledge.
2. Intuitive interface: We want to build a system with a shallow onramp, one that anyone can figure out and engage with in meaningful ways. We wish for there to be little gap between a newcomer’s existing knowledge (i.e. current knowledge) and the knowledge they need to operate the product (i.e. target knowledge). Our interface will provide community members with an uninterrupted experience, where they are well-supported to complete tasks with relative ease and confidence.
3. Multilingual: In a world where all people can share in the sum of all knowledge, it is critical that Wikimedia products are designed as multilingual interfaces and experiences. From the beginning, the Abstract Wikipedia team will prioritize designing for and testing in the native languages of our community-members. Our objective is to move at the speed of trust with all populations, but especially with small Wikis and underrepresented groups.
4. Balance: We wish to acknowledge that many of our community members will be veterans of Wikipedia, Wikidata, or any of the movement’s other projects. Many of them will be highly-skilled at navigating technical interfaces and desire customizability similar to Wikipedia or other open source projects. The challenge before us is prioritizing the needs of non-programmers and those with little-to-no coding experience while keeping in mind the expectations of veterans. This will require flexibility and openness in our designs, and filtering design decisions through multiple lenses in order to ensure that we are not locked into any one particular user experience.

We have implemented these principles by doing the following:

• Intentionally minimizing the use of technical jargon and providing a glossary of terms
• Providing tooltips and informative microcopy on every interface
• Focused attention on information hierarchy so as not to overwhelm users with information
• User testing with nontechnical people and iterating to meet their needs and expectations
• User testing with technical people to ensure we are not causing irritation or creating unnecessary roadblocks to their progress -Multilingual design and usability testing (testing is in the planning process)

How to use Wikifunctions

Creating a function

Let’s say we wanted to create a function that determined if two words rhymed. The “two words” are two inputs of type string, and the output is a single “yes or no”, which is a type Boolean. To create a function that could do this, we would need to tell the system, “Hey this function accepts two inputs, both have to be , and the function’s output will be a .”

This process is known as “defining” the function, and it is the first step in creating a function.

Our team believes that the UI and UX for defining a function must be as inclusive as possible because, technically, anyone can do it. It requires no coding or composition writing. Defining a function can be thought of as a request: “Hey I want a function that does X” translates to “the function accepts this input, returns this output”. We also provide a way to add documentation so that the user can further explain what they’re looking for.

Design challenge 1: The function editor

We initially planned to design a “function editor” that provided an end-to-end experience, meaning users could define, implement, and test in one environment. Since we were designing for technical and nontechnical people, we hypothesized that unifying the experience would:

• Simplify the concept of what a function needs
• Provide a contextual experience which would be helpful for nontechnical users
• Keep related tasks together
• Allow more senior editors to move in and out of the process as they pleased
• Not slow down the technical audience from using the editor as they wanted, while providing guardrails for nontechnical users

Early drafts and variations of an end-to-end function editor

The UX was inspired by Miro, Figma, Wikidata query builder, and Typeform.

Pivoting — we tested these mockups with technical non-Wikimedians and Wikimedians, and the results were overwhelmingly positive. However, as our developers were building this design, we realized that the underlying design of the system was incompatible with this approach. Function definitions, implementations, and tests are all separate objects in Wikifunctions, and each object can refer to other objects within it, such as data types (e.g. strings, Booleans, or other functions). Due to the complexity allowed for each object, we decided it’s best to keep each creation experience separate. Objects also have their own pages, so any object you create automatically publishes as a Page.

Today the function editor is a place only for defining the function, in other words part one of creating a function.

The function editor as it currently exists today

Design challenge 2: A function page

When you publish a function that you’ve defined, it is saved and published as a function page. This is the page where all parts of a function live. We framed this page as an entry point for the community to learn about, use, and/or edit a function. The goal of this page is to help people progressively edit a function or simply engage with one. The challenge of this page, and a Wikifunctions motif, is that the page must appeal to both non-technical and technical people.

Our short term goals were to

• Help all users UNDERSTAND the function so that they can edit or use it
• ENCOURAGE users with coding backgrounds to make technical contributions
• ENCOURAGE users who are not coders to make non-technical contributions
• Usability test a Figma prototype of the product solution with people who have a strong coding background

Our hopes and dreams were to

• Encourage everyone that views a function to try using it
• Support coders who want to make technical contributions
• Encourage users who don’t code to make non-technical contributions because functions need names, aliases, translations of the name/input labels, and documentation

We aimed to avoid

• Confusing, discouraging, overwhelming, or intimidating non-coders
• Unintentionally signaling to non-coders that this isn’t a space for them
• Irritating coders by slowing them down or hiding important actions

The function page as it currently exists today. A user lands on a function page’s ‘About’ tab and can toggle between ‘About’ and ‘Details’.

A user can learn more about the technical details of the function on the ‘Details’ tab, such as how many available implementations and verified testers it has, and what the definition is.

How to get involved with Wikifunctions

• Subscribe to our mailing list to get weekly updates from our product lead Denny Vrandecic. Read past newsletters on Meta-wiki!
• Join our Telegram community or our IRC channel (mirrored) for in-depth conversations on Wikifunctions
• Follow us on Twitter for team and product updates
• Email me (avardhana@wikimedia.org) if you’re interested in product testing

Thank you to Nicolas Ayoub, Amin Al Hazwani, Sudhanshu Gautam, Genoveva Galarza Heredero, David Martin, Nick Wilson, and Denny Vrandečić for their contributions!

Originally published at https://design.wikimedia.org on March 14, 2022.

Designing Wikifunctions, brick by brick was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

Prologue

On the discussion page about Nellie Bly’s article, communication isn’t the easiest in normal circumstances. You have to hunt for, and then decipher, little clues scattered around the page to initiate a conversation, and then navigate a confusing interface with seemingly-arcane glyphs to orient yourself in the conversation, all while trying to write what you actually came to discuss. A few months back, I wanted to add a photo of The Girl Puzzle, a monument which is located on Roosevelt Island, New York that memorializes Nellie Bly. I think that this image will show how revered she still is for her journalistic work and, since there are very few images of her, it is a nice representation of her likeness. I walk to the monument on the north edge of this island situated within the center of Manhattan, take a photo and prepare to post it on Wikimedia Commons so that it can be linked and posted on the Wikipedia article.

This is where I got stuck. Am I allowed to post this? Are there copyright issues? I visited the discussion page on the English Wikipedia and tried to start a new topic. It’s not easy, but ultimately I realized that I needed to click the “add new section” tab in the interface and ask about inserting the image into the article. After some conversation I wrote “I can take a photo, but would that make it alright to post it or do I need to get approval from the artist?” I didn’t know the rules around image rights. I magically got responses from other contributors, but I didn’t even notice that they wrote back to me until a month later when I checked the page. Eventually, I’m guided to a number of different policy pages, and from these I figured out that the best way to get the image up is to get an appropriately licensed photo from the monument’s owners, and then go back to the article to post the image and its associated information.

On its own, figuring out the best way to get and include an image on the article was painful, but figuring that out while trying to navigate the obscure conventions for using the discussion interface, now that was excruciating. On-wiki communication has gotten a lot better for me personally over time because I have at my disposal colleagues and long-term contributors who I actually know and can help me to learn. That’s great, but for others it has not gone so well; potential contributors just give up. They just stop trying. By their thousands. The Editing Team set out to make moments like this less painful, so that as a contributor, you can just focus on the point that you are trying to get across and improve the encyclopedia.

Making easier-to-use tools

In the first installment in this series, I shared how we in the Editing Team conducted research and collaborated with community members to identify a design strategy for the project. The heart of our strategy involves us diving deeply into the asynchronous collaboration experience. To do this, our approach is to first make it easier for contributors to use the tools and then, when we have confirmed that the tools are useful, follow that up by making it easier for folks to discover them in the first place.

We undertook a virtual “listening tour”, where we heard from contributors with a variety of experience levels on Talk Pages share what their interactions there were like. From this, we identified four initial opportunities to improve the Talk Page experience that I will walk you through today. Our hypothesis was that addressing these problems would mean that contributors have a more positive and productive experience.

Problem 1: There’s no clear way to engage

If you look at this discussion page here, you can see that it’s very challenging to identify what to even click to reply to these discussions.

You need to click the “edit” button to reply. Not obvious. What you’d expect here is a clear call to action, something that says “reply” on it that you can press and initiate the workflow for writing back to someone in a conversation. So that’s what we did. We made a minimal, non-invasive change, which was to add inline links with a more obvious word: “Reply”.

Though this one change seems simple, it was hugely technically challenging to implement within the constraints that the Wikipedia software gives us. Previous approaches had been hamstrung, either by making trivial systems that don’t work for all the hundreds of language communities that are already part of the Wikimedia movement, or by trying to replace discussion pages and all the functionalities that our communities have built into them over the decades that Wikimedia has been going.

As Principal Engineer Ed Sanders puts it, “Existing wikitext talk pages are completely unstructured. That means that all the information is thrown together in what we’ve been calling a wikitext soup. And while humans may be able to parse it out by looking at the structure of the conversation, it’s not easy for the software to do this.” For us to extract useful information from the page we needed to build a new tool which we called the Comment Parser.

The Comment Parser works by breaking up that flat page into structured data. There are lots of edge cases to consider here: hundreds of timestamp formats, multi-line comments with mixed indentation, strange username links. By rolling this out slowly to different communities we’ve been able to identify more edge cases and improve the accuracy of the parser. It helps us to know where comments start and end so we can add the individual reply links to conversations. It also allows us to insert your reply in the correct place with the correct amount of indentation.

For those who had used the Structured Discussions system, a key difference is this implementation approach is that, according to Ed, “all the information is extracted from the page, and not the tool, so we can apply this insight to edits made with the full page editor.” The Editing Team’s Product Manager noted that “a key requirement when we spoke to the community was compatibility with existing workflows, so being agnostic about which editor was used means we can do things like show notifications regardless of what editing interface people are using to trigger them.”

Problem 2: The interface for writing is confusing

With the reply button, you now have a nice entry point into the workflow. That said, actually writing a reply was still confusing for a handful of reasons, including the fact that there’s manual threading, the ability to edit other people’s comments by accident, edit-centric language, and you needed to know the wikitext conventions for signing and dating your posts.

The same problems also applied to starting new topics:

We addressed this by making it possible to reply and sign your name automatically. This alone would have made a huge difference, because it’s frankly a real pain to have to write “ ~~~~ “ after every single post. We built upon this moment to reduce the complexity of mentioning another Wikipedian, formatting your response, and identifying your comment in the discussion. Again, we made this change inline and more contextual, so that it just felt like an organic conversation experience.

For adding new topics we made a similar, more obvious workflow:

Problem 3: Missing Notifications

As a contributor, you were now in the situation where you could start and reply to topics, but in terms of getting notified if someone replied to you, well you got nothing, unless that person knew to use one of the many user-generated templates to @-mention you.

In fact it was even worse than nothing — you would sometimes get a small amount of notifications based on a set of rules so confusing that you’d have no idea if the other person is getting notified. So, we fixed this with a series of iterations on notifications. We created topic subscriptions, so you can choose if and how you want to be informed about activity within the conversation that you are in. If you’ve used the Structured Discussions tool, you’ll be familiar with these, but you will now get notifications for normal talk page edits, even if the other contributor isn’t using our tools.

Once we validated the effectiveness of that feature, we heard from contributors that there are just some conversations that they want to be automatically added to, so that’s what we’ve been working on recently. We have the technology in place and are still checking with test users to ensure that the tools aren’t over-communicating to contributors in such a way that it feels noisy and unwelcome.

Problem 4: Talk pages look like articles.

In English there’s a metaphorical idiom of “ the elephant in the room,” which means that there’s something very important that needs to be discussed but it’s probably going to be a little awkward to talk about it. For talk pages, that elephant is the fact that article pages look very similar to talk pages, and it confuses people who are new to the concept of a talk page. Headings are the same, the table of contents is the same, and there’s no other indication that this is a place where people can have conversations. This is where we are now. We are beginning to explore ways to make it more obvious what this page is all about by emphasizing elements on the page so that you can easily see what conversations are active, who is talking in them, and how you can get involved in them. This will be addressed in a future release.

Feedback

We’ve heard a lot of positive feedback about these improvements. Here are some nice quotes:

“…I like the fact that it’s very stripped down and doesn’t feel data intense… The colors are pleasing to the eye. Simple to navigate interface. Overall, it works the way I’d expect it to.”

- Usability Test Participant

“I think this is the best new feature since years and finally an approach to simplify discussion pages without breaking the workflow of experienced users. So thank you very much for your effort.”

- User:Hgzh, German Wikipedia

This feedback is crucial to the success of these tools. Peter explained that “As a fundamentally social and community-driven project, conversation is a core part of Wikipedia. It is a core practice that shapes Wikipedia’s policies, culture and ultimately, the knowledge that is and is NOT represented within it.”

I want testers and skeptics to try these tools out so that I can refine and if necessary rethink the design proposal. What’s particularly great about having a beta release channel is that it offers an opportunity to get things wrong and then quickly throw up a new proposal for review. Try out the tools yourself by going into Beta or global preferences and turning on Discussions Tools. Let us know what you think, it’s a discussion, afterall.

Epilogue

Through the help of the Roosevelt Island Operating Corporation, I was successfully able to add the image of The Girl Puzzle to Wikimedia Commons and Wikipedia.

Thanks to: Ed Sanders, Sherry Snyder, Peter Pelberg, James Forrester, Bethany Gerdemann, David Lynch, David Chan, Bartosz Dziewoński, Val Puffet-Michel and Esther Akinloose Unless otherwise credited, all images in this post are licensed under CC-BY-SA-4.0.

Originally published at https://design.wikimedia.org on February 8, 2022.