At Wellcome’s Data team we’ve worked to find new ways of facilitating data sharing between funders of scientific research. We believe better data sharing will help unlock new insight into the research landscape — particularly around diversity and research culture.

We’ve developed the Funder Data Platform to be a secure space for data sharing and analysis with partners. Organisations can upload and grant access to data for the purpose of collaborative projects. Researchers can then use analysis tools like Jupyter notebooks in a secure, closed environment. Once a project is completed, aggregate insights can be approved to leave the platform.

Creating a product for data sharing carries inherent risk. Due to this sensitivity, early work on the product was mostly driven by business and legal needs. Wellcome initially funded a small team of two developers to explore what was technically feasible. After the technical prototype we brought in a full product team with design, user research and delivery management expertise to develop the platform further.

The Funder Data Platform is now live, though with closed access, with its first project, Criteria which involves global funders including Welcome, Novo Nordisk, the Michael Smith Foundation and India Alliance. The goal of Criteria is to look at factors that explain differences in how women are funded compared to men, using data about which researchers have and have not funded.

The Funder Data Platform and Criteria project have been developed to support the Research on Research Institute and their mission to improve how we fund, practice, evaluate and communicate research.

Taking the Funder Data Platform from technical prototype to a product that meets user needs and satisfies legal terms and conditions required us to evolve our ways of working. Here are some of our lessons learned from developing the platform.

Focusing on user needs and being transparent with business stakeholders

As we’ve built the platform we’ve had to balance user, business and legal needs. We realised after the technical prototype was created that we had to separate these needs from each other to create focus in the team and with stakeholders.

Each quarter we set explicit OKRs (objectives and key results) related to specific flows that help users achieve a goal in the product, such as setting up a project, having the right tools for data analysis in the workspace or exporting results from the platform.

We brought project and legal stakeholders in as collaborators at design phases to help us understand and map any constraints. Working in this way helped us get the right feedback from experts early, which then gave us the ability and space to design flows that worked for users. Though the platform was not in production with active users, we ran regular monthly usability sessions as designed and developed each section of the platform. This helped us to stay focused and prioritised on user needs.

Breaking down the service into smaller flows and reducing complexity

With a product that has many legal complexities and where the team is working from discovery, we needed to be very clear about what we were working on when, and why.

Co-designing flows and mapping with users and stakeholders meant that we got better feedback than if we had just a conversation, or only decided to share once we’d finished designing and building the product feature.

We worked on setting objectives together as a team so we understood what we wanted to achieve before we got started on the work. We also shortened our operating rhythm so there were more opportunities for us to parts of the team like design and engineering to collaborate, feedback and adjust plans if needed.

Being clear about user roles and understanding the legal intricacies of building a data sharing product

With the Funder Data Platform we’ve tried to make processes related to data sharing that are inherently complex, simpler for users. We’ve achieved this partly by having clear user roles and boundaries around which users can upload and grant access to data, and which others can join projects and work on data in our closed environment.

We also spent time with our legal colleagues mapping out all of the legal checkpoints in the product to understand where we needed to add friction in the platform.

Overall we tried to create a sustainable working culture in the team by being clear that one platform won’t solve all the challenges with data sharing. We hope that the Funder Data Platform will go some way to breaking down barriers to collaboration between funders of research so we can use data to improve research culture and the research landscape.

The Funder Data Platform: building tools for better data sharing in science funding was originally published in Wellcome Data on Medium, where people are continuing the conversation by highlighting and responding to this story.

We focused on solving this problem in collaboration with our internal Insights and Analysis team for Wellcome and with partner organisations before deciding to release Reach more widely.

We found that evaluation teams wanted tools to help them measure the influence academic research was having on policy making institutions. We noticed that it is often challenging to track how scientific evidence makes its way into policy making. Institutions like the UK Government and the World Health Organisation have hundreds of thousands of policy documents available — it’s a heavily manual task to search through them to find evidence of our funded research.

At Wellcome we have some established methods for collecting evidence of policy influence from our funded research such as end of scheme reporting and via word of mouth. Through these methods we found great examples of how funded research was being put into policy and practice by government and global health organisations.

One example is from Kenya. The KEMRI Research Programme — a collaboration between the Kenyan Medical Research Institute, Wellcome and Oxford University launched a research programme to improve maternal health in 2005. Their research was cited in the World Health Organisation and with advocacy efforts from the KEMRI team influenced the development of new Kenyan national guidelines of paediatric care.

In Wellcome Data Labs we wanted to build a tool that would aid the discovery of evidence based policy making and be a step in the process of assessing research influence for evaluators, researchers and funding institutions.

Reach searches through hundreds of thousands of publicly available documents from the World Health Organisation, UK Government, UK Parliament, Medecins San Frontieres, NICE and UNICEF to find citations of scientific research publications. Users also have the ability to search through the corpus of policy documents from the global health organisations that we have collated.

In Wellcome Data Labs we have a commitment to openness, open data and transparency. Our GitHub repository is open and available to view. We get our database of published scientific research from European PubMed Central and we scrape policy documents only from openly available global health organisations and governments.

Building Reach has been a multidisciplinary effort from the team. Data scientists built the initial machine learning text parsing model and engineers built data pipelines that run the product. As Reach is based upon a machine learning model we’ve done a lot of work internally as a team on thinking about ethics and unintended consequences of releasing the product into the world. You can read more about our algorithmic fairness review here.

We ran user research sessions with people from funding, research and policy spheres as we developed Reach. As a technical team this increased our awareness of the fact that policy making is created in a myriad of ways and is influenced by a mixture of advocacy, evidence and timing.

We know that citations do not tell the whole story of how policy has been created but we hope that Reach becomes a starting point for people wanting to track how scientific research influences global health and governments.

Reach is in an ‘alpha’ stage, so this is our first iteration of solving this problem for users. This means that there could be bugs or results produced that are not 100% right. Our goal has been to release Reach so we can learn from usage and improve. We will be continually developing the product to provide best results.

To get in touch, participate in our research or provide any feedback after using Reach contact us here.

Introducing Reach: find and track research being put into action was originally published in Wellcome Data on Medium, where people are continuing the conversation by highlighting and responding to this story.

AI ethics is a hot topic in the tech industry. As a result of work by pioneering researchers like Joy Buolamwini, we’re learning more about how algorithms can discriminate against underrepresented groups, most alarmingly ethnic and gender minorities.

While AI and machine learning hold great promise, many are concerned about the impact new technology will have on society. Giants of the tech industry like Google and Facebook, government and academia are all trying to figure out how to ensure AI is used ethically and responsibly, with varying degrees of success.

Increasing the diversity of the workforce in tech is regularly suggested as a way of to mitigating against AI bias. Representation for people of colour and women in large tech companies are woeful. Less than 5% of employees in tech roles at Facebook and Google are black. At Twitter 30% of it’s leadership are women but only 15% of women are in engineering roles. The UK also has a problem with underrepresentation with only 8.5% of senior leaders being from ethnic minority backgrounds.

Increasing diversity is a long overdue first step, but not enough to mitigate against bias

Better diversity in tech is long overdue. However diversity alone will not be the solution to AI bias, ethics and irresponsible tech development.

We can’t expect people of colour and gender minorities to bear the weight of being the moral and ethical centre for tech companies. This expectation becomes a bigger problem if people from underrepresented backgrounds are only hired into entry or mid level positions with little power over company culture or the overall direction of product development.

The tech industry also needs to be careful of ‘diversity washing’. This would be a scenario where the overall diversity statistics of tech companies improve but people of colour and gender minorities not in senior roles and little change is made to non inclusive working cultures.

Grace Callcott from Projects By IF summarised the problem perfectly on Twitter:

We need a wider cultural change in the tech industry that supports openness, inclusivity and ethical product development

We need tech companies to think about the stress cases and unintended consequences of their products much earlier on.

Tech companies need to recognise the ethical trade offs they are making throughout the product development cycle and get better at being open about them. Unethical tech is rarely the result of one big decision but a series of small ones made over time.

I’ve worked in product teams for the last couple of years. It’s easy to see how technologists like data scientists and software engineers can become preoccupied with getting a product in a decent shape to ship over thinking about the long term consequences of their work.

We need to ensure that the creators of technology have the time and space to critically reflect on what they are building.

We also need to create a culture in the tech industry where this reflection is actively encouraged and people across specialisms and pay grades feel comfortable challenging ethical consequences of decisions.

We need to involve product leads, engineers and data scientists in the conversation about tech ethics

I’ve attended many good and thoughtful events about AI where mostly social scientists and non tech practitioners discuss ethics.

I’d love to see more software developers, data scientists, data engineers and product leads at these events, sharing their own experiences building products and understanding latest thinking in ethics.

Support communities and start-ups that are doing things differently

Finally, there’s an emerging group of organisations and communities working differently, embracing a culture of openness and beginning to change ways of working in tech.

At Wellcome Data Labs we’ve embedded an ethics and user researcher in our product team, helping us consider unintended consequences of our products and mitigate against them.

The energy company Bulb are one of the few startups I’ve seen take time to be open about the tech ethics and business trade offs they’re making as their company grows.

Projects by IF and Comuzi are using design principles and prototyping to explore practical ways of informing people about automated decision making; and YSYS are creating a community of diverse start-up founders, creatives and technologists.

We need to support organisations like these that are doing things differently and push for wider cultural change in tech so we see more inclusive, open and ethical tech development in the future.

Follow me on Twitter @DuhaneyDawn

A couple of months ago I left the Civil Service after two years working at the heart of data in government.

I’ve had the pleasure to be part of teams that are building capability, demonstrating the opportunity data science brings to operational and policy problems, and setting the strategy for how we use data across the public sector.

What’s working well — lots to be positive about

There’s a great group of motivated, skilled practitioners in the data science community changing the way data analysis happens in government. The community is a supportive peer network that crosses departmental boundaries in a way that’s not often seen in academia or industry.

DfT are using natural language processing to speed up the way consultation responses are analysed in policy teams, GDS are using neural networks to improve the discoverability of content on GOV.UK and UKHO are using deep learning to identify objects in satellite data.

The best projects I’ve seen have come from deliberate collaboration, a blend of specialisms and a flexible mindset. Contrary to what you may have heard, data scientists are not ‘unicorns’.

In data science teams people take on a variety of roles. Sometimes these roles are shared across multidisciplinary specialists and other times they are done by an individual alone.

• Data advocators — articulating how data science techniques can help solve messy problems and gaining support for projects
• Data sharing brokers — getting the data needed and managing relationships with custodians
• User researchers — ensuring that what is built meets user needs and requirements are correct
• Data cleaners — cleansing and structuring messy data sets
• Machine learning experts — building and deploying algorithms
• Data storytellers — communicating findings to decision makers and telling a compelling narrative about the problem they have solved

Many data scientists in government are part of the analytical professions. However, there is a difference between the culture of analysis and data science.

Traditional analysis is often a linear process — a decision maker requests a specific piece of analysis and the outcome is delivered. Data science is iterative; assumptions and models are tested multiple times and achieving success requires input from subject matter experts and end users.

Increasingly, data science teams in government are working in iterative ways. This approach

creates a better relationship between non technical and technical groups, a shared understanding of the problem to be solved and ultimately better products.

What could be better — improving culture and getting infrastructure fixed

Before the Civil Service I worked at the Open Data Institute. We regularly talked about data in terms of cultural challenges and I even co-wrote a report on how to bring about change for open data in government.

Though prepared with prior knowledge, I was genuinely surprised at how difficult it was to remove cultural barriers to data sharing, even when there was a shared goal that had real business and user value.

There is (rightly) a lot of focus on effective legal and regulatory frameworks for data. However, so much of the work needed to unlock better use of data in government is about changing mindsets, data advocacy and getting the basics of data infrastructure in place.

While there’s interest in artificial intelligence and emerging innovation, not as many want to fix the underlying data infrastructure challenges because, let’s face it, metadata will never be ‘sexy’.

Despite the challenge, there are teams looking at how to improve the foundations of technology and data in the public sector. MHCLG’s Local Digital Fund is a great example of working in the open and bringing communities together to #fixtheplumbing.

We also need to be mindful of creating silos where departments are building similar solutions to shared problems. Projects like RAP — designed to automate the process of producing official statistics — have been successful at creating change by actively creating cross-government communities, sharing progress and working in the open.

Many understand the value of data in the abstract, but few really see how the data they hold can help achieve their own objectives. There is still a case for increasing data literacy to create shared understanding of the ‘art of the possible’.

At the same time data specialists should feel comfortable de-mystifying and communicating the value of their work in a way that non technical groups can engage with.

What’s next?

The ‘data sector’ is a place where there’s an understanding of problems but few answers yet. We need to feel comfortable with uncertainty while testing interventions to tackle challenges.

When designing interventions, we need to make sure a diverse group of users are involved. This includes front line staff, data policy teams, analytical communities, heads of departments, data scientists and subject matter experts.

And, to avoid reinventing the wheel there needs to be continued cross government direction, support and community building.

Thanks to the brilliant data scientists, user researchers, product and delivery managers I’ve worked with in the last couple of years. You can follow me on Twitter @DuhaneyDawn.