<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Statistics Without Borders on Medium]]></title>
        <description><![CDATA[Stories by Statistics Without Borders on Medium]]></description>
        <link>https://medium.com/@statisticswithoutborders?source=rss-362bc366e67a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*v_38_Wb3_Q0t0rTF</url>
            <title>Stories by Statistics Without Borders on Medium</title>
            <link>https://medium.com/@statisticswithoutborders?source=rss-362bc366e67a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 22:00:05 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@statisticswithoutborders/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Designing and Developing a Centralized Analytics Dashboard]]></title>
            <link>https://medium.com/@statisticswithoutborders/designing-and-developing-a-centralized-analytics-dashboard-cc2fa2201324?source=rss-362bc366e67a------2</link>
            <guid isPermaLink="false">https://medium.com/p/cc2fa2201324</guid>
            <dc:creator><![CDATA[Statistics Without Borders]]></dc:creator>
            <pubDate>Tue, 20 Oct 2026 18:01:31 GMT</pubDate>
            <atom:updated>2026-10-20T18:07:14.020Z</atom:updated>
            <content:encoded><![CDATA[<p>Statistics Without Borders (SWB) worked with <a href="https://energy-base.org/">BASE Foundation</a>, founded as the Basel Agency for Sustainable Energy, to design and develop a centralized analytics dashboard for monitoring and evaluating program performance. The dashboard enables BASE to change how they engage with data and supports improved decision-making, strengthened operational oversight, and improved funder and partner communications. This blog post explores the purpose for and the outcome from BASE’s collaboration with SWB.</p><h4><strong>About BASE</strong></h4><p>BASE is a Swiss not-for-profit organization and specialized partner of the United Nations Environment Programme. BASE works across the world developing innovative and actionable financial strategies and market-driven solutions to encourage investment in climate change solutions. Combining expertise in technology, markets, economics, finance, and business development, BASE envisions a world where markets are transformed and climate change solutions are the norm.</p><p>Through its Your Virtual Cold Chain Assistant (Your VCCA) initiative, BASE supports cooling companies in low- and middle-income countries to offer solar-powered cold storage solutions to smallholder farmers. Farmers can pay a small fee per day per crate of fresh crops they store in a cold room. This business model reduces the need of upfront investments and offers farmers an alternative to liquidating their crops at the end of a market day.</p><p>To ensure that cold rooms are operated efficiently and crop spoilage is minimized, BASE developed Coldtivate, an open-source mobile application that helps cold room companies to digitalize the inventory. The app records cold room users, crop amounts checked in and out, and collected cooling fees. The Coldtivate app is available globally and is free to use to incentivize utilization at cold rooms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/302/1*LR3-muh4WYjgZu1YL9RU2A.png" /></figure><h4><strong>Project Aims and Scope</strong></h4><p>Sudharsana Kannan, SWB project and client manager, said BASE had “a large amount of operational data but needed a centralized dashboard to turn that data into clear, actionable insights, particularly for their non-technical team members.”</p><p>Similarly, Olusegun Ewemooje, SWB statistical consultant, said BASE was generating a growing volume of operational, marketplace, and impact data through its Your VCCA initiative and the Coldtivate application. However, this information was spread across multiple datasets, queries, and reporting systems, making it “difficult to obtain a clear and consistent picture of program performance. As a result, tracking key performance indicators, measuring impact, and supporting timely decision-making became increasingly challenging.”</p><p>It was important to address these issues because the project directly supports smallholder farmers by providing cold-storage solutions that help to reduce postharvest losses, increase incomes, and improve livelihoods, Ewemooje said. “Without a reliable and centralized reporting system, it was difficult to accurately monitor cold-room utilization, user engagement, marketplace activity, environmental benefits, and farmer outcomes across different countries. Having access to trusted and timely information was critical for strategic planning, reporting to funders and partners, demonstrating impact, and ensuring that decisions were based on evidence rather than assumptions.”</p><p>In turn, the SWB volunteers worked with BASE to design and develop a comprehensive reporting and analytics solution using Google’s Looker Studio, Ewemooje said. This work included integrating and organizing data from multiple datasets, conducting exploratory data analysis, cleaning and validating data sources, reviewing data quality, and developing a multi-page dashboard tailored to different stakeholder needs. The SWB volunteers also assessed dashboard technologies and recommended approaches that would provide sustainable and scalable reporting capabilities.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*KnDJQoljXMQxj87qYMCrKg.png" /></figure><h4><strong>Challenges Faced</strong></h4><p>The SWB volunteers encountered several challenges during the course of this project.</p><p>“We faced complex SQL logic, inconsistencies across datasets, evolving requirements, data quality issues, and database connectivity challenges,” Kannan said.</p><p>What’s more, the SWB volunteers had to navigate tight delivery timelines and capacity constraints that limited implementation of some requested enhancements, Ewemooje said.</p><p>To tackle these challenges, the SWB volunteers<strong> </strong>“divided the work into structured phases, validated queries and metrics, worked closely with the client to clarify business logic, and continuously tested and refined the dashboard,” Kannan said.</p><p>“The team conducted extensive exploratory data analysis and validation exercises to identify data inconsistencies and quality concerns,” Ewemooje said. “Data sources were cleaned, standardized, and reorganized before they were incorporated into the dashboard environment.”</p><p>In addition, the development of the dashboard was guided by an approved Miro board design framework, Ewemooje said. This design framework helped to maintain consistency, improve communication among stakeholders, and ensure that the final delivery met BASE’s reporting needs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Gq-ooUkKVp76W1bqCIk8qQ.png" /></figure><h4><strong>Project Outcome</strong></h4><p>This SWB project resulted in a centralized dashboard that makes Coldtivate<strong> </strong>operational data easier to understand and provides a foundation for ongoing reporting and analysis, Kannan said.</p><p>Further, the dashboard enables BASE to monitor cold-room utilization and produce movement in near real time, Ewemooje said. BASE can use the dashboard to track user engagement and company activities across multiple countries and to measure key impact indicators, including beneficiary reach, postharvest loss reduction, and revenue improvements. BASE can also use the dashboard to strengthen operational oversight, to support evidence-based decision-making, and to generate consistent reports for funders, partners, and management teams.</p><p>Beyond the dashboard itself, this SWB project also produced optimized database queries, documented metric definitions, and an evaluation framework, Ewemooje said.</p><p>Jina Yazdanpanah, technical product manager at BASE, said, “Prior to this engagement, accessing data collected through our Coldtivate mobile application required non-technical team members to submit individual data requests to the technical team who would then manually prepare and export the relevant dataset.”</p><p>This process created a persistent bottleneck, Yazdanpanah said, as technical staff spent a disproportionate amount of time on ad hoc data pulls instead of higher-value work and non-technical colleagues were left with limited, fragmented visibility into platform activity. Coverage was often incomplete, turnaround times varied, and the lack of self-serve access meant that data rarely informed real-time decision-making during partner calls or site reviews.</p><p>In contrast, with SWB’s centralized dashboard, BASE has been able to change how it engages with data, Yazdanpanah said. All key platform metrics are now consolidated in a single, always-accessible interface that any team member can navigate independently.</p><p>“The practical impact is immediate,” Yazdanpanah said. “For example, before a check-in call with a piloting partner, we can now review a full picture of that site’s operations at a glance. During calls, we can dynamically filter the dashboard in real time — drilling into specific date ranges, sites, or activity types as topics arise — which has elevated the quality of our troubleshooting and support conversations considerably.”</p><p>The dashboard has also reduced the barrier to data engagement across the BASE team, Yazdanpanah said. Non-technical colleagues are now empowered to run their own analyses, draw independent conclusions, and bring evidence-based perspectives into strategic discussions without having to rely on technical intermediaries.</p><h4><strong>Key Takeaways</strong></h4><p>Regarding the experience of working on this SWB project, Ewemooje said, “One of the most rewarding aspects of the project was transforming a complex collection of operational, marketplace, and impact data into a single platform that clearly communicates the story behind the numbers. It was especially satisfying to see how technical activities such as data validation, query optimization, and dashboard design could directly contribute to improving the lives of smallholder farmers.”</p><p>Of the collaboration with SWB,<strong> </strong>Yazdanpanah said, “The SWB team brought a combination of technical rigor and genuine curiosity about our work. They took the time to understand the complex and comprehensive operational context behind our data. The collaboration was well-structured, communicative, and delivered on its commitments.”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cc2fa2201324" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Determining Installed Solar Cooker Capacity]]></title>
            <link>https://medium.com/@statisticswithoutborders/determining-installed-solar-cooker-capacity-02f17d6bee60?source=rss-362bc366e67a------2</link>
            <guid isPermaLink="false">https://medium.com/p/02f17d6bee60</guid>
            <dc:creator><![CDATA[Statistics Without Borders]]></dc:creator>
            <pubDate>Fri, 21 Aug 2026 19:17:07 GMT</pubDate>
            <atom:updated>2026-08-21T19:17:07.878Z</atom:updated>
            <content:encoded><![CDATA[<p>Statistics Without Borders (SWB) worked with <a href="https://www.solarcookers.org/">Solar Cookers International</a> (SCI) to develop more robust cooking-capacity metrics. These metrics improve SCI’s insight into potential opportunities to expand solar-cooking distribution, providing environmental benefits and economic empowerment. This blog post explores SCI’s goals for and the outcome from its collaboration with SWB.</p><h4><strong>About SCI</strong></h4><p>SCI is a global nonprofit that promotes solar cooking for families worldwide through advocacy, research, and capacity building. The solar cooker method provides affordable, clean, and sustainable energy for cooking. In encouraging climate-friendly solar cooking, SCI aims to positively impact human health, the environment, women’s empowerment, and economic well-being.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/853/1*L_ER1wrWJtJvU0XdRL9C7A.png" /></figure><h4><strong>Project Goals and Scope</strong></h4><p>SCI maintains a global solar cooking <a href="https://www.solarcookers.org/map-of-solar-cookers/">map</a> that displays the distribution of solar cookers worldwide. From its collaboration with SWB, SCI wanted to know the worldwide installed solar cooker capacity in watts per type of solar cooker that was represented in the SCI map. The results of this investigation could then illustrate which areas of the world are currently underserved and could gain the most benefit from increased capacity in future.</p><p>Frankie Sanjana, SWB project and client manager, said SCI “knew roughly where solar cookers had been installed around the world, but they could not say how much cooking capacity that represented.”</p><p>The International Renewable Energy Agency had asked SCI for exactly this figure, in watts and by country, and SCI did not have a way to produce it, Sanjana explained. In turn, the SWB volunteers aggregated SCI’s underlying data across different types of solar cookers to develop a metric for the installed solar cooking capacity in watts by country. This was built in a structure that SCI can update each year. The volunteers also developed a set of maps and charts to tell the story, including how capacity has changed over time.</p><p>“We used exploratory tools such as mapping, frequencies, and graphical tools (e.g., histograms, density plots), as well as inferential techniques, including cluster analysis of nations by cooker frequency, cross-correlation coefficients for cooker frequency and CO2 emissions, and estimation of relationships between changes in cooker counts and wattage with national GDP,” Holmes Finch, SWB statistical consultant, said. The volunteers also examined relationships between cooker frequency counts and solar radiation exposure by country.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*giEtboDqAUEBHk9ledrANw.png" /><figcaption>Photo courtesy of Eco Impact Hub CBO</figcaption></figure><h4><strong>Difficulties Encountered</strong></h4><p>The SWB volunteers faced some challenges when working on this project.</p><p>One challenge was that SCI’s underlying data was incomplete as some installations undercounted because reporting is voluntary and some installations overcounted because cookers reach the end of their life but still remain in the records, Sanjana explained.</p><p>In addition, Sanjana said, “Cooker types were often unknown or mixed, wattage was measured against different standards in different projects, and since the data was incomplete, we searched online registries to ensure as complete a dataset as possible with publicly available information.”</p><p>Further, Finch said, “The team had to identify complementary datasets, including CO2 emissions, solar radiation, population, and national GDP.”</p><p>Of the whole process, Sanjana said, “We were explicit about every assumption we made and why. We went through the registries carefully to enrich the data while checking for duplication. And we produced both a straightforward snapshot of current capacity and a more detailed view over time, so the analysis works for a range of audiences.”</p><h4><strong>The Outcome</strong></h4><p>“As SWB began analyzing the data, they identified unexpected spikes and declines in solar cooker deployments,” Sara Rosen, SCI program and operations manager, said. SWB and SCI discovered that many of these dramatic shifts were tied to carbon finance projects. SCI also realized that a significant number of carbon-funded initiatives had not been captured in its own database.</p><p>“What emerged was striking,” Rosen said. “Wherever carbon finance had been introduced, it had the power to create entirely new solar cooking ecosystems, often deploying thousands of cookers at a time in places where solar cooking previously had little presence. It can help establish lasting markets and normalize clean cooking technologies within communities.”</p><p>This insight has fundamentally changed how SCI thinks about scaling impact, Rosen said. “Today, we see carbon finance as one of the most promising pathways for accelerating the transition away from polluting cooking fuels. It offers a way to introduce solar cooking at meaningful scale while simultaneously creating local jobs, reducing household fuel costs, decreasing time spent collecting firewood, limiting deforestation, and avoiding greenhouse gas emissions.”</p><p>As traditional development funding is becoming increasingly constrained, this shift in perspective has been particularly important, Rosen said. “Rather than relying solely on donor-funded interventions, we are now actively exploring how carbon finance can help build self-sustaining solar cooking markets in multiple countries.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/718/1*SZM0YofeoZm8Rkbco9gj4Q.png" /><figcaption>Photo courtesy of Eco Impact Hub CBO</figcaption></figure><h4><strong>Final Thoughts</strong></h4><p>Regarding the experience of working on this SWB project, Finch said, “SCI was able to take our work and immediately apply it in-house and as a tool to highlight their work to donors. It was rewarding to be able to help SCI help people across the world by providing clean, safe cookers that replace the open fires that many of them were using to prepare their food previously.”</p><p>Of the collaboration with SWB,<strong> </strong>Rosen said, “The results of the project were valuable, but what really stood out was the experience of working with the SWB team. They struck an excellent balance between being professional and approachable, reliable and flexible. Most importantly, they were genuinely invested in the work.”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=02f17d6bee60" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Helping to Advance Shared Transport Through Data Analysis]]></title>
            <link>https://medium.com/@statisticswithoutborders/helping-to-advance-shared-transport-through-data-analysis-bab05a751370?source=rss-362bc366e67a------2</link>
            <guid isPermaLink="false">https://medium.com/p/bab05a751370</guid>
            <dc:creator><![CDATA[Statistics Without Borders]]></dc:creator>
            <pubDate>Fri, 24 Apr 2026 16:40:45 GMT</pubDate>
            <atom:updated>2026-04-24T16:40:45.030Z</atom:updated>
            <content:encoded><![CDATA[<p>Statistics Without Borders (SWB) worked with <a href="https://www.como.org.uk/">Collaborative Mobility UK</a> (CoMoUK) to analyze the organization’s 2024 Car Club Survey data. This blog post explores CoMoUK’s objectives for and the results from their collaboration with SWB.</p><h4><strong>About CoMoUK</strong></h4><p>CoMoUK is the United Kingdom’s national charity for shared transport. This organization is committed to advancing shared transport for the social, economic, and environmental benefits of the public. Shared transport refers to schemes, such as car clubs and bike share, where people can use a mode of transport without having to own it. CoMoUK supports shared modes including car share, bike share, e-scooter share, lift share, and mobility hubs. To further these public benefits, CoMoUK works collaboratively with national, regional, transport, and local authorities, as well as the private sector, and promotes shared transport through research, operator accreditation, stakeholder engagement, policy work, and advocacy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WX-DMWegs8HtFr7RBu7_cg.png" /></figure><h4><strong>Project Objectives and Scope</strong></h4><p>CoMoUK sought outside analysis of its 2024 Car Club Survey data. Eren Kuru, SWB statistical consultant, explained that the large dataset collected through the car club survey included a mix of quantitative and qualitative responses and geographically distributed data across the United Kingdom. CoMoUK was faced with transforming this data into clear, actionable insights to inform how shared mobility solutions are developed and scaled across the United Kingdom.</p><p>“Without a structured statistical approach, it was difficult to fully understand user behavior, identify meaningful patterns, and translate the findings into evidence that could support policy development and operational decision-making,” Kuru said.</p><p>In turn, SWB was brought in to analyze the car club users’ usage patterns and satisfaction levels, as well as explore key events, such as adoption and frequency of use. SWB was also tasked with creating user personas by segmenting the users based on demographics to determine which groups are using this service and their reasons for doing so. Lastly, SWB was asked to analyze the car club survey data by location to help find patterns specific to different regions.</p><p>“The scope of SWB’s work covered the full analytical pipeline, from data preparation to final reporting,” Kuru said.</p><p>The SWB volunteers cleaned and structured the survey dataset, conducted descriptive and cross-tabulation analyses, and performed demographic and geographic segmentation to better understand user behavior across different regions, Kuru explained. The SWB volunteers also analyzed open-ended responses to capture qualitative insights, and they developed visualizations — including geographic mapping — to support interpretation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GcJEnAWMRvWzWgWhPTnCUg.png" /></figure><h4><strong>The Challenges</strong></h4><p>When working on this project, the SWB volunteers encountered several challenges.</p><p>Gautam Gogoi, SWB project and client manager, said, “We had to analyze the required data and come up with insights. It required data handling skills and data comprehension skills.”</p><p>Further, Kuru said, “The dataset was complex, combining quantitative survey data, qualitative responses, and geographic information, which required careful integration and consistent structuring.”</p><p>Another challenge included deriving and validating latitude and longitude information for the geographic analysis to enable accurate mapping and spatial insights, Kuru said. In addition, ensuring data quality more broadly, especially when handling missing or inconsistent responses, also posed a challenge.</p><p>To address these challenges, the SWB volunteers implemented a structured and collaborative workflow across the team, Kuru explained. The SWB volunteers carefully cleaned and standardized the dataset to ensure consistency and reliability for analysis. The SWB team applied quality assurance processes at each stage to review outputs, resolve inconsistencies, and ensure that all findings met professional statistical standards. Lastly, the SWB volunteers made sure to clearly allocate tasks and have regular communication to coordinate the volunteers’ work and maintain alignment throughout the project.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/898/1*nhA5Rah1M1grAGvIiwmBhw.png" /></figure><h4><strong>The Results</strong></h4><p>The final deliverables of this SWB project included detailed statistical analyses, user persona segmentation, thematic insights from open-ended responses, and advanced geographic visualizations using postcode-level data, Kuru said. These outputs enabled CoMoUK to better understand user behavior, regional demand patterns, service satisfaction, and barriers to access.</p><p>What’s more, these findings can support policy development, improve service design, and guide the strategic expansion of shared mobility solutions, Kuru said. The results also contribute to broader sustainability goals, particularly by reinforcing the role of car clubs in reducing private car dependency and lowering CO₂ emissions.</p><p>“Overall, the project enabled the client to make more informed, data-driven decisions with real potential for environmental and societal impact,” Kuru said.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S9q3IIk2sr5eBP0mtP-UkA.png" /></figure><h4><strong>Final Reflections</strong></h4><p>Regarding the experience of working on this SWB project, Gogoi said, “This was my first project with SWB, and I really enjoyed being part of the entire process and seeing it through to successful completion.”</p><p>Similarly, Kuru said, “The most memorable aspect of this project was seeing how the data reflected real human experiences and societal impact. Beyond the quantitative findings, the qualitative responses highlighted how car clubs support people in very practical and meaningful ways, including access to healthcare, hospital visits, and everyday needs for more vulnerable groups.”</p><p>Of the collaboration with SWB, Patrick Utz, head of consultancy and research at CoMoUK, said, <strong>“</strong>CoMoUK benefitted from additional analysis of the survey data that we collect from car sharing users in the UK. In many ways, SWB’s analysis confirmed our results that users of car sharing (also known as car clubs) travel more environmentally friendly and save money through reduced car travel and ownership. SWB also helped us think about different types of car-sharing users through clustering responses, using methods that we had not deployed before. Ultimately, SWB’s feedback on our survey helped us refine our questionnaire to improve data collection with subsequent rounds of surveys.”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bab05a751370" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Supporting Urban Sanitation Through Dashboard Development]]></title>
            <link>https://medium.com/@statisticswithoutborders/supporting-urban-sanitation-through-dashboard-development-b0426599edcc?source=rss-362bc366e67a------2</link>
            <guid isPermaLink="false">https://medium.com/p/b0426599edcc</guid>
            <dc:creator><![CDATA[Statistics Without Borders]]></dc:creator>
            <pubDate>Tue, 31 Mar 2026 19:12:41 GMT</pubDate>
            <atom:updated>2026-03-31T19:13:43.724Z</atom:updated>
            <content:encoded><![CDATA[<p>Statistics Without Borders (SWB) collaborated with <a href="https://www.oursoil.org/">SOIL </a>to develop a data monitoring dashboard. This blog post explores SOIL’s goals for this endeavor and the final result produced from the SWB collaboration.</p><h4><strong>About SOIL</strong></h4><p>SOIL is a nonprofit research and development organization in Haiti that works to design, test, and implement sustainable and equitable solutions to the sanitation crisis. SOIL’s mission is to ensure that safe and sustainable sanitation services are accessible to families in urban Haiti, creating a model for urban sanitation that is replicable nationally and throughout the world.</p><p>To support full-cycle sanitation services, SOIL provides toilets and removes waste from communities, and it also treats the collected waste and transforms it into compost. The compost is then sold to support agriculture, reforestation, and climate change mitigation in Haiti.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/833/1*trDQyYOdqQzCm9DnqRo4CQ.png" /></figure><h4><strong>The Issues at Hand</strong></h4><p>SOIL was experiencing data quality challenges in their operational datasets, Jael Garrido, SWB statistical consultant, said. “They collect household sanitation and demographic data through field agents, but issues such as missing values, inconsistent entries, and data input errors made it difficult to effectively monitor program performance.”</p><p>Further, Balachandar Ramalingam, SWB project and client manager, said, “SOIL was managing critical program data across multiple sources, but without a centralized system to bring it all together.”</p><p>Addressing these challenges was important because SOIL relies on accurate data to manage service delivery, track operational performance, and make informed program decisions, Garrido said. In addition, improving data quality helps to ensure that insights derived from the data are reliable and actionable.</p><h4><strong>The Scope of Work</strong></h4><p>In turn, SOIL’s Research team asked SWB to develop a data-monitoring dashboard to alert the organization when issues occur. The SWB volunteers designed and implemented an end-to-end dashboard solution that unified SOIL’s data into a single, interactive platform, Ramalingam explained. This included data modeling, KPI definition, visualization design, and full documentation. The SWB volunteers also led a structured handoff and training session to ensure long-term sustainability and client ownership.</p><p>“The focus was on identifying key data quality indicators and creating a practical visualization tool that staff could use to monitor data issues on a regular basis,” Garrido said.</p><h4><strong>Challenges Faced Along the Way</strong></h4><p>The SWB volunteers faced several challenges throughout this project. One of these challenges was ensuring that the dashboard remained simple and sustainable, given SOIL’s limited internal analytical capacity and the need for a solution that could be easily maintained, Garrido said.</p><p>Another challenge, Ramalingam said, was that the SWB volunteers worked with evolving datasets and changing definitions of key metrics.</p><p>To tackle these challenges, the SWB volunteers took an iterative, collaborative approach by validating assumptions early, aligning on metric definitions, and building flexible logic to handle data variability, Ramalingam said. The SWB volunteers also focused on transparency, so that SOIL could clearly understand how the dashboard works and adapt it as the organization’s needs evolve.</p><p>“By prioritizing usability and clarity, the team ensured that the final Looker dashboard aligned with the client’s operational needs,” Garrido said.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7-6V4AokuJYymDsuiq4XVg.png" /></figure><h4><strong>Results of the Collaboration</strong></h4><p>This SWB project resulted in a dashboard that “significantly reduces manual reporting effort and provides a centralized, real-time view of key performance indicators,” Ramalingam said.</p><p>The dashboard enables SOIL to spend less time compiling data and more time acting on insights, thereby improving both efficiency and decision-making, Ramalingam explained. What’s more, SOIL now has a scalable, user-friendly tool that the organization can independently maintain and expand, supported by clear documentation and training.</p><h4><strong>Participants’ Final Thoughts</strong></h4><p>Of working on this SWB project, Ramalingam said, “Seeing the client quickly adopt the dashboard and gain confidence in using it was the most rewarding part. During the handoff, it was clear the tool had shifted from being a deliverable to something they could rely on in their daily work.”</p><p>Similarly, Garrido said, “One of the most rewarding aspects was seeing how a well-designed visualization tool could translate complex data quality issues into clear, actionable insights for program staff. It highlighted the practical impact that statistical thinking can have in real operational contexts.”</p><p>Regarding the SWB collaboration, Maya Lubeck-Schricker, Director of Research and Innovation at SOIL, said, “We now have a data quality assurance dashboard that continuously monitors for missingness and irregular patterns across several of our operational data collection processes. Rather than stumbling onto problems when it’s already too late, we can catch them early — while there’s still time to course-correct before the data is needed.”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b0426599edcc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Helping to Build Climate Resilience at the Community Level]]></title>
            <link>https://medium.com/@statisticswithoutborders/helping-to-build-climate-resilience-at-the-community-level-df852f2f835d?source=rss-362bc366e67a------2</link>
            <guid isPermaLink="false">https://medium.com/p/df852f2f835d</guid>
            <dc:creator><![CDATA[Statistics Without Borders]]></dc:creator>
            <pubDate>Wed, 04 Feb 2026 17:35:45 GMT</pubDate>
            <atom:updated>2026-02-04T17:35:45.482Z</atom:updated>
            <content:encoded><![CDATA[<p>Statistics Without Borders (SWB) collaborated with <a href="https://crc.place/">Climate Resilient Communities</a> to identify gaps in Toronto climate resilience data. The SWB volunteers were brought in for data classification and gap assessment for different climate resilience metrics. This blog post explores Climate Resilient Communities’ goals for this project and the final results produced from the SWB collaboration.</p><h4><strong>About Climate Resilient Communities</strong></h4><p>Climate Resilient Communities (formerly known as Toronto Just Climate Resilience) is a Canadian not-for-profit project working under Transition Toronto. Transition Toronto serves as the Toronto chapter of the global Transition Movement, which is an international network of communities working to address the pressing environmental and resource challenges facing our planet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YEK-bkrYy9JpwyGMeD4w9Q.png" /></figure><h4><strong>Minding the Gaps</strong></h4><p>Using a data-driven and community-centric approach, the Climate Resilient Communities team strives to enhance climate resilience and advance climate justice in vulnerable and marginalized urban communities. Through working with SWB, Climate Resilient Communities aimed to identify gaps in Toronto climate resilience data based on the consolidated climate resilience measurement framework that the Climate Resilient Communities team created. The SWB volunteers were tasked with data classification and gap assessment for different climate resilience metrics, classifying datasets into different domains and identifying data gaps where citizen-generated data could provide value.</p><h4><strong>The Requirements</strong></h4><p>Sylvia Wang, SWB project and client manager, said the issue facing<strong> </strong>Climate Resilient Communities was that it needed to “leverage a vast range of publicly available data spanning the natural environment, energy, built environment, and social and governance domains.”</p><p>Wang said specialized expertise was required to classify these data, and given the scale of the data, “more intelligent and efficient methods” were essential to achieve this goal. It was important to address this issue as only through understanding and classifying the available data assets of the Canadian government and communities were the SWB volunteers able to “identify the data assets, make use of them, and be aware of existing gaps,” Wang explained.</p><h4><strong>Tackling the Challenges</strong></h4><p>The SWB volunteers had to overcome several challenges throughout this project. One such challenge was that the SWB volunteers had “only a small set of expert-labeled data with limited domain coverage,” resulting in class imbalance and adversely impacting both model training and evaluation, Wang said.</p><p>Shalini Gopalkrishnan, SWB statistical consultant, said another challenging aspect of this project was “getting accurate, local, fine-grained data that matches what communities care about.” Additional challenges included translating raw data into formats, visualizations, and narratives that are “meaningful to community members who may not have technical training” and ensuring that the insights and tools led to actual change, such as policy and interventions.</p><p>To address these challenges, “the volunteers were amazing” and they thought outside the box, Gopalkrishnan said. The SWB volunteer group was broken into smaller teams, with each team using different ways to address the challenges of this project. One team used logistical regression while another did testing with generative model building on top of climateBERT, and the volunteers applied several proxy variables and used interesting ways of combining domain-specific large language models to get data, Gopalkrishnan explained.</p><p>Wang said the volunteers also contributed by “manually labeling data and researching underrepresented domains to create larger, more balanced samples.”</p><p>The SWB volunteers ultimately presented four models, including machine learning approaches and a large language model, by which to classify datasets into the relevant domains, with the best model achieving 87 percent accuracy, Wang said.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v1UtCgvy0XkEFWaGHKh4Ag.png" /></figure><h4><strong>Final thoughts</strong></h4><p>Wang said working on SWB projects often involves engaging with cutting-edge technology, which provides volunteers with valuable learning and research experiences and helps in finding solutions for real-world challenges. The SWB volunteers’ work on this particular project “explored potential classification methods and highlighted the future direction of leveraging commercial AI models for developing a climate data asset taxonomy.”</p><p>Wang noted that when the SWB volunteer team took over this project, large language models were still new, and few people had expertise in the area. Still, the SWB volunteers approached this project with curiosity and a willingness to experiment, applying well-established methods as they explored how to build large language models.</p><p>Regarding the experience of working on this SWB project, Wang said, “What I find most inspiring is the way volunteers from such diverse backgrounds — university students, researchers, and professionals — came together, united by their motivation and dedication.”</p><p>Of the collaboration with SWB, Helena Yu, strategic lead at Climate Resilient Communities, said, “What stands out most about the SWB volunteers is their genuine investment in our success. They bring creativity to every challenge, passion for humanitarian impact, and an unwavering dedication that has transformed how we approach our work.”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=df852f2f835d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Harnessing the Power of NLP to Drive Cultural Sustainability]]></title>
            <link>https://medium.com/@statisticswithoutborders/harnessing-the-power-of-nlp-to-drive-cultural-sustainability-390f3fd03a10?source=rss-362bc366e67a------2</link>
            <guid isPermaLink="false">https://medium.com/p/390f3fd03a10</guid>
            <dc:creator><![CDATA[Statistics Without Borders]]></dc:creator>
            <pubDate>Tue, 20 Jan 2026 17:11:15 GMT</pubDate>
            <atom:updated>2026-01-20T19:20:50.624Z</atom:updated>
            <content:encoded><![CDATA[<p>Statistics Without Borders (SWB) collaborated with <a href="https://cultureforclimate.scot/">Culture for Climate Scotland </a>to streamline the organization’s analysis of free-text survey answers using Natural Language Processing (NLP) techniques and the development of a web-based automated reporting tool. This blog post explores Culture for Climate Scotland’s goal with this project and the final outcomes produced from the SWB collaboration.</p><h4><strong>About Culture for Climate Scotland</strong></h4><p>Culture for Climate Scotland (formerly known as Creative Carbon Scotland) is a nonprofit organization at the intersection of culture and sustainability. This organization believes the role of the arts, screen, cultural, and creative industries is an essential contribution to the development of a more environmentally sustainable Scotland, and it provides services to more than 130 cultural organizations in Scotland to help reduce their carbon footprints.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Xg6RfKEKpremvHJiLXINtw.png" /></figure><h4><strong>The Issue</strong></h4><p>Each year, Culture for Climate Scotland sends out surveys (including multiple-choice and free-text-based questions) and generates sustainability reports (including analyses of carbon footprints and recommendations). The manual labor required to extract key information from free-text answers, however, became a significant bottleneck, limiting Culture for Climate Scotland’s ability to scale its impact.</p><p>Eric Oulster, SWB statistical consultant, explained that a client submits information about metrics related to their carbon emissions, as well as “some detailed written responses about their future plans to reduce emissions and their contributions to helping other organizations and people reduce emissions.” Culture for Climate Scotland then provides structured feedback to these organizations on how to manage their climate change policy, and this feedback is provided in a climate change report card.</p><p>“The responsibility for analyzing all the text data rested on one dedicated person, which made the process extremely time consuming,” Evgeniya Polyakova<strong>,</strong> SWB project and client manager, said.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ITd5CLhLPyriHCbCvP_eKA.png" /></figure><h4><strong>The Scope of Work</strong></h4><p>In turn, this project focused on two key areas: the analysis of free-text answers using NLP techniques and the development of a web-based automated reporting tool. Through the use of NLP, the SWB team sought to develop a robust solution to analyze and extract valuable insights from the free-text answers. With the web-based automated reporting tool, the SWB team aimed to integrate the NLP process, automate the reporting, and streamline Culture for Climate Scotland’s analysis workflow.</p><p>The use of large language models for this project caused Culture for Climate Scotland to be “understandably anxious about the carbon impact of using large language models, so they also asked us to compile a report to better understand the energy usage that is associated with running such a model,” Oulster said.</p><p>The SWB volunteers’ work on this project involved developing “a code-based system that took long-form responses and returned point-form notes about each organization” for Culture for Climate Scotland’s report-writing process, Oulster said.</p><p>To process the long-form responses, the SWB volunteers used an existing pre-trained large language model and “fine-tuned it to extract insights relevant to the survey goals,” Polyakova<strong> </strong>said. Based on the model’s output, the SWB volunteers used data visualization of the results to make the findings clearer and more accessible to Culture for Climate Scotland.</p><p>The SWB volunteers also helped to build a system to categorize different company-wide initiatives into one of several categories, allowing Culture for Climate Scotland to more easily track the general actions organizations were taking, Oulster said. In addition, the SWB volunteers provided a report detailing the approximate electricity usage required by their large language model-based solution to give Culture for Climate Scotland a better idea as to whether the energy costs were worth the time savings.</p><h4><strong>The Challenges</strong></h4><p>The SWB volunteers encountered several challenges throughout this project. “While we originally committed to trying smaller models that are more energy efficient, we ultimately didn’t see the performance we had wanted from these models when compared to larger parameter models,” Oulster said.</p><p>Similarly, Polyakova said, “The first and most difficult challenge was finding a model that could accurately summarize the long-text responses from the survey.” The goal was to identify a model that delivered high-quality outputs, was affordable to use, and caused minimal harm to the environment.</p><p>Polyakova explained that, to achieve this, several models were tested, and the prompts were adjusted multiple times until the SWB volunteers found the most reliable option. The SWB team worked closely together, meeting frequently, and staying in constant communication with Culture for Climate Scotland to review intermediate results. Ultimately, the Gemini model gave the best and most consistent results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jXBgCUMa-dfznoqOCWBuhw.png" /></figure><h4><strong>The Outcome</strong></h4><p>This project resulted in a ready-to-use Python script for processing text data, which Culture for Climate Scotland can now apply to future surveys, Polyakova said. “Using the model made it possible to analyze open-ended responses more efficiently, saving time and reducing manual work.”</p><p>Moreover, SWB’s report on electrical consumption “determined that relying on the model for summarization would consume less electricity than the background electricity cost of running the computer to compile the reports manually,” Oulster said. This finding helped to justify the model’s implementation from a carbon emissions perspective.</p><p>“Being able to give a reasonable estimate about how much energy the entire process consumed gave me a better understanding about the energy costs of the AI revolution and also gave me a better appreciation about the costs and benefits of automation from a climate-based perspective,” Oulster said.</p><p>Further, SWB’s work at building out categories for organizations’ different emission types also helped Culture for Climate Scotland develop internal statistics to look at what actions companies were taking to lower their carbon footprint, Oulster added.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4JJM11RS8SYnA2Ig1xdT3w.png" /></figure><h4><strong>The Takeaway</strong></h4><p>Describing the collaboration with SWB, Matthew Belsey, environmental reporting and data manager for Culture for Climate Scotland, said, “We really appreciated how the team listened to our concerns about the environmental impact of the work, taking steps to ensure the processes were efficient and producing a report to dig into the concerns about energy usage.”</p><p>Regarding the experience of working on this SWB project, Polyakova said, “I learned that the success of a project really depends on how well the team understands each other. When people are ready to listen, cooperate, and work together as a team, it makes a huge difference and often decides how successful the project will be.”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=390f3fd03a10" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Evaluating the Statistical Reliability of Asociación Civil Sustentar’s Cleanliness Quality Index]]></title>
            <link>https://medium.com/@statisticswithoutborders/evaluating-the-statistical-reliability-of-asociaci%C3%B3n-civil-sustentars-cleanliness-quality-index-7e87cdffa553?source=rss-362bc366e67a------2</link>
            <guid isPermaLink="false">https://medium.com/p/7e87cdffa553</guid>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[quantitative-research]]></category>
            <category><![CDATA[ecosystem]]></category>
            <category><![CDATA[environmental-impact]]></category>
            <category><![CDATA[social-impact]]></category>
            <dc:creator><![CDATA[Statistics Without Borders]]></dc:creator>
            <pubDate>Sat, 06 Dec 2025 15:04:01 GMT</pubDate>
            <atom:updated>2025-12-06T15:24:46.869Z</atom:updated>
            <content:encoded><![CDATA[<p>Statistics Without Borders (SWB) recently collaborated with Asociación Civil Sustentar to evaluate the methodology used to generate the group’s Cleanliness Quality Index. This blog post covers the project’s motivation and some of its key findings. These findings offer valuable insights to any organization producing and publishing quantitative assessments.</p><h3>About Asociación Civil Sustentar</h3><p>Asociación Civil Sustentar is an Argentinian non-profit organization that works towards sustainable development at a local and regional level. Committed to sustainability, impact-generation through knowledge products, action implementation, technical assistance, and capacity building, this organization focuses on three areas of work: innovation for sustainability, sustainable mobility, and a circular economy. Asociación Civil Sustentar’s projects integrate environmental, social, and economic factors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DLroEl-FsceVCsSlugWvRg.jpeg" /><figcaption>Collecting data</figcaption></figure><h3><strong>The Problem to be Solved</strong></h3><p>One of Asociación Civil Sustentar’s main projects is the Urban Hygiene Observatory of Buenos Aires City. The Urban Hygiene Observatory generates objective information and provides technical assistance to support decision making in waste management and urban hygiene.</p><p>To objectively measure the cleanliness of streets and sidewalks of city blocks and provide quality information for decision making, the Observatory constructed the Cleanliness Quality Index. By working with Statistics Without Borders (SWB), Asociación Civil Sustentar aimed to evaluate the Cleanliness Quality Index methodology to ensure credibility, reliability, and standardization.</p><p>Jorge Vulej, SWB statistical consultant, said,<strong> </strong>“The city has been using this survey to measure urban cleanliness since 2018,” and Asociación Civil Sustentar needed to validate their Cleanliness Quality Index as it “directly influences the city’s decisions about resource allocation and urban management policies.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W8uJuy7S2R5t4jcc8q9l5g.png" /><figcaption>Educating at the Buenos Aires Planetarium</figcaption></figure><h3><strong>SWB’s Role in the Project</strong></h3><p>The SWB volunteers’ work on this project involved a comprehensive evaluation of the entire Cleanliness Quality Index survey, Vulej said, including “analyzing data from nearly 5,000 randomly sampled city blocks with up to 306 observations per block.” Vulej explained that the SWB volunteers validated four subscales and the overall cleanliness scale, examined the survey methodology, evaluated the statistical reliability and validity of both the subscales and overall scale, and provided practical recommendations for improving data collection and integration with city services.</p><h3><strong>Challenges Encountered During the Project</strong></h3><p>Vulej noted that the SWB volunteers faced several challenges throughout this project. The Cleanliness Quality Index survey is based on observational data which required evaluators to observe and assess a wide range of street cleanliness elements, Vulej explained, and this type of survey “doesn’t fit traditional statistical methods designed to evaluate the validity and reliability of social, attitudinal, political, marketing, or similar types of ‘construct’ questionnaires.”</p><p>What’s more, unlike most conventional surveys, this survey has missing data patterns built into its design, Vulej said. For example, recycling bin observations could only be recorded where bins actually existed. Moreover, traditional reliability measures, such as Cronbach’s alpha, did not work for this observational scale as the items being measured were not supposed to correlate highly with each other. For instance, finding graffiti does not mean trash bins are overflowing. In addition, the team had to evaluate scale items with vastly different numbers of valid observations across the dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n1pxRKzDra4JhMXeFvVccg.jpeg" /><figcaption>At work in the streets of Buenos Aires</figcaption></figure><h3><strong>Overcoming the Challenges</strong></h3><p>To address these challenges, the SWB volunteers “implemented specialized statistical techniques appropriate for this type of survey,” Vulej said. The SWB volunteers used K-Nearest Neighbors imputation to handle missing data by identifying similar locations and filling gaps based on those patterns, and they employed Partial Least Squares regression which is suited for formative scales where items define a concept rather than reflect it. They also examined various metrics including R-squared values to measure effectiveness and Variance Inflation Factor to check for item redundancy, and they conducted test-retest reliability checks using data from multiple survey waves.</p><p>Vulej said the work on this project is an example of how “not all measurement tools and projects fit traditional statistical approaches.”</p><h3><strong>Final Results of the Project</strong></h3><p>The SWB volunteers’ evaluation confirmed that the Cleanliness Quality Index survey is statistically sound and reliable, and it “gave the Buenos Aires City Government confidence that its cleanliness measurement tool and process work effectively,” Vulej said. The city now has statistical evidence that the tool reliably captures urban cleanliness and can be trusted as a source of objective information for policy decisions. The SWB recommendations also provided a roadmap for further improvements, including public dashboards, citizen feedback integration, and linking cleanliness data directly to operational decisions.</p><h3><strong>Before Versus After</strong></h3><p>Francesca Visconti Diccianni, analyst at Asociación Civil Sustentar, said, “Before the collaboration with SWB, we had a strong statistical tool to measure city cleanliness. After the work with SWB, we now have an index endorsed with external and unbiased expert endorsement, giving us greater clarity on opportunities within the project and increased confidence in the Cleanliness Quality Index. This will help us escalate the tool in other municipalities of the region.”</p><p>Of working on this SWB project, Vulej said, “Buenos Aires is my hometown,” and it was “meaningful to make a small contribution to the quality of life in a place that remains close to my heart.”</p><p>As to whether Asociación Civil Sustentar, would recommend SWB to another organization, Visconti Diccianni said, “We would highly recommend SWB to other organizations looking for statistical analysis, objective validation and expert guidance on one of their projects.”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7e87cdffa553" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Bridging Data and Biodiversity: Statistics Without Border’s Collaboration with IPÊ in Brazil]]></title>
            <link>https://medium.com/@statisticswithoutborders/bridging-data-and-biodiversity-statistics-without-borders-collaboration-with-ip%C3%AA-in-brazil-bc7c107f8d12?source=rss-362bc366e67a------2</link>
            <guid isPermaLink="false">https://medium.com/p/bc7c107f8d12</guid>
            <category><![CDATA[social-impact]]></category>
            <category><![CDATA[volunteering]]></category>
            <category><![CDATA[data-for-good]]></category>
            <category><![CDATA[nonprofit]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Statistics Without Borders]]></dc:creator>
            <pubDate>Mon, 13 Oct 2025 22:28:45 GMT</pubDate>
            <atom:updated>2025-10-13T22:28:45.668Z</atom:updated>
            <content:encoded><![CDATA[<p>Statistics Without Borders (SWB) recently partnered with IPÊ — Institute for Ecological Research — on a project focused on reviewing IPÊ’s research data management approaches and technologies. In this blog post, Steve Godfrey from SWB and Alexandre Uezu from IPÊ share the project’s motivation and some of its key findings. These insights offer value to organizations managing large or diverse datasets.</p><p><strong>About IPÊ</strong></p><p>IPÊ is one of the largest environmental NGOs in Brazil, dedicated to biodiversity conservation grounded in scientific research. Based in Nazaré Paulista, its work spans ecological research, professional training, environmental education, and the development of sustainable income-generation programs and businesses that promote socio-environmental responsibility among communities, entrepreneurs, and influencers.</p><p>A key part of IPÊ is the Socio-environmental Intelligence Laboratory (LIS), which aims to expand knowledge on the interactions between people, biodiversity, ecosystems, and landscapes. Its goal is to inform better decisions for environmental conservation and sustainable development.</p><p><strong>Project Topic and Objectives</strong></p><p>The LIS team at IPÊ is conducting a variety of exciting research projects using advanced technologies. These projects explore how people interact with the environment, biodiversity, ecosystems, and landscapes. Tools such as satellites, drones, wildlife cameras, acoustic monitors, and hydrological devices are used to gather data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aUAEzUOVebciV_mehnw8Cw.png" /><figcaption>Camargos Mariana and Minas Gerais Parcela using drones to collect more data.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/671/1*FfOPd0_x-G0RrxRBcS5LiA.png" /><figcaption>IPE collecting both sound and insect DNA samples through the DNA trap.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TH_ZAoeJuTVFl2AjEya58w.jpeg" /><figcaption>IPE Researchers</figcaption></figure><p>However, these cutting-edge tools and diverse research topics present challenges. First, the wide range of data types can require different data governance approaches and storage platforms. Second, the tools generate large volumes of data, which can strain existing infrastructure.</p><p><strong>Problem Statement</strong></p><p>More specifically, the challenge was:</p><p>This rapid growth in data made it difficult for LIS to manage the large and diverse volume, creating difficulties such as:</p><p>· Large storage demands, often hitting account limits</p><p>· Inconsistent data management practices across data types</p><p>· Project-specific storage systems that hinder consistency and data reuse, resulting in underutilized datasets</p><p><strong>Project Approach</strong></p><p>To address these challenges, SWB and IPÊ developed a project plan to:</p><ol><li>Review IPÊ’s current administrative and technical approaches to data management, and research best practices from similar organizations.</li><li>Recommend alternative strategies, including their benefits, limitations, and estimated costs.</li><li>Build and deliver a proof-of-concept for one alternative approach.</li></ol><p>While LIS manages a variety of data across its projects, the technical portion of this collaboration focused on one dataset from the Doce River watershed. This dataset includes a rich mix of information: thematic maps, high-resolution drone images, LiDAR point clouds, community survey data, biodiversity metrics, and more.</p><p><strong>Key Findings</strong></p><p>The project’s findings and recommendations were grouped into two main tracks:</p><p><strong>Track 1: Strengthening Data Governance</strong></p><p>Given the growing volume of research data, there was a clear need for short-term solutions to ease current constraints. At the same time, the project recognized and encouraged the organic efforts already emerging across LIS research teams — these grassroots initiatives form a strong foundation for broader institutional improvements.</p><p>Some specific recommendations that may benefit other organizations include:</p><p><strong>Establishing data management best practices:</strong></p><ul><li>Define data policy coverage</li><li>Set policies for data disposition after project completion</li><li>Create storage guidelines for different data types and platforms</li><li>Develop standardized directory structures for project files</li><li>Offer training on data best practices</li></ul><p><strong>Fostering shared data and code version control:</strong></p><ul><li>Identify shared input datasets across research projects</li><li>Adopt a version control system for code (e.g., Git)</li><li>Explore tools that facilitate shared data use</li><li>Pilot collaborative data-sharing models across projects</li></ul><p><strong>Track 2: Exploring Scalable Technical Solutions</strong></p><p>Track 2 focused on testing how cloud-based data platforms could complement LIS’s current infrastructure. A data lake model, in particular, was explored for its benefits:</p><ul><li>Scales efficiently for large and diverse data formats</li><li>Encourages data sharing and standardization</li><li>Supports tiered storage (e.g., hot vs. cold/archive) for cost-effective data management</li><li>Enables advanced analytics by integrating storage and compute capabilities</li></ul><p>Of course, all technical solutions come with trade-offs. Adding a new platform introduces both costs and complexity. For this reason, the team recommended pairing any cloud solution rollout with improvements to governance policies.</p><p><strong>IPÊ’s Next Steps</strong></p><p>Since partnering with SWB, IPÊ has begun enhancing database governance and implementing best practices for data use, particularly in ongoing projects.</p><p>We have also structured a database architecture that integrates both physically stored data and cloud-based solutions, depending on the type of data and its frequency of use.</p><p>This project has also generated valuable insights into how to better structure the entire data lifecycle — from creation and acquisition to storage, analysis, and public dissemination.</p><p>We collect and analyze socio-environmental data, including tropical biodiversity (fauna and flora), land use dynamics, ecosystem service provision, climate change, and bioeconomy.</p><p>The work carried out with SWB will strengthen the use of this information to support more informed decision-making, promote nature conservation, and help shape more sustainable landscapes across Brazil’s biomes.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bc7c107f8d12" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to foretell a flood: Predicting floods using machine learning]]></title>
            <link>https://medium.com/@statisticswithoutborders/how-to-foretell-a-flood-predicting-floods-using-machine-learning-2ef65b0bcfc3?source=rss-362bc366e67a------2</link>
            <guid isPermaLink="false">https://medium.com/p/2ef65b0bcfc3</guid>
            <category><![CDATA[forecasting]]></category>
            <category><![CDATA[floods]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[climate-change]]></category>
            <dc:creator><![CDATA[Statistics Without Borders]]></dc:creator>
            <pubDate>Wed, 24 Apr 2024 17:30:11 GMT</pubDate>
            <atom:updated>2024-04-24T17:50:02.629Z</atom:updated>
            <content:encoded><![CDATA[<p><em>In a new series of posts, expert volunteers from Statistics Without Borders break down the role of data and statistics in crucial domains.</em></p><p>“We’ve never seen anything like it…People were taken by surprise, and some didn’t have time to escape this deadly trap.” That’s what Daniel Scioli, the governor of Buenos Aires, told <a href="https://www.bbc.com/news/world-latin-america-22023196">the BBC</a> when he described flash floods that killed <a href="https://floodlist.com/america/official-figures-la-plata-floods-argentina">89 people</a> and displaced thousands in La Plata and Buenos Aires in Argentina.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*icZSeH-D0y82ml-6CJ67Ng.jpeg" /><figcaption><a href="https://visibleearth.nasa.gov/images/80875/flooding-in-la-plata/80875f">NASA image courtesy</a> NASA/GSFC/METI/ERSDAC/JAROS, and U.S./Japan ASTER Science Team.</figcaption></figure><p>This flood was especially tragic, but floods are unfortunately very commonplace around the world. Indeed, floods might be the most common natural disaster: they have affected <a href="https://www.who.int/health-topics/floods#tab=tab_1">over 2 billion people worldwide</a> between 1998 and 2017.</p><p>The aftermath of floods is catastrophic in many different ways. Floods are extremely deadly: people who survive are often injured and suffer serious damage to their homes and belongings. Critical public infrastructure, including public health facilities, sustains serious damage. Damaged infrastructure makes cleanup, recovery, and humanitarian aid much more difficult and dangerous.</p><p>Flood forecasting is an important tool to respond to floods: if we can predict where a flood is likely to occur, we can help mitigate the risks of floods.</p><p>In this article, I’ll offer some of my expertise on flood forecasting. As a volunteer for <a href="http://statisticswithoutborders.org/">Statistics Without Borders</a>, I love using data for good: I want to help spread valuable knowledge as widely as possible to help people across the world predict floods and prevent disasters.</p><p>Flood forecasters use sophisticated models to predict where floods are most likely. Here, I’ll contrast two major models for flood forecasting: machine learning and physical models of flood forecasting. I’ll also go into Google’s recently launched flood-forecasting tool and compare it with the Global Flood Awareness System, helping you understand their similarities and differences.</p><h3><strong><em>Machine learning vs. physical models</em></strong></h3><p>The most commonly used flood forecasting models currently rely on physics-driven or rule-based models. These methods typically incorporate known relationships and equations based on hydrological and meteorological principles. The information provided by standard models is often limited to explicit rules and predefined relationships, which may not capture the full complexity of the hydrological system.</p><p>In contrast, machine learning-based flood forecasting leverages the power of data, allowing for more dynamic and adaptable models. Machine learning models, particularly deep learning approaches, can learn complex patterns and relationships from data. This can enable them to provide more nuanced and data-driven insights into flood prediction, potentially uncovering relationships missed by physics-driven models.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ThfM1uCAwkDsgOXO" /><figcaption><em>A four-quadrant graph comparing </em>modeling approaches, based on data dependency and complexity of relationships modelled. <em>Image: Jessica Rapson</em></figcaption></figure><h3><strong><em>Using machine learning in flood forecasting</em></strong></h3><p>The most common motivation for using machine learning in flood forecasting is to increase the accuracy of predictions. This is often possible since machine learning enables more complex relationships to be modeled. It might also be the case that physical models do not describe the entire relationship between the data used and flood outcomes. There may be latent relationships within the data which indicate a flooding hazard but are not explicitly part of the physical models or are too complex to be detected through statistical models.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wrestaSeTbA8zEEA" /><figcaption>An overview of ML applications in the flood forecasting process. Image: Jessica Rapson.</figcaption></figure><p>It is also possible to use machine learning to help collect model data. The underlying data used to predict flooding can be generated using machine learning-based methods. For example, AI-based computer vision models can be applied to satellite images to detect key terrain features or detect historic regions affected by flooding.</p><h3><strong><em>Data used in flood forecasting</em></strong></h3><p>The usefulness of machine learning for flood forecasting is fundamentally limited by the availability of data. Fields that have near-infinite data availability, such as text or image analysis, can build highly complex and accurate models without the need to explicitly consider the physical structure underlying the data relationships. This has been hugely advantageous for firms such as OpenAI, the company behind ChatGPT, that primarily builds models that operate in these big data fields.</p><p>Flood forecasting, however, operates in a much more limited data landscape. Unlike text or image data, which can be accessed in massive amounts from the Internet where it already exists with relative ease, the data relevant flooding often requires special meteorological or remote sensing tools to access. Further, events that cause flooding, such as cyclones, occur infrequently and have only been consistently measured in recent decades, meaning that there often is not sufficient data to accurately generate predictions from data alone.</p><p>This is why flood forecasters often turn to physics-driven models. Data is also very important to physics-driven models, but it is used differently than in data-driven models. While in data-driven models, the data itself is used to determine relationships between the outcome variable of interest (i.e. flooding) and the predictor data, physics-driven models instead use data as inputs to pre-determined mathematical equations which are derived from known hydrological relationships.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Zq75esY26bAnxVNq" /><figcaption><em>The relationship between risk, hazard, exposure, and vulnerability. Image: Jessica Rapson.</em></figcaption></figure><p>Flood forecasts are typically concerned with risk. Risk refers to how individuals or systems are likely to be affected by a given hazard such as flooding. It is treated as a combination of hazard (the flood itself), exposure (the degree to which individuals are potentially affected by flooding), and vulnerability (the ways that flooding may affect certain individuals more than others). Data in all three of these areas is required to provide a holistic assessment of risk.</p><h3><strong><em>Google Flood Forecasts</em></strong></h3><p><a href="https://ai4sibook.org/wp-content/uploads/2022/08/Flood_Forecasting.pdf">Google’s flood forecasting model</a> is a good example of how machine learning can be used in flood predictions. It uses two different models: (1) a thresholding model and (2) a manifold model. Streamflow forecasting is done with long short-term memory (LSTM) networks, while flood extent is computed with thresholding and manifold models. The manifold model, developed by Google, is a machine-learning alternative to hydraulic modeling of flood extent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7KUxD20eHFVapbf9" /><figcaption><em>Predicted </em><a href="https://en.wikipedia.org/wiki/F-score"><em>F1 Score</em></a><em> for Google’s streamflow forecasting model. Image source: Source: </em><a href="https://arxiv.org/pdf/2307.16104.pdf"><em>https://arxiv.org/pdf/2307.16104.pdf</em></a></figcaption></figure><p>The thresholding model computes a flood extent map. It does not require a <a href="https://www.usgs.gov/faqs/what-digital-elevation-model-dem">digital elevation model (DEM)</a> and is easier to implement and deploy at scale. Each pixel in the inundation map is treated as a separate classification task, predicting whether the pixel will be inundated or not. It forecasts only flood extent; the manifold model can be used when a DEM is available and computes a map of both extent and depth. The manifold algorithm identifies boundaries of the inundated areas of the input inundation map. The results showed that machine learning models were more accurate than physical models, though they require more data.</p><p>The Google flood forecasts have also been benchmarked using the same criteria as the Global Flood Awareness System (GloFAS) — the current state-of-the-art physics-driven model for global flood forecasting — and <a href="https://doi.org/10.48550/arXiv.2307.16104">have shown to be more accurate or similarly accurate compared to this model</a>, even for rivers where no data on streamflow can be collected. This model uses publicly available data and its code is available as an <a href="https://doi.org/10.21105/joss.04050">open-source package</a>. Its predictions can be accessed through the <a href="https://sites.research.google/floods/l/0/0/3">Google Flood Hub website</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*O4kTYqtuxS5CN8K9" /><figcaption><em>Comparing F1 Scores for Google’s streamflow forecasting model with GloFAS. </em><a href="https://arxiv.org/pdf/2307.16104.pdf"><em>Source</em></a><em>.</em></figcaption></figure><h3>The bigger picture:</h3><p>Floods are clearly among the most pressing challenges today. Flood forecasting saves lives, protects community infrastructure, and is less expensive than cleanup and recovery from these certain, recurring disasters.</p><p>Ultimately, whether physics-driven or machine learning models are better suited for a given flood prediction task depends on the specific nature of the task at hand. For an issue as high-stakes as flooding, prediction accuracy is not the only metric that forecasters care about. Being able to produce physically explainable predictions and make predictions well before any flooding is also important.</p><p>Machine learning models may also suffer from data biases that do not affect physics-driven models. For example, less-developed regions may have less historic flooding data available that can be used to train machine learning models, making them systematically less accurate for these regions. These trade-offs must be carefully considered when deciding to use machine learning-based flood forecasts.</p><p><a href="http://jess-rapson.com/"><em>Jess Rapson</em></a><em> is a machine learning engineer with a multidisciplinary background in both statistical science and public policy, focusing on applications of AI/ML in optimizing decision-making processes for organizations that serve the public. She has a Master of Public Policy degree from the University of Toronto and a Master of Science in Statistical Science degree from the University of Oxford.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2ef65b0bcfc3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[SWB Project Highlight: IMPACT Initiatives]]></title>
            <link>https://medium.com/@statisticswithoutborders/swb-project-highlight-impact-initiatives-285fbf413ef8?source=rss-362bc366e67a------2</link>
            <guid isPermaLink="false">https://medium.com/p/285fbf413ef8</guid>
            <dc:creator><![CDATA[Statistics Without Borders]]></dc:creator>
            <pubDate>Mon, 12 Feb 2024 09:29:55 GMT</pubDate>
            <atom:updated>2024-02-13T03:16:50.445Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://www.statisticswithoutborders.org/">Statistics Without Borders (SWB)</a> and <a href="https://www.impact-initiatives.org/">IMPACT Initiatives</a> have collaborated on several projects, covering both the methodology of collecting household survey data and the analysis of selected results. This blog posts covers a recent collaboration focusing on the impact of survey design effects in some of IMPACT’s Multi-Sector Needs Assessments (MSNAs) measuring humanitarian needs in crisis-affected countries.</p><p>Steve Godfrey from SWB and Olivier Cecchi (Senior Research Specialist Data Science &amp; Innovation at IMPACT Initiatives) present the project and its results below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*w7AS7LpcYgR3WwKA" /></figure><h3>About IMPACT Initiatives</h3><p><em>Created in 2010, IMPACT is a Geneva-based NGO and the largest independent data provider in contexts of crisis. It aims to support a range of stakeholders in making better, more informed decisions in humanitarian, stabilization, and development settings.</em></p><ul><li>IMPACT takes an initiative-based approach to structuring its work and programming: REACH — strengthens evidence-based humanitarian decision-making through efficient data collection, management, and analysis,</li><li>AGORA — promotes localized and multi-sectoral aid action in support of the recovery and stabilization of crisis-affected communities,</li><li>PANDA — improves humanitarian and development interventions through program design, assessments, and evaluation.</li></ul><p>The work around needs assessments is done within REACH, and aims to provide data collection, analytical assessment, and analysis in over 20 countries across the Middle East, Africa, Asia, Europe, and Latin America.</p><h3>About MSNAs</h3><p>As noted, one of IMPACT’s flagship programs is Multi-sectoral Needs Assessments (MSNAs) in countries experiencing humanitarian challenges and crises. MSNAs are conducted through nationwide household surveys and cover a large spectrum of indicators including topics such as food security, water availability, safety and security, and healthcare, but always with a set of 15 core indicators.</p><p>These assessments are designed to be comprehensive, independent, and impartial to support annual humanitarian response planning and prioritization. Partners using MSNAs include non-governmental organizations (NGOs), international groups, governments, and other organizations who rely on data and analysis to effectively and fairly provide humanitarian aid.</p><h3>Project’s topic and objectives</h3><p>This SWB-IMPACT project’s primary topic addressed design effects of these MSNA household surveys, which IMPACT was interested in exploring to determine how to improve its research design, and in particular, the sampling strategy and size estimation regarding core indicators in all countries. More specifically, IMPACT sought data on the Intra-Class Correlation (ICC) values found in MSNA results across countries and topics, to use them in upcoming surveys. In addition, feedback was needed on generally applying a single Intra-Class Correlation (ICC) value across countries and topics.</p><h3>Survey design effect and Intra-Class Correlation (ICC)</h3><p>Design effect refers to the (almost always present) increase in variance arising from a complex survey-sampling methodology versus a simple unrestricted random sample. Complexity in sampling methodology occurs when stratification or clustering is used to select respondents. For example, to manage data-collection costs and maintain representation, household survey samples might be stratified along demographic or geographic criteria (such as urban or rural residents) and then clustered (such as x urban and y rural households). The variance of data collected using this complex approach will not be the same as a simple random selection, since these samples are not independent. Therefore, variance needs to be mathematically adjusted to account for these effects, thus resulting in “survey design effects.”</p><p>Quantifying design effects is important to accurately calculate precision and infer statistical significance from survey results, after data collection. In addition, design effects can be integrated prior to data collection, in sampling design to determine the number of clusters and surveys needed to achieve a desired precision level.</p><p>One factor directly linked to the design effect is the Intra-Class Correlation (ICC) which refers to the correlation among respondents in the same cluster. This correlation influences sampling methodology by affecting the responses distribution. If respondents in a cluster are highly correlated along some dimension — such as income level — then adding one more respondent to the cluster may provide very little additional information since their income will essentially match others. In this case, the survey’s design effect will be high since all respondents in each cluster are likely to have similar responses to the income question.</p><h3>Design effects and IMPACT’s work</h3><p>Handling design effects in the context of IMPACT’s work with MSNAs is not straightforward. These household surveys aim to collect data in some of the world’s most challenging economic and humanitarian contexts, while continuing to maintain good levels of precision. Beyond survey environments, the difficulty of maintaining result precision is increasing given that these surveys cover a wide range of topics including displacements, life events, livelihood, child labor, health, and protection, which are topics that can exhibit varying degrees of intraclass correlation. Still, IMPACT remains committed to expert survey methodology, as poor design-effect estimate prior to data collection leads to gathering less precise and reliable information to support effective humanitarian action.</p><h3>Why Statistics without Borders?</h3><p>IMPACT is a mature and sophisticated analytical organization who has developed a robust methodology to design surveys and analyze results. What role can SWB fill with such an entity? In this case, IMPACT felt that independent input from knowledgeable resources into a specific question within their methodological framework would be helpful. SWB’s feedback and recommendations could be viewed as an external expert’s input among an extensive knowledge base on the broader topic. From its perspective, the SWB project team also saw its output as observations and considerations that the client might consider adopting.</p><p>SWB was able to recruit a volunteer team consisting of professionals and academics experienced in survey statistics, as well as quantitative analysts with limited survey experience but expertise in other areas. Education and experience diversity among the project team benefits both the output and individual team members, by applying broad perspectives to the project and by providing volunteer learning opportunities.</p><h3>SWB’s project output</h3><p>The SWB team addressed the topic along two tracks:</p><ol><li>A review of academic and related research into household survey design and ICCs, particularly in emerging markets,</li><li>An ex-post analysis of IMPACT’s data from selected MSNA surveys, to calculate actual observed ICCs and design effects (see Table 1 for a list of these countries and the research).</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/609/1*SQzhFJNprrodLLIaOTLTtw.png" /></figure><p>General observations from the literature research were that:</p><ul><li>Understanding and capturing design effects are critical for a statistically sound use of survey results,</li><li>Similar emerging-market household surveys often observe high degrees of ICC and design effect variability, including among topics within the same survey,</li><li>Varied approaches are employed to estimate design effects when ICC data are unavailable, including borrowing from similar studies and informed assumptions.</li></ul><p>SWB’s analysis of IMPACT historical survey data found an expected wide ICC range, confirming IMPACT’s own observations. This variation was readily apparent when comparing topics within a country’s data across survey topics. This meant that it was not possible that one ICC could be employed for all topics in country X, while another ICC could be employed across all topics in country Y. A comparison across countries was even more challenging, and the team was not able to identify strong ICC patterns by topic that spanned all countries. In other words, it was not possible to make statements such as: “education questions should use one ICC assumption across all countries while aid questions should use another also across all countries.”</p><p>In summary, the SWB team’s feedback was that ICCs and related design effects seem to be highly varied with such household surveys. This can be seen in IMPACT’s data as well as the published results from other organizations. However, for practical reasons, IMPACT may not be able to incorporate this finding into its pre-collection survey design and may need to continue with its approach of applying one ICC level. Indeed, considering the challenges of collecting extensive survey data in these contexts, there may be a limit to how many households IMPACT can survey. While ICCs can provide guidance on expected precision, other practical factors such as collection cost or respondent availability are also important. In addition, the household survey is broad by design, and the number of cluster respondents is still likely to depend on some assumed ICC average even if they are expected to widely vary by topic.</p><p>That said, post-collection data analysis and statements of statistical precision and significance should still account for the observed design effects.</p><p>As previously noted, IMPACT has deep technical expertise including extensive survey-methodology knowledge developed from academic expertise and practical experience. The SWB team’s findings support much of what the organization had already observed and incorporated into their survey methodology. Still, having this this independent review has been helpful to update their post-survey learnings.</p><h3>IMPACT’s use of the project’s findings</h3><p>Given the unique context of household data collection within the scope of its work, utilizing similar Intra-Class Correlation (ICC) values from comparable surveys to determine the design effect of MSNAs is unrealistic. However, because they are conducted annually and encompass a consistent set of core indicators, computing ICC values across different years in the same country could provide valuable insights.</p><p>Thus, IMPACT aims to establish an ICC database for MSNAs spanning on multiple years. This database would facilitate an assessment of the assumption that ICC remains consistent for the same indicator in each country over time. If this assumption holds true, it becomes possible to gain insights into mitigating the impact of high ICC values on research design. Various strategies that may be considered include:</p><ul><li>Adjusting the overall sample size to enhance the precision of indicators displaying high ICCs,</li><li>Anticipating indicator precisions more effectively foreseeing better the precision levels of our findings after the assessment.</li><li>Reassessing the data collection process for certain indicators; in cases where ICCs are exceptionally high, alternative data collection methods like key informant interviews, secondary data analysis, or remote sensing may be explored as more suitable alternatives to household surveys.</li></ul><h3>References</h3><p>For those interested in learning more, here are some useful references. The UN publication is quite comprehensive but rather difficult to find online (reach out to Steve at <a href="/service/mailto:stephengodfrey223@gmail.com">stephengodfrey223@gmail.com</a> if you want help finding it).</p><ul><li><a href="https://reliefweb.int/report/world/impact-2022-annual-report">IMPACT 2022 Annual Report</a></li><li>United Nations. Statistical Division, and National Household Survey Capability Programme. Household Surveys in Developing and Transition Countries. Vol. 96. United Nations Publications, 200</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=285fbf413ef8" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>