Using Data to Improve Student Outcomes

𝐓𝐡𝐞 𝐒𝐞𝐜𝐫𝐞𝐭 𝐭𝐨 𝐓𝐫𝐚𝐢𝐧𝐢𝐧𝐠 𝐓𝐡𝐚𝐭 𝐀𝐜𝐭𝐮𝐚𝐥𝐥𝐲 𝐖𝐨𝐫𝐤𝐬? 𝐒𝐭𝐚𝐫𝐭 𝐚𝐭 𝐭𝐡𝐞 𝐄𝐧𝐝. 🏁 I used to think my job as an L&D professional started with a syllabus. I was wrong. Recently, I was tasked with building a learning solution for our Talent Acquisition (TA) team. The goal wasn’t just to "train recruiters"—it was to solve a business problem. Instead of looking at what they needed to know (Level 2), I started with what the business needed to achieve (Kirkpatrick Level 4). The "Reverse" Approach I didn’t start with slides. I started by analyzing Voice of the Customer (VOC) survey results, focusing on various metrics from both Hiring Managers and Candidates. Working Backwards: ✅ Level 4 (Results): I defined the business KPI. ✅ Level 3 (Behavior): Based on the VOC metrics, I identified the specific actions recruiters needed to change—specifically around "Precision Intake" and "Candidate Experience Management." ✅ Level 2 & 1 (Learning & Reaction): Only then did I design the actual training content that addressed those specific behavior gaps. The Result? The training didn't feel like a chore; it felt like a solution. Because I built it based on the actual metrics revealed in the VOC surveys, the TA team saw immediate value, and the business saw a measurable shift in hiring efficiency. The Lesson: If you want your learning solutions to be more than just "check-the-box" exercises, stop asking "What should we teach?" and start asking "What does the data say I need to solve?" How do you use VOC data to shape your enablement programs? 👇 #LearningAndDevelopment #InstructionalDesign #TalentAcquisition #KirkpatrickModel #Enablement #DataDrivenLD #BusinessImpact

Medical education has traditionally followed a one size fits all model, yet clinical competency develops differently for every trainee. A new New England Journal of Medicine perspective highlights how AI enabled precision education can continuously assess performance, identify skill gaps early, and personalize training across medical school, residency, and lifelong learning. By aggregating learner data, AI can map learning curves, guide targeted simulation and rotations, and provide real time coaching and feedback. The goal is more consistent training, earlier competency development, and ultimately safer patient care. Read more: https://lnkd.in/gxVmDKK2 Follow Zain Khalpey, MD, PhD, FACS for more on Ai & Healthcare. #MedicalEducation #AIinHealthcare #PrecisionEducation #MedEd #AIinMedicine #DigitalHealth #FutureOfMedicine #PhysicianTraining #HealthcareInnovation #MedicalTraining #ClinicalExcellence #HealthTech #AcademicMedicine #LifelongLearning #MedicalAI #HealthcareLeadership #InnovationInMedicine #NextGenMedicine #MedTech #MedicalInnovation

If you don’t measure it, you can’t improve it. When I first started Youth Leads UK, I thought impact was obvious. Young people told us they felt more confident, more connected, more skilled - surely that was enough? But here’s the truth: passion convinces hearts, evidence convinces funders, partners, and policymakers. Without data, it’s harder to scale what works and fix what doesn’t. That’s why I’ve been working with Charlotte and Bethan at Bean Research to rethink our approach. They’ve helped us move from collecting feedback here and there, to building a clear, consistent framework that shows both the numbers and the human stories. A few lessons I’ve taken away from this journey: ↳ Focus on what really matters - not 20 measures, just the few that capture confidence, skills, and opportunity. ↳ Make it simple and consistent - small surveys, quick reflections, and clear tools beat complicated forms that nobody fills in. ↳ Pair data with voices - stats prove scale, but quotes prove depth. Together they paint the full picture. Impact measurement isn’t about ticking boxes. It’s about making sure our time, energy, and resources actually change lives - and being able to show it with confidence. 👉 If you’re leading a project, what’s the one impact you most want to measure, and why?

I’ve been using n8n to connect my Learning Record Store (LRS) with real-world user interactions. Right now, when an xAPI statement (learner interaction data) comes in, it can trigger a robot to dance when it scans for specific data in the LRS. Next, I’m layering in Slack messages that respond to specific learner interaction data. It’s a simple way to demonstrate a bigger idea. When we collect granular xAPI data from learning in the flow of work, we can actually do something with it. For example, a customer service simulation could be delivered directly in Slack as a link or interactive chat. The rep completes the scenario right where they work. Each response, such as how they phrase answers, how quickly they respond, and whether they resolve the issue, sends detailed xAPI data to your LRS. That data does not stop there. It could connect with performance data from real customer calls. If those calls show that a rep struggles with empathy or tone, the system can automatically generate a custom simulation to practice that specific skill. After completing it, the rep receives personalized feedback or follow-up practice in Slack based on what the system detected. This could be done in so many different ways like with GenAI to create adaptive practice or add an agent with memory that connects chat data, call insights, and internal systems to deliver coaching that feels timely and contextual. This moves learning from a single event to a continuous, adaptive experience that fits naturally into how people already work. #xAPI #learningdesign #learningintheflowofwork #LRS #GenAI #n8n #instructionaldesign #learninganddevelopment #futureoflearning

You've just launched a reskilling program aimed at boosting digital literacy across your organization. Now, the big question is: how do you measure its success? To answer that, a combination of hard data and real-world feedback is key. Take the example of AT&T, which famously invested $1 billion in reskilling its workforce for the digital age. They tracked success through KPIs like training completion rates and skill acquisition. Post-training, they saw a marked increase in employees' ability to handle new technologies, evidenced by improved performance metrics. But metrics only tell part of the story. Gathering qualitative feedback is equally important. IBM, for instance, uses surveys and pulse checks to gauge how employees feel about their upskilling efforts. This feedback allows them to tweak programs in real-time, ensuring that learning remains relevant and engaging. Lastly, consider long-term evaluation. Adobe ties reskilling outcomes to annual performance reviews, allowing them to see if the new skills are leading to sustained improvements. This holistic approach—combining KPIs, feedback, and long-term tracking—ensures that reskilling initiatives not only deliver immediate results but also contribute to lasting change. Are you ready to measure the true impact of your reskilling efforts? #hr #chro #reskilling #datainsights #employeedevelopment #employeeskilling

"...Digital Personalized Learning (DPL) emerges as a promising and cost-effective alternative for math remediation. DPL leverages Artificial Intelligence (AI) and machine learning to provide students with adaptive instruction tailored to their competency levels, known as "Teaching at the Right Level" (TARL). The basic principle of TARL is to adapt instruction to match students' needs based on their prior knowledge. This adaptation enhances knowledge retention and motivation, while providing a strong foundation for future learning. Adaptive Learning is a promising mechanism to improve student skills and their perceptions about those skills, known as perceived self-efficacy, which is often associated with academic performance, especially in mathematics. DPL also offers pedagogical strategies and regular data for assessment, accessible through various devices with internet access." https://lnkd.in/dM5YBRti

Your project started without a baseline? Welcome to 90% of real-world Monitoring and Evaluation. Most programmes launch with urgency, political pressure, or donor timelines, not perfect data systems. That doesn’t mean you can’t measure change. It just means you need to reconstruct the “before” using the tools seasoned evaluators rely on: 🔹 Start with what already exists Intake forms, early reports, planning documents, grant proposals, even if they weren’t created for MEL, they often contain reference points you can extract. 🔹 Use recall methods strategically Ask participants and staff to describe conditions before the intervention, but anchor their memory to major events: ↳ “Before the school opened…” ↳“Before the water point was installed…” This reduces bias and increases accuracy. 🔹 Pull secondary data to fill the gaps Census tables, ministry surveys, NGO assessments, anything close in geography and timeframe can provide a credible reference. 🔹 Triangulate relentlessly Never rely on one source. Cross-check community recall with government data, staff insights, and documentation. Retrospective baselines aren’t shortcuts. They’re structured, defensible methods for rebuilding the past and they’re what experienced evaluators use when perfection isn’t possible (which is most of the time). 🔥 If you want more practical MEL techniques like this with no jargon, no theory-only talk, join my mailing list for weekly insights that will sharpen your practice. #Baseline

For the past decade many #learning organizations dreamt of adopting the Netflix model. But it was just a dream and frankly, that model would have been a bad-fit in the corporate learning space anyway. However, something interesting just happened that I believe tech minded Learning Leaders / LXM, LXP solution providers should pay attention to. Netflix just published a really interesting Tech Blog article on their foundation model for personalized recommendations. It offers valuable insights that can inform the redesign of learner experiences. Key Takeaways: Centralized Learner Modeling: Netflix transitioned from multiple specialized models to a unified foundation model, centralizing user preference learning. * Application in Learning: Develop a centralized learner model that aggregates data from various learning activities, enabling consistent personalization across different courses and modules. Data-Centric Approach: Emphasizing high-quality, large-scale data over intricate feature engineering, Netflix’s model benefits from end-to-end learning. * App. In Lrg: Prioritize collecting comprehensive learner interaction data (e.g., quiz attempts, forum participation) to inform adaptive learning paths and content recommendations. Interaction Tokenization: Netflix tokenizes user interactions to capture meaningful sequences, similar to language models. * App. In Lrg: Implement tokenization of learning activities to identify patterns (e.g., common misconceptions, preferred learning sequences) that can guide personalized content delivery. Scalable Personalization: The foundation model allows for scalable personalization across Netflix’s vast user base. * App. In Lrg: Design learning systems that can scale personalization efforts, accommodating diverse learner profiles and adapting to evolving educational needs. Interaction tokenization (IT) sounded very similar to LRS (Lrg Record Store). IT is the process of converting user activities (e.g., watching a video, taking a quiz, clicking “next,” participating in a forum, pausing content, revisiting materials) into “tokens”—discrete data units. These tokens form sequences that can be analyzed like language to model and predict learner behavior or preferences. It’s like treating a learner’s journey as a sentence, where: Each “word” is an interaction (e.g., “view_video,” “attempt_quiz,” “fail_question_2”). The “sentence” is a learning path. The model learns from many such “sentences” to predict and personalize future experiences. LRS is the source: It captures and stores granular learning data in a structured format using xAPI statements. Tokenization is the next layer: Once you have data in the LRS, tokenization transforms these raw interactions into meaningful sequences for: Personalization Predictive analytics Content recommendation Learner path modeling (like Netflix does) Really interesting stuff. Give the article a read- link in comments.

I wish we talked more about ROI in the nonprofit world. Funding is limited, so we have to ensure it creates the greatest possible impact. Right? But I still often see vague metrics like: We “reached” or “served” X number of people. That’s an output, not an outcome. It’s fine to measure outputs: number of trainings, meetings, distributions, etc. But it’s more important to measure the outCOMES: How did people’s lives measurably improve as a result of this work? I get that it can be hard to measure outcomes, especially in areas that don’t lend themselves to quantitative impact (governance, capacity building, etc.) But if the goal is to improve people’s economic power, then that’s what we should measure. And we should do it in a way that: ➡️ 1) Shows the ROI based on program cost ➡️ 2) Calculates attribution–i.e., how much revenue change is attributable to your organization’s work, vs. external market factors? ➡️ 3) Doesn’t assume unrealistic lasting impact. (For instance, TechnoServe assumes continued revenue improvement as a result of our work for THREE years afterward--no more. We often see indications that the impact lasts longer. But until we have more data from post-project evaluations--too rarely funded--we use a conservative, low-end estimate.) 🔸🔸🔸 So here's a quick breakdown of TechnoServe's 2025 ROI: ♦️ This past year, the people TechnoServe worked with gained an average $5.70 in additional revenue for every $1 we spent working with them 👇 ♦️ The ROI of all our closed projects last year ranged from over 20-to-1 to less than 1. ♦️ For very low-ROI projects: Some were simply failures. Others were pilots that we hope will attain a positive ROI as they scale. ♦️ For very high-ROI projects: Some may not reflect true cost-effectiveness--e.g., they might have had unusually low starting points due to COVID. Others may be truly impactful projects, where we’ll seek to replicate successful elements where we can. But we’ll be looking at the WHY behind all these scores to see what we can improve or scale. 👉 I think the development world owes it to the people fighting poverty worldwide to find what works and fix what doesn't. And that starts with working as hard as possible to measure things right.

Monitoring and Evaluation (M&E) systems form the backbone of program accountability, learning, and improvement. This document, Developing a Monitoring and Evaluation Plan, offers a step-by-step guide for creating robust and responsive M&E frameworks tailored to the complexities of humanitarian and development programs. It emphasizes the importance of aligning indicators, data collection methods, and reporting processes with program goals to ensure reliable and actionable insights. The content covers critical components of an effective M&E plan, including defining SMART indicators, setting baselines and targets, and establishing data acquisition and reporting methods. Humanitarian professionals will benefit from its practical focus on data quality, emphasizing validity, reliability, and timeliness as essential criteria for ensuring the credibility of findings. Additionally, the guide explores various data collection techniques, from surveys to focus group discussions, offering strategies to select the most appropriate methods for different contexts. This document serves as a comprehensive resource for M&E practitioners committed to optimizing program performance. By mastering the tools and principles presented, professionals can design M&E systems that drive evidence-based decision-making, enhance program accountability, and foster meaningful impact in humanitarian interventions.