Building Scalable ML Pipelines: Lessons from Our Grace Hopper Workshop
By Rashmi Raghunandan, Julie Quintero, and Anusha Jamkhandi
In September 2023, we had the opportunity to lead a hands-on workshop at the Grace Hopper Celebration (GHC), one of the largest gatherings of women and non-binary technologists in the world. Our goal? Share what we’ve learned building scalable machine learning systems — and give attendees a chance to get their hands dirty with real-world tools in a low-stakes, highly interactive setting.
Why GHC, Why This Workshop?
We believe deeply in creating space for underrepresented voices in tech. GHC has always been more than just a conference — it’s a celebration of inclusion, mentorship, and building pathways for those historically left out of the tech conversation.
As technologists who spend our day-to-day wrangling production ML systems, we wanted to share a behind-the-scenes view into how modern ML workflows actually get built. But more importantly, we wanted attendees to walk away with practical skills, a sense of empowerment, and a belief that this work is within their reach too.
Demystifying ML Pipelines
We designed our workshop around a realistic scenario using synthetic customer data, where attendees walked through the full pipeline — from data ingestion to model training and inference.
We used Prefect to orchestrate our workflows, and built everything to run on Google BigQuery. Our goal was to show how production-grade systems can still be approachable, even for folks just beginning to dip their toes into ML infrastructure.
Here’s what we covered:
- Intro to Prefect for orchestrating Python-based workflows
- Building ETL pipelines to ingest synthetic user data into BigQuery
- Exploring behavioral patterns in data
- Engineering features to support ML models
- Training a recommendation model (friend suggestions) end-to-end
All with an emphasis on reproducibility, privacy (hello, synthetic data!), and thoughtful system design.
The Real Impact
What surprised us most wasn’t just the turnout — 430 people showed up — but the depth of the questions. Attendees weren’t just curious about the “how,” but also the “why”: why we chose certain tools, how we handle failures in production, and how we think about building systems that are fair, robust, and maintainable.
Get Rashmi Raghunandan’s stories in your inbox
Join Medium for free to get updates from this writer.
It reminded us that the appetite for learning about ML infrastructure is real. And it reinforced the value of creating learning spaces where people feel safe asking foundational questions.
Try It Yourself
Curious what we built? We’ve made all the materials public, including the slides and code. Whether you’re just starting out or looking to build your own scalable ML stack, we hope it’s helpful.
🔗 Workshop materials on GitHub
💬 What Attendees Had to Say
The most rewarding part of our Grace Hopper workshop wasn’t just sharing technical knowledge — it was connecting with an audience genuinely excited to learn, ask thoughtful questions, and explore new ideas in ML systems.
Here’s a sample of what attendees shared with us after the session:
“I really loved the workshop and wanted to thank you for sharing your knowledge. It was great how you gave us a chance to work with a dataset and provided all the resources on GitHub to learn more. What I found most interesting was learning about the Prefect dataflow pipeline.”
“I totally enjoyed it! Streamlining a pipeline makes it so much easier than doing every job manually when working with ML models.”
“Thank you so much for your session. It was a really great experience. I got to learn a lot, and I hope to take this learning forward and innovate with new ideas.”
“Thanks for the amazing talk on the ML pipeline. I truly appreciate the effort you’ve all put into it. As someone new to ML, you all made it approachable!”
“I learnt a lot and was inspired by your eloquence and solid knowledge about your work. Thanks for sticking around to answer my questions about Prefect and embeddings!”
“Congratulations on speaking at #GHC23! Hope you had a wonderful conference. Keep inspiring others!”
These messages were a powerful reminder: when you open the door to technical knowledge with generosity and clarity, you’re not just teaching — you’re helping build a more inclusive and empowered tech community.
