<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[vouch-engineering - Medium]]></title>
        <description><![CDATA[Engineering @ Vouch - Medium]]></description>
        <link>https://medium.com/vouch-engineering?source=rss----d4246e699295---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>vouch-engineering - Medium</title>
            <link>https://medium.com/vouch-engineering?source=rss----d4246e699295---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 18:22:43 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/vouch-engineering" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Engineering Spotlight: EZ Habimana]]></title>
            <link>https://medium.com/vouch-engineering/engineering-spotlight-ez-habimana-c05ffb606e33?source=rss----d4246e699295---4</link>
            <guid isPermaLink="false">https://medium.com/p/c05ffb606e33</guid>
            <dc:creator><![CDATA[Engineering @ Vouch]]></dc:creator>
            <pubDate>Fri, 28 Mar 2025 16:17:07 GMT</pubDate>
            <atom:updated>2025-03-28T16:17:07.774Z</atom:updated>
            <content:encoded><![CDATA[<p>EZ Habimana | Senior Software Engineer</p><h3><strong>Tell us a bit about your background — who you are, how did you get into tech/engineering?</strong></h3><p>I immigrated to the U.S. to pursue a degree in STEM with the goal of becoming a doctor. I had a scholarship to study pre-med, and everything I planned pointed toward medical school. But before my freshman year officially started, while I was still in ESL classes, I had a conversation with a medical school professor that completely changed my direction. He broke down the path to medical school, and I realized how complex and expensive it would be — especially for someone who wasn’t a permanent resident or U.S. citizen.</p><p>I started exploring other STEM paths that were both accessible and interesting, and that’s when I discovered computer science. It helped that my father had taken computer-related courses back in the 1980s, and that connection inspired me. Math had always been a strong subject for me, and computer science felt like a natural fit.</p><p>But I’ll be honest — it was intimidating at first. I’d barely touched a computer before college, and walking into a computer lab where everyone else was flying across keyboards was overwhelming. I had only used a computer in internet cafés back home, mostly to check emails. Over time though, I came to understand that programming was about logic, problem-solving, and creativity — not just typing fast or using Excel. That’s when everything started to click.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*orovnDK1F7mCabog" /></figure><p>Toward the end of college, the concept of “big data” started gaining traction, and I became interested in becoming a data engineer. After graduating, I landed a job at a data brokering company working as an ETL engineer. The tools were mostly proprietary or quite old-school — Perl script, internal ETL tools — which eventually pushed me to look for something more modern and flexible.</p><p>I transitioned into full stack development in the e-commerce space. It was a big switch, but also a great decision. I was finally working with widely used programming languages and frameworks that opened doors across the industry. And moving from a large organization to a fast-paced startup gave me more ownership over my work and a much steeper learning curve. That experience helped me grow quickly.</p><p>From there, I jumped into customer support software — still a startup, still fast-moving — but this time I was working with cloud-native, serverless technologies. It was my first experience moving fully into the cloud, and that opened up a whole new set of skills.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JCDRf2FyzLyIDS9h" /></figure><p>Eventually, I found my way to Vouch, where I’ve been diving into a brand-new space — insurance. It’s been rewarding to learn how the industry works, especially since insurance touches so many parts of life and business. At Vouch, I feel like I’m contributing to something that has a real impact. The team moves quickly, the tech stack is modern, and the culture encourages you to speak up, share ideas, and take ownership of your work.</p><h3><strong>What’s a fun fact about you that others might not know?</strong></h3><p>I love learning languages! I speak French and my native language, Kinyarwanda, and I’m currently learning Spanish. I also love to travel — especially exploring nature, new cultures, and different foods. I’ve been to Japan, several countries in Central and South America, and a couple of countries in Africa. Outside of work, I also enjoy watching sports, especially soccer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*OtQjNDlvEZomV_SH" /></figure><h3><strong>What advice would you give to someone just starting in Engineering?</strong></h3><p>Keep learning. Seriously. The pace of change in tech is constant — what you used in school might be outdated in just a few years. Be open-minded to new tools, frameworks, and practices. Every new project is an opportunity to try something different. Explore alternative libraries, experiment with new approaches, and push yourself to learn beyond what you already know.</p><p>Also, don’t be afraid to change industries. Software engineering applies to everything — from healthcare to retail to construction. Working in different domains helps you appreciate how tech supports other areas and gives you a broader view of problem-solving.</p><p>One more thing: if you ever experience a mass layoff, don’t let it shake your confidence. It happens, and you’re not alone. I’ve been through it, and the people you meet during those tough times often stay in your network and help you grow. Sometimes, a layoff is the push you need to shift directions, learn something new, or find a role that fits you even better.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c05ffb606e33" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vouch-engineering/engineering-spotlight-ez-habimana-c05ffb606e33">Engineering Spotlight: EZ Habimana</a> was originally published in <a href="https://medium.com/vouch-engineering">vouch-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Engineering Spotlight: Shaun Payne]]></title>
            <link>https://medium.com/vouch-engineering/engineering-spotlight-shaun-payne-4e3aeba53a25?source=rss----d4246e699295---4</link>
            <guid isPermaLink="false">https://medium.com/p/4e3aeba53a25</guid>
            <dc:creator><![CDATA[Engineering @ Vouch]]></dc:creator>
            <pubDate>Thu, 12 Dec 2024 18:10:50 GMT</pubDate>
            <atom:updated>2024-12-12T18:09:14.202Z</atom:updated>
            <content:encoded><![CDATA[<p>Shaun Payne | Software Engineer</p><h4><strong>Tell us a bit about yourself? Who you are and what’s your background?</strong></h4><p>Hello and welcome, I’m Shaun, a Software Engineer at Vouch. I’m one who always loved playing with the latest emerging technology but took the scenic route on the way over to engineering. I have an innate drive for learning and a constant curiosity about what the future holds. Originally from Pleasanton, CA, I’ve since nestled into San Francisco, where I enjoy the tech community and the abundance of personal stories from people trying to, or succeeding at, affecting change.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tSFhcTvPWz7t12d6B05T3Q.jpeg" /></figure><h4><strong>Can you share a bit about your journey to engineering?</strong></h4><p>My path to engineering started in a completely different world: operations. I spent years leading teams and building processes, thriving on the challenges of problem-solving and helping others grow. Operations felt like home — but sometimes, a new home comes calling.</p><p>In my role previous to Vouch, I had my first exposure to working alongside a fully functioning engineering team in the same location. I seized the chance to observe engineers at work and understand what they did, all while managing an eclectic team of full-timers, part-timers, contractors, and on-demand workers. At one point I was told we needed to speed up the delivery timeline and was given free rein to hire more on-demand workers. I politely declined and told my boss, “I’ll do it faster — with no people.”</p><p>I recalled a tool I previously used to automate mouse and keyboard actions. That Thursday, I spent a few hours coding a script to interpret the data and handle the task. On Friday, I set up a spare computer, launched the script, and watched it go. At first, colleagues thought it was a training video, but when they realized it was live automation, word spread quickly. People stopped by to snap pictures and videos as the program completed the two-week project in just nine hours.</p><p>While I understand now, as an engineer, I would tackle the task differently, it still captivated my coworkers and earned me the inaugural Employee of the Month award. More importantly, it became a pivotal life moment, grabbed hold of my interests, and set a new career North Star.</p><p>I realized many of the items I liked about operations, helping others and solving challenging puzzles, were also waiting for me in engineering. I began mapping out a career switch and, when COVID hit, seized the opportunity. I went back to school to deepen my technical knowledge and enrolled in a coding bootcamp to sharpen my skills. Then I got the opportunity to join Vouch, which signaled the official commencement of my new future.</p><h4><strong>What initially attracted you to Vouch and how has your experience been so far?</strong></h4><p>I was originally looking for two things. One, I had just previously come from operations for a tech-forward insurance company, so I wanted to find something relatively similar with opportunities to work alongside insurance operations. Two, I wanted to find a place that, in good or tough times, wasn’t going to lose sight of what’s truly critical, the people. That second one can be a bit of a tall order and also easy to portray a facade to someone looking in through the window instead of being inside the house. There is no loss in shooting high, but if you aim low, you’ll end up low.</p><p>The experience has been exhilarating! Vouch originally worked with my knowledge and understanding to place me in a position to thrive. I was given ample opportunity to interact directly with internal operations teams and teams alike, understand their pain points, and be able to work on solutioning and implementation, blurring the lines of engineer and PM in some cases. Utilizing my previous knowledge while ramping on engineering principles catapulted my ability to soak in understanding and learn (and deliver).</p><p>I recently joined a new team under unique circumstances — the manager was initially unknown, so the director spoke directly with potential team members. During our conversations, I shared details about my goals, some of which I considered minor. When I joined, my new manager referenced those details, emphasizing how Vouch and its leadership had worked to place me in a role aligned with my aspirations. It’s one thing to hope leadership is paying attention; it’s another to see them actively listen and take action.</p><p>I could keep going with examples, but all to say, I’ve found a place that treats me as Shaun and is willing to meet me in the middle to make sure we’re all successful.</p><h4><strong>What’s life like outside Vouch?</strong></h4><p>Mountains, learning new things, and contemplating the future.</p><p>I grew up visiting backcountry Yosemite, which later developed into leading groups on those same excursions. As of late, you’ll find me hiking the local hills of San Francisco. The mountains are a rare retreat — a place to recharge, find inspiration, and reflect on life.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/461/1*m1cEDzWJNQo6bBKs8Pplhw.jpeg" /></figure><p>I’m also passionate about learning and am constantly diving into new hobbies or skills. Whether it’s experimenting with programming side projects, reading about emerging technologies, or picking up a random DIY project, I enjoy staying curious and challenging myself to grow.</p><p>And when I’m not exploring or learning, I love thinking about the bigger picture — pondering what the future holds for technology, society, and the world. I find myself drawn to questions that stretch the imagination: <em>Is knowledge finite? What would existence look like if humanity understood 0.01%, 0.1%, 1%, or even 10% of all current knowledge?</em> I think about how our world might evolve if the average job requires an intellect beyond today’s human average, or whether we could one day accumulate enough information to reconstruct the past so perfectly that someone, somewhere, might even be reading my thoughts right now.</p><p>Of course, I also enjoy life’s simpler pleasures: spending time with friends, discovering new SF dining spots, and unwinding with a good sci-fi flick.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4e3aeba53a25" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vouch-engineering/engineering-spotlight-shaun-payne-4e3aeba53a25">Engineering Spotlight: Shaun Payne</a> was originally published in <a href="https://medium.com/vouch-engineering">vouch-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Leveraging the Adapter Pattern to Decompose the Monolith]]></title>
            <link>https://medium.com/vouch-engineering/leveraging-the-adapter-pattern-to-decompose-the-monolith-c6bd6256a117?source=rss----d4246e699295---4</link>
            <guid isPermaLink="false">https://medium.com/p/c6bd6256a117</guid>
            <category><![CDATA[hexagonal-architecture]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[broker-platform]]></category>
            <category><![CDATA[carrier-platform]]></category>
            <dc:creator><![CDATA[Engineering @ Vouch]]></dc:creator>
            <pubDate>Mon, 11 Nov 2024 17:58:02 GMT</pubDate>
            <atom:updated>2024-11-11T17:12:28.687Z</atom:updated>
            <content:encoded><![CDATA[<p>Alex Mann | Senior Software Engineer</p><p>Vouch launched our Carrier Platform to beta this summer. It represents the culmination of a herculean amount of work across almost every vertical in the company, all focused on the same goal: bring Vouch’s core insurance product offerings to the bleeding edge of the industry. The Carrier Platform isn’t just an engineering accomplishment. It also marks a change in Vouch’s disposition towards the insurance marketplace in the business sense and a meaningful step in the company’s plan to move away from our old monolithic architecture. Before, we didn’t have a clear distinction between broker and carrier — concepts that are core to the industry. We’ve realized that these two subunits of our business have wildly different concerns, and in order to start decomposing our existing monolith and move to a microservice pattern we had to start drawing conceptual domain boundaries between them. In short, we needed the carrier, but we also needed the broker, and neither one can have an undue influence on how the other operates. In this post we’ll take a look at what that decomposition looks like, primarily from the broker’s perspective, and how those conceptual domain boundaries were drawn.</p><p>There’s no guarantee that Vouch’s Carrier Platform has the same interface for making requests for coverage as any other platform. In fact, we know in a few cases already that there are non-trivial differences. The same will be true across the system from the underlying data models, to the APIs that drive the applications, to the very insurance products themselves that are being sold. A broker shouldn’t choose one paradigm as gospel. That would preclude the possibility of integrating with all the others.</p><p>Our introspection has most often come in the form of asking ourselves the question, “Is this how a broker would want this feature to work?” Sometimes the answer isn’t simple. These things don’t exist independently of one another. Meaning, what a broker <em>is</em> will inevitably be influenced by what a carrier <em>is</em>. It’s a goal of both Vouch platforms to be able to integrate with external partners: our carrier should be able to accept requests and produce quotes for external brokers, and our broker should be able to give requests and receive quotes from external carriers. There are no hard and fast rules for how these integration points should work, which puts the onus on us to be prepared for anything.</p><p>So if we have to write some bespoke code for every new partner integration anyway, what does it matter? It’s true, there will be some of that work. And this is where those conceptual domain boundaries go from being technical design aspirations to real, tangible force multipliers. The key is in making sure the core broker behavior remains only <a href="https://en.wikipedia.org/wiki/Loose_coupling#:~:text=Coupling%20refers%20to%20the%20degree,which%20provides%20the%20required%20behavior.">loosely coupled</a> with the carrier-specific functionality. All engineers are familiar with concepts like tight versus loose coupling, encapsulation, and abstraction. This should be nothing new. We’re just formalizing a rule across the broker domain to enforce the separation between broker and carrier specifics with a pattern referred to as <a href="https://en.wikipedia.org/wiki/Hexagonal_architecture_(software)">hexagonal architecture</a>. It’s the line on which we’ve split the monolith in two.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wkENL1u7SI6-6pF_" /></figure><p>For every broker service that will be communicating with carriers in some way, we’re establishing a base set of “actions” that represent all the ways our digital broker needs to communicate with carriers. One of these actions might be “ingest quote”. Our core broker logic shouldn’t know the particulars of whether the quote is fetched through polling some REST endpoint or given to us via a webhook. The core logic knows: given some identifier, for some carrier, the digital broker needs this quote. The carrier specific work is separate from the core of the broker application, and depending on which carrier we’re ingesting from at runtime, we can call the classes that encapsulate that carrier’s behavior (referred to as “Adapters”).</p><p>The result is that a change to how a carrier does business only impacts a tiny portion of the application and doesn’t risk service interruptions for any other carriers. And onboarding a new carrier involves just building the functionality to interact with them in the ways that they prescribe. The core application is agnostic and unchanged. The product is easy to reason about and, therefore, more maintainable. Engineers don’t need to keep the full context of carrier interactions in their heads when working with these services. Individual Carrier Adapters can be mocked out to test the core application, and each adapter can be tested independently. We also achieve high code reuse by that core logic being the same for every onboarded partner.</p><p>Complexity is the norm in both software and insurance. That makes it especially important for us to find ways to scale our engineering efforts. This, at the end of the day, is the driving force behind our vision for the Vouch Digital Broker. We’re building this to help realize that goal of offering a first-class digital broker experience. We’re doing it in a way that enables our team to ship quickly and efficiently and maximizes the impact to the business.</p><blockquote>When we say “Carrier Platform”, we mean an independent software system responsible for accepting requests for coverage and producing quotes. The Vouch Carrier Platform is itself an example of microservice oriented architecture and was the first big step in decomposing our old monolithic system. The “Broker Platform” is an analogous system meant to handle the generation and submission of those requests for coverage and the consumption and dissemination of the quotes that a carrier would produce.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c6bd6256a117" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vouch-engineering/leveraging-the-adapter-pattern-to-decompose-the-monolith-c6bd6256a117">Leveraging the Adapter Pattern to Decompose the Monolith</a> was originally published in <a href="https://medium.com/vouch-engineering">vouch-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Engineering Spotlight: Connie Wang]]></title>
            <link>https://medium.com/vouch-engineering/engineering-spotlight-connie-wang-11caef516697?source=rss----d4246e699295---4</link>
            <guid isPermaLink="false">https://medium.com/p/11caef516697</guid>
            <dc:creator><![CDATA[Engineering @ Vouch]]></dc:creator>
            <pubDate>Tue, 07 Nov 2023 18:34:52 GMT</pubDate>
            <atom:updated>2023-11-07T18:34:52.033Z</atom:updated>
            <content:encoded><![CDATA[<p>Connie Wang | Senior Software Engineer</p><p><strong>Tell me a bit about yourself? Who you are and what’s your background?</strong></p><p>Hey everyone, I’m Connie! I’m a Senior Software Engineer on the Carrier Management team at Vouch, where I work on backend services that manage and enable our proprietary insurance products. I was born and raised in New Jersey, went to college in Chicago, and now I live in Manhattan, New York.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/460/0*hSaAp_4zGHUSiJXC" /></figure><p><strong>Why did you choose to be an engineer?</strong></p><p>My career journey was a little unconventional — I studied Piano Performance and Cognitive Science as my undergraduate majors at Northwestern University. I was first introduced to computer science through my cognitive modeling and human-computer interaction classes. I became interested in the technical aspects of this work and what practical applications these theories have for the digital world we live in today. I took some basic computer science courses and realized that the process of learning a computer language and building something felt very similar to me as the process of studying and performing music. It was a fascinating parallel to discover, so I decided to explore this further after graduating by enrolling in a bootcamp program for full stack web development.</p><p>The bootcamp was an intense but exhilarating three months of fully immersing my brain in programming — there were many nights where I dreamt about my code and the projects I was working on. As crazy as it was, I loved the experience and was excited to take these new concepts and apply them to the real world.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ycLqukLu1pufK62y" /></figure><p><strong>Tell me a bit about your overall career journey. How did you get to where you are now?</strong></p><p>After graduating from my bootcamp, I joined what was then a seven person seed stage startup. Our engineering team consisted of three people, we worked with our product person who was also our designer every day, and we had daily standups with the entire company including our CEO. I loved the early stage startup experience — it was so exciting participating in the full lifecycle of a feature and watching my code go out to production every day.</p><p>I quickly learned the difficulty of finding product-market fit, and the challenging role funding played in the prioritization of features and product direction. I also learned humility and how important it was to set aside pride and being willing to start all over again, which our team did one pivotal summer where we built a fully functioning marketplace in three months.</p><p>Our company went through several rounds of funding, and we grew into a team of over 60 employees. Through this process I encountered many difficulties related to scaling, both on the technical and organizational sides. It was a really exciting and fun journey, but after four years, I was ready for a change and found Vouch!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8SyNN26AgpwuP6Np" /></figure><p><strong>How’s your experience been at Vouch since you joined us?</strong></p><p>Initially what drew me to Vouch was the people and culture that I felt during my interviews. I immediately felt welcomed and supported, even during stressful technical interviews which is a rarity! During the onboarding process, I was happy to discover that my initial feelings were indeed correct. I was so impressed with how thoughtfully everyone talked about Vouch’s values and culture and it gave me confidence that I made the right choice in joining Vouch.</p><p>During my time here at Vouch, I’ve had the opportunity to work on a pretty wide variety of projects. Something cool we’ve been working on in the past year is getting started on work to decompose our Ruby on Rails monolith into Typescript services. It’s been a great learning experience being part of that process, from the planning to the implementation and rollout. Another great learning experience and one that is more unique of Vouch is being able to work directly with our actuarial team and work on projects to support their work. All in all, I can confidently say that I have a much better understanding of insurance now than when I first started (which was close to nothing), and that has turned out to have some very helpful applications in my personal life!</p><p><strong>What’s life like outside of Vouch?</strong></p><p>Outside of work I like to explore new places via travel and hiking. I also try to stay involved with local community organizations in New York City. I was previously on the volunteer board for a nonprofit organization called Taiwanese American Professionals, where we hosted various cultural and social events throughout the year. While I am no longer on the board I still like to volunteer and support various initiatives when I can.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=11caef516697" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vouch-engineering/engineering-spotlight-connie-wang-11caef516697">Engineering Spotlight: Connie Wang</a> was originally published in <a href="https://medium.com/vouch-engineering">vouch-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Implementation of a System-Wide Event Bus]]></title>
            <link>https://medium.com/vouch-engineering/implementation-of-a-system-wide-event-bus-094e4db54ab4?source=rss----d4246e699295---4</link>
            <guid isPermaLink="false">https://medium.com/p/094e4db54ab4</guid>
            <dc:creator><![CDATA[Engineering @ Vouch]]></dc:creator>
            <pubDate>Mon, 23 Oct 2023 16:17:21 GMT</pubDate>
            <atom:updated>2023-10-23T16:32:29.417Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Ryan | Senior Software Engineer</strong></p><p>One of Vouch’s strategic advantages is its ability to use technology to bring new insurance products to market quickly. In order to serve the innovative companies of the future, Vouch needs to take what otherwise would be a long 6 month manual process of bringing new insurance products to market and do it in a matter of weeks. That way, when a bunch of new-age generative AI companies need insurance coverage for when their AI model goes off the rails, Vouch is the first to offer products to support them 👍</p><p>But Vouch itself realizes that not all jobs can be fully automated with technology (at least not yet!). Instead, there’s a lot of value in enabling humans to be supercharged with technology. And that is definitely the case for Vouch as an insure-tech that deals with manual and asynchronous processes.</p><p>To enable critical business processes between underwriters, actuaries, brokers, and insurance advisors, Vouch engineering made the strategic decision to introduce an event-driven design into our foundational architecture.</p><p>Besides asynchronous processes, another big part of enabling these internal employees to be successful is to enable them to work not only with the tools they are experts in but also with the data they need. This near real-time data synchronization problem is another big reason why we decided on an event-driven system in our backend infrastructure.</p><p>Last but not least, we are also leveraging this event-driven system to embark on the journey of breaking apart our Ruby on Rails monolithic architecture (which was a great fit in our early startup days) into a TypeScript service-oriented architecture to allow our organization to grow scalably.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m9yTQhbEFm7lf5WYVQ639w.png" /></figure><p>The process of bringing this event-driven system online was a great example of how Vouch does engineering.</p><p>After centralizing on the idea of bringing an event-driven system via cross-level discussions between ICs, managers, and CTO, our team elected a tech lead to do a deep dive into our options from hosting open-source tools to fully managed cloud solutions.</p><p>We ultimately landed on using a cloud managed Kafka solution to leverage a battle-tested open source solution that did not require significant amounts of time to stand up infrastructure for. After selecting the technology, we started our design process which is a combination of written technical design as well as fast prototype code where text doesn’t tell as a compelling story as a little bit of working code can.</p><p>Since we agreed upon the event-driven system as a group to start, getting the team aligned and adopting this centralized infrastructure has been super easy, with teams coming to the tech foundations team as soon as they see use-cases where our new event system could accelerate their work.</p><p>That being said, we’re still pretty early on our journey of seeing the full value enabling this asynchronous, domain-driven, data synchronizing system-wide event bus. So if you’d like to be involved in building the future of insurance innovation come join me on the tech foundations team of Vouch!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=094e4db54ab4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vouch-engineering/implementation-of-a-system-wide-event-bus-094e4db54ab4">Implementation of a System-Wide Event Bus</a> was originally published in <a href="https://medium.com/vouch-engineering">vouch-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[2023 Vouch Engineering Offsite]]></title>
            <link>https://medium.com/vouch-engineering/2023-vouch-engineering-offsite-f953c2822adb?source=rss----d4246e699295---4</link>
            <guid isPermaLink="false">https://medium.com/p/f953c2822adb</guid>
            <dc:creator><![CDATA[Engineering @ Vouch]]></dc:creator>
            <pubDate>Wed, 02 Aug 2023 16:37:38 GMT</pubDate>
            <atom:updated>2023-08-02T16:19:08.197Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Steven Banks | Lead Software Engineer</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Nrg04GTDeqbPseSEI8XRKQ.jpeg" /></figure><p>The 2023 Vouch Engineering Offsite meeting took place from June 14–16 at “The Den” in Columbus, Ohio. I had only been at Vouch for about two months, so I had only seen a few of my co-workers in person at a collaboration event a few weeks back. This was going to be the entire engineering team in one place at the same time. I did not know what to expect, but I was excited.</p><p>The day kicked off with a spotlight session in which a representative from every team presented their team’s responsibilities and revealed a team mascot. This was highly entertaining as each team described why their mascot represented the team. I am on the Client Servicing team and our mascot was the platypus, with its adaptability accurately reflecting our team’s diverse skill set and capacity to work across diverse platforms, languages, and technologies.</p><p>Each session on the first day had some sort of presentation followed by an interactive part where the entire group got to participate. This was a great idea to get involvement and discussion from everyone and not just sit through lectures. The first session concluded with a game of Vouch Jeopardy with some work-related categories to test our knowledge. The game was a great way to stir up some competition and see how much we could recall about Vouch engineering and insurance in general.</p><p>The second session featured our Senior Staff Engineer Don Valdez giving a talk about System Architecture and the process for migrating from a monolith to a microservices architecture. It ended with each table taking a domain concept, trying to describe all the actions/verbs of that domain, with the end goal of trying to understand what a microservice for that domain might look like.</p><p>After a good lunch we reconvened and discussed developer experience and observability and monitoring across our vast array of services and dependencies. It can be a bit overwhelming to think of all the pieces/parts that go into Vouch’s technology stack. We concluded the session with each table attempting to diagram all of those services and dependencies, and thinking through things like throughput, single points of failure, and communication links between services.</p><p>The last bit of the day was a fun team-building exercise where the entire group split up into groups of designers and builders. Our job as teams was to build a lego hedgehog. Seems easy, but the catch was that only the lead designer could see how the hedgehog was to be built. They had to describe it using only words to the lead engineer, who then had to take those directions to the team of builders. Describing a lego construction with no pictures seemed like a near impossible task. And we were given a 20 minute time limit to do it all! Well some teams succeeded and others not so much, but each team had a great time in the process!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UuzO_vqto9yW4Z5SJ8mtrw.jpeg" /></figure><p>Day 2 was all about the hackathon. Ideas were compiled beforehand and people chose which idea they wanted to work on. We split the groups between two locations so that there was room to work. There were five members of my team and we worked on a CLI tool to spin up all the dependent services required for a single application or workflow when running our software locally. We worked on it for a maximum of about six hours with each team member making contributions along the way. The entire group reconvened over at the Den and we began presenting our work.</p><p>The team presentations were impressive, to say the least. In less than a full day of work, every group was able to accomplish a lot. Some of the projects worked on were a proprietary feature flag solution, a health check for all our services that goes beyond standard heartbeat functionality, a ChatGPT solution to get information about potential customers, as well as helper projects such as auditing/updating system tests, a document marathon to update stale documentation, generating automated test data, and adding more automation to our deploy process.</p><p>There are initiatives to expand this hackathon in the future to allow for more time. It’s exciting to think about how much the engineering team could improve their work environment and create innovative solutions if given more time. This would be time well spent, for sure.</p><p>The day concluded with everyone heading over to 16-Bit Bar+Arcade for the Engineering Cocktail Hour. This was a unique location. It was completely open to the street and inside were a bunch of retro pinball machines and arcade games. What a great venue for this group! Good times were had by all, and who could pass up a game or two of the classic NBA Jam or NFL Blitz?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hYEQXY5L_bbfvyo-hb3ImQ.jpeg" /></figure><p>After this, each team headed to a local restaurant to have a team dinner. Our team headed to “The Pearl”, a fancy steak/seafood restaurant. This was an enjoyable time to connect with members of your team in an informal setting. After dinner, some folks stayed and hung out afterwards in the area.</p><p>I did not know what to expect going into this meeting but I was pleasantly surprised. I think the best part about all of it was just seeing my coworkers face to face and getting a chance to get to know them better. That level of interaction was priceless and somewhat difficult to come by in a remote working environment. Seeing folks over Zoom is just not the same, so the value that an offsite meeting like this brings is immeasurable and definitely brings the team closer together. Overall these couple days were a smashing success, and I’m already looking forward to the next one.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f953c2822adb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vouch-engineering/2023-vouch-engineering-offsite-f953c2822adb">2023 Vouch Engineering Offsite</a> was originally published in <a href="https://medium.com/vouch-engineering">vouch-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Engineering Spotlight: Brant Morton]]></title>
            <link>https://medium.com/vouch-engineering/engineering-spotlight-brant-morton-99857db04f73?source=rss----d4246e699295---4</link>
            <guid isPermaLink="false">https://medium.com/p/99857db04f73</guid>
            <dc:creator><![CDATA[Engineering @ Vouch]]></dc:creator>
            <pubDate>Tue, 20 Jun 2023 15:59:44 GMT</pubDate>
            <atom:updated>2023-06-20T15:59:44.689Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Brant Morton | Senior Software Engineer</strong></p><p><strong>Tell me a bit about yourself- who you are and what’s your background?</strong></p><p>Hey, I’m Brant! I’m a Senior Software Engineer at Vouch on our Insurance Distribution team, where I spend my time building out our Insurance API. I was born and raised in Westlake, Louisiana, and after a few years of traveling around the lower 48, my wife and I landed at our current home base outside of Portland, Oregon.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5HIFQRtelW8xRyX7MjVqBQ.jpeg" /></figure><p><strong>Why did you choose to be a Software Engineer? Did you know you always wanted to be an engineer? What was your career journey like?</strong></p><p>As a kid, I was always interested in STEM subjects, and growing up along the Gulf Coast, I had a great deal of exposure to the oil and gas industry. This combination led me to believe that the most fulfilling career path for me would be as a Chemical Engineer in an industrial plant. Thinking I knew exactly what I wanted, I pursued a bachelor’s degree in Chemical Engineering from McNeese State University, took a few internships, and went on to work at an oil refinery in my hometown after graduation. I quickly realized that the career path I was on, was not where I wanted to be.</p><p>I wanted to solve hard problems. I wanted to express creativity. I wanted to build things. The internet told me that I might like Software Engineering, but I didn’t want to go back to school for Computer Science. Fast forward a few months — I took a trip to San Francisco, and my Uber driver told me that he was an Engineer and had co-workers that were self-taught developers. This sent me down a rabbit-hole of online resources on learning how to code, and all of the sudden, I knew exactly what I wanted to do. I began learning the basics of Python through online learning platforms. Eventually, I’d pick up HTML, CSS, and JavaScript fundamentals. After my day job, I’d work on side projects and build websites for local businesses to have a portfolio that would one day impress an employer. After ten months of effort and hundreds of job applications, I finally landed my first Software Engineering position with an EdTech SAAS startup.</p><p>I worked remotely for this company for a few years as a Full Stack developer. The company had a fantastic engineering culture, and I had great mentors along the way. It was a wonderful opportunity to build impactful products with new technologies, and learn what it’s like to work in a fast-paced startup. Eventually, I felt that I was ready for new challenges, so I joined Vouch as a Senior Engineer.</p><p><strong>What do you like to do outside of work? Any hobbies or activities?</strong></p><p>I spend most of my free time hiking around the magical Pacific Northwest with my wonderful wife and our mischievous dogs. I also spend a lot of time bouncing around local coffee shops and pondering on who pulls the best espresso, which is an endless journey in this part of the United States. When I’m not home, I’m probably camping in my van across our country’s vast wilderness, at the driving range trying to fix my golf swing, or catching a baseball game.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/378/1*8FV5FFit-U0P5vSResOHGQ.jpeg" /></figure><p><strong>Why did you choose to join Vouch? What attracted you to Vouch &amp; was it what you expected? What are you looking forward to?</strong></p><p>While working at my last company, a Vouch recruiter reached out and asked if I’d be interested in a role working in a technology stack that I was familiar with. I hadn’t heard of Vouch at the time, but Vouch’s engagement in the startup ecosystem impressed me. The recruiter really sold me on the company’s values, and it was apparent that Vouch’s commitment to putting people first and facing challenges head on were directly aligned with what I wanted in my next company. I was interested enough to go through the interview process.</p><p>After going through the loop and meeting all of the folks I’d be working with, it was immediately clear that this was a place that I’d want to work. Two things were consistently apparent in the interviews:</p><ol><li>These people were <em>really</em> collaborative and always willing to help others. It was evident that I would have the support needed to continue to grow in my engineering career.</li><li>I would be challenged in the best way. Challenging the status quo of business insurance is in the fabric of the company. People at Vouch are ready for tough problems. They’ll beat them and learn a lot along the way.</li></ol><p>After a year at the company, the aspects of Vouch that initially attracted me have proven to remain my favorite parts of working here. I’ve encountered endless amount of growth opportunities, and I’ve been so supported along the way. The business is growing rapidly, and as we move into the next phase of growth, I’m looking forward to even more complex challenges we have on the horizon.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=99857db04f73" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vouch-engineering/engineering-spotlight-brant-morton-99857db04f73">Engineering Spotlight: Brant Morton</a> was originally published in <a href="https://medium.com/vouch-engineering">vouch-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Vouch launched an Internship Program]]></title>
            <link>https://medium.com/vouch-engineering/how-vouch-launched-an-internship-program-4298082a3416?source=rss----d4246e699295---4</link>
            <guid isPermaLink="false">https://medium.com/p/4298082a3416</guid>
            <dc:creator><![CDATA[Engineering @ Vouch]]></dc:creator>
            <pubDate>Fri, 05 May 2023 16:21:16 GMT</pubDate>
            <atom:updated>2023-05-05T16:21:16.111Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Judy Situ | Senior Recruiting Coordinator</strong></p><p>As a recruiting coordinator, you typically work behind the scenes throughout the hiring process. I wasn’t expecting it to be any different when I started with Vouch in January 2021, but I’m glad that I was wrong. About two or three months after I started with the company, I was given an opportunity to build and manage our summer internship program. Taking this on was daunting since there wasn’t an existing process in place, but I was excited by the challenge.</p><p>With an approved headcount and the support of my team, off I went. Unfortunately, I had no idea where to start. Imposter syndrome was a frequent and unwelcome visitor, but one of my teammates gave me some great words of comfort. They said, “Anything you come up with this year will only be an improvement from last year.” It was hilariously simple, but it gave me the confidence to trust in myself to continue moving forward. So with that mantra in mind, I journeyed on a little less nervous and a little more self-assured. That comment stuck with me throughout that first year, and still does.</p><p>The first year is a bit of a blur. I spent a lot of my time researching and gathering ideas on how to manage a successful internship program. Some clear snapshot memories I have are reviewing resumes, begging schools on Handshake to approve our company, and reaching out to various school programs and clubs to diversify the range of sources for applicants. In the end, what came out of it was a clumsily sketched blueprint for a 10-week summer internship program and a backbone for the hiring process behind that.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c68s6yP_0w6edfJ-lH5x9Q.jpeg" /></figure><p>A very common term you hear associated with start-ups is “scrappy”. There truly is no other word more fitting than “scrappy” to describe that first year of creating the Vouch Internship Program. Despite the challenges, we got the program launched and ended up with six fantastic interns that summer. One of them was given a part-time position until the end of the year, and three others moved into full-time roles with the company.</p><p>Looking back, it was far from perfect, but I’m incredibly proud of what we accomplished that year.If I could give one piece of advice on creating an internship program or something similar from scratch, it would be documentation. One of the most helpful things we did during that first year was to document every step of the process. We recorded ideas from brainstorming and research sessions, created various templates and scripts, and jotted down timelines and feedback. This documentation was important because it became our roadmap for the following years. We used it to identify pain points and plan ways to improve the hiring experience for both candidates and hiring managers.</p><p>Since launching the program we’ve continued to look for ways to refine and enhance it each year. We’ve learned from our mistakes, implemented new ideas, and made improvements along the way.</p><p>In 2021, we were primarily focused on throwing together an intern hiring process as well as roughly sketching out what the 10-week internship program would entail. It wasn’t much, but it gave us something to work off of for the next year. In 2022, we worked on creating more structure for the hiring managers during the hiring process. This included things like stricter timeliness, and direction on creating interview panels and making decisions. This year we’re collaborating more closely with our HR partners and Office Ops with the intention of providing a meaningful and intentional 10-week summer internship program experience for our interns.</p><p>A few highlights we offer in the internship program:</p><ul><li>A comprehensive and structured onboarding plan to make sure our interns feel supported from day 1</li><li>Direct mentorship and valuable development opportunities</li><li>Celebrating Intern Appreciation Day</li><li>Wrapping up with Intern Presentations on projects that they worked on over the summer</li></ul><p>The most rewarding perk that came out of this experience was, and still is, the opportunity to build something from scratch and watch it evolve over time. That’s certainly been the case with our internship program at Vouch. It started out as a blueprint of ideas, but with scrappy persistence and a commitment to providing the best hiring and internship experience, the summer internship program now has a structure with many support systems that help it run. Looking back and seeing how the hiring process went this year, as well as the steps we’re currently taking to focus on the growth of the intern, I am immensely proud of what we have built. At the same time, I fully expect that the process we worked with this year will look different next year, but that’s all the more exciting because it can only improve.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4298082a3416" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vouch-engineering/how-vouch-launched-an-internship-program-4298082a3416">How Vouch launched an Internship Program</a> was originally published in <a href="https://medium.com/vouch-engineering">vouch-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introducing Quadipy: A Python package for transforming structured data into RDF graph format]]></title>
            <link>https://medium.com/vouch-engineering/introducing-quadipy-a-python-package-for-transforming-structured-data-into-rdf-graph-format-9c074c5b5704?source=rss----d4246e699295---4</link>
            <guid isPermaLink="false">https://medium.com/p/9c074c5b5704</guid>
            <dc:creator><![CDATA[Engineering @ Vouch]]></dc:creator>
            <pubDate>Tue, 18 Apr 2023 15:58:50 GMT</pubDate>
            <atom:updated>2023-04-18T16:34:11.409Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pnLG3p51FgOE9WrKnnYtpw.png" /></figure><p>Vouch is excited to announce the release of <a href="https://github.com/vouch-insurance/quadipy">Quadipy</a>, Vouch’s first open-source project. Built by the Vouch engineering team, Quadipy is a Python package that makes it easy to transform structured data into RDF graph format.</p><p>Quadipy was designed to help developers build a config-based ingestion pipeline to an RDF data store, similar to FiveTran or Stitch, but for RDF. Quadipy doesn’t handle connections to different systems, but it allows you to configure the RDF data you want to create from any data source. By leveraging RDFLib, Quadipy can pythonically structure RDF data, making it easy to work with and manipulate.</p><p>We believe that Quadipy can be a powerful tool for anyone who needs to work with RDF data. Whether you’re building a knowledge graph or working with other kinds of structured data, Quadipy makes it easy to convert that data into RDF graph format, opening up new possibilities for data analysis and exploration.</p><p>We’ve used this config-based system to build out our internal knowledge graph at Vouch, and gave a talk on the subject at KGC’22: “<a href="https://www.youtube.com/watch?v=MTZD5VTV4NI">Modeling the startup ecosystem using a config-based knowledge graph.</a>”</p><p>So what can you do with Quadipy? Here are just a few examples:</p><ul><li><strong>Build a knowledge graph:</strong> Quadipy makes it easy to convert any structured data into RDF graph format, which is perfect for building knowledge graphs. By leveraging the power of RDF, you can create a network of relationships between entities and unlock new insights from your data.</li><li><strong>Ingest data from multiple sources:</strong> Quadipy allows you to configure the RDF data you want to create from any data source. This means you can ingest data from multiple sources and transform it into a unified RDF graph format.</li><li><strong>Perform data analysis and exploration:</strong> Once your data is in RDF graph format, you can use a variety of tools and techniques to perform data analysis and exploration. For example, you could use SPARQL queries to explore relationships between entities, or use graph visualization tools to gain new insights from your data.</li></ul><p>We believe that Quadipy can be a powerful tool for anyone who needs to work with RDF data. Whether you’re building a knowledge graph, ingesting data from multiple sources, or performing data analysis and exploration, Quadipy makes it easy to convert your data into RDF graph format.</p><p>To get started with Quadipy, check out <a href="https://github.com/vouch-insurance/quadipy">our repo on GitHub</a> or the <a href="http://quadipy.vouch.us/">documentation</a>. We hope that you’ll find Quadipy useful and that you’ll consider contributing to the project. We’re excited to see what new things people will build with Quadipy!</p><p><em>A big shout out to Avy Faingezicht, Nabeel Zewail, Shuyu Wang, Nikki Hu, Emily Ekdahl, and Conor Mongey for their contributions to the project, their dedication and hard work have been invaluable to its success.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9c074c5b5704" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vouch-engineering/introducing-quadipy-a-python-package-for-transforming-structured-data-into-rdf-graph-format-9c074c5b5704">Introducing Quadipy: A Python package for transforming structured data into RDF graph format</a> was originally published in <a href="https://medium.com/vouch-engineering">vouch-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Advantages of Blue-Green Deployment with dbt and Snowflake]]></title>
            <link>https://medium.com/vouch-engineering/the-advantages-of-blue-green-deployment-with-dbt-and-snowflake-8777c05562f3?source=rss----d4246e699295---4</link>
            <guid isPermaLink="false">https://medium.com/p/8777c05562f3</guid>
            <category><![CDATA[blue-green-deployment]]></category>
            <category><![CDATA[dbt-labs]]></category>
            <category><![CDATA[snowflake]]></category>
            <dc:creator><![CDATA[Jenna Allen]]></dc:creator>
            <pubDate>Thu, 09 Mar 2023 20:06:51 GMT</pubDate>
            <atom:updated>2023-03-09T20:06:51.143Z</atom:updated>
            <content:encoded><![CDATA[<p>Jenna Allen | Senior Data Analytics Engineer, Data</p><p>Data quality is crucial at any data-driven organization like Vouch. Poor data quality leads to inaccurate or incomplete insights, which in turn lead to poor decision making and a loss of credibility with stakeholders. To ensure the accuracy and completeness of our data, we recently updated our data modeling pipelines to use a <a href="https://en.wikipedia.org/wiki/Blue-green_deployment">blue-green deployment process</a>. In a blue-green deployment, we run dbt models in a separate database, and only “graduate” data to user-facing databases for models that are computed successfully, isolating data users when either upstream systems or our modeling code produce bad outputs.</p><p>In this post, we will cover what dbt is, some of the problems that we solved with blue-green deployment, and how we set it up.</p><p>Problems we aimed to solve</p><ol><li>When our pipeline fails, it can either fail to build the models or tests can fail. For us, we most frequently encountered failing tests, which meant we were exposing our end users to bad outputs. We wanted a way to build and test the data in an isolated environment to ensure quality before it reached our end users rather than building and testing in production.</li><li>We wanted a way to implement “build-breaking” tests depending on their impact on the pipeline. Build-breaking tests are those that will stop the pipeline from running if they fail, while less severe tests will only generate an alert. dbt allows users to configure test severity as ‘warning’ or ‘error’. However, we found that warnings were being ignored and getting lost in Airflow run logs. Errors would stop the entire pipeline from running. We had a need to be alerted in an obvious way about some test failures that shouldn’t stop the entire pipeline.</li><li>We wanted a way to reduce the sense of urgency experienced by the data on-call person when tests failed. Classifying tests as “build-breaking” or “not build-breaking” allows engineers to have a clearer understanding of the impact of any issues that are identified by the tests and to take appropriate action to fix them.</li></ol><p>At Vouch, we use ELT, where data is extracted, loaded into the database, and then transformed. We use dbt’s open-source command-line tools to build and maintain reliable data pipelines, which provides a way to develop, test, and deploy database changes and enables the definition of data models (i.e. tables and views) across the organization. dbt handles the transformations or the ‘T’ in this ELT process. Using dbt’s open source library, raw data is modified, cleaned, and standardized for use by various stakeholders. The dbt testing framework allows engineers and analysts to define tests (e.g. not null, unique, accepted values, etc.) for their data models. These tests can be used to identify issues with the data pipeline and to ensure that changes to the pipeline don’t break existing models.</p><p><strong>Blue-green deployment to the rescue</strong></p><p>One way to address the problems outlined above is through the use of a blue-green deployment. The motivation for using this solution came from two great “how to” blogs; one from <a href="https://discourse.getdbt.com/t/performing-a-blue-green-deploy-of-your-dbt-project-on-snowflake/1349">dbt Discourse</a> and the other from <a href="https://blog.montrealanalytics.com/blue-green-deployment-with-dbt-and-snowflake-922f1c658011">Montreal Analytics</a>. Blue-green deployments are a technique used to safely build and test dbt models in a separate database (green/stage environment), and only graduate data to user-facing databases (blue/production environment) for models that are computed successfully. Our basic architecture looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wRMNBrm1khnT5XZ8" /></figure><p>We first build and test all of our models in a database called ‘unaudited’. If all models and “build-breaking” tests are successful, we use a dbt macro to execute Snowflake’s ‘swap with’ function to swap the ‘unaudited’ (staging) database with the ‘analytics’ (production) database. If models fail to build or “build-breaking” tests fail, the swap doesn’t occur. This leaves the data in the user-facing production environment stale, but devoid of bad outputs.</p><p>This is what the process looks like in our dbt Airflow DAG:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Pf6SFa0uPrDSEIK8" /></figure><p>Not shown here, but also an integral part of the workflow is the alerting that happens if models fail to build or tests fail. We recently implemented <a href="https://hub.getdbt.com/elementary-data/elementary/latest/">Elementary</a> data checks to handle our data observability needs.</p><p>Ultimately, this architecture gives us much more control over how we build and test dbt models. The blue-green environment in combination with “build-breaking” vs. “non-build-breaking” tests allow us to protect our end users from bad outputs and to differentiate between minor data problems and major ones, while not losing sight of the minor ones.</p><p>Blue-green deployments can be a valuable tool for organizations looking to improve the quality of their data and reduce the risk of errors. By using this technique in conjunction with dbt-core’s functionality, organizations can ensure that their data is accurate and up-to-date, allowing them to make more informed decisions and build a stronger reputation with stakeholders.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8777c05562f3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vouch-engineering/the-advantages-of-blue-green-deployment-with-dbt-and-snowflake-8777c05562f3">The Advantages of Blue-Green Deployment with dbt and Snowflake</a> was originally published in <a href="https://medium.com/vouch-engineering">vouch-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>