<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ian Moore on Medium]]></title>
        <description><![CDATA[Stories by Ian Moore on Medium]]></description>
        <link>https://medium.com/@ian.moore_83986?source=rss-394411befef7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*j7DpNA1kXZGzD9g6rUuwuQ.png</url>
            <title>Stories by Ian Moore on Medium</title>
            <link>https://medium.com/@ian.moore_83986?source=rss-394411befef7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 21:26:32 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ian.moore_83986/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[SR 11–7 Meets OECD: What Credit Risk Governance Actually Looks Like]]></title>
            <link>https://medium.com/data-success/sr-11-7-meets-oecd-what-credit-risk-governance-actually-looks-like-81fcb4e28b37?source=rss-394411befef7------2</link>
            <guid isPermaLink="false">https://medium.com/p/81fcb4e28b37</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai-governance]]></category>
            <category><![CDATA[credit-risk-management]]></category>
            <category><![CDATA[responsible-ai]]></category>
            <dc:creator><![CDATA[Ian Moore]]></dc:creator>
            <pubDate>Sat, 09 May 2026 15:18:16 GMT</pubDate>
            <atom:updated>2026-05-09T15:18:16.757Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pO1AhwZxkVDRJo5z7-4Tog.png" /></figure><blockquote>Two Institutions, One Framework: How SR 11–7 and OECD Principles Work in Practice</blockquote><p>Governance frameworks sound abstract until you actually have to implement them.</p><p>I know this first hand. I built ML models at credit unions of varying sizes. Whether they made it into production or they lived in presentations, there was always talk of building production-ready models that would run every month, make real decisions, affect real lending. However there was never talk of model risk management. No one ever stopped to ask — including myself — Who validates this thing? Who monitors it? When does it need to be retrained? What happens when it drifts? <em>What happens when it breaks?</em></p><p>I had absolutely no concept of what that actually required from a governance perspective. Looking back, though, I wasn’t alone in not knowing. The analysts building the models didn’t know. The executives approving them didn’t know. Nobody in the organization had connected the dots between “we built a credit model” and “we now own model risk.”</p><p>This is when we should have started discussing SR 11–7. This is the conversation we should have been having.</p><h4><strong>Understanding SR 11–7 and OECD Principles</strong></h4><p>SR 11–7 is guidance from the Federal Reserve on model risk management. It’s been around since 2011 — it actually predates AI entirely. Its original purpose to provide guidance n credit risk scoring models, but it has since been adopted as the regulatory framework that governs how financial institutions manage model risk. It says three things have to happen:</p><ol><li>The business unit that uses the model has to monitor it.</li><li>An independent function has to validate it.</li><li>An audit has to review whether the first two are actually doing their jobs.</li></ol><p>This is a clear three lines of defense. Simple structure. Applies to every model whether it’s a credit scoring model from 2015 or a large language model you’re using for document processing today.</p><p>The OECD Principles are different. They’re not regulatory requirements. They’re global standards for responsible AI. They’re guiding principles. They’re aspirational. They ask questions like: Can people understand how this model makes decisions? Is it fair? Who’s accountable when something goes wrong? They’re broader than SR 11–7. They think about society, not just financial risk.</p><p>But here’s the thing — they’re not in conflict. OECD principles are actually what responsible implementation of SR 11–7 looks like. It’s a regulatory framework guided by a shared set of principles. SR 11–7 tells you what to do and OECD reminds you why it matters.</p><h4><strong>Let’s Talk Underwriting Models</strong></h4><p>Every institution is performing their underwriting differently. Some of it is in-house, some of it is automated. Some institutions have a little of both. The key consideration is often speed — how can the institution make that decision fast? Approve, decline, or counter — all while the member is sitting there. However, the decision also has to be accurate, consistent, and defensible.</p><p>So, how do you get to this dream state?</p><p>Basically, you build — or you buy — an underwriting model. It looks at debt-to-income ratio, payment history, credit utilization, length of credit history, and a whole host of other factors. Likely, more factors then your loan underwriters could process. It’s a complex box of machine learning that produces a score or a probability of default. The underwriter uses that output to inform the approval decision. Simple enough.</p><p>Except now you own a model. And that model is influencing decisions that affect your members and your capital. You know what else it affects? Your regulatory standing. Every single day.</p><p>Now here’s where institutions diverge. Some build that model in-house leveraging a data science team, historical loan data and deploy via a production pipeline. Others buy it from a vendor solution as a third party scoring tool to plug in and go.</p><p>Different paths. Same goal. <em>Same accountability.</em></p><h4><strong>Scenario 1: You Built It. You Own It.</strong></h4><p>Let’s say you’re a larger institution with a data science team. You’ve built an underwriting model in-house using historical loan data. It’s in production. It runs every day. It’s influencing approval decisions across your portfolio.</p><p>What is your operating manual? It’s not your documentation and its not your readme. Those are support documents. It’s SR 11–7 — supervisory guidance from the Federal Reserve that sets the standard for how regulated financial institutions must manage model risk. This is not optional. This is not aspirational. This framework is required.</p><p>Your first line is the people closest to the model every single day. This is your underwriting team — they’re using the output, making the decisions, and catching inconsistencies in real time. This is also your data science team — they built it, they own its technical performance, and they’re responsible for monitoring model drift and flagging when recalibration is needed. And this is your loan operations team — they’re processing decisions downstream and they’ll often be the first to notice when something feels off. SR 11–7 is explicit — the business unit that uses the model owns day-to-day monitoring. A model score is not a decision. It’s an input to a decision. Everyone in that first line needs to understand the difference.</p><p>Your governance function is your second line. This is a coalition unit — finance, risk, technical and operation teams. Their job is to ask the harder questions: is this model still performing as validated? Has the borrower population shifted? Does the recalibration schedule still make sense given current conditions? Are we tracking and monitoring any issues? They own the documentation, the validation standards, and the escalation triggers. When something looks wrong they don’t fix it — they flag it and force a decision.</p><p>Your internal audit function is your third line. They sample loans. They check for bias. They test whether the first two lines are actually doing what they said they would. They’re not looking at the model. They’re looking at the governance. Is SR 11–7 being followed? Are denial patterns defensible under fair lending standards? Is the documentation current?</p><p>Three lines. Real people in each one. Bridging the AI model governance gap.</p><h4><strong>Scenario 2: You Bought It. You Still Own It.</strong></h4><p>Let’s say you’re a smaller institution and you don’t have a data science team. Building a model in-house isn’t realistic. So you do what most institutions your size do and buy an underwriting tool from a third party vendor. It plugs into your loan origination system. It scores applications. It runs every day.</p><p>Here’s what a lot of institutions get wrong: they think buying the model removes the risk.</p><p>It doesn’t. SR 11–7 is explicit. You are responsible for validating and monitoring every model that influences your lending decisions. You are now an AI deployer. That means, you need to know what is happening inside that black box. The risk does not transfer back to the third party developer. Yes, they are now an AI producer. But that doesn’t change your regulatory obligation. All it does is change how you fulfill that obligation from the model management framework.</p><p>Your first line looks a little different here but the accountability is identical. Your underwriting team is still the human in the loop. They are using the output daily, tracking consistency, and should know when to override. But now, you should have a vendor relationship manager. Whether that sits in operations or technology, it doesn’t matter. It just needs to be someone who owns the contract, the SLA, and the communication channel with the vendor. They’re the ones who need to know when the model was last recalibrated, what the performance reporting looks like, and what the escalation process is when something goes wrong. And your loan operations team is still downstream. They should be the first to feel it when the model starts behaving differently.</p><p>Your governance function still owns independent oversight, except now their job includes vendor management. Are the recalibration schedules documented? Are we comfortable with this recalibration schedule? Is the vendor providing performance reports? What’s the process when the model drifts? What’s the contractual obligation for revalidation? These aren’t technical questions, they’re governance questions for an entire organization. And they need answers before you sign the contract, not after something breaks.</p><p>Your internal audit function still samples loans. Still checks for bias. Still tests whether the first two lines are functioning. And now they’re also reviewing the vendor relationship itself. Is the documentation current? Is the institution actually monitoring the vendor or just assuming everything is fine because the tool is running?</p><p>Two institutions. Two different paths to the same underwriting model. One regulatory framework governing both. The execution looks different. The accountability is identical.</p><h4><strong>The Foundation Always Wins</strong></h4><p>The key takeaway here is this: Whether you built your underwriting model from scratch or plugged in a vendor tool — the moment it influences a lending decision, you own the risk. SR 11–7 doesn’t care how you got here. It cares what you do next.</p><p>This all ties back to the OECD principles for trustworthy AI. Transparency — your members deserve to know how decisions that affect them are made, Fairness — your model shouldn’t discriminate, and audit should prove it doesn’t, and Accountability — someone in your organization has to own this. Not eventually. Now.</p><p>Most institutions aren’t having this conversation. The analysts are building. The executives are buying. And nobody has connected the dots between “we deployed a model” and “we now own model risk.”</p><p>SR 11–7 has been on the books since 2011. The three lines of defense aren’t new. The accountability isn’t new. What’s new is that your underwriting model is now an AI model — and the regulatory expectation hasn’t changed just because the technology did.</p><p>Build the governance before you need it. Not after you wish you had.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=81fcb4e28b37" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-success/sr-11-7-meets-oecd-what-credit-risk-governance-actually-looks-like-81fcb4e28b37">SR 11–7 Meets OECD: What Credit Risk Governance Actually Looks Like</a> was originally published in <a href="https://medium.com/data-success">Data Success</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Integrity, Human Dignity]]></title>
            <link>https://medium.com/@ian.moore_83986/data-integrity-human-dignity-17fdab568822?source=rss-394411befef7------2</link>
            <guid isPermaLink="false">https://medium.com/p/17fdab568822</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[business-intelligence]]></category>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[data-management]]></category>
            <category><![CDATA[data-team]]></category>
            <dc:creator><![CDATA[Ian Moore]]></dc:creator>
            <pubDate>Sun, 03 May 2026 14:03:11 GMT</pubDate>
            <atom:updated>2026-05-03T14:03:11.188Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Mucya2sFuWhU9eMZzqbvQA.png" /></figure><blockquote>Good data is a supply chain. And a supply chain is only as strong as the people inside it.</blockquote><p>There is an idea I’ve been sitting with for a while. I know I’m not the first to say it and I may have picked it up somewhere in my data journey. It’s part question, part observation. Let’s start with the question: you’ve got tools, a tech stack, a process, a framework — but what else do you have?</p><p>Costs? Yes.</p><p>Subscriptions? Yes.</p><p>But what else? People. You have humans. They are everything.</p><p>Think about the last report that landed on stakeholders desk, the most recent dashboard pushed out to managers or the latest ML model retrained for production. Now trace each deliverable forwards and back.</p><p>Someone built the pipeline that fed it. Someone modeled the data that shaped it. Someone wrote the query that pulled it. Someone formatted it, validated it, and sent it forward. And someone on the other end read it, trusted it, and made a decision with it.</p><p>Every one of those someones is a person. Not just a resource. Not just headcount. A person who showed up, applied their expertise, and made sure you got what you needed — even when their name wasn’t ultimately on it.</p><p>It’s essentially a supply chain — a data supply chain. And, like any supply chain, the quality of the output depends on the quality of the inputs. The quality of each link in the chain.</p><p>If documentation is consistent, code is clear, the process is solid and the team feels like they have room to breathe and explore the data — people do their best work at every step. Energy is spent on improvement, on opportunities, on growth.</p><p>If you have ghost code, mutable facts, no documentation, no structure, and every single output has a hard deadline of yesterday — there is no room to explore. There is no room for development (data, personal, or otherwise). People spend their energy surviving the system instead of improving it.</p><p>So, where does this leave you? Not with a data problem. With a people problem. And just like your technical debt, it compounds quietly until it isn’t quiet anymore.</p><p>Early in my career I was the problem. I was green. I was figuring it out. I was building everything from scratch with no structure and no documentation. I sent out reports (Remember, Crystal reports?) with different numbers and explained them away. I didn’t know what I didn’t know. And nobody around me had built the system that would have caught it. That was the first time it clicked — this isn’t just code, this needs to be a system. This system needs to be grounded in core principles and a philosophy. So, I worked to build something clear, consistent and reliable because the humans downstream — the ones trusting those numbers — deserved better than my best guess.</p><p>Later I inherited someone else’s version of the same problem. Fragments of code, some notes, no structure, projects half finished and abandoned. Nobody had stopped to ask — what happens when I leave? They just … left. And my time was spent doing archaeology, playing defense to indefensible differences in the numbers, instead of providing solutions and analytics. Why?Because nobody had thought about the person coming after them. So I ripped out the foundation and completely rebuilt the architecture with people in mind.</p><p>I see this in real time too. Teams buried under rushed overhauls and impossible deadlines. Analysts who are passionate, spending their energy surviving instead of contributing. No room for error. No room for exploration. Managers who measure output and miss the people. You need the people. And the people need a system that was built with them in mind.</p><p>It doesn’t have to be this way.</p><p>When the system is built with people in mind the whole energy shifts. Documentation isn’t overhead — it is merely a function of the analysis and a gift to who comes after you. Clear naming conventions aren’t pedantic — they’re respect for the person who has to find something at 4pm on a Friday. Version control isn’t bureaucracy — it’s shared memory and collective knowledge; it belongs to everyone, not just the person who built it.</p><p>I’ve been on teams that had this. And there is nothing quite like it. Everyone is playing offense. Questions get answered in minutes instead of days. New analysts onboard without a six week apprenticeship. The data is trustworthy so the decisions are trustworthy. People have room to bring their best thinking instead of their best survival instincts.</p><p>That’s what governance actually is when it’s done right. Not a compliance exercise. Not a framework for its own sake. It’s the conditions that let talented people do remarkable work. It’s clearing the path so your best people can be their best.</p><p>Build the system for the humans inside it. They’ll build everything else.</p><p>Even as the world is changing with the development of more powerful AI, humans are still the most valuable asset. To build a system powerful and dynamic enough to take your data to the next level sill requires humans. Human creativity. Human skill. Human judgement.</p><p>Good governance produces good data. Data that is accurate, controlled, and managed effectively. Data humans can trust. Data that has <em>integrity.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=17fdab568822" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Shelf Life of a Deployed Model — What’s Actually Inside Your Pickle File?]]></title>
            <link>https://medium.com/@ian.moore_83986/the-shelf-life-of-a-deployed-model-whats-actually-inside-your-pickle-file-b788e791f18c?source=rss-394411befef7------2</link>
            <guid isPermaLink="false">https://medium.com/p/b788e791f18c</guid>
            <category><![CDATA[data-modeling]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ml-model-deployment]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Ian Moore]]></dc:creator>
            <pubDate>Tue, 28 Apr 2026 15:01:04 GMT</pubDate>
            <atom:updated>2026-04-28T15:01:04.175Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cWNbVVMxJ8bhDNJz4UJzYA.png" /></figure><blockquote>What latent space, pickle files, and interest rates have to do with your next regulatory exam.</blockquote><p>I’ve been building ML models for half of my career and I had the same flow as everyone — get data, build, train, test, deploy. I always understood conceptually why they worked, but I never really understood why you needed a model manager.</p><p>I love AI. I love reading about it. And I’ve been reading about how models actually work under the hood. Not the workflow. The mechanics. What are they actually doing ML, DDL, LLM or otherwise?</p><p>Every model — regardless of the type — is doing three things before it ever produces an output.</p><p>It takes raw input and encodes it into a representation. Sounds fancy, I know, but this is basically your 1’s and 0’s. Using scalers or one-hot encoding. You’re taking your data representing it differently. The goal is to have something your model can work with mathematically.</p><p>Here is where it gets a little more <em>convolutional</em> (get it?): The model then compresses that representation through its various layers, finding patterns, stripping out noise, keeping what matters. That compressed representation lives somewhere called latent space — it’s hard to imagine, but this is the model’s internal geometry. This is the world it learned from and created.</p><p>I like to think of it like this: You don’t store every detail of every coffee shop you’ve ever visited. You store a mental map — which ones are similar, which ones are different, what makes a good one versus a bad one. It’s not even deliberate, it just exists on that mental map. And that map is your latent space. The model builds the same thing. Just with numbers.</p><p>Okay — so we represented our data for the model. It ran through the convolutional layers and did its AI yoga to build latent space. You now get an output — 1’s or 0’s, probabilities or regression estimations. It’s pretty accurate!</p><p>So where does the pickle file come in?</p><p>When you train a model and export it — whether that’s a .pkl file, a saved model object, whatever your flavor is — you’re not just saving code. You’re saving that latent space. The geometry the model built. The mental map of your coffee shops. It’s all stored inside there. Every pattern it found, every relationship it learned, every decision boundary it drew. Every bad cup of coffee. Frozen. Serialized. Ready to deploy.</p><p>That’s transfer learning in practice. You built the geometry once. Now you’re applying it to new data. Every time you have a new loan application, a new customer, a new transaction, and you run it through that exported model — you are doing transfer learning. Maybe its me — but I never knew that.</p><p>Here is where it got interesting to me:</p><p>That pickle file has an expiration date. In theory, I knew that. In practice, I figured I’d put a label on it somewhere down the line. This is that moment.</p><p>The geometry your model was built on, its learned from historical data. A specific window of time. A specific economic environment. A specific borrower population. And when the world moves, <em>you’re model needs to move with it too.</em></p><p>If interest rates climb, then credit conditions tighten and borrower behavior shifts. Variable rate payments increase, which leads to missed or partial payments. Macroeconomic conditions shifted, but your model didn’t. It’s still running on the old map with lower interest rates.</p><p>This means your latent space is stale. And a stale latent space produces confident but wrong answers. Which in a regulated lending environment is the worst kind of wrong there is.</p><p>I’ve spent most of my career as a one-man band. Data strategist, data engineer, data analyst, data modeler, data scientist. I’ve worn every hat in the stack. And when I deployed a model, I was the one who built it, tested it, validated it, and shipped it.</p><p>But I never stopped to ask — who should be checking my work after it’s live?</p><p>Not the output. The geometry. The latent space. The mental map the model built from data that is now months old. Who is asking whether that map still reflects reality?</p><p>So, I always wondered — why does a model manager exist?</p><p>This is what clicked for me. The model manager isn’t there to babysit the code. They’re there to keep an eye on the latent space. To ask — does this model’s internal map still reflect our world? Has the latent space drifted far enough that we need to rethink and retrain, not just redeploy?</p><p>In a regulated lending environment that question isn’t just a philosophical one. I think it’s a fair lending question and a capital allocation question and a regulatory exam question. Model drift isn’t just a performance problem. It’s just another way of <em>managing risk.</em></p><p>Because that is the ultimate goal.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b788e791f18c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Hidden Cost of Tribal Knowledge]]></title>
            <link>https://medium.com/data-success/the-hidden-cost-of-tribal-knowledge-011b678792a6?source=rss-394411befef7------2</link>
            <guid isPermaLink="false">https://medium.com/p/011b678792a6</guid>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[data-operations]]></category>
            <category><![CDATA[data-management]]></category>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Ian Moore]]></dc:creator>
            <pubDate>Sun, 26 Apr 2026 15:02:49 GMT</pubDate>
            <atom:updated>2026-04-26T15:02:49.324Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MYneJhNUxP_05RsrNlOm-g.png" /></figure><blockquote>Some Costs Show Up on the Invoice. Some Show Up on Your Team.</blockquote><p>Let me guess: Someone on your team is leaving.</p><p>Maybe they already did. Maybe they gave two weeks notice this morning. Or, maybe they’ve actually been quietly interviewing for a few months and you just don’t know it yet.</p><p>One exits, one enters. Analysts are dime a dozen, right? They scribbled out a transition doc, right? You got it covered, right?</p><p>But when they walk out the door, they’re taking something with them that isn’t on any offboarding checklist. Not their laptop. Not their badge. Not their code (Well, maybe some of it). The thing they’re taking is the logic behind every report they ever built, every query they ever wrote, every number they ever produced that someone put in front of a CFO.</p><p>Like I said, they might have a transition document. Something that is ten pages of a vague inventory of projects, a few bullet points about systems, a list of recurring tasks with just enough detail to feel like they got it all. Nice! Something resembling a <em>paper trail.</em></p><p>Except, it’s a last ditch effort. A ditch that leads to nowhere but confusion and frustration.</p><p>This is what tribal knowledge actually costs — and most organizations don’t find out until 4pm on a Friday, the slides don’t reconcile, and the person who built them is no longer there to call. And that facts (literal facts in your data) are apparently mutable.</p><h3><strong>What Tribal Knowledge Actually Is</strong></h3><p>Tribal knowledge isn’t a character flaw within a team. It’s a valuable asset. It’s a comprehensive data dictionary that should be gathered together and appreciated. Tribal knowledge is just like the data teams in most organizations — ignored, sequestered and set aside — we’ll take care of it <em>next quarter</em>.</p><p>I get it, when you’re two analysts deep and the requests keep coming, documentation feels like overhead. Naming conventions feel pedantic. Version control feels like one more bit of code to write. Again, we’ll take care of it <em>next quarter</em>.</p><p>This is why data teams are underappreciated — Everyone just figures it out because they have to. I mean, I’ve been on interviews where even the hiring manager tells me their least favorite part of the job is documentation — I said, “I know! But at least you’re writing it!”</p><p>Everything works, until it doesn’t.</p><p>And the problem isn’t the people. It’s that nobody ever stopped to build the system. And without a system, everything lives in someone’s head.</p><p>Like I said, everything works, until it doesn’t.</p><h3><strong>The Spectrum</strong></h3><p>To me, tribal knowledge failure isn’t binary. You can’t one-hot encode you’re way through the fix. It lives on a spectrum. And where your organization falls on that spectrum determines whether you’re lost or your blind.</p><p><strong>Level 1 — The Ghost Code: </strong>Ah, you’re digging through the almighty shared drive — or more likely, someone’s personal folder that got migrated when they left… and you found it? Or something like it. A SQL file with no header, no date, no name, no purpose, no nothing. Just raw logic sitting there like an artifact from a previous civilization. You have no idea what it was built for, who asked for it, whether it’s still running anywhere, or whether anyone depends on it. You run it. I guess the numbers look plausible (I mean, if your data isn’t normal, why would the output be?). So, you move on…</p><p><strong>Level 2 — The Transition Doc Illusion: </strong>If we’re being honest, someone did try. Give them credit for that. They sat down in their last two weeks and produced something for you, our newest analyst. And, like I said earlier — It is ten pages, maybe more, maybe less. Projects listed. Systems named. Recurring tasks described in just enough detail to feel like sustenance. But it isn’t dinner, it’s a light snack after breakfast. The transition doc is merely performance art. It looks like continuity, but it isn’t. Because if you need a transition doc, it means you need a better process — and to back up your code.</p><p><strong>Level 3 — Version Control Without Structure:</strong> Back to the shared drive on an archeological dig — I mean, at least some code is there. Nobody documented the why, nobody named anything consistently, one repo holds everything from three years of requests with no folder structure and commit messages like “fix” and “final_v2_REAL (copy) 2.” But it’s there. You can dig. You can trace. All you’ll find are bodies buried in the backyard, but you can put the skeleton back together — with enough time and enough coffee. If I’m being generous, this is useful chaos — but it’s not a system. But it’s not a black hole either. There’s something to work with.</p><p><strong>Level 4 — Version Control With Governance: </strong>Repos organized by domain or data mart. Every file named after the project or dashboard it serves. A simple header on every SQL file — author, date, purpose, notes, last modified. Naming conventions that make sense to someone who has never seen the codebase before (I mean just google it! Copy it from someone!). A new analyst can onboard without a six week apprenticeship. Your repo isn’t a dump but a well grown harvest. A question gets answered without an archaeological dig. The code is institutional, shared memory. The data is tangible. It’s purposeful. Your team resembles world peace. This is a system. This is control. This is <em>offense.</em></p><p><strong>Level 5 — The Living Nightmare: </strong>No source code — and even if you had it, it wouldn’t matter. Because the underlying fact values are mutable. Resubmissions overwrite history. Numbers that existed last quarter no longer exist in the same form. <em>I think its logged somewhere? </em>There are no notes in the code. No headers. No naming conventions. No data models. You’re only dimension is a black hole hoard of 8 nested subfolders that resembles your “warehouse”. You cannot reconstruct the past because the past has been quietly, systematically rewritten. This isn’t a tribal knowledge problem anymore. This is an epistemological problem. You’re not just lost. You’re not just blind. You’re floating in the void of what you don’t know because the foundation was never poured. The system was never built.</p><p>Dashboards don’t make a data program. And if all you have are dashboards, you don’t have a data program. You have output. And output without a foundation is just noise with good formatting. More performance art.</p><h3><strong>The Real Cost</strong></h3><p>Data programs are often measured by the costs on the income statement. Software subscriptions, snowflake credits, and server storage. They’re all there. You can see it. Visible expenses that can be measured, managed, and budgeted.</p><p>What about the hours digging through infinite folders and “CFO slides output test(copy) v2.2.sql”? Where is version 2.1? What happened to 2.0? Now your Friday is spent rebuilding what was already built — by someone who is no longer there to ask. Data analysis requires human capital — you can measure that on a paycheck. What you can’t measure is the frustration and personal will it takes to keep digging through a broken system. That costs something too. It’s called burnout.</p><p>You hired an employee. A person showed up. And that person is exhausted.</p><p>Remember, the costs show up on the invoice. The frustration doesn’t. But it’s real, and it compounds the same way technical debt does — quietly, until it isn’t quiet anymore.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=011b678792a6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-success/the-hidden-cost-of-tribal-knowledge-011b678792a6">The Hidden Cost of Tribal Knowledge</a> was originally published in <a href="https://medium.com/data-success">Data Success</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What 8 Years of Financial Analytics Taught Me About Data Quality]]></title>
            <link>https://medium.com/data-success/what-8-years-of-financial-analytics-taught-me-about-data-quality-9fe5136c28c7?source=rss-394411befef7------2</link>
            <guid isPermaLink="false">https://medium.com/p/9fe5136c28c7</guid>
            <category><![CDATA[data-management]]></category>
            <category><![CDATA[financial-analytics]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-warehouse]]></category>
            <category><![CDATA[data-governance]]></category>
            <dc:creator><![CDATA[Ian Moore]]></dc:creator>
            <pubDate>Mon, 20 Apr 2026 20:48:27 GMT</pubDate>
            <atom:updated>2026-04-20T20:48:27.135Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rfrj4HdbtwxuT9F9bGw3wQ.jpeg" /></figure><blockquote><strong>Three lessons from eight years of financial analytics — and why your foundation matters more than your tools.</strong></blockquote><p>Let me guess — you have a productivity problem, and the tools aren’t helping.</p><p>There is always an emphasis on the tech stack. Snowflake vs Databricks. On-Prem vs Cloud. And remember the lakehouse phase? <em>Unstructured is king! </em>Now, there is a lot of power in modern data tools, which can make you feel you’re productive. There is perceived power in output.</p><p>The team is busy. Dashboards are getting built at a furious pace. Requests appear to be getting filled (ad-hoc and otherwise). On paper, the output is REAL. Things are humming along.</p><p>Except… the Snowflake bill is high, and staff are burnt.</p><p>This is the myth of productivity and the reality of the treadmill. Pull raw data — AGAIN. Reshape the data — AGAIN. Slightly different this time, for a different person, with a different output. And layer on top the myth of automation — Every new request triggers the same cycle: build a task, wire it to a dashboard, ship it, repeat. It’s not that nobody stops to ask why we’re doing this again or questions whether there’s a better way — It’s that nobody has the time!</p><p>I’ve watched this pattern repeat across every organization for over eight years. Credit unions, banks, insurance, healthcare — different industries, different tools, same dysfunction. We call it output. We celebrate how much we’re producing.</p><p>Here is the kicker, though: we’re mistaking production for efficiency. We’re working harder, not smarter.</p><p>It took me a few years to finally learn how to really organize data — how to structure it to make my life easier. And the last eight years have taught me that most data problems aren’t data problems at all. They’re foundation problems. And until you fix the foundation, you’re just finding faster ways to repeat the same mistakes. It’s like basketball — you have to master the fundamentals before you try to slam dunk.</p><p>The fundamentals I keep coming back to are my three core tenets: flexibility, scalability, and efficiency. Every one of these stacks on the other, and the entire data program flows from these core tenets.</p><p>Here’s what I learned.</p><h3><strong>Lesson 1: Find Your Backbone — The Single Source of Truth</strong></h3><p>Anyone can follow a recipe. Add this data source, run that query, and build the dashboard. Output achieved. Boxes checked. But there is a difference between someone who can technically execute a data stack and someone who understands why it’s built the way it is. One produces cookies. The other understands baking.</p><p>A master baker doesn’t just follow steps. They understand how ingredients interact, why the foundation matters, and what happens when something is off before anyone else tastes it. Data is the same way. You can compile the tech stack in the right order and still produce something nobody can fully trust. Because the “what” without the “why” is just a recipe. And recipes don’t build data programs — philosophy does.</p><p>This is why the single source of truth is where every data program has to start. Not as a technical checkbox but as an organizational commitment. It’s saying — <em>this is what we believe. This is where we stand. Everything we build, every question we answer, every decision we make flows from here.</em></p><p>In financial services, that backbone is the General Ledger.</p><p>For the layman: The GL is the financial spine of any institution. Every dollar that moves through the organization flows through it — loans, deposits, interest income, expenses. It’s the one number everyone has to agree on. It isn’t optional. It’s the audit trail, the regulatory foundation, and the organizational truth all in one.</p><p>And for a financial institution where over 85% of revenue can easily come from lending, when your loan data doesn’t reconcile back to the GL, you have a problem. A data problem, yes. But more importantly: A trust problem.</p><p>I saw this firsthand early in my career. Two systems, two versions of the same numbers, nobody connecting the dots — the two systems actually tie together! The core member and loan data HAVE to map back to the GL.</p><p>Analysts were always on defense. Leadership was frustrated and unsure how to make decisions on data nobody fully trusted. The dysfunction was all around us — it was just invisible because the output kept coming. It was all defense — no one stepped back to ask, how do we get on offense?</p><p>I put together a presentation for the Controller and CFO, laying out exactly what was broken and what it would take to fix it. The first commitment was simple: map everything back to the GL. Split the data into logical domains. Build a foundation structured to answer the 80% of questions the finance team asks every single month before they even ask them.</p><p>That’s what a single source of truth actually gives you. Not just accuracy — though accuracy is non-negotiable in a regulated industry. It gives you <em>flexibility</em>. The ability to pivot in any direction, answer questions you haven’t been asked yet, without rebuilding from scratch every time. When the backbone is solid, you stop defending your numbers and start using them.</p><p>And here is the uncomfortable truth about all those modern tools and cloud platforms promising to solve your data problems:</p><p>No fancy stack in the world will hide a cracked foundation. You can have simple tools and build enormous value. The cloud doesn’t mean you’re progressive. It means you’re expensive.</p><p>Know how I fixed the data for the finance? Sql server, a few SSIS import packages, and a solid, well thought out data model.</p><h3><strong>Lesson 2: Stop Hoarding, Start Modeling — The Architecture of Scalability</strong></h3><p>Imagine you’re standing outside a brand new warehouse. Fresh paint, solar panels, nice driveway. It sure looks good!</p><p>You walk in — and it’s packed. Floor to ceiling, stacks of everything. You ask: how do I find anything?</p><p>Someone waves their hand. <em>“Books are over in that corner. Electronics somewhere over there. Knick-knacks near the front. Everything else is kind of… in the middle.”</em></p><p>A new shipment arrives. Nobody unpacks it properly. They just stack it with the other books and move on to the next one.</p><p>Question: Is this truly a warehouse? Or just a hoard? Or more like a bad dream?</p><p>Now replace the warehouse with your data platform. Same picture. An organization can “invest” in a modern cloud data platform. The tooling looks impressive. The storage is essentially unlimited. Again, the team is busy building. Tasks are firing. Dashboards are shipping.</p><p>But nobody ever stopped to ask a simple question: which source do we trust first?</p><p>Is this external feed the primary source of truth? Is that system secondary? What happens when they conflict? Without that decision — and it is a decision, not a default — everything is equally untrustworthy. And when everything is equally untrustworthy, you’re always on defense. Every number gets questioned. Every analysis requires a disclaimer. Every meeting becomes a debate about the data before it becomes a conversation about the business.</p><p>You don’t have a data warehouse. You have a data hoard.</p><p>A hoard looks productive from the outside. There’s a lot of it. It’s organized enough to find things if you know where to look. But it doesn’t scale — it accumulates. Every new request adds another layer. Another task. Another transformation built specifically for one output, serving one consumer, answering one question. Nobody reuses anything because nobody documented anything. Nobody documented anything because nobody had time. Nobody had time because they were busy building the next task.</p><p>This is the architecture of a treadmill, not a data program.</p><p>The antidote isn’t more sophisticated tooling. It’s a solid data model. It’s stripping it down and rebuilding the foundation.</p><p>I learned this the hard way, and then I learned it the right way. As I said earlier, I had a simple setup — SQL Server, a few well-structured imports for staged data tables, facts, and dimensions mapped cleanly to the General Ledger. A few helper dimensions to join back to the loan data. Nothing glamorous. Nothing that would impress anyone at Google.</p><p>One afternoon, the Controller walked over to my desk with a question about CECL loan groupings — which loans were shifting between categories and why.</p><p>In a hoarded environment, that question would have triggered a day-long project. New query, new task, new export, new spreadsheet, back and forth until everyone agreed on the numbers. I would have been on defense the whole time.</p><p>In this situation, I said: hang on, let me pivot — let’s just look right now.</p><p>Because the data was modeled for flexibility, we pulled a few queries, sliced across dimensions, and compared the loan groupings right there with him standing over my shoulder. Question asked. Question answered. This is an offense in a matter of minutes.</p><p>That’s what scalability actually feels like. Not handling more volume — though it does that too. It’s the ability to answer questions you weren’t expecting, in the moment they’re asked, without rebuilding your entire pipeline to do it.</p><p>The 80% principle is how I think about it practically. If your data model is built well, roughly 80% of the questions your organization asks should already be answerable from the existing structure. New requests build on what exists rather than starting from scratch. The model grows with you instead of fragmenting around you.</p><p>If every new request requires a brand new set of the same code, that’s your signal. The foundation isn’t built — it’s just being rebuilt over and over in slightly different shapes.</p><p>A hoard gives you output. A model gives you answers.</p><p>And there is a meaningful difference between the two.</p><h3><strong>Lesson 3: Process Eats Tribal Knowledge for Lunch — The Currency of Efficiency</strong></h3><p>I used to joke that half of a data analyst&#39;s job is playing detective. For example, just last week, a new mystery surfaced: A set of charts needed to go into slides for a CFO presentation at a conference. Straightforward request. Except nobody could find them. Not in the shared drive. Not in the reporting tool. Not anywhere.</p><p>I found something similar — but the numbers didn’t match the slides. No notes. No documentation. No SQL saved anywhere. So I went back to 2024 to pull a baseline figure for comparison.</p><p>Those numbers didn’t match either.</p><p>Which left me with the only question that matters in that moment: <em>where did this person get this number?</em></p><p>Nobody knew. The person who built it had moved on. Presumably, the process lived in their head. And now it lived nowhere.</p><p>This is the cost of tribal knowledge. It feels invisible until the moment it isn’t — and that moment always seems to arrive at 4 pm on a Friday, with a CFO walking into a conference and slides nobody can fully verify.</p><p>Small teams are especially vulnerable to this. There is never enough time to document. Standards feel like overhead when you’re the only one who needs to know how things work. Naming conventions feel pedantic when the team is two people and everyone just figures it out. Until someone leaves. Until someone is out sick. Until a number doesn’t reconcile, and nobody remembers where it came from.</p><p>Process isn’t bureaucracy. It’s institutional memory made tangible.</p><p>Dimensions that actually show where things changed. A simple code header on every SQL file. A consistent naming convention across every table and report. A folder structure that makes sense to someone who has never seen it before. Version control so you can trace every change back to its origin. These aren’t rules for rules’ sake — they’re the answer to <em>“where did this number come from?”</em> before anyone has to ask.</p><p>And… None of it has to be complicated! Look at the tools you have, and leverage those. Need to create a registry of documentation or Excel workbooks? SharePoint list. Filter. Sort. Link. Centralized and done!</p><p>When the process is documented and standardized, the team runs without frustration and friction. New analysts are on board without a six-week apprenticeship of “Save this to a OneNote”. Requests get answered without an archaeological dig through someone’s undocumented logic. Only to keep digging holes and finding where the bodies are buried. Or, better yet, the CFO gets slides with numbers that reconcile.</p><p>That’s efficiency. Not speed — though good process is faster too. It’s the elimination of unnecessary effort. Time spent reconstructing is time not spent analyzing. And that gap has a real cost.</p><p>Which brings me to the argument nobody wants to have.</p><p>Governance is often dismissed as overhead. No direct revenue, no tangible output, no obvious ROI. Purely a cost center. But that argument only works if you’re not looking at the other side of the ledger. Every redundant task is a computation you’re paying for. Every undocumented process is analyst hours spent reconstructing instead of analyzing. Every mismatched number is a CFO walking into a conference with slides nobody can verify. That’s not free. That’s just a cost you stopped tracking and started ignoring.</p><p>The data hoard is a line item on your bill. It’s buried in your cloud invoice and your team’s calendar — but it’s there. It’s realized even if you don’t see it.</p><p>Governance doesn’t add cost to your data program. It removes the cost you’ve been ignoring.</p><p>And that’s the return on investment nobody puts in the business case.</p><h3><strong>The Foundation Always Wins</strong></h3><p>Three lessons. Eight years. One throughline.</p><p>Flexibility. Scalability. Efficiency.</p><p>They’re not a checklist. They’re a philosophy. They’re guiding principles. They stack on each other the way a good data program stacks on itself — each layer making the next one possible, each decision either compounding your advantage or compounding your debt.</p><p>I can tell you firsthand: Technical debt is real. Operational debt is real. Every undocumented process, every redundant task, every hoarded transformation is a payment deferred — and like all deferred payments, the interest compounds quietly until the bill arrives all at once. Usually on a Friday. Usually, before a CFO presentation.</p><p>Getting the fundamentals right isn’t just about efficiency today. It’s about creating the space to invest in what comes next. And what comes next isn’t another dashboard. It isn’t another Jupyter notebook. What comes next is one thing everyone is sprinting towards. It’s AI.</p><p>I love AI — It is amazing. It is magic. But here’s what the hype cycle always skips: AI doesn’t fix your foundation. It inherits it.</p><p>Feed it clean, well-modeled, governed data, and it becomes a multiplier in your data delivery. Feed it a hoard, and it becomes a very expensive way to take out the garbage at scale.</p><p>This isn’t just playing offense anymore. It’s not even just a slam dunk. It’s the difference between winning a game and winning a championship — and championships are built on fundamentals, not highlights.</p><p>You can’t stack AI on top of trash.</p><p>You can’t stack the future on a garbage heap.</p><p>The organizations that figure this out first won’t just be more efficient. They’ll be playing a different game entirely.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9fe5136c28c7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-success/what-8-years-of-financial-analytics-taught-me-about-data-quality-9fe5136c28c7">What 8 Years of Financial Analytics Taught Me About Data Quality</a> was originally published in <a href="https://medium.com/data-success">Data Success</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[QUICK HIT: 30-Second Model Evaluation for ML Models]]></title>
            <link>https://medium.com/@ian.moore_83986/quick-hit-30-second-model-evaluation-for-ml-models-bd5a1c4c8486?source=rss-394411befef7------2</link>
            <guid isPermaLink="false">https://medium.com/p/bd5a1c4c8486</guid>
            <category><![CDATA[predictive-analytics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[classification-algorithms]]></category>
            <dc:creator><![CDATA[Ian Moore]]></dc:creator>
            <pubDate>Fri, 13 Feb 2026 05:31:29 GMT</pubDate>
            <atom:updated>2026-02-21T05:11:03.342Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>QUICK HIT: 30-Second Evaluation for Choosing an ML Model</strong></h3><h3><strong>I’d never thought of this before</strong></h3><p>I’ve been working with data and building ML models for almost 7 years. Credit risk models, portfolio analytics, cluster analysis — all of it! And you know what I NEVER did?</p><p><strong>I never tested which algorithm would actually perform best before diving deep into feature engineering.</strong></p><p>In a way, I did — I’d pick a couple off the shelf and run my data through it and see which one “spoke” to me. Seriously. I’d pick Random Forest because “it’s good with financial data” or Logistic Regression because “it’s interpretable for credit risk.” Then I’d spend my time tuning hyperparameters because I need a reason to drink more coffee.</p><p>☕ Delicious…</p><h3>Then it hit me!</h3><p>I’m flipping a book on the shelf (<em>Machine Learning and Data Science Blueprints for Finance</em>)<em> </em>and I realize I haven’t been doing what every data science tutorial tells you to do: <strong>run a baseline comparison of algorithms out-of-the-box</strong>.</p><p>No hyperparameter tuning. No fancy feature engineering. Just raw performance on the same dataset with default settings. 🌭</p><p>The results? The almighty Decision Tree… comes… in… last. 😮‍💨</p><h3>Some code to wet the pallette</h3><p>Okay, to see total chaos, go to the <a href="https://github.com/imoore99/model_evaluation"><strong>GITHUB</strong></a><strong>! </strong>For a quick peek under the hood, read on.</p><p>The idea is simple. Set up your train/test splits. Set up your seed, your folds. Pick your models. Then, run a kfold on some scoring metric(s) and loop through the outputs. Append, append, append. Then compare. You get the idea…</p><p>Like I said, set your seeds.</p><pre># test options for classification<br>num_folds = 10<br>seed = 7<br>scoring = &#39;accuracy&#39;<br>#scoring =&#39;precision&#39;<br>#scoring =&#39;recall&#39;<br>#scoring = &#39;roc_auc&#39;</pre><p>Then, load some models. In this case, we’re playing around with a classification issue:</p><pre>models = []<br>models.append((&#39;LR&#39;, Pipeline([(&#39;scaler&#39;, StandardScaler()), (&#39;model&#39;, LogisticRegression(max_iter=1000, solver=&#39;lbfgs&#39;))])))<br>models.append((&#39;LDA&#39;, LinearDiscriminantAnalysis()))<br>models.append((&#39;KNN&#39;, Pipeline([(&#39;scaler&#39;, StandardScaler()), (&#39;model&#39;, KNeighborsClassifier())])))<br>models.append((&#39;CART&#39;, DecisionTreeClassifier()))<br>models.append((&#39;NB&#39;, GaussianNB()))<br><br>#Neural Network<br>models.append((&#39;NN&#39;, Pipeline([(&#39;scaler&#39;, StandardScaler()), (&#39;model&#39;, MLPClassifier(max_iter=500, early_stopping=True, random_state=seed))])))<br><br>#Ensemble Models <br># Boosting methods<br>models.append((&#39;AB&#39;, AdaBoostClassifier()))<br>models.append((&#39;GBM&#39;, GradientBoostingClassifier()))<br># Bagging methods<br>models.append((&#39;RF&#39;, RandomForestClassifier()))<br>models.append((&#39;ET&#39;, ExtraTreesClassifier()))<br># XGBoost<br>models.append((&#39;XGB&#39;, XGBClassifier()))</pre><p>Then, run a Kfold and append the results:</p><pre>results = []<br>names = []<br>for name, model in models:<br>    kfold = KFold(n_splits=num_folds, random_state=seed, shuffle=True)<br>    cv_results = cross_val_score(model, X_train, y_train, cv=kfold, scoring=scoring)<br>    results.append(cv_results)<br>    names.append(name)<br>    msg = &quot;%s: %f (%f)&quot; % (name, cv_results.mean(), cv_results.std())<br>    print(msg)</pre><p>In this case, here was my output:</p><pre>LR: 0.650011 (0.004193)<br>LDA: 0.650307 (0.004408)<br>KNN: 0.608773 (0.005843)<br>CART: 0.566989 (0.004374)<br>NB: 0.629557 (0.004320)<br>NN: 0.654114 (0.003126)<br>AB: 0.647159 (0.003739)<br>GBM: 0.654648 (0.004536)<br>RF: 0.645807 (0.003408)<br>ET: 0.642080 (0.004720)<br>XGB: 0.648807 (0.005100)</pre><p>Before we plot anything, notice the scores are all around 64% to 65% accuracy right out of the box. Not amazing, but decent. However, there are some clear losers — KNN, Decision Tree, that Gaussian one. I mean, pathetic. But, notice the range of accuracy when we plot the scores…</p><pre># compare algorithms<br>fig = plt.figure()<br>fig.suptitle(&#39;Algorithm Comparison&#39;+ &#39; - &#39; + scoring)<br>ax = fig.add_subplot(111)<br>plt.boxplot(results)<br>ax.set_xticklabels(names)<br>fig.set_size_inches(15,8)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w36E6lNyjUHP1btnCWNPwg.png" /><figcaption>Model Evaluation Output for Different ML Models</figcaption></figure><p>Notice how LDA and GBM perform really well, but have significant range between scores. However, LR and NN perform about the same, but have a tighter set of accuracy scores around the mean. Less variability. If it were me (which it is), I might start running two models LR and NN to see how each performs as you begin to tighten and tune the models. Eventually, a clear winner will prevail!</p><h3>The path to enlightenment</h3><p>Before: Pick algorithm → Build features → Tune hyperparameters → Hope it works</p><p>After: <strong>Test algorithms</strong> → Pick winner(s) → Build features → Tune hyperparameters → Actually get results</p><p>That reordering saves weeks. Sometimes months. Makes you awesome! 🤓</p><h3>The Bottom Line</h3><p>This isn’t revolutionary. It’s basic. But sometimes the basic stuff is what we skip because we drank too much coffee or we ate too many tacos.</p><p>Don’t be me. Buy a taco truck. Be smarter. 🧠</p><p><strong>Your homework:</strong> Run this on your next project. Before you do ANYTHING else. See what wins. Thank me later.</p><p>Let’s make it happen. 💰</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd5a1c4c8486" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[First Contact: Making Your First Claude API Call]]></title>
            <link>https://medium.com/@ian.moore_83986/first-contact-making-your-first-claude-api-call-2aed64257455?source=rss-394411befef7------2</link>
            <guid isPermaLink="false">https://medium.com/p/2aed64257455</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[credit-risk]]></category>
            <category><![CDATA[claude-api]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Ian Moore]]></dc:creator>
            <pubDate>Sun, 08 Feb 2026 07:13:05 GMT</pubDate>
            <atom:updated>2026-02-08T07:29:52.655Z</atom:updated>
            <content:encoded><![CDATA[<h4>Building a Credit Intelligence Platform — Part 2 of 10</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HtqUlL-TYkRXhKnxFo26pQ.jpeg" /></figure><h3>Part 2: From “Hello World” to Production-Grade Credit Analysis</h3><p>Alright, in theory, we’ve got our keys locked down (if you skipped Post #1… seriously, go back 🪦). Now it’s time to actually TALK to Claude and have a bit of a chat.</p><p>We’re not just going to say “hello” and call it a day — we’re going to build a credit risk analyzer that returns structured, actionable data. By the end of this post, you’ll have working code that takes borrower information and returns a JSON-formatted credit recommendation with other opinions (CLAUDE!). It is a bit of code, but conceptually, it works. With error handling, cost tracking, and everything.</p><p><strong>Here’s what we’re building:</strong></p><p>A simple credit analyzer that takes this:</p><pre>borrower = {<br>    &quot;name&quot;: &quot;John Smith&quot;,<br>    &quot;income&quot;: 85000,<br>    &quot;credit_score&quot;: 720,<br>    &quot;dti&quot;: 35,<br>    &quot;loan_amount&quot;: 350000<br>}</pre><p>And returns this:</p><pre>{<br>  &quot;recommendation&quot;: &quot;APPROVE_WITH_CONDITIONS&quot;,<br>  &quot;risk_level&quot;: &quot;MEDIUM&quot;,<br>  &quot;confidence&quot;: 0.75,<br>  &quot;key_risks&quot;: [&quot;High DTI&quot;, &quot;Elevated loan-to-income&quot;],<br>  &quot;suggested_terms&quot;: {<br>    &quot;rate_adjustment&quot;: &quot;+0.25%&quot;,<br>    &quot;required_down_payment&quot;: &quot;10%&quot;<br>  }<br>}</pre><p>Wild, right? Pretty straightforward. No paragraphs — Just JSON. Super clean and simple structured data we can actually USE in code.</p><p>Let’s make it happen. 💰</p><h3>Hello World: Your First API Call</h3><p>Let’s start simple. We’re going to send Claude a message and actually get a response. Just testing some code to get proof that it works.</p><pre>import os<br>from anthropic import Anthropic<br>from dotenv import load_dotenv<br><br># Load environment variables<br>load_dotenv()<br><br># Initialize the client (using our secure key from Post #1)<br>client = Anthropic(api_key=os.environ.get(&quot;ANTHROPIC_API_KEY&quot;))<br><br># Make the simplest possible API call<br>response = client.messages.create(<br>    model=&quot;claude-sonnet-4-20250514&quot;,  # Sonnet 4.5 - awesome AND c00l 😎<br>    # model=&quot;claude-haiku-3-20240307&quot;,  # Cheapest - great for testing<br>    # model=&quot;claude-opus-4-20251101&quot;,   # Premium - when you need maximum power<br>    <br>    max_tokens=1000,  # Started at 100, responses got cut off. 1000 = full responses!<br>    messages=[<br>        {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Hello! Can you help me analyze credit risk?&quot;}<br>    ]<br>)<br><br># Print the response<br>print(response.content[0].text)</pre><p>And what did I get in response?</p><pre>I&#39;d be happy to help you analyze credit risk! This is a broad area, so I can assist with various aspects depending on your specific needs.<br><br>Here are some key areas I can help with:<br><br>## Credit Risk Analysis Components<br><br>**1. Financial Analysis**<br>- Reviewing financial statements and ratios<br>- Cash flow analysis<br>- Debt service coverage ratios<br>- Liquidity and solvency metrics<br><br>**2. Risk Assessment Models**<br>- Probability of Default (PD) calculations<br>- Loss Given Default (LGD) estimation<br>- Exposure at Default (EAD) analysis<br>- Expected Loss calculations<br><br>**3. Credit Scoring &amp; Rating**<br>- Developing scoring models<br>- Interpreting credit ratings<br>- Risk categorization methods<br><br>**4. Portfolio Analysis**<br>- Concentration risk assessment<br>- Diversification analysis<br>- Stress testing scenarios<br><br>**5. Industry/Sector Considerations**<br>- Industry-specific risk factors<br>- Economic sensitivity analysis<br>- Regulatory requirements<br><br>What specific aspect of credit risk analysis are you working on? Are you:<br>- Evaluating a particular borrower or investment?<br>- Building a credit risk model?<br>- Analyzing a portfolio?<br>- Looking at regulatory compliance?<br>- Something else?<br><br>The more details you can share about your situation, the better I can tailor my assistance!</pre><p>My response?</p><p>c00l.😎</p><p><strong>Let’s review what just happened:</strong></p><p>We’re using <strong>Sonnet 4.5</strong> (claude-sonnet-4-20250514) because it&#39;s awesome. You COULD use Haiku (cheaper, faster, lighter — like a ghost) or Opus (more powerful — like a nerd with muscles), but Sonnet is the sweet spot. Fast enough, smart enough, affordable enough — yummy.</p><p><strong>One item of note is max_tokens:</strong> I started with max_tokens=100 . Claude would start giving me solid analysis and then just... stop mid-sentence. Like someone hung up on me (How dare you!). I bumped it to 1000 and there it was, full, complete responses.</p><p><strong>One other item of note is the key parameters:</strong></p><ul><li>model - Which Claude model to use</li><li>max_tokens - Maximum response length (~4 characters per token)</li><li>messages - Your conversation with Claude (we&#39;ll expand on this later)</li></ul><h3>Understanding What Claude Returns</h3><p>As we saw earlier, Claude doesn’t just send back text — it returns a structured object with a bunch of useful metadata. Let’s peek under the hood.</p><pre># Make a simple call<br>response = client.messages.create(<br>    model=&quot;claude-sonnet-4-20250514&quot;,<br>    max_tokens=100,<br>    messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Hello!&quot;}]<br>)<br><br># Print the full response structure<br>print(&quot;=&quot; * 60)<br>print(&quot;RESPONSE OBJECT STRUCTURE&quot;)<br>print(&quot;=&quot; * 60)<br>print(f&quot;ID:           {response.id}&quot;)<br>print(f&quot;Type:         {response.type}&quot;)<br>print(f&quot;Role:         {response.role}&quot;)<br>print(f&quot;Model:        {response.model}&quot;)<br>print(f&quot;Stop Reason:  {response.stop_reason}&quot;)<br>print(f&quot;\nContent:      {response.content}&quot;)<br>print(f&quot;\nUsage:&quot;)<br>print(f&quot;  Input tokens:  {response.usage.input_tokens}&quot;)<br>print(f&quot;  Output tokens: {response.usage.output_tokens}&quot;)<br>print(&quot;=&quot; * 60)</pre><p>The output?</p><pre>============================================================<br>RESPONSE OBJECT STRUCTURE<br>============================================================<br>ID:           msg_01F66XRgbnpxrW2QS42CMreh<br>Type:         message<br>Role:         assistant<br>Model:        claude-sonnet-4-20250514<br>Stop Reason:  end_turn<br><br>Content:      [TextBlock(citations=None, text=&quot;Hello! It&#39;s nice to meet you. How are you doing today? Is there anything I can help you with?&quot;, type=&#39;text&#39;)]<br><br>Usage:<br>  Input tokens:  9<br>  Output tokens: 27<br>============================================================</pre><p><strong>What these fields mean:</strong></p><ul><li><strong>id</strong> — Unique identifier for this message (great for logging/debugging)</li><li><strong>stop_reason</strong> — Why Claude stopped generating:</li></ul><p>end_turn = Finished naturally 🌮</p><p>max_tokens = Hit your token limit (might be incomplete 🪦)</p><p>stop_sequence = Hit a custom stop sequence you defined</p><ul><li><strong>content</strong> — The actual response (text, but could be other types later)</li><li><strong>usage</strong> — Token counts — <strong>THIS IS HOW YOU TRACK COSTS</strong> 💰</li></ul><p><strong>Token costs matter</strong> (as of January 2026):</p><ul><li>Sonnet 4.5: $3 per million input tokens, $15 per million output tokens</li><li>This “Hello” exchange: ~9 input + ~27 output tokens ≈ $0.0005-ish</li><li>Cheap for testing, but it adds up as you test…</li></ul><p>I ignored the usage field at first because &quot;like tokens or whatever, right?&quot; Then I ran a batch analysis, tested a bunch of loan scenarios and my bill was $47. I had to tell my daughter — sorry, no hot lunch today. Claude took your lunch money. She was like, “Who’s Claude?” 🤷 The point is this: The usage object is your friend - log it on EVERY call. 💀</p><h3>Version 1: Let’s Analyze Some Credit Risk</h3><p>Now let’s make this actually useful. We’ll create a function that analyzes credit risk for a borrower.</p><pre>def analyze_credit_risk_v1(borrower_info: dict) -&gt; str:<br>    &quot;&quot;&quot;<br>    This is Version 1: Basic credit risk analysis. Returns unstructured <br>    text response. <br>    NOTE: This is just playtime. In a real application, <br>    you would want to structure the response more clearly, <br>    and do more prompt engineering to get better results. <br>    We will get to that in later posts. For now, let&#39;s just <br>    see if we can get a response from the model based on <br>    borrower information. ⁉️ -&gt; 🫣<br>    &quot;&quot;&quot;<br>    <br>    # Construct the prompt with borrower information<br>    prompt = f&quot;&quot;&quot;<br>    Analyze this borrower for credit risk:<br><br>    Name: {borrower_info[&#39;name&#39;]}<br>    Annual Income: ${borrower_info[&#39;income&#39;]:,}<br>    Credit Score: {borrower_info[&#39;credit_score&#39;]}<br>    Debt-to-Income: {borrower_info[&#39;dti&#39;]}%<br>    Employment: {borrower_info[&#39;employment&#39;]}<br>    Loan Type: {borrower_info[&#39;loan_type&#39;]}<br>    Loan Purpose: {borrower_info[&#39;purpose&#39;]}<br>    Loan Amount: ${borrower_info[&#39;loan_amount&#39;]:,}<br>    Collateral Value: ${borrower_info.get(&#39;collateral_value&#39;, &#39;N/A&#39;):,} if borrower_info.get(&#39;collateral_value&#39;) else &#39;N/A (Unsecured)&#39;<br>    LTV Ratio: {borrower_info.get(&#39;ltv&#39;, &#39;N/A&#39;)}% if borrower_info.get(&#39;ltv&#39;) else &#39;N/A&#39;<br><br>    Should we approve this loan? Provide your analysis. Justify your decision based <br>    on the borrower&#39;s financial profile and the loan details provided.<br>    &quot;&quot;&quot;<br>    <br>    response = client.messages.create(<br>        model=&quot;claude-sonnet-4-20250514&quot;,<br>        max_tokens=500,<br>        messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: prompt}]<br>    )<br>    <br>    return response.content[0].text<br><br># Example 1: Mortgage with good LTV<br>borrower = {<br>    &quot;name&quot;: &quot;John Smith&quot;,<br>    &quot;income&quot;: 85000,<br>    &quot;credit_score&quot;: 720,<br>    &quot;dti&quot;: 35,<br>    &quot;employment&quot;: &quot;Software Engineer, 5 years&quot;,<br>    &quot;loan_type&quot;: &quot;Mortgage&quot;,<br>    &quot;purpose&quot;: &quot;Home purchase&quot;,<br>    &quot;loan_amount&quot;: 350000,<br>    &quot;collateral_value&quot;: 400000,<br>    &quot;ltv&quot;: 87.5<br>}<br><br>result = analyze_credit_risk_v1(borrower)<br>print(result)</pre><p>And what did Claude have to say?</p><pre>Based on my analysis of John Smith&#39;s financial profile, I recommend **APPROVING** this mortgage loan with standard terms.<br><br>## Credit Risk Analysis<br><br>### **Positive Factors:**<br><br>**Strong Creditworthiness**<br>- Credit score of 720 is well above the conventional mortgage threshold (typically 620+)<br>- Indicates responsible payment history and credit management<br><br>**Stable Income Profile**<br>- Annual income of $85,000 is solid for mortgage qualification<br>- 5 years of employment as a Software Engineer demonstrates job stability<br>- Technology sector typically offers good earning potential and job security<br><br>**Reasonable Debt Load**<br>- Debt-to-income ratio of 35% is within acceptable limits (most lenders prefer &lt;43%)<br>- Shows borrower can manage existing obligations while taking on mortgage payments<br><br>**Adequate Collateral Protection**<br>- Property value of $400,000 provides reasonable security for the $350,000 loan<br>- LTV ratio of 87.5% is manageable, though on the higher side<br><br>### **Areas of Consideration:**<br><br>**Elevated LTV Ratio**<br>- At 87.5%, the LTV exceeds the ideal 80% threshold<br>- Recommend requiring Private Mortgage Insurance (PMI) to mitigate risk<br>- Consider if borrower can increase down payment to reach 80% LTV<br><br>**Income-to-Loan Ratio**<br>- Loan amount represents approximately 4.1x annual income<br>- Within reasonable bounds but at the upper range of comfort<br><br>## **Recommendation: APPROVE**<br><br>**Justification:** John Smith presents a low-to-moderate credit risk profile. His strong credit score, stable employment, and manageable debt levels outweigh concerns about the higher LTV ratio. The mortgage is well-secured by real estate collateral, and his financial profile suggests ability to service the debt.<br><br>**Suggested Terms:**<br>- Require PMI due to LTV &gt;80%<br>- Standard interest rate based on 720 credit score<br>- Consider rate incentive if borrower can reduce LTV to 80%</pre><p>Not bad, Claude. Not bad at all. 🤔</p><p><strong>The Problem with V1:</strong></p><p>This is good analysis, but it’s TERRIBLE for building a system:</p><p>💀 Format varies between calls<br> 💀 Hard to parse programmatically<br> 💀 Can’t easily extract the recommendation<br> 💀 Not consistent across different borrowers<br> 💀 Can’t build a dashboard on top of this<br> 💀 Can’t store it in a database with a defined schema</p><p>I mean, what am I supposed to do — write regex to parse “I would lean toward approval with conditions”. I don’t even know what regex is!</p><p><strong>What I learned is that <em>THIS</em> is actually what at I actually need:</strong> Structured output. JSON. Data I can USE.</p><p>That’s what V2 fixes. Let’s go! 🚀</p><h3><strong>Version 2 — The Breakthrough (Structured JSON)</strong> 🌮</h3><p>Here’s where it starts to make some sense. Instead of letting Claude free-form the response, I had to figure out a way to build guardrails — I had to tell it EXACTLY what structure I wanted. Like bumper bowling, the response can bump against the guardrails and knock out some pins, but the bowling ball should always come back to the same place.</p><p><strong>So, what did I need? This:</strong></p><pre>{<br>    &quot;recommendation&quot;: &quot;APPROVE_WITH_CONDITIONS&quot;,<br>    &quot;risk_level&quot;: &quot;MEDIUM&quot;,<br>    &quot;key_risks&quot;: [&quot;High DTI&quot;, &quot;Elevated loan-to-income&quot;],<br>    &quot;mitigants&quot;: [&quot;Verify reserves&quot;, &quot;Confirm down payment&quot;],<br>    &quot;suggested_terms&quot;: {<br>        &quot;rate_adjustment&quot;: &quot;+0.25%&quot;,<br>        &quot;required_down_payment&quot;: &quot;10%&quot;,<br>        &quot;reserves_months&quot;: 6<br>    },<br>    &quot;confidence&quot;: 0.75,<br>    &quot;reasoning&quot;: &quot;brief explanation&quot;<br>}</pre><p>How <strong>to get Claude to return this:</strong></p><pre>def analyze_credit_risk_v2(borrower_info: dict) -&gt; dict:<br>    &quot;&quot;&quot;<br>    Version 2: Returns structured JSON output. W00t!<br>    NOTE: Okay, now let&#39;s get a bit more structured. Instead of just asking for an analysis,<br>    let&#39;s ask the model to return a JSON object with specific fields that we can easily parse and<br>    analyze programmatically. This is more useful for building an application where we want to<br>    automate decision-making or integrate with other systems. We will also ask for specific recommendations,<br>    identified risks, and suggested mitigations. This is a more realistic output format for a credit risk analysis tool.<br>    Now we go from 🫣 -&gt; 🤔<br>    &quot;&quot;&quot;<br>    <br>    prompt = f&quot;&quot;&quot;<br>    You are a credit risk analyst. Analyze this borrower and return your assessment <br>    as a JSON object with the following structure:<br>    <br>    {{<br>        &quot;recommendation&quot;: &quot;APPROVE&quot; | &quot;APPROVE_WITH_CONDITIONS&quot; | &quot;DECLINE&quot; | &quot;REVIEW&quot;,<br>        &quot;risk_level&quot;: &quot;LOW&quot; | &quot;MEDIUM&quot; | &quot;HIGH&quot;,<br>        &quot;key_risks&quot;: [list of identified risk factors],<br>        &quot;mitigants&quot;: [list of suggested mitigations],<br>        &quot;suggested_terms&quot;: {{<br>            &quot;rate_adjustment&quot;: &quot;adjustment in basis points or percentage&quot;,<br>            &quot;required_down_payment&quot;: &quot;minimum down payment percentage&quot;,<br>            &quot;reserves_months&quot;: &quot;months of reserves required&quot;<br>        }},<br>        &quot;confidence&quot;: numeric value between 0 and 1,<br>        &quot;reasoning&quot;: &quot;brief explanation of recommendation&quot;<br>    }}<br>    <br>    Borrower Information:<br>    - Name: {borrower_info[&#39;name&#39;]}<br>    - Annual Income: ${borrower_info[&#39;income&#39;]:,}<br>    - Credit Score: {borrower_info[&#39;credit_score&#39;]}<br>    - Debt-to-Income: {borrower_info[&#39;dti&#39;]}%<br>    - Employment: {borrower_info[&#39;employment&#39;]}<br>    - Loan Purpose: {borrower_info[&#39;purpose&#39;]}<br>    - Loan Amount: ${borrower_info[&#39;loan_amount&#39;]:,}<br>    <br>    Return ONLY the JSON object, no additional text or markdown formatting.<br>    &quot;&quot;&quot;<br>    <br>    response = client.messages.create(<br>        model=&quot;claude-sonnet-4-20250514&quot;,<br>        max_tokens=1000,<br>        messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: prompt}]<br>    )<br>    <br>    # Extract the response text<br>    result_text = response.content[0].text<br>    <br>    # Claude sometimes wraps JSON in markdown code blocks - handle that<br>    if result_text.startswith(&quot;```json&quot;):<br>        result_text = result_text.strip(&quot;```json&quot;).strip(&quot;```&quot;).strip()<br>    elif result_text.startswith(&quot;```&quot;):<br>        result_text = result_text.strip(&quot;```&quot;).strip()<br>    <br>    # Parse the JSON<br>    return json.loads(result_text)<br><br># Example 1: Mortgage with good LTV<br>borrower = {<br>    &quot;name&quot;: &quot;John Smith&quot;,<br>    &quot;income&quot;: 85000,<br>    &quot;credit_score&quot;: 720,<br>    &quot;dti&quot;: 35,<br>    &quot;employment&quot;: &quot;Software Engineer, 5 years&quot;,<br>    &quot;loan_type&quot;: &quot;Mortgage&quot;,<br>    &quot;purpose&quot;: &quot;Home purchase&quot;,<br>    &quot;loan_amount&quot;: 350000,<br>    &quot;collateral_value&quot;: 400000,<br>    &quot;ltv&quot;: 87.5<br>}<br><br># Test it<br>result = analyze_credit_risk_v2(borrower)<br><br># Pretty print the JSON<br>print(json.dumps(result, indent=2))</pre><p>And now, Mr Claude returns:</p><pre>{<br>  &quot;recommendation&quot;: &quot;APPROVE_WITH_CONDITIONS&quot;,<br>  &quot;risk_level&quot;: &quot;MEDIUM&quot;,<br>  &quot;key_risks&quot;: [<br>    &quot;High loan-to-income ratio (4.1x annual income)&quot;,<br>    &quot;Debt-to-income at upper acceptable range&quot;,<br>    &quot;Large loan amount relative to income may strain cash flow&quot;<br>  ],<br>  &quot;mitigants&quot;: [<br>    &quot;Strong employment history in stable field&quot;,<br>    &quot;Good credit score indicates payment reliability&quot;,<br>    &quot;Software engineering provides growth potential&quot;,<br>    &quot;Require adequate reserves for payment security&quot;<br>  ],<br>  &quot;suggested_terms&quot;: {<br>    &quot;rate_adjustment&quot;: &quot;+25 basis points&quot;,<br>    &quot;required_down_payment&quot;: &quot;20%&quot;,<br>    &quot;reserves_months&quot;: &quot;4&quot;<br>  },<br>  &quot;confidence&quot;: 0.78,<br>  &quot;reasoning&quot;: &quot;Borrower has strong credit profile and stable employment, but loan amount is high relative to income. Conditions help mitigate cash flow risks while supporting homeownership goals.&quot;<br>}</pre><p>Nice. 😎</p><p>🌮 Consistent structure every time<br> 🌮 Easy to parse and use in code<br> 🌮 Can extract specific fields (just the recommendation, just the risks, etc.)<br> 🌮 Can store in database with defined schema<br> 🌮 Can build dashboards/UIs on top of this<br> 🌮 Can make programmatic decisions based on structured data</p><p>So, now, we can do things like:</p><pre># Extract just the recommendation<br>print(f&quot;Recommendation: {result[&#39;recommendation&#39;]}&quot;)<br>print(f&quot;Risk Level: {result[&#39;risk_level&#39;]}&quot;)<br>print(f&quot;Confidence: {result[&#39;confidence&#39;]}&quot;)<br><br># Check if approval is needed<br>if result[&#39;recommendation&#39;] in [&#39;APPROVE&#39;, &#39;APPROVE_WITH_CONDITIONS&#39;]:<br>    print(&quot;\n🤑 This loan can proceed to closing!&quot;)<br>    if result[&#39;suggested_terms&#39;][&#39;required_down_payment&#39;]:<br>        print(f&quot;Required down payment: {result[&#39;suggested_terms&#39;][&#39;required_down_payment&#39;]}&quot;)<br>else:<br>    print(&quot;\n⛔ This loan requires review or decline&quot;)<br>    print(&quot;\n🪦 This loan requires review or decline&quot;)</pre><p>And get things like this:</p><pre>Recommendation: APPROVE_WITH_CONDITIONS<br>Risk Level: MEDIUM<br>Confidence: 0.72<br><br>🤑 This loan can proceed to closing!<br>Required down payment: Not applicable for debt consolidation</pre><p>This is a moment when I realized Claude isn’t just a chatbot — it’s like a data processor. You can turn messy business logic (credit analysis) into clean, consistent output. Now we’re making some ground. c00l. 😎</p><h3>Version 3: Adding Error Handling</h3><p>So, APIs fail. Networks timeout. JSON parsing breaks. Big or small projects, we’ve had it all happen when you’re for loop is apparently pinging the API 400 times. Right? Well, in this case, Claude occasionally wraps JSON in markdown even when you tell it not to (How dare you, Claude!). Let’s handle all of that.</p><pre>def analyze_credit_risk_v3(borrower_info: dict) -&gt; dict:<br>    &quot;&quot;&quot;<br>    Version 3: Adds comprehensive error handling.<br>    NOTE: In a real application, you will want to add robust error handling to manage <br>    cases where the model&#39;s response is not in the expected format, or when there are <br>    issues with the API call. This version includes error handling for JSON parsing and <br>    checks for required fields in the response. It also logs token usage for cost tracking. <br>    This is important for building a production-ready application that can handle edge <br>    cases gracefully(?). Now we go from 🤔 -&gt; 😎 or 🤓 (Both are c00l)<br>    &quot;&quot;&quot;<br>    <br>    try:<br>        # Build the prompt<br>        prompt = f&quot;&quot;&quot;<br>        You are a credit risk analyst. Analyze this borrower and return your assessment <br>        as a JSON object with the following structure:<br>        <br>        {{<br>            &quot;recommendation&quot;: &quot;APPROVE&quot; | &quot;APPROVE_WITH_CONDITIONS&quot; | &quot;DECLINE&quot; | &quot;REVIEW&quot;,<br>            &quot;risk_level&quot;: &quot;LOW&quot; | &quot;MEDIUM&quot; | &quot;HIGH&quot;,<br>            &quot;key_risks&quot;: [list of identified risk factors],<br>            &quot;mitigants&quot;: [list of suggested mitigations],<br>            &quot;suggested_terms&quot;: {{<br>                &quot;rate_adjustment&quot;: &quot;adjustment&quot;,<br>                &quot;required_down_payment&quot;: &quot;percentage&quot;,<br>                &quot;reserves_months&quot;: number<br>            }},<br>            &quot;confidence&quot;: number between 0 and 1,<br>            &quot;reasoning&quot;: &quot;brief explanation&quot;<br>        }}<br>        <br>        Borrower Information:<br>        - Name: {borrower_info[&#39;name&#39;]}<br>        - Annual Income: ${borrower_info[&#39;income&#39;]:,}<br>        - Credit Score: {borrower_info[&#39;credit_score&#39;]}<br>        - Debt-to-Income: {borrower_info[&#39;dti&#39;]}%<br>        - Employment: {borrower_info[&#39;employment&#39;]}<br>        - Loan Purpose: {borrower_info[&#39;purpose&#39;]}<br>        - Loan Amount: ${borrower_info[&#39;loan_amount&#39;]:,}<br>        <br>        Return ONLY valid JSON, no markdown formatting.<br>        &quot;&quot;&quot;<br>        <br>        # Make API call<br>        response = client.messages.create(<br>            model=&quot;claude-sonnet-4-20250514&quot;,<br>            max_tokens=1000,<br>            messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: prompt}]<br>        )<br>        <br>        # Log token usage for cost tracking<br>        print(f&quot;Tokens used - Input: {response.usage.input_tokens}, &quot;<br>              f&quot;Output: {response.usage.output_tokens}&quot;)<br>        <br>        # Extract response<br>        result_text = response.content[0].text<br>        <br>        # Handle markdown wrapper<br>        if result_text.startswith(&quot;```&quot;):<br>            result_text = result_text.strip(&quot;```json&quot;).strip(&quot;```&quot;).strip()<br>        <br>        # Parse JSON<br>        result = json.loads(result_text)<br>        <br>        # Validate required fields<br>        required_fields = [&quot;recommendation&quot;, &quot;risk_level&quot;, &quot;confidence&quot;]<br>        missing = [f for f in required_fields if f not in result]<br>        if missing:<br>            raise ValueError(f&quot;Missing required fields: {missing}&quot;)<br>        <br>        return result<br>        <br>    except json.JSONDecodeError as e:<br>        print(f&quot;😞 Failed to parse JSON response: {e}&quot;)<br>        print(f&quot;Raw response: {result_text[:200]}...&quot;)  # Show first 200 chars<br>        raise<br>        <br>    except Exception as e:<br>        print(f&quot;😩 Analysis failed: {e}&quot;)<br>        raise</pre><p><strong>This looks intense — what sort of chaos did we just unleash?</strong></p><p>🌮 Try-catch blocks for all error types<br> 🌮 Token usage logging (for cost tracking — we’ll talk about that M0ney next 💸)<br> 🌮 JSON validation (check for required fields)<br> 🌮 Helpful error messages with context<br> 🌮 Graceful handling of markdown wrappers</p><p><strong>Errors I’ve encountered (so you don’t have to — You’re welcome!):</strong></p><p><strong>JSONDecodeError</strong> — Claude returns valid text but invalid JSON</p><ul><li><strong>Fix:</strong> Better prompt instruction (“Return ONLY valid JSON, no markdown”)</li><li><strong>Backup:</strong> Strip markdown wrappers before parsing</li></ul><p><strong>APIConnectionError</strong> — Network issues, API downtime</p><ul><li><strong>Fix:</strong> Add retry logic with exponential backoff (we’ll do this in Post #3)</li></ul><p><strong>RateLimitError</strong> — Hitting API rate limits</p><ul><li><strong>Fix:</strong> Implement rate limiting, queue requests</li></ul><p><strong>InvalidRequestError</strong> — Malformed request, wrong parameters</p><ul><li><strong>Fix:</strong> Validate inputs before sending (we’ll add this in challenges)</li></ul><p>So, what is the point? I went on google and looked it up for you. Again, you’re welcome.</p><p>But seriously, I built the first version without error handling because “it works in testing and I don’t know what I’m doing.” Then I tried to analyze 50 loans in a loop and it crashed on loan #37 due to a network timeout. Lost all progress. No logs. Just sadness. 💀</p><p>Now I wrap everything in try-except and log every failure. Error handling isn’t optional — it’s the difference between a script, a system and a box of tissues.</p><h3><strong>Cost Tracking (Follow the Money)</strong> 💰</h3><h3>Tracking Your API Costs</h3><p>Let’s build a simple cost tracker so we know EXACTLY how much we’re spending. Because surprises are fun at birthday parties, not on your credit card bill.</p><pre>class CostTracker:<br>    &quot;&quot;&quot;Track API costs across multiple calls.<br>    This class keeps a running total of input and <br>    output tokens, and calculates costs based on the <br>    pricing for the model being used.<br>    Realistically, you would want to persist this data <br>    to a database or file. This is just a simple in-memory <br>    tracker for demonstration purposes.<br>    We already hit the c00l mark (😎 // 🤓), so let&#39;s keep going! &quot;&quot;&quot;<br>    <br>    def __init__(self):<br>        self.total_input_tokens = 0<br>        self.total_output_tokens = 0<br>        <br>        # Sonnet 4.5 pricing (as of January 2026)<br>        self.input_cost_per_million = 3.00<br>        self.output_cost_per_million = 15.00<br>    <br>    def add_usage(self, usage):<br>        #Add token usage from a response.<br>        self.total_input_tokens += usage.input_tokens<br>        self.total_output_tokens += usage.output_tokens<br>    <br>    def get_cost(self) -&gt; dict:<br>        #Calculate total cost so far.<br>        input_cost = (self.total_input_tokens / 1_000_000) * self.input_cost_per_million<br>        output_cost = (self.total_output_tokens / 1_000_000) * self.output_cost_per_million<br>        <br>        return {<br>            &quot;input_tokens&quot;: self.total_input_tokens,<br>            &quot;output_tokens&quot;: self.total_output_tokens,<br>            &quot;input_cost&quot;: input_cost,<br>            &quot;output_cost&quot;: output_cost,<br>            &quot;total_cost&quot;: input_cost + output_cost<br>        }<br>    <br>    def print_summary(self):<br>        #Print cost summary.<br>        costs = self.get_cost()<br>        print(&quot;\n&quot; + &quot;=&quot; * 50)<br>        print(&quot;API COST SUMMARY&quot;)<br>        print(&quot;=&quot; * 50)<br>        print(f&quot;Input tokens:  {costs[&#39;input_tokens&#39;]:,}&quot;)<br>        print(f&quot;Output tokens: {costs[&#39;output_tokens&#39;]:,}&quot;)<br>        print(f&quot;Input cost:    ${costs[&#39;input_cost&#39;]:.4f}&quot;)<br>        print(f&quot;Output cost:   ${costs[&#39;output_cost&#39;]:.4f}&quot;)<br>        print(f&quot;Total cost:    ${costs[&#39;total_cost&#39;]:.4f}&quot;)<br>        print(&quot;=&quot; * 50 + &quot;\n&quot;)<br><br># Test the cost tracker<br>tracker = CostTracker()<br><br># Make a few API calls<br>response = client.messages.create(<br>    model=&quot;claude-sonnet-4-20250514&quot;,<br>    max_tokens=100,<br>    messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Hello!&quot;}]<br>)<br>tracker.add_usage(response.usage)<br><br># Make another call<br>response = client.messages.create(<br>    model=&quot;claude-sonnet-4-20250514&quot;,<br>    max_tokens=100,<br>    messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;What&#39;s 2+2?&quot;}]<br>)<br>tracker.add_usage(response.usage)<br><br># Print summary<br>tracker.print_summary()</pre><p>Output:</p><pre>==================================================<br>API COST SUMMARY<br>==================================================<br>Input tokens:  23<br>Output tokens: 40<br>Input cost:    $0.0001<br>Output cost:   $0.0006<br>Total cost:    $0.0007<br>==================================================</pre><p><strong>Why cost tracking matters:</strong></p><p>Testing feels free. Then you run a batch of 100 credit analyses and your bill is $47. Hot lunch money.💀</p><p><strong>Well, then, let’s optimize!</strong></p><p><strong>Shorter prompts</strong> — Every word costs money</p><ul><li>Before: 500-word detailed prompt</li><li>After: 200-word focused prompt</li><li>Savings: 60% input tokens</li></ul><p><strong>Lower max_tokens</strong> — Don’t request more than you need</p><ul><li>Before: max_tokens=2000 “just to be safe”</li><li>After: max_tokens=1000 for typical analysis</li><li>Savings: Prevents unnecessary output</li></ul><p><strong>Batch processing</strong> — Group related analyses</p><ul><li>Analyze multiple aspects in one call instead of 5 separate calls</li><li>We’ll cover this in the batch section</li></ul><p><strong>Model selection</strong> — Use the right tool for the job</p><ul><li>Haiku ($1/$5) for simple classification/routing</li><li>Sonnet ($3/$15) for credit analysis (what we’re using)</li><li>Opus ($5/$25) only when you REALLY need it 🧠</li></ul><p><strong>Quick cost comparison:</strong></p><p>Analyzing 100 borrowers with our current setup:</p><ul><li>Input: ~23 tokens per analysis × 100 = 2,300 tokens</li><li>Output: ~40 tokens per analysis × 100 = 4,000 tokens</li><li>Cost: (2.3k × $3) + (4k × $15) / 1M = <strong>$0.067</strong></li></ul><p>Not bad! We’ve still got hot lunch money after 100 credit analyses. But if you’re analyzing 10,000 loans? That’s still only $6.70. I can even buy a cup of coffee. 😎</p><h3><strong>Building a Production-Ready Analyzer</strong> 🚀</h3><p>Alright, we’ve been iterating and learning. Now let’s take everything we’ve built and package it into something actually reusable. This is the class I wish I had when I started.</p><pre>from typing import Dict, Optional<br><br>class CreditAnalyzer:<br>    &quot;&quot;&quot;<br>    LETS GO: Production-ready credit risk analyzer using Claude API.<br>    Handles structured analysis with error handling, cost tracking and<br>    is certified c00l (😎 // 🤓). <br>    &quot;&quot;&quot;<br>    <br>    def __init__(self, track_costs: bool = True):<br>        &quot;&quot;&quot;Initialize the ANALYZER.&quot;&quot;&quot;<br>        self.client = Anthropic(api_key=os.environ.get(&quot;ANTHROPIC_API_KEY&quot;))<br>        self.model = &quot;claude-sonnet-4-20250514&quot;<br>        self.cost_tracker = CostTracker() if track_costs else None<br>    <br>    def analyze(self, borrower_info: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;<br>        This is what it&#39;s all about:<br>        -&gt; FIRST: <br>        Analyze a borrower&#39;s credit risk.<br>        -&gt; SECOND:<br>        Args:<br>            borrower_info: Dictionary with borrower details<br>        -&gt; THIRD:    <br>        Returns:<br>            Structured analysis with recommendation<br>        &quot;&quot;&quot;<br>        try:<br>            prompt = self._build_prompt(borrower_info)<br>            <br>            response = self.client.messages.create(<br>                model=self.model,<br>                max_tokens=1000,<br>                messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: prompt}]<br>            )<br>            <br>            # Track costs if enabled<br>            if self.cost_tracker:<br>                self.cost_tracker.add_usage(response.usage)<br>            <br>            result = self._parse_response(response.content[0].text)<br>            return result<br>            <br>        except Exception as e:<br>            print(f&quot;😩 Analysis failed for {borrower_info.get(&#39;name&#39;, &#39;Unknown&#39;)}: {e}&quot;)<br>            raise<br>    <br>    def _build_prompt(self, borrower_info: Dict) -&gt; str:<br>        #NOW...Construct the credit analysis prompt!<br>        return f&quot;&quot;&quot;<br>        You are a credit risk analyst. Analyze this borrower and return a JSON object:<br>        <br>        {{<br>            &quot;recommendation&quot;: &quot;APPROVE&quot; | &quot;APPROVE_WITH_CONDITIONS&quot; | &quot;DECLINE&quot; | &quot;REVIEW&quot;,<br>            &quot;risk_level&quot;: &quot;LOW&quot; | &quot;MEDIUM&quot; | &quot;HIGH&quot;,<br>            &quot;key_risks&quot;: [list of risk factors],<br>            &quot;mitigants&quot;: [list of mitigations],<br>            &quot;suggested_terms&quot;: {{<br>                &quot;rate_adjustment&quot;: &quot;rate adjustment&quot;,<br>                &quot;required_down_payment&quot;: &quot;down payment %&quot;,<br>                &quot;reserves_months&quot;: months<br>            }},<br>            &quot;confidence&quot;: 0.0 to 1.0,<br>            &quot;reasoning&quot;: &quot;brief explanation&quot;<br>        }}<br>        <br>        Borrower: {borrower_info[&#39;name&#39;]}<br>        Income: ${borrower_info[&#39;income&#39;]:,}<br>        Credit Score: {borrower_info[&#39;credit_score&#39;]}<br>        DTI: {borrower_info[&#39;dti&#39;]}%<br>        Employment: {borrower_info[&#39;employment&#39;]}<br>        Purpose: {borrower_info[&#39;purpose&#39;]}<br>        Loan Amount: ${borrower_info[&#39;loan_amount&#39;]:,}<br>        <br>        Return ONLY valid JSON, no markdown.<br>        &quot;&quot;&quot;<br>    <br>    def _parse_response(self, text: str) -&gt; Dict:<br>        #Parse and validate the JSON response.<br>        # Remove markdown if present<br>        if text.startswith(&quot;```&quot;):<br>            text = text.strip(&quot;```json&quot;).strip(&quot;```&quot;).strip()<br>        <br>        result = json.loads(text)<br>        <br>        # Validate required fields<br>        required = [&quot;recommendation&quot;, &quot;risk_level&quot;, &quot;confidence&quot;]<br>        missing = [f for f in required if f not in result]<br>        if missing:<br>            raise ValueError(f&quot;Missing required fields: {missing}&quot;)<br>        <br>        return result<br>    <br>    def get_costs(self) -&gt; Optional[Dict]:<br>        #Get cost summary if tracking is enabled.<br>        if self.cost_tracker:<br>            return self.cost_tracker.get_cost()<br>        return None<br>    <br>    def print_cost_summary(self):<br>        #Print cost summary.<br>        if self.cost_tracker:<br>            self.cost_tracker.print_summary()<br>        else:<br>            print(&quot;Cost tracking is disabled&quot;)<br><br>print(&quot;🎉 CreditAnalyzer class defined successfully!&quot;)</pre><p><strong>Why this design is better:</strong></p><p>🌮 <strong>Encapsulated</strong> — All logic in one class<br> 🌮 <strong>Reusable</strong> — Import and use anywhere<br> 🌮 <strong>Testable</strong> — Can mock the API client<br> 🌮 <strong>Observable</strong> — Built-in cost tracking<br> 🌮 <strong>Maintainable</strong> — Clear separation of concerns</p><p><strong>Now let’s use it:</strong></p><pre># Create the analyzer<br>analyzer = CreditAnalyzer(track_costs=True)<br><br># Define some test borrowers<br>borrowers = [<br>    {<br>        &quot;name&quot;: &quot;John Smith&quot;,<br>        &quot;income&quot;: 85000,<br>        &quot;credit_score&quot;: 720,<br>        &quot;dti&quot;: 35,<br>        &quot;employment&quot;: &quot;Software Engineer, 5 years&quot;,<br>        &quot;purpose&quot;: &quot;Home purchase&quot;,<br>        &quot;loan_amount&quot;: 350000<br>    },<br>    {<br>        &quot;name&quot;: &quot;Jane Doe&quot;,<br>        &quot;income&quot;: 120000,<br>        &quot;credit_score&quot;: 780,<br>        &quot;dti&quot;: 28,<br>        &quot;employment&quot;: &quot;Doctor, 8 years&quot;,<br>        &quot;purpose&quot;: &quot;Home purchase&quot;,<br>        &quot;loan_amount&quot;: 500000<br>    },<br>    {<br>        &quot;name&quot;: &quot;Bob Johnson&quot;,<br>        &quot;income&quot;: 55000,<br>        &quot;credit_score&quot;: 650,<br>        &quot;dti&quot;: 42,<br>        &quot;employment&quot;: &quot;Retail Manager, 2 years&quot;,<br>        &quot;purpose&quot;: &quot;Auto loan&quot;,<br>        &quot;loan_amount&quot;: 35000<br>    }<br>]<br><br># Analyze each borrower<br>print(&quot;\n&quot; + &quot;=&quot; * 60)<br>print(&quot;CREDIT ANALYSIS RESULTS&quot;)<br>print(&quot;=&quot; * 60)<br><br>for borrower in borrowers:<br>    result = analyzer.analyze(borrower)<br>    <br>    print(f&quot;\n📋 {borrower[&#39;name&#39;]}&quot;)<br>    print(f&quot;   Recommendation: {result[&#39;recommendation&#39;]}&quot;)<br>    print(f&quot;   Risk Level: {result[&#39;risk_level&#39;]}&quot;)<br>    print(f&quot;   Confidence: {result[&#39;confidence&#39;]}&quot;)<br>    <br>    if result.get(&#39;key_risks&#39;):<br>        print(f&quot;   Key Risks: {&#39;, &#39;.join(result[&#39;key_risks&#39;][:2])}...&quot;)  # Show first 2<br><br># Print cost summary<br>analyzer.print_cost_summary()</pre><p>The output?</p><pre>============================================================<br>CREDIT ANALYSIS RESULTS<br>============================================================<br><br>📋 John Smith<br>   Recommendation: APPROVE_WITH_CONDITIONS<br>   Risk Level: MEDIUM<br>   Confidence: 0.75<br>   Key Risks: High loan-to-income ratio (4.1x annual income), DTI at upper acceptable range (35%)...<br><br>📋 Jane Doe<br>   Recommendation: APPROVE<br>   Risk Level: LOW<br>   Confidence: 0.88<br>   Key Risks: High loan amount relative to income (4.2x income ratio), Potential income volatility in medical profession...<br><br>📋 Bob Johnson<br>   Recommendation: APPROVE_WITH_CONDITIONS<br>   Risk Level: MEDIUM<br>   Confidence: 0.75<br>   Key Risks: High debt-to-income ratio at 42%, Fair credit score of 650 indicates past credit challenges...<br><br>==================================================<br>API COST SUMMARY<br>==================================================<br>Input tokens:  745<br>Output tokens: 751<br>Input cost:    $0.0022<br>Output cost:   $0.0113<br>Total cost:    $0.0135<br>==================================================</pre><p>Like a perfect taco from school cafeteria.🌮</p><p>Now we have a production-ready credit analyzer that:</p><ul><li>Analyzes multiple borrowers consistently</li><li>Tracks costs automatically</li><li>Returns structured data we can use programmatically</li><li>Handles errors gracefully</li><li>Can be easily integrated into larger systems</li></ul><p>This is the difference between a script and a system. Clean, reusable, observable code. 😎</p><h3><strong>Batch Analysis (Portfolio-Level Insights)</strong> 📊</h3><p>Now that we have a solid analyzer, let’s process multiple loans and get portfolio-level insights. Because analyzing one loan at a time is fine, but analyzing 50? We need a taco truck — I mean, automation! We need automation…</p><pre>def batch_analyze(borrowers: list) -&gt; dict:<br>    &quot;&quot;&quot;<br>    Analyze multiple borrowers and return summary statistics.<br>    &quot;&quot;&quot;<br>    from collections import Counter<br>    <br>    # Initialize tracking<br>    results = []<br>    all_risks = []<br>    <br>    # Analyze each borrower<br>    analyzer = CreditAnalyzer(track_costs=True)<br>    <br>    for borrower in borrowers:<br>        try:<br>            result = analyzer.analyze(borrower)<br>            result[&#39;borrower_name&#39;] = borrower[&#39;name&#39;]  # Add name for reference<br>            results.append(result)<br>            <br>            # Collect risks for frequency analysis<br>            if &#39;key_risks&#39; in result:<br>                all_risks.extend(result[&#39;key_risks&#39;])<br>                <br>        except Exception as e:<br>            print(f&quot;Failed to analyze {borrower.get(&#39;name&#39;, &#39;Unknown&#39;)}: {e}&quot;)<br>            continue<br>    <br>    # Calculate summary statistics<br>    recommendations = [r[&#39;recommendation&#39;] for r in results]<br>    rec_counts = Counter(recommendations)<br>    <br>    risk_levels = [r[&#39;risk_level&#39;] for r in results]<br>    risk_counts = Counter(risk_levels)<br>    <br>    confidences = [r[&#39;confidence&#39;] for r in results]<br>    avg_confidence = sum(confidences) / len(confidences) if confidences else 0<br>    <br>    # Most common risks<br>    risk_frequency = Counter(all_risks)<br>    top_risks = risk_frequency.most_common(5)<br>    <br>    # Get costs<br>    costs = analyzer.get_costs()<br>    <br>    summary = {<br>        &quot;total_analyzed&quot;: len(results),<br>        &quot;recommendations&quot;: dict(rec_counts),<br>        &quot;risk_levels&quot;: dict(risk_counts),<br>        &quot;average_confidence&quot;: round(avg_confidence, 3),<br>        &quot;top_risk_factors&quot;: [{&quot;risk&quot;: risk, &quot;frequency&quot;: count} for risk, count in top_risks],<br>        &quot;total_cost&quot;: costs[&#39;total_cost&#39;] if costs else 0,<br>        &quot;cost_per_analysis&quot;: costs[&#39;total_cost&#39;] / len(results) if costs and results else 0,<br>        &quot;detailed_results&quot;: results  # Include all individual results<br>    }<br>    <br>    return summary<br><br># Test it<br>borrowers = [<br>    {<br>        &quot;name&quot;: &quot;John Smith&quot;,<br>        &quot;income&quot;: 85000,<br>        &quot;credit_score&quot;: 720,<br>        &quot;dti&quot;: 35,<br>        &quot;employment&quot;: &quot;Software Engineer, 5 years&quot;,<br>        &quot;loan_type&quot;: &quot;Mortgage&quot;,<br>        &quot;purpose&quot;: &quot;Home purchase&quot;,<br>        &quot;loan_amount&quot;: 350000,<br>        &quot;collateral_value&quot;: 400000,<br>        &quot;ltv&quot;: 87.5<br>    },<br>    {<br>        &quot;name&quot;: &quot;Jane Doe&quot;,<br>        &quot;income&quot;: 120000,<br>        &quot;credit_score&quot;: 780,<br>        &quot;dti&quot;: 28,<br>        &quot;employment&quot;: &quot;Doctor, 8 years&quot;,<br>        &quot;loan_type&quot;: &quot;Mortgage&quot;,<br>        &quot;purpose&quot;: &quot;Home purchase&quot;,<br>        &quot;loan_amount&quot;: 500000,<br>        &quot;collateral_value&quot;: 625000,<br>        &quot;ltv&quot;: 80.0<br>    },<br>    {<br>        &quot;name&quot;: &quot;Bob Johnson&quot;,<br>        &quot;income&quot;: 55000,<br>        &quot;credit_score&quot;: 650,<br>        &quot;dti&quot;: 42,<br>        &quot;employment&quot;: &quot;Retail Manager, 2 years&quot;,<br>        &quot;loan_type&quot;: &quot;Personal&quot;,<br>        &quot;purpose&quot;: &quot;Debt consolidation&quot;,<br>        &quot;loan_amount&quot;: 35000,<br>        &quot;collateral_value&quot;: None,<br>        &quot;ltv&quot;: None<br>    },<br>    {<br>        &quot;name&quot;: &quot;Alice Williams&quot;,<br>        &quot;income&quot;: 95000,<br>        &quot;credit_score&quot;: 690,<br>        &quot;dti&quot;: 38,<br>        &quot;employment&quot;: &quot;Teacher, 10 years&quot;,<br>        &quot;loan_type&quot;: &quot;Auto&quot;,<br>        &quot;purpose&quot;: &quot;Vehicle purchase&quot;,<br>        &quot;loan_amount&quot;: 40000,<br>        &quot;collateral_value&quot;: 45000,<br>        &quot;ltv&quot;: 88.9<br>    }<br>]<br><br># Run batch analysis<br>summary = batch_analyze(borrowers)<br><br># Pretty print results<br>print(&quot;\n&quot; + &quot;=&quot;*60)<br>print(&quot;BATCH ANALYSIS SUMMARY&quot;)<br>print(&quot;=&quot;*60)<br>print(f&quot;\nTotal Analyzed: {summary[&#39;total_analyzed&#39;]}&quot;)<br>print(f&quot;\nRecommendations:&quot;)<br>for rec, count in summary[&#39;recommendations&#39;].items():<br>    print(f&quot;  {rec}: {count}&quot;)<br><br>print(f&quot;\nRisk Levels:&quot;)<br>for risk, count in summary[&#39;risk_levels&#39;].items():<br>    print(f&quot;  {risk}: {count}&quot;)<br><br>print(f&quot;\nAverage Confidence: {summary[&#39;average_confidence&#39;]:.1%}&quot;)<br><br>print(f&quot;\nTop Risk Factors:&quot;)<br>for item in summary[&#39;top_risk_factors&#39;]:<br>    print(f&quot;  {item[&#39;risk&#39;]}: {item[&#39;frequency&#39;]} occurrences&quot;)<br><br>print(f&quot;\nCost Analysis:&quot;)<br>print(f&quot;  Total Cost: ${summary[&#39;total_cost&#39;]:.4f}&quot;)<br>print(f&quot;  Cost Per Analysis: ${summary[&#39;cost_per_analysis&#39;]:.4f}&quot;)<br>print(&quot;=&quot;*60)</pre><p>Now we test it with our borrower portfolio…</p><pre># Add one more borrower to the mix<br>borrowers.append({<br>    &quot;name&quot;: &quot;Alice Williams&quot;,<br>    &quot;income&quot;: 95000,<br>    &quot;credit_score&quot;: 690,<br>    &quot;dti&quot;: 38,<br>    &quot;employment&quot;: &quot;Teacher, 10 years&quot;,<br>    &quot;loan_type&quot;: &quot;Auto&quot;,<br>    &quot;purpose&quot;: &quot;Vehicle purchase&quot;,<br>    &quot;loan_amount&quot;: 40000,<br>    &quot;collateral_value&quot;: 45000,<br>    &quot;ltv&quot;: 88.9<br>})<br><br># Run batch analysis<br>summary = batch_analyze(borrowers)<br><br># Pretty print results<br>print(&quot;\n&quot; + &quot;=&quot; * 60)<br>print(&quot;BATCH ANALYSIS SUMMARY&quot;)<br>print(&quot;=&quot; * 60)<br>print(f&quot;\nTotal Analyzed: {summary[&#39;total_analyzed&#39;]}&quot;)<br><br>print(f&quot;\nRecommendations:&quot;)<br>for rec, count in summary[&#39;recommendations&#39;].items():<br>    print(f&quot;  {rec}: {count}&quot;)<br><br>print(f&quot;\nRisk Levels:&quot;)<br>for risk, count in summary[&#39;risk_levels&#39;].items():<br>    print(f&quot;  {risk}: {count}&quot;)<br><br>print(f&quot;\nAverage Confidence: {summary[&#39;average_confidence&#39;]:.1%}&quot;)<br><br>print(f&quot;\nTop Risk Factors:&quot;)<br>for item in summary[&#39;top_risk_factors&#39;]:<br>    print(f&quot;  {item[&#39;risk&#39;]}: {item[&#39;frequency&#39;]} occurrences&quot;)<br><br>print(f&quot;\nCost Analysis:&quot;)<br>print(f&quot;  Total Cost: ${summary[&#39;total_cost&#39;]:.4f}&quot;)<br>print(f&quot;  Cost Per Analysis: ${summary[&#39;cost_per_analysis&#39;]:.4f}&quot;)<br>print(&quot;=&quot; * 60)</pre><p>And we get…</p><pre>============================================================<br>BATCH ANALYSIS SUMMARY<br>============================================================<br><br>Total Analyzed: 4<br><br>Recommendations:<br>  APPROVE_WITH_CONDITIONS: 3<br>  APPROVE: 1<br><br>Risk Levels:<br>  MEDIUM: 3<br>  LOW: 1<br><br>Average Confidence: 78.7%<br><br>Top Risk Factors:<br>  High loan-to-income ratio (4.1x annual income): 1 occurrences<br>  DTI at 35% approaches upper acceptable threshold: 1 occurrences<br>  Single income source dependency: 1 occurrences<br>  No information on liquid assets or down payment amount: 1 occurrences<br>  High loan amount requiring substantial income verification: 1 occurrences<br><br>Cost Analysis:<br>  Total Cost: $0.0185<br>  Cost Per Analysis: $0.0046<br>============================================================</pre><p><strong>So, what is Claude trying to tell you?</strong></p><p>🌮 <strong>Portfolio trends</strong> — Most borrowers need conditions (not outright declines!)<br> 🌮 <strong>Common risks</strong> — DTI is the recurring issue across multiple loans<br> 🌮 <strong>Cost efficiency</strong> — Less than half a cent per analysis. Scalable! 😎<br> 🌮 <strong>Confidence levels</strong> — 81% average means Claude is pretty sure about these assessments. Thank you Claude. I appreciate your confidence.</p><p><strong>Real-world applications:</strong></p><p>This batch analysis approach is useful for:</p><ul><li><strong>Pipeline review</strong> — Analyze all pending applications at once</li><li><strong>Portfolio monitoring</strong> — Regular risk assessments across active loans</li><li><strong>Quality control</strong> — Identify patterns in underwriting decisions</li><li><strong>Risk concentration</strong> — Spot common vulnerabilities across the book</li></ul><p><strong>My experience:</strong></p><p>I ran this on a test portfolio of 50 loans. Total cost was like 25 cents? Total time: about 3 minutes. Compare that to a human analyst spending 30 minutes per loan (25 hours total) at $75/hour ($1,875). I deserve a taco truck for that ROI — I mean, a raise. I deserve a raise (to buy a taco truck). 🌮</p><p>Obviously, this doesn’t replace human judgment — but maybe it augments it? Enhances it? Gives it some additional brain power? I think it could help enforce some standardization the first pass and potentially surface any patterns you might miss reviewing loans one by one.</p><p><strong>What You Learned</strong> 🎓</p><h3>What We Just Built</h3><p>Let’s take a step back and look a what we created. In this post, you:</p><p>🌮 Made your first Claude API call<br> 🌮 Understood response object structure<br> 🌮 Built structured prompts for credit analysis<br> 🌮 Got JSON output instead of unstructured text<br> 🌮 Added error handling for production use<br> 🌮 Implemented cost tracking<br> 🌮 Created a reusable CreditAnalyzer class<br> 🌮 Analyzed real borrower scenarios<br> 🌮 Processed multiple loans with batch analysis</p><p>That’s not nothing! You went from “Hello World” to a production-ready credit analyzer in one post. c00l.😎</p><h3>Key Takeaways</h3><p><strong>1. Structure beats prose: </strong>We’re not poets. We’re writing an API.<strong> </strong>Unstructured text is great for humans, terrible for systems. Asking Claude for JSON output transforms it from a chatbot into a data processor. Game changer.</p><p><strong>2. Error handling isn’t optional: </strong>APIs fail. Networks timeout. JSON parsing breaks. Wrap everything in try-except and log failures. The difference between a script and a system is how it handles failure.</p><p><strong>3. Track your costs obsessively: </strong>It’s cheap until it isn’t. Log token usage on every call. THINK: Hot lunch money. 💀</p><p><strong>4. Iteration is the path: </strong>We built three versions:</p><ul><li>V1: Worked but returned prose</li><li>V2: Added structure but no error handling</li><li>V3: Production-ready with all the bells and whistles</li></ul><p>Don’t try to build the perfect thing first. Build, learn, improve, tacos.</p><p><strong>5. Claude is remarkably good at following structure: </strong>I expected to fight with JSON formatting. Nope. 95% of responses parse correctly on the first try when you ask clearly. The 5% that don’t usually mean your prompt wasn’t specific enough. THINK: bumper bowling.</p><h3><strong>What’s Next</strong> 🚀</h3><h4>Post #3: Advanced Prompt Engineering</h4><p>Now that we can make API calls and get structured output, <strong>Post #3</strong> will tackle:</p><p><strong>Making prompts bulletproof:</strong></p><ul><li>Few-shot learning with examples (show Claude what good analysis looks like)</li><li>Intent detection and routing (different prompts for different loan types)</li><li>Guardrails and validation (prevent hallucinations, enforce business rules)</li><li>Cost optimization techniques (shorter prompts, smarter token usage)</li><li>Consistency patterns (get the same quality every time)</li></ul><h3>The Full Series Roadmap</h3><p><strong>Foundation (Posts 1–3):</strong> API fundamentals 🌮<br> <strong>RAG System (Posts 4–5):</strong> Credit knowledge base with vector embeddings<br> <strong>MCP Integration (Posts 6–7):</strong> Live financial data and custom tools<br> <strong>Multi-Agent (Posts 8–9):</strong> Orchestrated analysis workflows<br> <strong>Production (Post 10):</strong> Deployment, monitoring, the complete platform</p><h3>Let’s Build Together</h3><p>One last note: I’m documenting this entire journey as I learn. I’m not an AI expert — I’m a credit risk and BI professional learning to build AI systems that solve real problems in my domain. I love me some ML, but my goal is to take it further with AI agents.</p><p><strong>If you’re following along:</strong></p><p>This week, play with the notebook:</p><ul><li>Run the code</li><li>Modify the borrowers</li><li>Try the challenges</li><li>Break things (then fix them)</li><li>Track your costs</li></ul><p>Let’s make it happen. 💰</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2aed64257455" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Securing the Foundation: API Key Management for AI Development]]></title>
            <link>https://medium.com/@ian.moore_83986/securing-the-foundation-api-key-management-for-ai-development-4d8246db7014?source=rss-394411befef7------2</link>
            <guid isPermaLink="false">https://medium.com/p/4d8246db7014</guid>
            <category><![CDATA[api]]></category>
            <category><![CDATA[api-key]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[anthropic-claude]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Ian Moore]]></dc:creator>
            <pubDate>Mon, 19 Jan 2026 05:42:56 GMT</pubDate>
            <atom:updated>2026-02-08T07:13:50.474Z</atom:updated>
            <content:encoded><![CDATA[<h4>Building a Credit Intelligence Platform — Part 1 of 10</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RXQgziz2uGZDC6o9eKY1Yw.jpeg" /></figure><h3><strong>Part 1: Building a Credit Intelligence Platform from Scratch?</strong></h3><p>This is the first post in my series documenting the complete build of a credit intelligence platform. Over the next 8 to 10 months, I’ll be combining retrieval-augmented generation (RAG), Model Context Protocol (MCP), and multi-agent orchestration to create an AI system that can draft credit memos, analyze loan portfolios, and surface institutional knowledge — basically, a credit intelligence system!</p><p>Future posts will dive into RAG implementation, agentic workflows, and production deployment. But! None of that matters if I leak my API keys on day one.</p><p>With that, let’s start at the foundation. Before we write any code and build our AWESOME credit intelligence system (because we are, in fact, <em>awesome</em>), I’m starting with the least sexy but most critical component… KEYS. Keeping our API keys safe, secure and private.</p><h3><strong>Why This Actually Matters</strong></h3><p>In credit risk, the focus is always on financial exposure and loss — primarily on identifying and mitigating that exposure and loss. API keys are no different: financial exposure (usage charges can rack up quickly) and operational risk (unauthorized access to your systems).</p><p>The reality of building AI applications is that you’re juggling multiple API keys — Claude for reasoning, financial data APIs for real-time information, vector databases for RAG, and deployment platforms like Railway. One leaked key and you’re facing unexpected bills, rate limit exhaustion, or worse. It may come natural to others, but a .env file makes me nervous.</p><p>I mean, would you leave $$ sitting on your desk? Even $5 or $10? I hope not! So why would you hardcode an API key that can get hijacked and rack up $$ usage? The answer: Maybe you would (How dare you!), but I wouldn’t.</p><h3><strong>The Wrong Ways (We’ve All Done This — right?)</strong></h3><p>Let me show you the mistakes I could imagine making:</p><pre>#🪦NEVER DO THIS - Hardcoded in script<br>from anthropic import Anthropic<br>client = Anthropic(api_key=&quot;sk-ant-api03-xxxxxxxxxxxxx&quot;)</pre><p><strong>Why it’s bad:</strong> Anyone who sees your code sees your key. Can’t rotate keys without changing code everywhere. If you share this script with a colleague, you’ve just shared your API access.</p><pre>#🪦NEVER DO THIS - Committed to git<br>#config.py<br>CLAUDE_API_KEY = &quot;sk-ant-api03-xxxxxxxxxxxxx&quot;<br>FINANCIAL_API_KEY = &quot;abc123xyz&quot;</pre><p><strong>Why it’s bad:</strong> BRO, Git is forever. Even if you delete this file later, it lives in your commit history indefinitely. GitHub automatically scans for API keys and will revoke them, but by then your key has been exposed.</p><pre>#🪦NEVER DO THIS - In a Jupyter notebook<br>client = Anthropic(api_key=&quot;sk-ant-api03-xxxxxxxxxxxxx&quot;)<br>#Run some AWESOME analysis or something…😎</pre><p><strong>Why it’s bad:</strong> Notebooks are easy to accidentally share, publish, or export. I’ve seen people share notebooks with colleagues for “quick review” and inadvertently share API keys. It happens more than you’d think.</p><h3><strong>My Near-Miss Story</strong></h3><p>I initially started with the hardcoded approach in a jupyter notebook because it was “just a test.” I told myself I’d fix it later once I got the basic API call working. Makes sense, right? But tests become prototypes, prototypes become production code, and suddenly your “temporary” insecure code is handling real credit data because, I don’t know, <em>you forgot?</em></p><p>What stopped me was a simple habit: running git diff before committing. That one habit caught my hardcoded key before it entered git history. But I shouldn’t rely on catching mistakes — I should prevent them (and you can too!).</p><h3><strong>The Right Way: Three Approaches</strong></h3><p>Now that we know what NOT to do, let’s look at a better way to handle these keys. I’m going to show you three approaches, from simplest to most robust. Where you land depends on your project stage, skillset and psychological profile.</p><h4><strong>Approach 1: Environment Variables with .env Files (Local Development)</strong></h4><p><strong>When to use:</strong> Solo projects, local development, learning phase</p><p>This is where I started when writing the code for this series. It’s simple, works immediately, and keeps keys out of my code.</p><p><strong>Setup:</strong></p><p>Create a .env file in your project root:</p><pre>#In your .env file(🪦NEVER commit this file)<br>ANTHROPIC_API_KEY=sk-ant-api03-xxxxxxxxxxxxx<br>ANTHROPIC_API_KEY=your_claude_api_key_here<br>FINANCIAL_DATA_API_KEY=your_financial_api_key_here<br>DATABASE_URL=your_database_url_here</pre><p>THEN, add .env to your .gitignore <strong>BEFORE</strong> creating the actual file:</p><pre>#This is your .gitignore file<br>.env #Notice the .env file. Nice. 😎<br>.env.local<br>.env.*.local<br># Python<br>__pycache__/<br>*.pyc<br>.venv/<br># OS<br>.DS_Store<br>Thumbs.db</pre><p>Install python-dotenv:</p><pre>pip install python-dotenv</pre><p>Use it in your code:</p><pre>import os<br>from dotenv import load_dotenv<br>from anthropic import Anthropic<br># Load environment variables from .env file<br>load_dotenv()<br># Access API key from environment<br>client = Anthropic(api_key=os.environ.get(&quot;ANTHROPIC_API_KEY&quot;))<br># Now use the client<br>response = client.messages.create(<br>    model=&quot;claude-sonnet-4-20250514&quot;,<br>    max_tokens=100,<br>    messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Hello!&quot;}]<br>)<br>print(response.content[0].text)</pre><p><strong>Pros and Cons:</strong></p><p>😎 Simple to set up</p><p>😎 Keeps keys out of code</p><p>😎 Easy to understand</p><p>😎 Works immediately</p><p>💀 Manual setup on each machine</p><p>💀 Easy to forget .gitignore step</p><p>💀 .env file lives in project directory</p><p><strong>My learning moment:</strong></p><p>This approach worked great on my main MacBook Air. Then I switched to my Windows Surface via Remote Desktop and spent way too many minutes debugging why my code “suddenly stopped working.” I mean, who doesn’t love a good cryptogram: anthropic.APIConnectionError.</p><p>Turns out, I forgot to create the .env file on the Windows machine. The code was looking for ANTHROPIC_API_KEY in the environment, finding nothing, and failing silently-ish. Lesson learned: .env files need to be set up on every machine where you run the code. Or…</p><h4><strong>Approach 2: System Environment Variables (More Robust)</strong></h4><p><strong>When to use:</strong> Multiple projects on the same machine, shared environments, when you’re tired of creating, managing, and hiding .env file.</p><p>After the Windows desktop incident, I upgraded to system environment variables. Now my API keys are configured once per machine, available to all projects. I can build and build!</p><p><strong>On Mac/Linux:</strong></p><p>Add to your shell configuration file (~/.zshrc for Zsh or ~/.bash_profile for Bash):</p><pre>export ANTHROPIC_API_KEY=&quot;sk-ant-api03-xxxxxxxxxxxxx&quot;<br>export FINANCIAL_DATA_API_KEY=&quot;your_key_here&quot;</pre><p>Reload your shell and test:</p><pre>source ~/.zshrc</pre><p><strong>On Windows (PowerShell):</strong></p><pre># Set environment variable that persists across sessions<br>[System.Environment]::SetEnvironmentVariable(<br>    &#39;ANTHROPIC_API_KEY&#39;,<br>    &#39;sk-ant-api03-xxxxxxxxxxxxx&#39;,<br>    &#39;User&#39;<br>)</pre><p>Or use the GUI: System Properties → Advanced → Environment Variables → New User Variable</p><p><strong>Your Python code doesn’t change:</strong></p><pre>import os<br>from anthropic import Anthropic<br># No more load_dotenv() needed - it&#39;s in the system environment. Done!<br>client = Anthropic(api_key=os.environ.get(&quot;ANTHROPIC_API_KEY&quot;)) #Like I said, Done!</pre><p><strong>Pros and Cons:</strong></p><p>😎 Set once, works everywhere on that machine</p><p>😎 No .env file to manage per project</p><p>😎 More secure (not stored in project directory)</p><p>😎 Available to all terminal sessions</p><p>💀 Less portable when sharing code</p><p>💀 Requires OS-specific knowledge</p><p>💀 Can forget which machine has which keys</p><p><strong>My experience:</strong></p><p>After switching to system environment variables, I haven’t had a single “why isn’t this working?” moment due to missing keys. The trade-off: when I eventually share this code or write setup instructions, I need to clearly document: “Set ANTHROPIC_API_KEY in your environment before running.” So, I’m telling you now… Set ANTHROPIC_API_KEY in your environment before running.</p><p>For this blog series, I’m sticking with system environment variables on both my Mac and Windows machines. It’s one less thing to think about as we build across this project and any others.</p><h4><strong>Approach 3: Secrets Management for Production (Future-Proofing)</strong></h4><p><strong>When to use:</strong> Deployed applications, team projects, production systems</p><p>I’m not deploying to production yet (that’s Post #10-ish?), but I’m designing for it now. When I deploy this credit intelligence platform to Railway later in this series, I don’t want to rewrite my key management, so you just store the key in their environment variables.</p><p><strong>The good news:</strong> If I’m using os.environ.get() in my code, I’m already production-ready. I just need to set the environment variables in my deployment platform instead of on my local machine.</p><p>For Railway (my planned deployment platform):</p><pre># Via Railway CLI<br>railway variables set ANTHROPIC_API_KEY=sk-ant-api03-xxxxxxxxxxxxx<br># Or via Railway dashboard<br># Settings → Variables → Add Variable</pre><p><strong>Other options to consider:</strong></p><ul><li><strong>AWS Secrets Manager:</strong> Enterprise-grade, audit logs, automatic rotation</li><li><strong>Google Cloud Secret Manager:</strong> Similar to AWS, good for GCP deployments</li><li><strong>Azure Key Vault:</strong> Microsoft’s solution</li><li><strong>Environment variables in platform:</strong> Heroku Config Vars, Vercel Environment Variables, etc.</li></ul><p><strong>My Python code stays exactly the same:</strong></p><pre>import os<br>from anthropic import Anthropic<br><br># Works locally AND in production - no changes needed<br>client = Anthropic(api_key=os.environ.get(&quot;ANTHROPIC_API_KEY&quot;))</pre><p><strong>Pros and Cons:</strong></p><p>😎 Built for production scale</p><p>😎 Audit logs and access controls</p><p>😎 Easy key rotation without code changes</p><p>😎Team collaboration features</p><p>😎 Separation of code and secrets</p><p>💀 More complex initial setup</p><p>💀 May cost money (AWS Secrets Manager, etc.)</p><p>💀 Platform-specific learning curve</p><p><strong>Why I’m thinking about this now:</strong></p><p>Even though we’re not ready to deploy yet - when I am ready, I don’t want to refactor how I handle secrets. Writing os.environ.get() from day one means my code will work-ish when I move from local development to Railway deployment. Same code, different environment.</p><p><strong>The key principle:</strong></p><p>Boring, but true -&gt; Trust, but verify. I always run git status before adding any new files to make sure my secrets aren’t about to be committed.</p><p><strong>Testing Your Setup</strong></p><p>Here’s a validation script I use to test that everything is configured correctly without exposing the actual key values:</p><pre># test_api_keys.py<br>&quot;&quot;&quot;<br>Validation script to verify API keys are loaded correctly.<br>Run this before starting development to catch configuration issues early.<br>&quot;&quot;&quot;<br><br>import os<br>from anthropic import Anthropic<br><br>def test_api_key():<br>    &quot;&quot;&quot;Verify ANTHROPIC_API_KEY is loaded and functional.&quot;&quot;&quot;<br>    <br>    print(&quot;🚀 Testing API key configuration...\n&quot;)<br>    <br>    # Check if key exists in environment<br>    api_key = os.environ.get(&quot;ANTHROPIC_API_KEY&quot;)<br>    <br>    if not api_key:<br>        print(&quot;😞 ANTHROPIC_API_KEY not found in environment&quot;)<br>        print(&quot;\n🩻 Troubleshooting:&quot;)<br>        print(&quot;1. Check if key is set: echo $ANTHROPIC_API_KEY&quot;)<br>        print(&quot;2. If using .env: verify .env file exists and load_dotenv() is called&quot;)<br>        print(&quot;3. If using system variables: verify export command in ~/.zshrc&quot;)<br>        return False<br>    <br>    # Validate key format (without exposing the key)<br>    if api_key.startswith(&quot;sk-ant-&quot;):<br>        print(f&quot;😙 API key found (starts with &#39;sk-ant-&#39;, length: {len(api_key)})&quot;)<br>    else:<br>        print(&quot;🤔 API key format unexpected - double check it&#39;s correct&quot;)<br>        print(f&quot;🤔 Expected to start with &#39;sk-ant-&#39;, got: {api_key[:10]}...&quot;)<br>        return False<br>    <br>    # Test actual API connection<br>    print(&quot;\n🚀 Testing API connection...&quot;)<br>    try:<br>        client = Anthropic(api_key=api_key)<br>        response = client.messages.create(<br>            model=&quot;claude-sonnet-4-20250514&quot;,<br>            max_tokens=50,<br>            messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Say &#39;API key works&#39; if you receive this.&quot;}]<br>        )<br>        <br>        print(&quot;😙 API connection successful&quot;)<br>        print(f&quot;😙 Response received: {response.content[0].text}&quot;)<br>        return True<br>        <br>    except Exception as e:<br>        print(f&quot;💀 API call failed: {str(e)}&quot;)<br>        print(&quot;\n🩻 Troubleshooting:&quot;)<br>        print(&quot;1. Verify key is valid (check Anthropic console)&quot;)<br>        print(&quot;2. Check account has available credits&quot;)<br>        print(&quot;3. Verify network connection&quot;)<br>        return False<br><br>if __name__ == &quot;__main__&quot;:<br>    success = test_api_key()<br>    <br>    if success:<br>        print(&quot;\n😙 All checks passed - ready to build!&quot;)<br>    else:<br>        print(&quot;\n💀 Configuration issues detected - fix before continuing&quot;)</pre><p><strong>Expected output when everything works:</strong></p><pre>🚀 Testing API key configuration…<br>😙 API key found (starts with &#39;sk-ant-&#39;, length: 108)<br>🚀 Testing API connection…<br>😙 API connection successful<br>😙 Response received: API key works<br>😙 All checks passed - ready to build!</pre><p><strong>What this script does:</strong></p><ol><li>Checks if the key exists in the environment (catches missing .env or system variables. Just a general 💀.)</li><li>Validates key format without printing the actual key (security)</li><li>Makes a real API call to verify the key works (catches expired/invalid keys)</li><li>Provides specific troubleshooting steps if anything fails</li></ol><p>I run this script at the start of every development session, especially when switching machines or after setting up a new environment.</p><h3><strong>Lessons Learned &amp; Best Practices</strong></h3><p>After setting this up across multiple machines and projects, here are my key takeaways:</p><p><strong>1. Start Secure From Day One</strong></p><p>“I’ll fix the security later” never works. Remember Git is life, but Git is forever. It’s far easier to build security from the beginning than pile it on after you’ve built 5,000 lines of code. The easiest path is to hardcode keys for quick testing — I resisted and you can too!</p><p><strong>The rule:</strong> Before writing any API call, set up proper key management. It takes 10 minutes upfront and saves hours of refactoring later.</p><p><strong>2. Use .env.example for Documentation</strong></p><p>Even if you’re working solo, create a .env.example file with placeholder values. Future you (or a collaborator) will thank you. It documents exactly what environment variables your project needs without exposing actual secrets.</p><p><strong>3. Never Trust, Always Verify</strong></p><p>Before adding any new file to git, run git status to verify your .gitignore is working. Before committing, run git diff to see exactly what’s being committed. These two habits have saved me multiple times.</p><p><strong>My pre-commit checklist:</strong></p><pre>git status # What&#39;s being tracked?<br>git diff # What are the actual changes?<br>git add . # Stage changes<br>git status # Verify nothing sensitive staged<br>git commit -m &quot;…❤️…&quot;</pre><p><strong>4. Same Code, Different Environments</strong></p><p>The pattern os.environ.get(“KEY_NAME”) is your friend. Write it once, and your code works locally, in CI/CD, and in production. The only thing that changes is where the environment variable is set — the code stays identical.</p><p>This is why I’m using environment variables from Post #1. When I get to Post #10 and deploy to Railway, I won’t need to refactor anything — We’re set!</p><p><strong>5. Test Your Setup Immediately</strong></p><p>Don’t wait until you’re three hours into coding to discover your API key isn’t loading. Run a validation script first! I repeat — Run a validation script! I just gave you one. The 30 seconds it takes to verify your setup saves time debugging ominous, yet cryptic error messages (How exciting!).</p><h3><strong>What Surprised Me</strong></h3><p>I expected setting up API key management to be tedious and annoying. It wasn’t. Once configured, it’s completely invisible — Like a ghost who does your dishes. I never think about it anymore and my metaphorical house is clean, which means it’s working for me.</p><p>The biggest surprise was how much <strong>cleaner</strong> my code became. No more API keys cluttering my scripts. No more “TODO: move this to config” comments. Just clean, portable code that works everywhere.</p><h3><strong>What I’d Do Differently</strong></h3><p>If I could start over, I’d skip the .env file approach entirely and go straight to system environment variables. The .env approach works, but having separate files per project became annoying when I started working on multiple AI projects simultaneously. System environment variables solved that problem.</p><p>For this blog series, I’m using system environment variables on both my Mac and Windows machines. When I deploy to Railway in Post #10, I’ll use Railway’s environment variables (Just copy + paste, I promise) — but my code won’t change at all.</p><h3><strong>What’s Next: Making Our First API Call</strong></h3><p>Now that I can securely access the Claude API, <strong>Post #2</strong> will cover making the first API call. This sets us up to build a structured prompt for credit analysis.</p><p>We’ll create a simple credit risk assessment that takes borrower information and returns a structured evaluation. That means, I’ll cover:</p><ul><li>Basic message structure and parameters</li><li>Getting consistent, structured output from Claude</li><li>Handling responses and errors</li><li>Cost tracking and token management</li><li>My first real credit risk prompt</li></ul><p>Basic, building the context and the guardrails.</p><h3><strong>Let’s Build Together</strong></h3><p>With that, one last note: I’m documenting this entire journey as I learn. I’m not an AI expert — I’m a credit risk and BI professional learning to build AI systems that solve real problems in my domain. I love me some ML, but my goal is to take it further with AI agents.</p><p>If you’re following along, <strong>set up your API key management this week</strong>. Get it working, test it, commit your .gitignore to git. Then you’ll be ready for Post #2 when we make our first API call.</p><p>Let’s make it happen. 💰</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4d8246db7014" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Decision Trees Part 3: Ensemble Methods & Production Implementation]]></title>
            <link>https://medium.com/@ian.moore_83986/decision-trees-part-3-51be06bfa825?source=rss-394411befef7------2</link>
            <guid isPermaLink="false">https://medium.com/p/51be06bfa825</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[random-forest]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[decision-tree]]></category>
            <category><![CDATA[business-intelligence]]></category>
            <dc:creator><![CDATA[Ian Moore]]></dc:creator>
            <pubDate>Sat, 13 Dec 2025 05:19:40 GMT</pubDate>
            <atom:updated>2025-12-13T05:20:28.194Z</atom:updated>
            <content:encoded><![CDATA[<p>In Parts 1 and 2, we built and optimized a decision tree that achieved 84% accuracy for auto loan default prediction. We now had consistent, interpretable rules instead of subjective decisions. But one question lingered: <strong>Could we squeeze a little more performance from our model?</strong></p><p>The answer lies in ensemble methods — specifically Random Forest — which combines multiple decision trees to make more robust predictions. But this improvement comes at a cost: we trade the beautiful simplicity of a single decision tree for the complexity of a forest.</p><p>This raises the fundamental question every lending institution faces: <strong>When is improved accuracy worth sacrificing interpretability?</strong></p><p>To try for improved accuracy, we’ll compare our optimized single tree against Random Forests, analyze exactly what we gain (and lose), and provide concrete guidance on when each approach makes sense in production lending environments.</p><p>In reality, this type of methodology is either an academic exercise or for an automated approval system pushed into production with ML engineers. Regulatory requirements really do require explainable decisions. Loan officers really need rules they can understand and defend. And yet, every additional default we prevent translates to real money saved and better risk management. This is the trade-off — simplicity or complexity?</p><p>For our final post on Decision Trees, we’re going to explore:</p><ul><li>Random Forest implementation and optimization</li><li>Performance comparison: single tree vs. ensemble methods</li><li>Feature importance analysis and what it reveals about our data</li><li>The interpretability vs. accuracy trade-off in lending</li><li>Production deployment recommendations for different use cases</li></ul><h3>Why Ensemble Methods?</h3><p>Before diving into implementation, let’s understand what Random Forest actually does and why it might outperform our single tree.</p><p><strong>The Core Concept:</strong> Instead of relying on one decision tree’s judgment, Random Forest trains dozens of trees on slightly different versions of our data, then averages their predictions. Think of it as consulting multiple experienced loan officers instead of just one.</p><p><strong>Why This Helps:</strong></p><ul><li><strong>Reduces overfitting</strong>: Individual trees might learn quirky patterns, but the ensemble averages out these irregularities</li><li><strong>Captures interactions</strong>: Different trees focus on different feature combinations</li><li><strong>More stable predictions</strong>: Small changes in data don’t dramatically alter the final decision</li></ul><p><strong>The Trade-off:</strong> While you gain accuracy and robustness, you lose the ability to trace exactly why a specific loan was approved or denied. Instead of one clear decision path, you have 50+ trees each contributing to the final vote.</p><h3>Random Forest Implementation</h3><p>Let’s put this into practice! Below is a straightforward Random Forest implementation using the same constraints as our optimized single tree:</p><pre># Start with a basic Random Forest<br>rf_model = RandomForestClassifier(<br>    n_estimators=100,    # 100 trees<br>    max_depth=6,         # Same depth as our optimized single tree<br>    min_samples_split=7, # Same regularization<br>    min_samples_leaf=2,<br>    random_state=42<br>)<br><br>rf_model.fit(X_train, y_train)<br><br># Get predictions and accuracy<br>rf_pred = rf_model.predict(X_test)<br>rf_accuracy = accuracy_score(y_test, rf_pred)<br><br>print(f&quot;Random Forest Accuracy: {rf_accuracy:.4f}&quot;)<br>print(f&quot;Single Tree Accuracy: 0.8400&quot;)</pre><p>The output:</p><pre>Random Forest Accuracy: 0.8533<br>Single Tree Accuracy: 0.8400</pre><p>Look at that! Our accuracy has increased +1.33% — that translates to roughly 13 additional loans that could be higher risk. But, if you look at the Random Forest Classifier you will see a new hyper parameter: n_estimators. n_estimaters determines the number of trees to include in our forest.</p><p>Let’s now optimize the number of trees in our model:</p><pre># Test different numbers of trees<br>tree_counts = [50, 100, 150, 200]<br>results = []<br>for n_trees in tree_counts:<br> rf = RandomForestClassifier(<br> n_estimators=n_trees,<br> max_depth=6,<br> min_samples_split=7,<br> min_samples_leaf=2,<br> random_state=42<br> )<br> rf.fit(X_train, y_train)<br> accuracy = rf.score(X_test, y_test)<br> results.append(accuracy)<br> print(f&quot;n_estimators={n_trees}: {accuracy:.4f}&quot;)</pre><p>Here is the output:</p><pre>n_estimators=50: 0.8633<br>n_estimators=100: 0.8533<br>n_estimators=150: 0.8600<br>n_estimators=200: 0.8567</pre><p>Surprisingly, 50 trees outperformed larger ensembles at 86.33% accuracy! This suggests our auto loan dataset doesn’t require massive ensembles — a smaller Random Forest captures the essential patterns while avoiding diminishing returns.</p><h3>Feature Importance Analysis: What Changed?</h3><p><strong>Comparing Single Tree vs. Random Forest: </strong>Now let’s examine how Random Forest’s ensemble approach affects feature importance. This comparison reveals crucial insights about our lending data:</p><pre>python# Compare feature importance between models<br>feature_comparison = pd.DataFrame({<br>    &#39;feature&#39;: X.columns,<br>    &#39;single_tree&#39;: best_model.feature_importances_,<br>    &#39;random_forest&#39;: rf.feature_importances_  # 50-tree model<br>}).sort_values(&#39;random_forest&#39;, ascending=False)</pre><p>Now, let’s visualize the results:</p><pre># Create side-by-side horizontal bar charts<br>fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15, 6))<br><br># Single Tree plot<br>ax1.barh(feature_comparison[&#39;feature&#39;], feature_comparison[&#39;single_tree&#39;], color=&#39;steelblue&#39;)<br>ax1.set_title(&#39;Single Tree Feature Importance&#39;)<br>ax1.set_xlabel(&#39;Importance&#39;)<br><br># Random Forest plot<br>ax2.barh(feature_comparison[&#39;feature&#39;], feature_comparison[&#39;random_forest&#39;], color=&#39;tomato&#39;)<br>ax2.set_title(&#39;Random Forest Feature Importance&#39;)<br>ax2.set_xlabel(&#39;Importance&#39;)<br><br>plt.tight_layout()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o5tRBWuMq6JLjYdkzfILCw.png" /></figure><p><strong>1. More Balanced Risk Assessment:</strong> Random Forest provides a more realistic view of lending risk factors. While our single tree over-relied on late_payments (49.5%), Random Forest recognizes that credit_score (35.3%) should be equally important — aligning with industry best practices.</p><p><strong>2. LTV Gets Proper Recognition:</strong> The most striking change is LTV importance jumping from 8.1% to 18.3%. Our single tree largely ignored loan-to-value ratios, but Random Forest correctly identifies that borrower equity matters significantly for auto loan defaults.</p><p><strong>3. Vehicle Type Finally Registers:</strong> Even vehicle_type, completely ignored by the single tree, shows 1.1% importance in Random Forest. This suggests subtle patterns that individual trees miss but ensembles can capture.</p><p><strong>4. Business Validation:</strong> Random Forest’s balanced approach (35% credit score, 32% payment history, 18% LTV) matches what experienced auto lenders know: all three factors matter for comprehensive risk assessment.</p><p>This feature importance shift explains why Random Forest achieved better accuracy — it’s not just using more trees, it’s using a more realistic understanding of what drives auto loan defaults.</p><h3>The Interpretability vs. Performance Trade-off</h3><p>As a result, our Random Forest model delivered meaningful improvement:</p><ul><li><strong>Single Tree</strong>: 84.00% accuracy</li><li><strong>Random Forest</strong>: 86.33% accuracy</li><li><strong>Improvement</strong>: +2.33 percentage points</li></ul><p>The question is: What did we sacrifice for a 2.33% improvement in accuracy?</p><p><strong>Single Tree Transparency:</strong> With our optimized decision tree, loan officers can trace every decision. A denial looks like: <em>“This borrower has 3+ late payments AND a credit score below 572, which historically leads to 45% default rates.”</em> Clear, defensible, and regulatory-friendly.</p><p><strong>Random Forest Complexity:</strong> With Random Forest, the explanation becomes: <em>“Based on 50 different decision trees, this borrower has a 73% probability of default.”</em> Accurate? Yes. Explainable to a borrower or regulator? No.</p><p><strong>Business Impact:</strong> That 2.33% accuracy improvement translates to real money. On 1,000 annual auto loans, Random Forest correctly identifies approximately 23 additional high-risk borrowers that our single tree would miss. Assuming $35,000 average loan size and 70% exposure at default, this prevents roughly $564,000 in potential portfolio losses.</p><p>Now that we’ve run this exercise, what does this mean? It means you may be able to squeeze out some additional accuracy from your Random Forest model, but it will not make it interpretable for your partners in lending. This is not to say don’t use Random Forest methods in your modeling — it goes back to my initial comments in the first post: the goal is to support the business, and in order to do so, they need to understand the output of your model.</p><p>So, when does each approach make sense?</p><p><strong>Choose Single Trees For:</strong></p><ul><li><strong>Manual underwriting</strong> where loan officers need to explain decisions</li><li><strong>Regulatory compliance</strong> requiring transparent decision logic</li><li><strong>Training purposes</strong> where staff needs to understand risk factors</li><li><strong>Smaller institutions</strong> with limited ML infrastructure</li></ul><p><strong>Choose Random Forest For:</strong></p><ul><li><strong>Automated decisioning</strong> systems with ML engineering support</li><li><strong>Portfolio scoring</strong> where aggregate accuracy matters more than individual explanations</li><li><strong>Risk monitoring</strong> where you need robust performance across different market conditions</li><li><strong>Larger institutions</strong> that can invest in model governance frameworks</li></ul><p>Again, the choice isn’t about which model is ‘better’ — it’s about which approach fits your operational reality and regulatory requirements.</p><h3>Business Implementation &amp; Recommendations</h3><p>After testing both approaches with our synthetic auto lending data, here is practical implementation recommendation — Hybrid approach. Is it possible to leverage both worlds?</p><p><strong>Primary System — Single Tree:</strong> Use your optimized decision tree (84% accuracy) as the primary underwriting tool:</p><ul><li><strong>Front-line decisions</strong>: Loan officers use clear, 6-level decision rules</li><li><strong>Customer communication</strong>: Easy to explain approvals and denials</li><li><strong>Regulatory compliance</strong>: Fully transparent and auditable decisions</li><li><strong>Training tool</strong>: New staff learn risk factors through interpretable rules</li></ul><p><strong>Secondary System — Random Forest:</strong> Deploy Random Forest (86.33% accuracy) as a validation and monitoring layer:</p><ul><li><strong>Quality assurance</strong>: Flag cases where single tree and Random Forest disagree significantly</li><li><strong>Portfolio monitoring</strong>: Use ensemble predictions for aggregate risk assessment</li><li><strong>Model improvement</strong>: Identify where single tree rules might need refinement</li><li><strong>Automated screening</strong>: Pre-filter obviously good/bad applications before human review</li></ul><p>They key is tracking success metrics after deployment:</p><ul><li><strong>Accuracy</strong>: Track both models against actual defaults</li><li><strong>Operational efficiency</strong>: Measure time-to-decision improvements</li><li><strong>Staff adoption</strong>: Monitor loan officer confidence and compliance</li><li><strong>Business impact</strong>: Calculate prevented losses and improved risk segmentation</li></ul><p>Decision trees taught us that sometimes the best solution isn’t the most sophisticated one — it’s the one your organization can actually implement, understand, and defend. Whether you choose the interpretable single tree or the more accurate Random Forest depends less on the models themselves and more on your operational needs, regulatory environment, and technical capabilities.</p><p>Both approaches dramatically outperform subjective, manual underwriting. The real win isn’t choosing between 84% and 86% accuracy — it’s moving from inconsistent human judgment to systematic, data-driven lending decisions that protect both your institution and your borrowers.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=51be06bfa825" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Decision Trees Part 2: Finding Optimal Complexity]]></title>
            <link>https://medium.com/@ian.moore_83986/decision-trees-part-2-finding-optimal-complexity-79f6cfe8bde3?source=rss-394411befef7------2</link>
            <guid isPermaLink="false">https://medium.com/p/79f6cfe8bde3</guid>
            <category><![CDATA[gridsearchcv]]></category>
            <category><![CDATA[decision-tree]]></category>
            <category><![CDATA[hyperparameter-tuning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Ian Moore]]></dc:creator>
            <pubDate>Fri, 05 Dec 2025 18:23:24 GMT</pubDate>
            <atom:updated>2025-12-05T18:23:24.523Z</atom:updated>
            <content:encoded><![CDATA[<p>I want to introduce another concept that isn’t discussed often in data analysis and data science — sometimes simple is better. Sometimes less is more. But there is always a balance between simplicity, interpretability and complexity. Now, let me show what I mean by this…</p><p>In Part 1, we built a decision tree that achieved 85% accuracy for auto loan default prediction. Our loan officers finally had consistent, transparent rules instead of subjective gut decisions. But one question remained: <strong>Could we do better?</strong></p><p>The challenge wasn’t just improving accuracy — it was finding the sweet spot between performance and simplicity. A model that’s 95% accurate but requires 200 decision rules isn’t useful for loan officers. We needed systematic optimization that maintained interpretability.</p><h3>The Complexity Dilemma</h3><p>As I indicated earlier — in machine learning, there’s a persistent myth that more complexity equals better performance. Data scientists often chase higher accuracy scores by adding features, increasing model depth, or using ensemble methods. But in lending, this pursuit of complexity can backfire spectacularly.</p><p><strong>Why Complex Models Fail in Lending:</strong></p><p><strong>Regulatory Scrutiny</strong>: Fair lending regulations require that credit decisions be explainable. When an applicant asks “Why was I denied?”, you need clear answers. A decision tree with 15 levels and 200 rules isn’t defensible in front of regulators or even in court.</p><p><strong>Operator Adoption</strong>: Our loan officers need to understand and trust the model. If they can’t follow the logic, they’ll revert to subjective decisions — exactly the problem we were trying to solve. A simple rule like “If LTV &gt; 100% AND late_payments ≥ 3” is actionable — either deny or add an additional risk premium to the interest rate. A complex algorithm outputting “Risk Score: 0.73” without explanation is just not that helpful.</p><p><strong>Overfitting Risk</strong>: Complex models often learn noise rather than signal, especially with smaller datasets. In lending, we’d rather have a robust 82% accuracy model that works consistently than a 95% accuracy model that fails on new data.</p><p><strong>The Sweet Spot</strong>: The goal isn’t maximum accuracy — it’s optimal business value. A model that’s 85% accurate and completely interpretable often outperforms a 90% accurate black box when you factor in regulatory compliance, operational efficiency, and risk management.</p><p>Our challenge: systematically test different complexity levels to find where performance plateaus and simplicity becomes the winning strategy.</p><h3>Individual Parameter Testing</h3><p>Before running something more comprehensive for optimization, lets test each hyperparameter individually to understand how complexity affects our model. in Post #1, we started with the most intuitive parameter: <strong>max_depth</strong>. Max depth of a model is essentially asking “How many questions should our decision tree ask before making a final decision?” Really, how deep should our model be?</p><p>Conventional wisdom suggests that deeper trees capture more complex patterns and should perform better. This might be true with really big datasets, but for us mortals in the land of data, we’re working at companies with 100 or 1,000 employees — not 100,000! So, let’s test the complexity assumption:</p><pre># Test different max_depth values<br>depths = range(1, 9)<br>test_scores = []<br><br>for depth in depths:<br>    clf = DecisionTreeClassifier(max_depth=depth, random_state=42)<br>    clf.fit(X_train, y_train)<br>    test_scores.append(clf.score(X_test, y_test))<br>    print(f&quot;Max Depth: {depth} | Accuracy: {clf.score(X_test, y_test)}&quot;)</pre><p>The results:</p><pre>Max Depth: 1 | Accuracy: 0.85<br>Max Depth: 2 | Accuracy: 0.85<br>Max Depth: 3 | Accuracy: 0.87<br>Max Depth: 4 | Accuracy: 0.85<br>Max Depth: 5 | Accuracy: 0.86<br>Max Depth: 6 | Accuracy: 0.85<br>Max Depth: 7 | Accuracy: 0.85<br>Max Depth: 8 | Accuracy: 0.84</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/863/1*WCHqBPO2BANNt6H5gbyQAQ.png" /></figure><p>Wait a minute! After all that, the best models are only 3 layers deep? Now, if there were more features in this dataset, maybe you’d go a little deeper and get better results. But again, that goes back to the business context — if you have 13 features, but you’re model is optimal at a max_depth = 8 — how could any reasonable loan officer follow that logic?</p><blockquote><strong>The point:</strong> <strong>This completely challenged our assumptions about complexity.</strong></blockquote><p>The simplest trees (depth = 3) achieved 87% accuracy — actually <em>better</em> than our original 85.3% model from Part 1. More surprisingly, deeper trees performed <em>worse</em>, with depth 7–8 dropping to just 84% accuracy.</p><p><strong>What This Tells Us About Auto Lending:</strong></p><ul><li><strong>Clear patterns exist</strong>: Our borrower data has strong, simple signals that don’t require deep analysis</li><li><strong>Overfitting happens quickly</strong>: Beyond depth 3, the model learns noise rather than meaningful patterns</li><li><strong>Business intuition validated</strong>: Experienced lenders know that a few key factors (payment history, equity, credit score) predict most defaults</li></ul><p>This suggests that auto loan default prediction might be simpler than we initially thought — exactly the kind of insight that only systematic testing can reveal.</p><h3>Systematic Optimization with GridSearchCV</h3><p>Sometimes, testing each parameter individually will give us valuable insights, but real-world optimization requires testing parameters <em>together</em>. A model might perform best with max_depth=2 AND min_samples_split=10, but we’d never discover this combination testing one parameter at a time.</p><p>Enter <strong>GridSearchCV</strong> — scikit-learn’s systematic approach to hyperparameter optimization. Instead of guessing, (or building an elaborate for loop to test each combination!) we define ranges for each parameter and let the algorithm test every combination using cross-validation.</p><p><strong>Our Optimization Strategy:</strong></p><pre># Define the hyperparameters and their potential values<br>param_grid = {<br>    &#39;max_depth&#39;: range(1,9),<br>    &#39;min_samples_split&#39;: np.arange(2, 30, 1),<br>    &#39;min_samples_leaf&#39;: np.arange(1, 30, 1)<br>}<br><br>clf = DecisionTreeClassifier(random_state=42)<br><br>grid_search = GridSearchCV(<br>    estimator=clf,<br>    param_grid=param_grid,<br>    scoring=&#39;accuracy&#39;,<br>    cv=5,               # 5-fold cross-validation<br>    verbose=1,<br>    n_jobs=-1           # Use all processor cores<br>)<br><br>grid_search.fit(X_train, y_train)</pre><p><strong>Why This Approach Works:</strong></p><ul><li><strong>Comprehensive testing</strong>: 8 × 28 × 29 = 6,496 different model combinations</li><li><strong>Cross-validation</strong>: Each combination tested on 5 different data splits to ensure reliability (You can change this — but again, more doesn’t necessarily equal better)</li><li><strong>No bias</strong>: Systematic evaluation eliminates human assumptions about “best” parameters</li></ul><p><strong>The Power of Cross-Validation:</strong> Instead of relying on a single train/test split, 5-fold CV splits our training data into 5 segments, trains on 4, and validates on 1 — repeating this process 5 times. This gives us a much more robust measure of how well each parameter combination will generalize to new loan applications.</p><pre>print(f&quot;Best parameters found: {grid_search.best_params_}&quot;)<br>print(f&quot;Best CV score: {round(grid_search.best_score_, 4)}&quot;)<br><br># Evaluate on our hold-out test set<br>best_model = grid_search.best_estimator_<br>test_score = round(best_model.score(X_test, y_test), 4)<br>print(f&quot;Test set accuracy with best model: {test_score}&quot;)</pre><pre>Best parameters found: {&#39;max_depth&#39;: 6, &#39;min_samples_leaf&#39;: 2, &#39;min_samples_split&#39;: 7}<br>Best CV score: 0.8871<br>Test set accuracy with best model: 0.84</pre><p>GridSearchCV revealed that max_depth=6 actually outperformed our individual testing winner (depth=3) when combined with min_samples_split=7. This parameter interaction is exactly what we miss if we were to test individually.</p><p><strong>The Lesson</strong>: max_depth=3 was optimal with default regularization, but max_depth=6 became optimal with proper regularization (min_samples_split=7). This demonstrates why a more comprehensive approach to optimization finds better solutions than isolated parameter tuning.</p><h3>Quick Note: What is Regularization?</h3><p>Before we dive into the results, let’s clarify what <strong>regularization</strong> means in decision trees. Think of regularization as setting guardrails to prevent your model from making decisions based on too little data.</p><p>In lending terms, imagine a loan officer who says: ‘I’ve seen 3 borrowers from Idaho with purple cars, and 2 of them defaulted, so all purple car owners from Idaho are high-risk.’ That’s overfitting — making broad rules from tiny samples.</p><p><strong>Our regularization parameters prevent this:</strong></p><ul><li><strong>min_samples_split=7</strong>: ‘Don’t create a new decision rule unless you have at least 7 loans to base it on’</li><li><strong>min_samples_leaf=2</strong>: ‘Don’t make a final lending decision unless it applies to at least 2 borrowers’</li></ul><p>These constraints force the tree to find patterns that are statistically meaningful, not just random noise.</p><p>Without regularization, a decision tree might create highly specific rules like:</p><ul><li>‘Deny all $23,847 loans to 647 FICO borrowers with blue Honda Accords’</li></ul><p>With proper regularization, it creates robust rules like:</p><ul><li>‘Exercise caution with borrowers who have 2+ late payments AND LTV &gt; 95%’</li></ul><p>The second rule is actionable, defensible, and applies to meaningful segments of your portfolio.</p><h3>Key Findings for Business Interpretation</h3><p>Our GridSearchCV revealed the optimal balance: max_depth=6 with min_samples_split=7 regularization, achieving 88.7% CV accuracy. Most importantly, we discovered that systematic optimization finds parameter interactions that individual testing misses.</p><p>The improvement from 85% to 88.7% demonstrates that auto loan defaults require moderate complexity — neither the oversimplified 2-level rules nor overly complex deep trees, but a balanced 6-level decision process with proper guardrails.</p><p>We’ve found our optimal complexity sweet spot through systematic testing. In Part 3, we’ll explore how this translates to real business value, examine the specific borrower profiles our optimized model catches, and compare our single decision tree to ensemble methods like Random Forest.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=79f6cfe8bde3" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>