<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jon Law on Medium]]></title>
        <description><![CDATA[Stories by Jon Law on Medium]]></description>
        <link>https://medium.com/@jonwlaw?source=rss-3364945644f7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*_LOHEgJoch2WSvFNolh-9g.jpeg</url>
            <title>Stories by Jon Law on Medium</title>
            <link>https://medium.com/@jonwlaw?source=rss-3364945644f7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 06:52:23 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jonwlaw/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[An Intuitive Introduction to the Hamilton–Jacobi–Bellman (HJB) Equation in Economics]]></title>
            <link>https://jonwlaw.medium.com/an-intuitive-introduction-to-the-hamilton-jacobi-bellman-hjb-equation-in-economics-8083e1551db4?source=rss-3364945644f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/8083e1551db4</guid>
            <category><![CDATA[bellman]]></category>
            <category><![CDATA[jacobi]]></category>
            <category><![CDATA[hjbarraza]]></category>
            <category><![CDATA[hamilton]]></category>
            <category><![CDATA[economics]]></category>
            <dc:creator><![CDATA[Jon Law]]></dc:creator>
            <pubDate>Thu, 07 Aug 2025 22:12:35 GMT</pubDate>
            <atom:updated>2025-08-07T22:12:35.320Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*yGYyhzq5cK2iO-uYnRey-Q.png" /><figcaption>Hamilton–Jacobi–Bellman (HJB) Equation</figcaption></figure><p>The HJB equation and setup is an important solution method for many dynamic programming and control theory problems. It’s less complicated than the name (and perhaps your professor) makes it out to be, and in this article I’ll focus on conveying <strong>an understanding of the equation’s purpose, setup, and real-world use cases. </strong>If you’re looking for proofs, consider that in “further reading!”</p><p>Fundamentally, the HJB equation is a setup that asks “what’s the best I can do at any point?” By solving the question this equation poses for inputs of our choice, we get that answer.</p><p><strong>Think about it like a video game:</strong> you have a starting level, different points where you can save your progress, and an ultimate destination.<strong> The HJB equations tells you the maximum score you can achieve from any saved position conditional on playing optimally from that point onward.</strong></p><p>Like so, it embodies Bellman’s Principle of Optimality (<a href="https://jonwlaw.medium.com/what-is-the-bellman-equation-in-economics-bellman-methods-rl-and-dynamic-programming-5b812054949f">you can read my article on it here</a>): <em>An optimal policy has the property that whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the first decision. </em>This just means that regardless of what you’ve done <em>up to this point, </em>if you’re on the best possible path, then the <em>rest</em> of the journey must also be on the best possible path.</p><p>Optimization is an essential mathematical tool and the notion of starting from a “saved” position is critical to its economic applications—think of consumers with a certain capital stock (their saved wealth), firms with an advertising budget, or countries with a certain economy. That breadth of application gives the HJB its practical use in solving optimization problems.</p><p>That all said: <strong>what’s the actual equation?</strong></p><p>While standard notation differs slightly depending upon the source, here’s a common standard form:</p><p>Variables:</p><ul><li>State<strong><em> X</em></strong></li><li>Time<strong><em> T</em></strong></li><li>Action <strong><em>U</em></strong></li><li>Value<strong><em> V</em></strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/956/1*yGYyhzq5cK2iO-uYnRey-Q.png" /><figcaption>Hamilton–Jacobi–Bellman (HJB) Equation</figcaption></figure><p><em>Note: this is a maximization problem because we’re trying to get the most “value” out of our objective (reward) function. You may see a minimization problem as well with the same setup—that just means we’re finding the smallest possible cost moving forward, and any maximization problem can be converted into a minimization problem and vice versa.</em></p><p><em>Note 2: the partial derivative of a variable with respect to time is commonly denoted as variable-dot, so you may notationally see V-dot.</em></p><p>This equation has two big parts: the <strong>rate of change of value<em> (∂V/∂t)</em></strong><em> </em>and the <strong>optimal response <em>(max u)</em></strong>, all set equal to zero.</p><p>Inside the optimal response function, we have two more parts: the <strong>immediate value function</strong> <strong><em>f(x,u,t)</em></strong> and the <strong>discounted future value function <em>(∂V/∂x*g(x,u,t)).</em></strong></p><p>Intuitively, these first two big parts describe a fundamental trade off: how value changes as time passes, and the action you can take to get the most value. Value usually decreases as time passes because we value things more in the present. Since this value is usually negative, we can subtract it from both sides to get this equation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/888/1*xgfakInIjSuT1dLzwxQizw.png" /><figcaption>Hamilton–Jacobi–Bellman (HJB) Equation</figcaption></figure><p>This brings us back to the golden rule of economics: marginal cost equals marginal benefit. The optimal path balances the <em>benefit</em> (maximizing u) against the <em>cost</em> (time passing), and this is the core structure of the HJB.</p><p>The two parts inside the max-u function describe the two components of the “benefit” side to the equation, or finding the optimal route moving forward, which consists of individual optimal actions to play in certain states. <strong><em>f(x,u,t)</em></strong> is the immediate reward—it’s what we get from taking action <em>u</em> in state <em>x</em> at time<em> t. </em><strong>(∂V/∂x)*g(x,u,t) </strong>is everything else—it multiplies the change in value from future states by how a certain action could change the state, or basically finds what the future payoff of this choice is, and then how that choice changes the situation. We’re not simulating every possible state-action payoff, we’re rather encoding all future possibilites into a single number: the marginal value of state x, or ∂V/∂x.</p><p>I want to focus on this idea of the HJB compressing future decisions into a single marginal value. Here’s a (hopefully) intuitive example:</p><p>Going back to our video game analogy, imagine a game with several different locations (states). At each location, you can choose to <em>fight</em> or <em>run</em> (actions). Each choice gives points immediately and moves you to a new location. The goal of the game is to maximize your points, and let’s say we can move twice, so we exist in three total locations/states.</p><p>You could brute force the game by trying every possible sequence: fight-run-fight, fight-fight-run, run-run-fight, and so on. Whichever sequence yields the most points is the maximization solution. This works, but takes a while, especially when dealing with far more complex problems.</p><p>Here’s the HJB approach: at each location, we ask “what’s the maximum number of points I could get from here to the end if I play <em>perfectly</em>?” At location A, that could be 48 points, at location B, 24 points, and at location C, 33 points. Now, when we’re at Location A and choosing between <em>fight</em> or <em>run, </em>we don’t need to run every payoff, because we already know how much points we’ll optimally get if we get to Location B or C. Thus, our calculations is simply the number of points we get now plus the expected payoff from the state we get to by playing a certain action. If there are two other states (B and C), then we’ll get two “point” values, and whichever is higher is the maximization solution. Like so, again, ∂V/∂x contains the solution for “all future decisions”, so you only need to consider the immediate payoff and this summed future value.</p><p>Let’s go one layer deeper and produce the derivative to understand this point in a two-period example problem.</p><p>In this sample problem, let’s say we start with a certain wealth in period one (W₁) and select a certain amount to consume (spend on things) called c₁. (To materialize this, say you get your paycheck and are deciding how much to spend when going out this weekend versus how much to save). You have a linear utility function u(c) = c such that your utility is equivalent to what you consume (you only enjoy what you spend, not what you save). Wealth grows in period 2, such that W₂ = 2(W₁ — c₁). W₁ — c₁ is your savings, so this just says that your savings double in value from period 1 to period 2.</p><p>Your goal is to maximize your utility (e.g., consumption) over the two periods.</p><p>Logically, we should consume <em>all</em> our wealth in period 2, since this is the last period of the game and we don’t get any utility by saving. So, in period 2:</p><p><strong>c₂ = W₂</strong> <em>(wealth equals consumption in period 2)</em></p><p><strong>V₂(W₂) = u(c₂) = u(W₂) = W₂</strong> <em>(our value in period 2 equals our utility, which equals our wealth)</em></p><p>∂V₂/∂W₂ = 1 <em>(taking the partial derivative w.r.t to W₂, we get a value of </em><strong><em>1).</em></strong></p><p>This “solves” period 2, since we logically know that we should consume all of our wealth, and the partial derivative confirms this answer (consume “1” or 100% of wealth).</p><p>To solve period 1, we know that our total utility equals the combined utility from both periods:</p><ul><li>u(c₁) + V₂(W₂) = c₁ + V₂(2(W₁ — c₁)) = c₁ + 2(W₁ — c₁)</li><li>= c₁ + 2W₁ — 2c₁ = 2W₁ — c₁</li></ul><p>And when taking the derivative with respect to c₁ and setting equal to zero, we get -1 = 0. This is impossible, so we don’t have an interior solution.</p><p>In checking the endpoints, we get:</p><ul><li>If c₁ = 0: Total utility = 2W₁ — 0 = 2W₁</li><li>If c₁ = W₁: Total utility = 2W₁ — W₁ = W₁</li></ul><p>Since 2W₁ &gt; W₁, the optimal choice is c₁ = 0*, or to consume nothing in period 1 and save it all for consumption in period 2 (since it will have doubled in value; should be intuitive).</p><p>So, knowing that optimal consumption c₁* = 0, V₁(W₁) = 2W₁ — 0 = 2W₁<strong>. </strong>Taking the partial derivative: ∂V₁/∂W₁ = 2</p><p>So, the partial derivative captures what we can manually solve for and logically gut-check in this problem: one <em>marginal</em> dollar in period 1 gives us two extra dollars in period 2, which equates to two extra units of utility.</p><p>Said differently (since this rule applies beyond two periods), the derivative says <strong>the optimal strategy turns each dollar into 2 dollars, e.g. two units of total utility</strong>.</p><p>Here’s a little recap:</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*yGYyhzq5cK2iO-uYnRey-Q.png" /></figure><p>Value w.r.t time (marginal cost) equals (given the “=0”) the sum of the immediate reward (action u in state x at time t) and the compressed marginal value taken from all future periods. Cumulatively, it’s the structure we need to solve many optimal control or optimization problems.</p><p>That all said, let’s get to a few useful clarifications and points that may be helpful in your understanding of the equation, and then I’ll cover solution methods for the HJ — generally speaking, how we go about actually solving this setup.</p><ol><li><strong><em>Why do we set it equal to zero?</em></strong></li></ol><p>Setting the equation equal to zero is like a no arbitrage condition — it says that there’s no room to “behave better.” If the left side were positive, we could increase our value just by waiting, while if it were negative, we’d be losing value by waiting, when we could simply act immediately to increase our value. Neither of these are optimal, and so the left side must equal zero.</p><p><strong><em>2. What’s the difference between ∂V/∂t and ∂V/∂x?</em></strong></p><p>∂V/∂t looks at how value changes as <em>time</em> passes, while ∂V/∂x looks at how value changes while <em>states</em> change.</p><p><strong><em>3. When are there corner/boundary solutions vs. interior solutions?</em></strong></p><p>Corner solutions occur when tradeoffs are so extreme (or linear, with a constant spread) that it’s optimal to go all in on one choice. Interior solutions conversely occur when marginal benefits are more balanced.</p><p>In the (very simple) example we looked at, the solution was a corner/boundary solution, not an interior solution. <a href="https://jonwlaw.medium.com/what-are-inner-versus-outer-problems-and-solutions-in-economics-bfd06be0c670">You can read my article on the clarifications here</a>—basically, once we add in constraints, we’ll usually get inside solutions.</p><p><strong><em>4. What if I have multiple state or control variables?</em></strong></p><p>The equation naturally extends over many state or control variables:</p><ul><li>Multiple states: ∂V/∂x₁, ∂V/∂x₂, … ∂V/∂xₙ</li><li>Multiple controls: max over u₁, u₂, … uₙ</li></ul><p><strong><em>5. What should f(x,u,t) and g(x,u,t) be in my problem?</em></strong></p><p>f(x,u,t) constitutes your immediate payoff, or what you get right now from taking an action u. g(x,u,t) is <em>how that action changes your state — </em>the law of motion for the system. E.g., with consumption/saving, that’s utility from consuming (f) and how consumption affects next period’s wealth.</p><p><strong><em>6. What if my problem has an infinite time horizon?</em></strong></p><p>The HJB still works, you just drop the ∂V/∂t term. The equation then becomes solely about immediate rewards measured against state improvements.</p><p><strong><em>7. Can I use this for discrete time problems?</em></strong></p><p>The HJB is for continuous time. For discrete time, you’d use the Bellman equation, which follows similar intuition but with a simpler structure.</p><p>That said, let’s get into solution methods for the HJB.</p><h3>Solution Methods</h3><p>I’ll cover five solution methods for HJB problems. Let’s consider the analogy of a busy, complex city for these solution methods, where we’re trying to find the optimal route through.</p><p>The first method is<strong> guess and check </strong>— say you’ve visited this city before, and so you already have a decent idea of what the best route looks like. Instead of trying every possible route, you can simply guess the route and then check to see if it’s actually the best.</p><p>Say you’re solving a consumption problem where the value function is V(W) = AW^a. We can make our guess by choosing some constant for A and a. Then we calculate the derivatives, substitute into the HJB, and solve the maximization (take derivative w.r.t c, set equal to 0). This gives us consumption as a function of W, and substituting back, we can find A and a. If this solution makes sense and satisfies the boundary conditions, we might have found our answer.</p><p>This works well for standard utility function, especially with linear dynamics, and anything similar to well-understand and straightforward problems as you may have solved in textbooks. When you’re dealing with weird functional forms or constraints, and/or multiple state variables, this probably won’t work out.</p><p>The second method covers <strong>analytical solutions. </strong>Going back to our city example, this is everything just working out well—say, a highway straight through the city.</p><p>Say we’re trying to minimize costs that are quadratic for your state and control (like <em>min ∫[x²+ u²]dt s.t. dx/dt = ax + bu</em>). Since quadratic functions have linear derivatives, when we optimize this, it all becomes linear algebra and the solution is always u = -Kx (control as a proportion of state), where K can be found by solving a simple equation. This also works well with log utility/specific production function , exponential utility/normal distributions, and infinite horizon, constant paramters problems.</p><p>Then, we have <strong>numerical methods</strong>—basically, brute forcing it with a program or computer of some sort. As opposed to analytical, where we calculate an exact solution, numerical methods iterate through all combinations to estimate the best solution. That’s like using a GPS system for our city drive that checks millions of routes and determines the best one. This ties into dynamic programming, which I have several articles on — if you’re unfamiliar, I’d suggest giving those a read.</p><p>A common numeric method is <strong>value function iteration</strong>. This is like starting with a very poor map of the city, then updating it by testing different routes until we arrive at the best one.</p><p>We could start with a guess, say V₀(x) = 0, and then improve it, whereas for each state x:</p><p>V₁(x) = max u{f(x,u) + β∫V₀(x&#39;)g(x,u,x&#39;)dx&#39;}</p><p>We repeat this with V₂, V₃, V₄, … etc. until they converge to a final policy V* which maximizes value at each state x. Here’s a more concrete example with consumption:</p><pre>Period 1 guess: V₁(W) = √W<br>Period 2: V₂(W) = max c{√c + β√(W-c)}<br>Period 3: V₃(W) = max c{√c + β√(W-c)}<br>continue until Vₙ ≈ Vₙ₊₁</pre><p>Another numerical method uses a grid, where instead of trying to test and solve for every possible state, we create a grid and then solve just at those grid points. In the consumption, instead of all possible continuous wealth state values, we could establish a grid at W = {0, 1, 2, 3, …, 100}, then approximate derivatives:</p><p>∂V/∂W ≈ [V(W+1) - V(W-1)]/2 ∂V/∂t ≈ [V(W,t+Δt) - V(W,t)]/Δ</p><p>which converts the HJB into a system of equations at each grid point. The accuracy isn’t perfect here and depends on the number of grid points—smaller state intervals in the grid will produce more exact results.</p><p>I walk through full python examples for things like stochastic simulations (say, the Monte Carlo method) and dynamic programming in other articles—check them out on my profile!</p><p>Our fourth solution method is <strong>perturbation methods</strong>, which involves applying a shock to a model. We can use this when our problem is close to something that we can solve analytically (exactly).</p><p>In our city analogy, say we know how to drive through the city optimally when it’s empty, but not during rush hour. Following this shock approach, we’d start with the empty city solution and then adjust for rush-hour traffic.</p><p>Here’s an example where we start with a deterministic problem and then add uncertainty.</p><pre>dx/dt = f(x,u)  →  Solve analytically →  Get x*(t), u*(t)</pre><pre>dx/dt = f(x,u) + σε(t)  (σ is a small shock)</pre><p>For our perturbation approach, we would assume the solution is close to the original deterministic one, and then substitute into the HJB and collect terms by powers of σ, then solve order by order whereas σ⁰ terms return the original problem, σ¹ terms give the first-order correction, and σ² terms give the second-order correction.</p><p>Practically, this could take the form of something like business cycles, where we start with a analytically solved steady-state growth model and then add small shocks to see how the economy fluctuates around its steady state.</p><p><strong><em>When to Use What HJB Solution Method</em></strong></p><p>Practically speaking, you’ll probably use a computer or code to solve these equations. Beyond that, however, if your problem is similar to a textbook case, you can try to guess and verify. If not, consider whether your equation has standard functional forms. If it does, you can look for analytical solutions in literature, if not, then perturbation methods could work, and finally, if you need insights, then quick numerical approximations work, while if you need exact answers, then numerical methods with convergence checks will work.</p><p>That wraps an introduction to the HJB structure, equation, common clarifications, and solution methods. Hope you found it useful &amp; thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8083e1551db4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What are “Inner” versus “Outer” Problems and Solutions In Economics?]]></title>
            <link>https://jonwlaw.medium.com/what-are-inner-versus-outer-problems-and-solutions-in-economics-bfd06be0c670?source=rss-3364945644f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/bfd06be0c670</guid>
            <category><![CDATA[outer-solutions]]></category>
            <category><![CDATA[optimal-control]]></category>
            <category><![CDATA[economics-homework-helper]]></category>
            <category><![CDATA[economics]]></category>
            <category><![CDATA[mathematics]]></category>
            <dc:creator><![CDATA[Jon Law]]></dc:creator>
            <pubDate>Tue, 29 Jul 2025 10:25:25 GMT</pubDate>
            <atom:updated>2025-08-07T22:13:08.212Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RL5bLqWgJC6VpCfNJeY1vQ.jpeg" /><figcaption>What are “Inner” versus “Outer” Problems and Solutions In Economics?</figcaption></figure><p>When learning about and solving optimization problems, you will hear the terms <strong>inner problems</strong>, <strong>outer problems</strong>, <strong>interior solutions</strong>, <strong>boundary solutions</strong>, and <strong>corner solutions</strong>.</p><p>These terms refer to certain components of the <strong>solution method</strong> and <strong>economic interpretations of optimal paths. </strong>Intuitively, “inside” solutions represent optimality trade-offs whereas marginal benefit equals marginal costs, while “outside” solutions present optimal extremes<strong><em> </em></strong>due to constraints or marginal benefit from, say, one good strictly dominating the marginal benefit of another good <em>(for these terms in relation to optimal control problems specifically, scroll down).</em></p><p><strong>Simply put:</strong> if optimal solutions hit a constraint or boundary, it’s an “outer” solution to an “outer” problem, and vice versa. Think about a corner solution as an “outside” solution because it hits the “outside” wall of a diagram. Anything not hitting that “outside” wall is an “inside” solution. See here:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Eu3Ox4ZdxCVNWK9nH8TBrA.png" /><figcaption>Inner Versus Outer Solutions in Economics</figcaption></figure><p>That said, let’s first focus on inside problems.</p><p><strong>Inside problem </strong>solutions<strong> </strong>yields <strong>interior (inner) solutions</strong>. These problems involve optimality choices where decision variables take a range of values strictly within <strong>feasible bounds</strong>, as opposed to at boundaries or constraints.</p><p>This includes a firm choosing a production quantity such that marginal cost equals marginal revenue and an optimal quantity being positive but below capacity limitations. It also includes a consumer allocating income across goods, with income constraints and limitations on the volume of each good that can be purchased and an investment decision where the optimal capital stock is neither zero nor a given upper bound.</p><p>These inside solutions are characterized in unconstrained prolbems by FOCs where derivatives equal zero — an interior optimum.</p><p>Next, outside problems, yielding boundary/corner solutions, happen when optimal choices hit constraints or boundaries. This could mean a firm optimally choosing to product zero units of a good, a consumer who spends all their income on one good, and an investment where an optimal choice is either no investment or the maximum possible investment.</p><p><strong>In optimal control problems, </strong>this inside/outside distinction refers to optimal control variables that satisfy standard Hamiltonian conditions with interior solutions (inside) and cases where control or state variables hit their bounds (outside), which require different solution methods to solve.</p><p>That’s it! Hope you found that helpful. You may also be interested in my articles about <a href="https://jonwlaw.medium.com/monte-carlo-simulations-an-intuitive-introduction-python-example-dd559070275b?source=user_profile_page---------4-------------3364945644f7----------------------">Monte Carlo Simulations</a>, <a href="https://jonwlaw.medium.com/what-is-the-bellman-equation-in-economics-bellman-methods-rl-and-dynamic-programming-5b812054949f?source=user_profile_page---------3-------------3364945644f7----------------------">the Bellman Equation</a>, and <a href="https://jonwlaw.medium.com/what-is-a-pontryagin-system-pontryagin-maximum-principle-control-theory-dynamic-optimization-e22b9d39bd41?source=user_profile_page---------2-------------3364945644f7----------------------">Pontryagin Systems in optimal control theory</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bfd06be0c670" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[An Introduction to Hamiltonian and Lagrangian Methods for Economists]]></title>
            <link>https://jonwlaw.medium.com/an-introduction-to-hamiltonian-and-lagrangian-methods-for-economists-eb4a2d89f0a6?source=rss-3364945644f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/eb4a2d89f0a6</guid>
            <category><![CDATA[economics]]></category>
            <category><![CDATA[hamiltonian]]></category>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[lagrangian]]></category>
            <dc:creator><![CDATA[Jon Law]]></dc:creator>
            <pubDate>Sun, 08 Jun 2025 11:32:40 GMT</pubDate>
            <atom:updated>2025-06-08T11:32:40.089Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/650/1*vdLuKPHmDLTGTNmhbfLzTQ.png" /><figcaption>A Hamiltonian &amp; Lagrangian General Form</figcaption></figure><p>You may remember high school physics class and being introduced to <strong>Newtonian</strong><em> </em>mechanics (F=ma, anyone?). If you stuck with physics, you would learn about two alternative, but complementary, approaches—<strong>Lagrangian</strong> and <strong>Hamiltonian</strong> classical mechanics.</p><p>So, how do we get from physics to economics? What’s all this about Hamiltonian and Lagrangian mechanics, and do we employ these methods in practical economic problems?</p><p>Those are the questions we’ll be answering in this article. We’ll start with the core connection between physics and economics and build up into applied methods via Lagrangian multipliers and the Hamiltonian.</p><blockquote>For a deeper dive into some of the components mentioned in this article, you can check out <a href="https://jonwlaw.medium.com/what-is-the-bellman-equation-in-economics-bellman-methods-rl-and-dynamic-programming-5b812054949f?source=user_profile_page---------2-------------3364945644f7----------------------">this article to learn about states, actions, policies, rewards, &amp; intertemporal optimization</a>, <a href="https://jonwlaw.medium.com/what-is-a-pontryagin-system-pontryagin-maximum-principle-control-theory-dynamic-optimization-e22b9d39bd41">this one for the Pontryagin Maximum and Control Theory</a>, and <a href="https://jonwlaw.medium.com/what-is-the-envelope-theorem-in-optimization-theory-2707d46b404c">this for the Envelope Theorem</a>.</blockquote><p>So, as with just about everything, our story starts with physics. Newtonian, Hamiltonian, and Lagrangian mechanics are all <strong>different approaches</strong> to describe the <strong>same mechanical systems</strong>—frameworks and approaches used to describe the same inherent truths. Depending upon the problem, one approach versus another may be faster or simpler, but they all <em>could</em> be used to solve the same problem.</p><p><strong>Newton’s</strong> approach was centered on <strong>forces </strong>(say, gravity!), represented by vectors, which are played forward in time in tiny increments to calculate things like trajectories.</p><p>A century later, <strong>Lagrange</strong> converted the problem into that can sometimes be solved analytically (calculating an <em>exact</em> answer) by finding the minimum or maximum of an integral, instead of having to integrate forces step-by-step. Like so, he framed classical mechanics as an optimization problem over continuous curves (hence the integrals), where you need to minimize the trade-off between kinetic and potential energy.</p><p>Then, <strong>Hamilton</strong> recast these dynamics in <strong>phase space</strong>—the set of all possible states—to build a single scalar function, of which its partial derivatives generate the same paths otherwise calculated through Newtonian and Lagrangian methods.<strong> </strong>Central to his approach are<strong> costate variables</strong>, which equal the marginal value of the state (effects from slight changes in the state). The Hamiltonian function’s FOCs give the law of motion of these costate variables.</p><p><strong>So, we have three approaches cementing three concepts</strong>: forward-looking forces (Newton), minimization and maximization problems subject to constraints (Lagrange), and optimization with marginal effects (Hamilton).</p><p>One conceptual layer deeper, each approach says that an <strong>evolving system</strong>—whether an orbiting planet, a firm, or a person—<em>could</em> behave or act in many ways, but that we can calculate the trajectory or behavior the system <em>will </em>actually<em> </em>follow, given all the things and constraints acting upon it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fihGZ_L_9hSxo8qs-HnmRQ.png" /><figcaption>Newtonians, Lagrangians, &amp; Hamiltonian Mechanics</figcaption></figure><p><strong>It is this core intuition that explains the deep parallels between physics and economics </strong>and why these mechanical systems are so useful in economic practice. Economics asks the same questions, albeit with different labels, about how systems evolve or behave over time.</p><p>In economics, we analyze and simulate prices, quantities, and behavior, which feature constraints, different things that <em>can</em> happen, and different things will <em>optimally</em> happen.</p><p><strong>Case in point:</strong> say we have a satellite is drifting around a planet and we’re trying to find the best flight path to conserve energy, which requires a balancing act between kinetic and potential energy. Say we also have a household can save or consume capital throughout its lifespan—it can give up a little consumption today to save more and enjoy the payoff later or vice versa.</p><p>Framing both these problems in the same light, <strong>we’re looking to minimize the lifetime resource loss subject to constraints</strong>—whether energy for the satellite or utility for the household.</p><p>Or, imagine an orbiting planet and say we’re tracking its location and momentum. If we plot the possible locations and momentums, then the planet becomes a point moving along a trajectory (orbit) in a two-dimensional <strong>phase portrait </strong>(graph of states, basically).</p><p>If we swap that language out from astronomy to economics, it barely changes—let’s make the horizontal axis the labour stock of a firm and the vertical axis the marginal value of labour (MPL). The phase portrait, or state space, now describes net present value instead of energy, with the optimal growth path being the arrow threading its way toward a steady state.</p><p>Like so, mathematical approaches used to describe changing systems and control how complex, constrained objects act is just as useful in economics as anywhere else.</p><p>That’s all a lot of theory—let’s proceed to what exactly the Hamiltonian and Lagrangian frameworks are and how they’re used in economics.</p><blockquote><strong>A Note on Newton: </strong>In practice, Hamiltonian and Lagrangians are explicit approaches and structures used for all kinds of economic problems that you will most likely have to solve during studies or research. Calculus and differential equations, obviously, are used everywhere—but it’s not the focus from here on out. For that reason, no more talk of Newtonian mechanics!</blockquote><p><strong>Lagrangian Optimization</strong> is a method for solving optimization problems featuring constraints (budget, time, etc.). It uses what are called <strong>Lagrangian multipliers</strong> to do this, which aid in calculating local maxima and minima. These points represent the solutions, whether the <strong>cost-minimizing bundle </strong>or<strong> </strong>the<strong> profit-maximizing strategy.</strong></p><p>Meanwhile, <strong>“The Hamiltonian”</strong> is a function used to solve <strong>optimal control problems</strong> in dynamic (changing) systems. <strong>Control theory </strong>is about designing inputs in such a way to achieve a desired outcome, while<strong> optimal</strong> <strong>Control theory</strong> is all about finding a “<strong>control</strong>”, or solution, to maximize an <strong>objective function</strong> over time. An objective function is kin to a reward function in programming—it’s like the goal of the game, and optimal control theory is about finding the best strategy for the game (<a href="https://jonwlaw.medium.com/what-is-a-pontryagin-system-pontryagin-maximum-principle-control-theory-dynamic-optimization-e22b9d39bd41">I go into more detail about that in this article</a>). Like so, when we’re dealing with a system, world, firm, or consumer that changes over time, the Hamiltonian is a tool to maximize some reward, profit, utility, etc.</p><blockquote>A note on <strong>Laws of Motion</strong>: if not familiar already, you’ll soon run into laws of motion. Laws of motion describe how variables change over time in economics, whether a market, asset, competitor, or demand. Think of them like the laws of physics, except we get to define and build our own rules like gravity and thermodynamics, which may then be applied to elements we’re modeling.</blockquote><p>Thus, both these methods are used for constrained optimization. When dealing with dynamic systems, it can be better to use the Hamiltonian, and when dealing with a static situation, it’s best to use Lagrangians. Here are a few common problems and the solution method we’d employ:</p><ul><li>Consumer choosing a bundle of goods today to maximize utility given a budget constraint. <strong><em>Lagrangian</em></strong>.</li><li>Firm picking factor prices to minimize cost today. <strong><em>Lagrangian.</em></strong></li><li>General equilibrium in a market where each firm is looking to maximize profit subject to resources constraints. <strong><em>Lagrangian.</em></strong></li><li>Intertemporal lifecycle utility optimization for households in an economy that can consume or save earnings. <strong><em>Hamiltonian.</em></strong></li><li>Firm picking how quickly to drill through an oil reservoir to maximize discounted lifetime profits. <strong><em>Hamiltonian.</em></strong></li><li>A nation deciding at what rate to print money to balance debt load and inflation. <strong><em>Hamiltonian</em></strong></li></ul><p>So, mostly-correct rule of thumb: for static or one-period problems, Lagrangians beat out Hamiltonians, while if it involves continuous time, the inverse applies (discrete time problems will often use a Bellman setup—<a href="https://jonwlaw.medium.com/what-is-the-bellman-equation-in-economics-bellman-methods-rl-and-dynamic-programming-5b812054949f?postPublishedType=repub">see here</a>).</p><blockquote>One note about solutions before we get into the actual setup and examples: for both methods, we’ll need to verify that solutions are both unique and actually exist. For Lagrangians, solution existence requires a continuous objective function and a non-empty feasible set (as defined by constraints). A theorem called the <strong>Weierstrass theorem</strong> then guarantees that at least one solution exists. Uniqueness requires additional conditions, like the objective function being strictly concave or convex. For Hamiltonians, verifying these conditions more complex — all that complexity is why it’s reasonable to stick with well-known functional forms like Cobb-Douglas and CES functions. With these, you don’t have to worry about no solutions, multiple solutions, or useless conclusions.</blockquote><h4>The Lagrangian</h4><p>The actual “Lagrange Multiplier” is a single value denoted by lambda <em>λ</em>. It’s does not represent a variable—instead, it’s a core part of the Lagrangian structure, both simplifiying calculations and representing the “shadow value”—the marginal cost of the constraint.</p><p>Here’s the general form of the Lagrangian:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N0qd0uSzKOUt7OYX3S9ifw.png" /><figcaption>Lagrangian General Form in Economics</figcaption></figure><p><em>x</em> is an input variable, <em>f(x)</em> is some objective (reward) function <em>f</em>, <em>g(x)</em> is the constraint, lambda is the Lagrangian multiplier, and this all forms the Lagrangian L.</p><p>With this setup, we’re trying to find the maximum or minumum of the function <em>f</em> subject to the constraint <em>g(x)=0. </em>That maximum or minimum is the point at which all partial derivatives are equal to zero, representing a<strong> stationary point.</strong></p><blockquote>A note on g(x)=0. The actual constraint is some function g(x) = c, like g(x)=$10 being the amount of money a consumer can spend. We can rewrite this as g(x) — c=0, which says that the value we spend must be equivalent to our constraint (thus, it cannot exceed, say, the amount of money we have). That’s why our constraint is g(x)=0, not just g(x).</blockquote><p>Since the inputs of the Lagrangian are lambda and x, this means we want to satisfy this condition:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UbTeo-58VCDW4u35VcJMHQ.png" /><figcaption>Lagrangian FOCs (1)</figcaption></figure><p>So, returning to the general form, we write out and solve the FOCs:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3yeHjjsWerB8sdh33Oh2Jw.png" /><figcaption>Lagrangian FOCs (2)</figcaption></figure><p>For the FOC with respect to <em>x</em>, we can’t simplify beyond <em>f(x) + λg(x) = 0 </em>without specifiying a functional form for <em>f(x)</em> and <em>g(x)</em>. For the FOC with respect to lambda, since lambda isn’t present in <em>f(x)</em> or <em>g(x)</em>, the condition simplifies to g(x) — you will always get the constraint for the FOC with respect to lambda. Setting both conditions equal to zero, we get:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/786/1*d31rsimvb_AV-Jqe4uRx7g.png" /><figcaption>Lagrangian FOCs (3)</figcaption></figure><p>If we <em>did</em> have a functional form for f(x) and g(x), here’s how we’d proceed:</p><ol><li>Compute the partial derivatives.</li><li>Solve the systems of equations.</li><li>Check that the x we find satisfies g(x)=0.</li><li>Check second-order conditions.</li><li>Interpret the lambda value, which represents the marginal effect of reducing the constraint by one unit (the shadow price/value).</li></ol><p>Here’s a quick example with functional forms. Let’s say we’re maximizing <em>f(x,y) = xy</em> subject to <em>g(x,y) = x + y — 10 = 0</em>. Here’s the math:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3JbIg2dKQhtupnrFNclJSw.png" /><figcaption>Lagrangian Example - Part 1</figcaption></figure><p>Now we can solve the system of equations with algebra:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dRK-NmWTy6Gcn-qXNrWCQg.png" /><figcaption>Lagrangian Example — Part 2</figcaption></figure><p>Thus, the xy-bundle optimally equals (5,5), and the shadow price is -5 ( ∂f/∂10 = 5), meaning that relaxing the constraint x + y = 10 (increasing the value of 10 to, say, 11) reduces the maximum value of f(x), which is currently 25, by 5.</p><p>This same approach can be used on a host of optimization problems, and can be adapted to even more problems through other building blocks like Hessian matrices, KKT conditions, Arrow &amp; Mangasarian conditions, and Euler equations.</p><p>All that said, next up is the Hamiltonian.</p><h4>The Hamiltonian</h4><p>As mentioned, the Hamiltonian is a function used in dynamic constrained optimization problems.</p><p>We start by writing out our maximization problem, whereas our agent (<a href="https://medium.com/@jonwlaw/what-is-the-bellman-equation-in-economics-bellman-methods-rl-and-dynamic-programming-5b812054949f?postPublishedType=repub">see this article for relevant terms</a>) selects the value of a control variable <em>u(t)</em> over time to affect a state <em>x(t)</em>. For the problem, we maximize an objective function (our reward) subject to constraints:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5mvDbaP9U4Bz2n-0byAU7A.png" /></figure><p>Within the objective function, we have a definite integral that adds up the pay-offs derived from each continuous moment in time. That payoff in <em>this</em> moment, in turn, is defined by a function F, which in reduced form depends upon the current state <em>x(t)</em>, the choice <em>u(t)</em>, and the time period <em>t</em>. The <em>e</em> block is a discount raised to a constant <em>p</em> - this is what discounts all future periods beyond the current period t up to the final period <em>T</em>.</p><p>Finally, the rightmost part of the function, also multiplied by a discount factor, is the <strong>terminal payoff</strong>—that’s a one-time reward or penalty dependent only on the final state at time<em> T</em>. It can be zero if there’s no such reward, and this just depends on the problem setup (if we’re dealing with an infinite time horizon optimization problem, then we also drop the Φ-term—it <em>only</em> applies in finite-horizon problems). Like so, <strong>the core part of the equation is the integral</strong>, which is really just a discounted reward stream given a current state, choice, and time.</p><p>Next, the objective function is subject to a law of motion <em>x(t)</em> - keep in mind that anything with a dot over represents the time-derivative of that variable, or the partial derivative with respect to <em>t</em>. That determines how choices (controls) translate into states, since choices do <em>not</em> deterministically predict states. We have a vector of these control variables representing all the various choices for its value, and a vector of state variables which evolves depending on the current state, chosen controls, and any exogenous shocks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XkoJFgL5WDFvDdtbpsPWRw.png" /></figure><p>With that setup in place, we can construct the Hamiltonian function. We start by introducing (this is why we looked at the Lagrangian first!) a time-varying Lagrangian multiplier λ(t).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xY10PmDnN8sSS1wucKUMlA.png" /></figure><p>This makes the term inside the brackets the current-value Lagrangian with the law of motion <em>g(x,u,t) </em>and the objective function <em>F(x,u,t)</em>. Now, we’ll integrate by parts to pull the constraint outside of the integral, with an incorporation of the costate equation through the maximum principle <em>(I’m skipping the derivation here to focus on the concept of the Hamiltonian—there are great videos are textbooks out there walking through this step by step!):</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BI89Gzlc9PaCCmmgcpmnGQ.png" /></figure><p>Now, we collect everything inside the integral and rename it the Hamiltonian, termed as H:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*4CKM40VJvlu2fR86UQc8_g.png" /></figure><p>Making the objective function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vrxU28RxELU00aIRfUHPJw.png" /></figure><p>With this setup, H already embeds the shadow, or marginal, value of slightly altering the state. <strong>This means that maximizing H at every continuous instant is the same as maximizing the whole intertemporal objective function </strong>and solving the problem.</p><p>Through the Hamiltonian, we can interpret the problem with the FOCs for the state, costate, and control equations, and the boundary conditions:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*aSxV2OLW129CPJNAzsyR3g.png" /></figure><p>The first FOC of <em>u(t)</em> implies that a control is chosen today, given today’s marginal effect, such that slight state change can’t raise or lower the Hamiltonian.</p><p>The next, of <em>x(t)</em>, says that the shadow price must change over time to keep track of how valuable a marginal unit of that state is.</p><p>The third FOC hardwires the original law of motion back in.</p><p>These three differential equations—the FOCs—still don’t give us a <em>unique</em> solution. Instead, since we have two first-order unknowns s(t) and λ(t), we need two anchors to pin down a unique trajectory. That means both <strong>a starting and a terminal (final) condition</strong>. Here they are:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/660/1*hYDIOXGNXyOl1pxqsgbxcw.png" /></figure><p>The initial state condition x(0) = x0 is the starting stock for the variable reprsented by x — that could be a firm’s capital stock, consumer’s wealth, pollution stock, or so on. This starting value has to be exogenouesly fixed since the solution method is fundamentally forward-looking. After the fixed initial state, <strong>the state differential equation can take over to produce a trajectory</strong> going forward.</p><p>The terminal boundary condition (calculated by merging a boundary term for the integration with the discounted terminal reward, varying x(T), and setting the variation of total payoff to zero) states that the <strong>shadow value of one more unit of the state at period T—the marginal state effect—equals the direct marginal utility</strong> that the terminal payoff assigns to that extra unit. Intuitively, we’re setting marginal values equal to one another just like we would anywhere else, such that in the last period keeping or discarding a state unit doesn’t matter.</p><p><strong>So, you can visualize these boundary conditions like the entrance and exit to the maze of possibilities within the phase space.</strong> With them, we can use the FOCs to solve forward and backward until the paths meet. That resulting path, often calculated numerically (in practice, meaning you let a computer guess a bunch and see what sticks!), is the optimal trajectory and the solution to the dynamic constrained optimization problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8ky-0gsck5RWaGPtMOePEA.png" /><figcaption>Hamiltonian Phase Space With Boundary Conditions</figcaption></figure><p>So, there you have it! Here’s a summary of what we’ve covered:</p><ul><li><strong>Physics roots: </strong>Newtonian forces, Lagrangian trade‑offs, and the Hamiltonian phase‑space.</li><li><strong>Economic mapping</strong>: states and constraints as framing optimization problems.</li><li><strong>Lagrangian optimization</strong>: static optimization with an objective function and constraints, solved with a Lagrangian multiplier.</li><li><strong>Lagrange FOCs:</strong> partial derivatives set to zero that deliver the bundle and shadow value optimum.</li><li><strong>The Hamiltonian &amp; optimal control theory</strong>: dynamic optimization, laws of motion, costate functions, and the phase space.</li><li><strong>Hamiltonian FOCs</strong>: state, costate, and control equations, along with the boundary conditions, determining the optimal path over time.</li></ul><p>Hopefully, this helps you with all kinds of constrained optimization problems, which are a powerful tool to understand and predict real-world behavior.</p><p><strong>If you liked this article, you’ll probably like </strong><a href="https://jonwlaw.medium.com/monte-carlo-simulations-an-intuitive-introduction-python-example-dd559070275b"><strong>this article I just wrote about Monte Carlo simulations</strong></a><strong>, </strong>which rely on similar concepts to what we’ve employed here and are a useful applied means of simulating the real world in economics, finance, and beyond.</p><p>Or, if you’re into economics, you’re in the right place, and make sure to check out my other articles <a href="https://jonwlaw.medium.com/list/economics-36ae74971581">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eb4a2d89f0a6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is a Pontryagin System? | Pontryagin Maximum Principle, Control Theory, & Dynamic Optimization]]></title>
            <link>https://jonwlaw.medium.com/what-is-a-pontryagin-system-pontryagin-maximum-principle-control-theory-dynamic-optimization-e22b9d39bd41?source=rss-3364945644f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/e22b9d39bd41</guid>
            <category><![CDATA[lev-pontryagin]]></category>
            <category><![CDATA[pontryagin]]></category>
            <category><![CDATA[optimal-control-theory]]></category>
            <dc:creator><![CDATA[Jon Law]]></dc:creator>
            <pubDate>Fri, 06 Jun 2025 10:38:53 GMT</pubDate>
            <atom:updated>2025-06-06T10:38:53.139Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is a Pontryagin System? | Pontryagin Maximum Principle, Optimal Control Theory, &amp; Dynamic Optimization</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*461sXrNLNcnGmOIYb4bBkQ.png" /><figcaption>Pontryagin — Optimal Control Framework</figcaption></figure><p>A Pontryagin system is a <strong>system of equations in optimal control theory</strong> used for finding optimal controls in dynamical systems. In this article, I’ll break down the concept, background (maximum principle), and use cases—no prior experience needed!</p><p>First, optimal control theory is part of control theory, which uses equations and algorithms to guide the behavior of dynamic systems (meaning systems with <strong>states</strong> that change—<a href="https://jonwlaw.medium.com/what-is-the-bellman-equation-in-economics-bellman-methods-rl-and-dynamic-programming-5b812054949f?source=user_profile_page---------1-------------3364945644f7----------------------">you can learn more here</a>) to achieve some desired outcome. It’s applied in all kinds of fields, from flight control systems to ensure aircraft stability to industrial robotics and autonomous vehicles.</p><p>Optimal control theory, in turn, focuses specifically on <strong>optimizing an objective function in a dynamic system</strong>, a dynamic system being a model that describes how the states of a system change over time. Similarly, an objective function is an expression defining a goal to be mathematically maximized (gain, profit, utility, etc.) or minimized (error, loss, unemployment, time etc). So, simply, if we frame this like a game, optimal control theory is about finding the best way to play the game, given the <strong>rules </strong>and the <strong>goal</strong> (get the most points, survive the longest, don’t lose to your little brother, or so on).</p><p>With that all in mind, we can get to <strong>Pontryagin’s maximum principle</strong>, formulated in 1956 by Russian mathematician Lev Pontryagin and co. It’s an approach or tool of sorts used in optimal control theory, and involves defining a Hamiltonian function (which is a problem setup of sorts if you’re not familiar—comparable for sake of understanding to a Lagrangian) and maximizing it along the optimal trajectory, in turn determined by solving a two-point boundary value problem.</p><p><strong>A Pontryagin system <em>is</em> this principle</strong>—a framework, or system of equations, used to find an optimal control. Like Lagrangians, first-order conditions, derivative tests, and any number of other mathematical processes, it’s a defined approach to solving complex, dynamic problems in an environment where states and state transition paths are known.</p><p>Specifically, the system of equations involves these components, which we’ve already introduced (see the above image):</p><ul><li><strong>Dynamic System</strong></li><li><strong>Hamiltonian</strong></li><li><strong>Costate Equations</strong></li><li><strong>Boundary Conditions</strong></li></ul><p>Like so, the state‑change rules feed directly into the Hamiltonian, with a costate equation being a vector of first order differential equations (Lagrange multipliers) that captures the marginal value of each state variable. As states evolve forward in time, this marginal effect (shadow price) is propagated backward from the terminal (boundary) condition, thus identifying the impact of one more unit of an input (state variable) on the objective function (reward).</p><p>As stated, this approach is used in all kinds of fields—whether helping a rocket reach a target orbit, an economy maximize GDP growth, or a person minimize the time needed to get to work.</p><p>Again, if you need a refresher on basic building blocks like states, actions, and dynamic optimization, <a href="https://jonwlaw.medium.com/monte-carlo-simulations-an-intuitive-introduction-python-example-dd559070275b">check out this recent article</a>.</p><p>Otherwise, if you’re into (or studying!) economics, you’re in the right place, and make sure to check out my other articles <a href="https://jonwlaw.medium.com/list/economics-36ae74971581">here</a>. <strong>Thanks for reading!</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e22b9d39bd41" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is the Bellman Equation in Economics? | Bellman Methods, RL, and Dynamic Programming]]></title>
            <link>https://jonwlaw.medium.com/what-is-the-bellman-equation-in-economics-bellman-methods-rl-and-dynamic-programming-5b812054949f?source=rss-3364945644f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/5b812054949f</guid>
            <category><![CDATA[bellman-equation]]></category>
            <category><![CDATA[economics]]></category>
            <category><![CDATA[bellman-methods]]></category>
            <category><![CDATA[dynammic-programming]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <dc:creator><![CDATA[Jon Law]]></dc:creator>
            <pubDate>Sat, 24 May 2025 04:06:27 GMT</pubDate>
            <atom:updated>2025-06-07T11:09:55.547Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is the Bellman Equation in Economics? | An Introduction to Bellman Methods, Reinforcement Learning, and Dynamic Programming</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6VQE0845J-cbxP_LOcMmjQ.png" /><figcaption>Bellman Value Iteration</figcaption></figure><p>In this article, I introduce the core concepts driving the Bellman equation and its use in dynamic programming and reinforcement learning (don’t worry if you’re not familiar with those terms!). Here’s a quick rundown of what you’ll learn:</p><ul><li><strong>Foundational concepts: </strong>states, actions, rewards, dynamic programming, and reinforcement learning.</li><li><strong>Bellman Equation struture:</strong> immediate rewards + discounted expected future values.</li><li><strong>Uncertainty in Bellman equations: </strong>probability distributions and expected values to model stochastic shocks.</li><li><strong>Policies versus actions: </strong>the difference between choices (actions) and strategies (decision rules/policies).</li><li><strong>Solution methods: </strong>value iteration as a computational approach for using Bellman equations to solve optimization problems.</li><li><strong>Two examples using the Bellman equation</strong>: a two-period machine maintenance example solved by hand, and an infinite-horizon consumption-savings model that we use numerical methods to solve.</li><li><strong>Implementation techniques: </strong>discretizing continous variables, grid-based optimization, and convergence criteria.</li></ul><blockquote>Also: if you want to learn some of the same concepts with a focus on one specifica application of the math, you can check out my <a href="https://jonwlaw.medium.com/monte-carlo-simulations-an-intuitive-introduction-python-example-dd559070275b">article about Monte Carlo simulations here</a>.</blockquote><p>That all said, let’s get into it. This is the general form of a Bellman equation (the standard way of expressing it). You may have seen it with different variables, or slight variations of to address a certain problem or method, but regardless, this is the structure it will nearly always maintain:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/698/1*5BLRrkJgGILLjOZMYFgjLg.png" /><figcaption>General Form of a Bellman Equation</figcaption></figure><p>The Bellman equation provides a structure to solve<em> </em><strong>intertemporal optimization problems</strong>. That’s a fancy way of saying it helps us figure out the <strong>single best decision or set of decisions to make</strong> in a situation where we have decisions to make that could affect the world in different ways, and affect <em>us</em> in different ways.</p><p>The initial Bellman principle was developed by Richard Bellman in the 1950s under the banner of dynamic programming, which is an umbrella-term for problem-solving methods that use <strong>recursive</strong> sub-problems to solve big problems. <em>Recursion</em> maens iterative results, whereas results from one equation are used as inputs to re-run the equation, and so on until a problem is solved.</p><p>Modern reinforcement learning methods then adopted and extend those earlier equations. Reinforcement learning is a subset of machine learning, generally involving approaches to getting <strong>agents (</strong>algorithms) to “discover” optimal decisions within an environment.</p><p>Let’s understand some of the basics of reinforcement learning to inform our understanding of the Bellman equation, and then we’ll return to that the general form equation:</p><h4><strong>Introducing States, Actions, &amp; Rewards</strong></h4><p>Generally, for any reinforcement learning or dynamic programming problem, we’ll construct a series of rules and equations to produce a “world” or “game” of sorts, and then build an algorithm to play that game <em>(if you’re familiar with game theory, you’ll find a lot of this familiar—think backward induction!)</em>.</p><p>If we build an algorithm to play the game in the right way and improve over time, it will figure out the optimal way to play the game and answer our question for us.</p><p>A cornerstone of this process is what are called <strong>states. </strong>Intuitively, you can substitute <strong><em>states</em></strong> for <strong><em>states of world. </em></strong>For example, if the weather can either be sunny or raining in the world we’re creating, then the two <em>states</em> or <em>states of word </em>are sunny and raining. States could also be different earnings values reported by a firm, interest rate changes potentially reported by the Federal Bank, or any number of other <strong>environmental changes. </strong>Transitions between states can occur <strong>deterministically</strong> (if x event happens, y state is <strong><em>transitioned into</em></strong>) or <strong>stochastically </strong>(a state which is transitioned into randomly selected from a given distribution).</p><p>In turn, <strong>actions</strong> are things that an algorithm can do given a set of states. If it’s raining, an action may be to get an umbrella, or when it’s sunny, to purchase sunscreen.</p><p>Combining these, a <strong>state-action pair</strong> is an action an agent can take within a state.</p><p>Finally, <strong>rewards</strong> are the cost or benefit garnered from applying actions to states—usually measured in <strong>utility</strong> for consumers and profit for <strong>firms. </strong>For example, if an algorithm we create incorrectly purchases suncreen when it’s raining (instead of an umbrella), then we could program <em>getting wet</em> as a negative reward, or something to avoid. This negative reward is a signal to the algorithm, telling it to potentially avoid that action in that state when other, better options are available.</p><p>So, reinforcement learning methods establish various states that stochastically occur and evolve over time. Algorithms perform actions in these states and compute a reward given the actions they took and the states in which they occured.</p><p>By repeating this process (“learning”) to maximize rewards, algorithms can find optimal approaches to any number of problems: portfolio optimization, risk analysis, economic simulations, or how to not get wet!</p><p>This is where Richard Bellman comes in. Bellman methods are the backbone of many reinforcement learning methods, <strong><em>as reinforcement learning algorithms are ways to solve or approximate the Bellman equations</em></strong>. Like I said at the start of the article, the Bellman equation provides a <strong>structure </strong>used<strong> </strong>to solve complex problems.</p><p>Specifically, the equation says that the value of a state or state–action pair equals the <strong>immediate reward</strong> plus the <strong>discounted value</strong> of what comes next. In an economics context, you’ve likely heard of discounted profit streams, whereas you want to maximize the value of not just profits currently, but the value of <em>all</em> profits over time given some discount factor (“I value money 5% more today versus tomorrow). As mentioned, it’s a <strong>recursive equation</strong>, meaning it breaks a large problem down into smaller sub-problems, and solves them incrementally using data from each solved sub-problem.</p><h4>Breaking Down the Bellman Equation</h4><p>Let’s return to the Bellman general form:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/698/1*5BLRrkJgGILLjOZMYFgjLg.png" /><figcaption>General Form of a Bellman Equation</figcaption></figure><p>V(s)is a general form notation—like f(x)—describing the right-hand side of the equation is a <strong>value function.<em> </em></strong>Conceptually, maximizing the value function represents the maximum expected lifetime payoff (utility or profit, depending on the context) that a decision maker can achieve given rules and constraints (states, actions, rewards, etc.).</p><p>Now, I just said that the Bellman equation combines immediate reward with the discounted value of what comes next. Let’s first understand the immediate reward portion of the equation:</p><p>s is the <strong>current value of the state</strong> we’re considering in our “world.” ais the <strong>decision variable</strong>, the unknown value that we’re adjusting to achieve the optimal outcome. For example, our current state could be wealth and our decision variable investment, and in that case we’re trying to figure out how much to optimally invest given our wealth. Like so, u(s,a)is the utility we get from choosing action ain state s. Because <em>s</em> refers to the current state and<em> a</em> the current input, this u(s,a)calculates the<em> immediate reward</em>.</p><p>This visualization represents many possible actions for a state s<em> (s</em>ome may produce the same state, others different states). Each state corresponds with a value once the immedate reward is added. In this case (note the bottom-right), the state corresponding to action 5 produces the highest value.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*v0HjDU4sDFHRqPPCgDZsWQ.png" /><figcaption>Actions and States in the Bellman Equation</figcaption></figure><p>But, here’s the catch:<strong> actions do not lead to a predefined next state with a probability of 1. </strong>Instead<strong>, </strong>An action a taken in state s<em>leads </em>to a state s&#39; according to a <strong>state transition probability distribution</strong>, P(s&#39;|s,a)(meaning s&#39;is conditional upon the state-action pair s,a. The action is an input to the function that defines the distribution of next states.</p><p>We’ll get to that in a second—let’s first touch on the discount factor.</p><p><em>β∈(0,1) </em>is the <strong>discount factor</strong> (the “<strong>time preference factor</strong>”). It determines the rate at which value today is compared to value tomorrow—a person with a year to live versus one with fifty years to live will likely value money very different, for example, and that spread is a difference in time preference. This time preference factor must be between 0 and 1, with 1 meaning you care about all future value equally to today, and 0 meaning you only care about current value.</p><p>s&#39; is the state of the next period, which is conditional (note the notation “|”) upon the current state-action pair s,a. For example, if we make a certain smart investment <em>a</em> today given a wealth <em>s</em> of $100,000, we may have $200,000 in the next state s&#39;(which could represent, say, the following year). V(<em>s′) </em>is the value of that next-period state.</p><p>Finally, the expectation E[⋅] <strong>integrates over uncertainty</strong>. Let’s go a bit deeper there. Here’s what it’s equivalent to:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/634/1*6Xf-lydRghOPQVnJ0CArPQ.png" /><figcaption>Uncertainty in the Bellman Equation</figcaption></figure><p>With P(s&#39;|s,a) being the probability of landing in state <em>s′ </em>after choosing action ain state s<em>. </em>This reflects the fundamental notion that the world is uncertain. For example (back to the weather!), there’s always some chance that our umbrella breaks. Let’s say it happens 10% of the time. This means that even when we choose the action of <strong><em>bringing an umbrella </em></strong>(action)<strong><em> </em></strong>on our walk when there are <strong><em>dark</em></strong> <strong><em>clouds</em></strong> (state) which produce rain 50% of the time, we’re not guaranteed to stay <strong><em>dry</em></strong> (our reward). Instead, bringing the umbrella means we stay dry 95% of the time, but get soaked the other 5%. If we value being dry versus wet at (100, 0), then our expected value is 95. Formally:</p><p><strong>P(dry) = P(no rain) + P(rain)*P(umbrella works) =0.5+0.5×0.9=0.95</strong></p><p>So, to reflect uncertainty, <em>E</em> sums the probability of landing in each state multipled by the value derived from the state. By integrating, we find the <strong>average</strong> or<strong> expected value</strong> from choosing that state-action pair s,a. We call that the <strong>continuation value</strong>.</p><p>Here’s a visualization, representing that when we choose an action ain state s, we are probably going to end up in a certain state s&#39;, but due to uncertainty, we could also end up in states s&#39;&#39;through s&#39;&#39;&#39;&#39;(the green bars representing the probability, or liklihood of occuring, with probabilities necessarily summing to 1). Each state brings about a different value:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_0uPjfTAW6uKModNEIAokg.png" /><figcaption>Visualizing State Uncertainty in the Bellman Equation</figcaption></figure><p>When we combine the two parts of the equation we’ve covered thus far, we get the current reward and the average value (reward) we’ll get in the next period given the actions we’re taking in this period.</p><p>The last component of the equation we haven’t touched on is the <strong><em>max </em></strong>notation. This means that given our current state<em> </em>s, we’re considering all feasible actions ain the set A(s)<em> </em>through<em> </em>the two components we just described. Like so, each action will produce a different immediate payoff + discounted continuation value. Since we’re trying to maximize value (“max”), we select the highest calculated value. Conceptually, the maximum utility or profit is the best action we can take given our current state and the randomness of the world.</p><p>Practically, and in many economic applications of the Bellman equation, we’ll need to consider many periods, or even infinite. We don’t have to change the equation to do this — instead, we just iterate through the one-period equation as many times as we need to. Here’s the equation for a multi-period Bellman setup:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_doyE-kMvbit1VsdqKkEkg.png" /><figcaption>Multi-Period Finite Horizon Bellman Equation</figcaption></figure><p>Whereas, the value of the current period <em>Vt</em> is calculated given a maximization of the immediate reward from the state-action pair in this period and the continuation value involving Vt<em>+1</em>. This repeats T-1 number of times (T being the end state) to solve the multi-period (intertemporal) optimization problem.</p><h4>Introducing Policies</h4><p>Now, I mentioned dynamic programming. More specifically, in economics, dynamic programming involves coded scripts (usually) employing Bellman equations to recursively iterate through various actions, states, and values.</p><p>A core concept in dynamic programming, and a practical component attached to Bellman equations that we haven’t touched on yet, are <strong>policies</strong>. Policies are the strategy component of optimization—functions that map states onto actions—and inform a program’s decision on which action to take given a certain state. Value functions let policies be improved based on the value they bring about.</p><blockquote>If you’ve played the board game Clue, each guess you make is likely a value function output. You may not have guessed the right answer, but your overarching “policy” or strategy moving forward is better because of your guess (your value function output). With enough guesses, you will settle on the correct answer.</blockquote><p>Polices differ from actions, such that if policies are the <strong>strategy</strong> to the game, then actions are the <strong>rules</strong>. Say we want an agent to move through a maze: possible <em>actions</em> may be to move left, right, up, or down, while the <em>policy</em> for that state may be to move left. A policy can also apply to all states, like we’ve discussed previously (say, always bring an umbrella if there are rainclouds).</p><p>Let’s map what we’ve learned about Bellman equations onto two examples of increasing complexity:</p><h4><strong><em>A One-Machine, Two-Period Maintenance Model</em></strong></h4><p>Say we want to optimally decide whether to let a machine in our factory produce goods or to fix it. There are two states, one in which the machine is in a good condition <strong><em>G</em></strong>, and the other for a bad condition,<em> </em><strong><em>B</em></strong>, which signals that the machine needs fixing. There are two possible actions—<strong>keep</strong> (to run the machine this period) and <strong>fix</strong> (pay to fix it in this period). For our instant reward function u(s,a), we apply these rules for every combination of states and actions:</p><ul><li><strong>s = G, a = keep</strong> → profit = 10</li><li><strong>s = B, a = keep </strong>→ profit = 2</li><li><strong>s = G, a = fix → </strong>profit = -8</li><li><strong>s = B, a = fix →</strong> profit = -8</li></ul><p>If we <em>keep</em> in the current period given a state of Good, there is a 60% probability the machine says in the Good state, and a 40% probability it transitions to the bad state (these are called “<strong>transition paths</strong>”). If it’s already in a bad state, it stays bad with a probability of 1, and if we fix instead of keep, the machine moves to G with a probability of 1.</p><p>Our discount factor (beta) is 0.95, meaning we value profit in the next period 95% much as we do profit today.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*M2qH6VZJ5ew_LJZ6BwdVxw.png" /><figcaption>States, Actions, and Optimal Policy with the Bellman Equation</figcaption></figure><p>So, we have two states, two actions, and the optimal policy (strategy) question of what actions our agent should play in each state. Let’s calculate the Bellman equations for the states of good and bad:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/880/1*siiQptpgBkTcwQ7FtoqSRA.png" /><figcaption>Bellman Equation — A One-Machine, Two-Period Maintenance Model</figcaption></figure><p>For state G, we can either “play” the action keep or the action fix. Let’s calculate outcomes for both:</p><ul><li><strong>keep: 10 + 0.95[0.6V(G)+0.4V(B)]</strong></li><li><strong>fix: -8 + 0.95V(G)</strong></li></ul><p>Here’s the same for state B:</p><ul><li><strong>keep: ​2 + 0.95V(B)</strong></li><li><strong>fix: −8 + 0.95V(G)​</strong></li></ul><p>Now, to solve, we’ll use guess-and-check (something the computer will do when creating scripts to solve problems with Bellman equations) where we guess a policy, solve, and compare. Let’s start by guessing a very reasonable policy (ruleset) of <strong>keep</strong> in state G and <strong>fix</strong> in state B (logically, fix a machine when it’s broken, and let it run when it’s not):</p><ul><li><strong>V(G) = 10 + 0.95[0.6V(G) + 0.4V(B)]</strong></li><li><strong>V(B) = −8 + 0.95V(G).</strong></li><li><strong>This produces: 0.43V(G) − 0.38V(B) = 10.</strong></li><li><strong>Substituting in V(B) = −8 + 0.95V(G): 0.43V(G) + 3.04−0.361V(G) = 10⟹ 0.069V(G) = 6.96 ⟹ V(G) ≈100.9.</strong></li><li><strong>Plugging V(G) into V(B): -8 + 0.95(100.9) = 87.8</strong></li></ul><p>To check whether this is an optimal policy, we want to make sure that the chosen actions yield the larger payoff in each state. We just calculated keep in state G and fix in state B, so we’ll compare that to fix in stage G and keep in state B.</p><ul><li><strong>keep(G) = 10 + 0.95[0.6(100.9) + 0.4(87.8)] ≈ 100.9</strong></li><li>fix(G) = −8 + 0.95V(G) = -8 + 0.95(100.9) = 87.9</li></ul><p>So, playing keep in state G does produce a higher payoff. We’ll do the same for state B:</p><ul><li>keep(B) = ​2 + 0.95V(B) = ​2 + 0.95V(87.8) = 85.4</li><li><strong>fix(B) = −8 + 0.95V(G)​ = -8 + 0.95(100.9) = 87.9</strong></li></ul><p>Like so, we’ve confirmed that our previous guess is the optimal <strong>policy, </strong>or the optimal “strategy” for this game.</p><p>This makes intuitive sense—as stated, we’re saying that when a machine is working (state G), we should run it (keep), and when it’s broken (B), we should fix it (fix). Paying -8 now is better than making only +2 in profit.</p><p>For our second example, we’ll work through something a little more relevant to an economics-related course or job. We’ll use a Python script to calculate the optimal policy with the Bellman equation instead of calculating by hand.</p><blockquote>A quick note: depending on where you are in your studies of economics and/or finance and computer science, you may or may not be comfortable with the notation and concepts involved in optimization problems of this sort. If not, I think it’s best to keep in mind that the notation can make problems feel significnatly more mathematically difficult than they are. Problems will (usually) be much easier if you take time to understand the question and are generally comfortable with the notation.</blockquote><h4>Infinite Horizon Consumption–Saving Problem</h4><p>For this problem, we’ll set up an agent deciding whether to consume or save. The agent’s current wealth is some positive number <strong>Wt​ ≥ 0</strong>. This (wealth) is the state.</p><p>For the action, the agent chooses how much to consume <strong><em>c</em></strong> in the current period <strong><em>t</em></strong>. The choice must exist on an interval between 0 and the wealth, which can be thought of as the budget constraint: <strong>ct​ ∈ [0,Wt​]</strong>. Wealth not consumed is saved (savings in current period = wealth in current period - consumption in curent period):</p><ul><li><strong>st ​= Wt​− ct</strong></li></ul><p>The immediate reward function u(s,a)takes the general form <strong>u(Wt, ct)</strong> since wealth is the state and consumption is the action. The specification of this function is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/764/1*wSY0EHqKQoEdQPqta3mA7w.png" /><figcaption>Bellman Immediate Reward Function in an Infinite Horizon Consumption–Saving Problem</figcaption></figure><p>With gamma (y) being a parameter subject to gamma &gt; 0 that is the notational norm in <strong>Constant Relative Risk Aversion (CRRA)</strong> utility models and represents relative <strong>risk aversion</strong>. The higher the value of gamma, the more risk-averse the agent is (in turn affecting consumption versus savings decisions).</p><p>The <strong>state transition path</strong>—how wealth “updates” in the next period—is the wealth saved in this period plus a stochastic (meaning random) shock. Intuitively, the amount of money you have tomorrow is modeled pretty well as much you save today plus some randomness, whereas you aren’t fully sure of your income tomorrow. Note that wealth saved in this period appreciates by a risk-free interest rate <em>r</em>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/664/1*yvFvdKfPBS8hvaETHbkl2g.png" /><figcaption>Bellman Equation State Transition Path in an Infinite Horizon Consumption–Saving Problem</figcaption></figure><p>The exogenous shock <strong><em>Yt+1</em></strong> is drawn (randomly selected) from a known distribution <strong>Fy. </strong>Fy (F with a subscript y) is a variable that can denote any distribution we choose (usually, you want to empirically match a distribution to an input). We could say, for example, that Fy represents a normal Gaussian distribution.</p><p>We can say that <strong>P(Wt+1​ | Wt​,ct​) is induced by Y ∼ Fy</strong>​, or that the wealth in the next period is conditional upon wealth and consumption in this period, and is also <em>dependent</em> upon Fy because Y is determined randomly by Fy and this randomness carries over into the Wt+1 calculation.</p><p>The discount factor is 0.952 given β ∈ (0,1). β, more specifically, equals 1/(1+rate), so an annual discount rate of 5% produces a beta value of 0.952, and so on.</p><p>That all said, this problem differs from the previous example, which had only two periods. Instead, here, we’re dealing with an <strong>infinite horizon optimization</strong>. With such a problem, we’ll <strong>converge</strong> to an answer over time through the recursive iteration method described previously.</p><p>Convergence is the process of <strong>guessing</strong> a value, <strong>plugging</strong> it back into the equation to find a higher value, and <strong>continuing</strong> - as we (well, our script) keeps plugging in previous values, the change in the V(x) output will get smaller and smaller, until it eventually “converges” to a fixed point <em>(again, I like the Clue board game analogy for this)</em>. Once we have that point, we can derive the action for each state which attains that maximum value. This resulting set of state-action pairs is the solution to the problem.</p><p>Ok, here’s the Bellman equation - in this case, value as a function of wealth (the state):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i2OuJDEJEarUFp-JQYDwyA.png" /><figcaption>Bellman Value Function in an Infinite Horizon Consumption–Saving Problem</figcaption></figure><p>The equation is just like the previous ones we’ve looked at - the immediate reward function (the fraction to the left of the “+”) is equal to u(c), the state and action (W,c) is used in the continuination value, and we’re maximizing subject to<em> </em>consumption<em> c </em>being a positive fraction of wealth <em>W.</em></p><p>We know that wealth in the next period is found through a random income shock in the next period added to appreciated saved wealth in the current period:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/930/1*MMQi_p3aTmswGEXypVtyeQ.png" /><figcaption>Next-period wealth in a Bellman Infinite Horizon Consumption–Saving Problem</figcaption></figure><p>Solving this fixed-point equation (meaning that one fixed set of values is the solution) gives the lifetime value V(⋅), and the consumption that attains this max in each wealth state is the optimal consumption policy c(W).</p><blockquote>A quick note: we’re using value iteration here, since we’re guessing-and-checking a certain value and updating our Bellman equation with it to converge on a lifetime value. You will also hear about policy iteration, which is another dynamic programming approach in which a policy is calculated and recursively improved in each iteration, instead of values. While policy iteration tends to converge faster, the best algorithm to use depends upon the problem at hand.</blockquote><p>Now, an analytical solution is impossible because income is random. This is why we have to use <strong>numerical</strong> methods, which are methods where we approximate an answer (convergence being one way of doing so).</p><p>With all this setup in mind, here are the steps we want our Python script to succcessfully execute value iteration with the Bellman equation:</p><ul><li><strong>Choose an inital guess V0<em>(W) = 0</em></strong></li><li><strong>Create a discrete “grid” for wealth.</strong></li><li><strong>For each point on the wealth grid, compute the Bellman equation to update the value.</strong></li><li><strong>Return the value as the new guess, and repeat.</strong></li><li><strong>Stop this process once the changes in output values get extremely small.</strong></li><li><strong>Return the input value that maximizes the function given the converged value.</strong></li></ul><p>A quick note on step two: wealth is fundamentally continous and can take on an infinite number of values. To get by this problem, we “<strong>discretize</strong>” the state space by defining a maximum, minumum, and iterval for the computer will consider. For example, we could set the minimum and maximum to 1 and 100, and the iterval to 1. Like so, the computer will consider wealth states of 1, 2, 3 … 100. We’ll create more than one grid (one for consumption as well), and so we create nested grids of sorts whereas each discrete wealth grid point contains a consumption grid within it, each representing a different equation to calculate and “run up the chain.”</p><p>Here’s the script, which we’ll break down (you can run it yourself in terminal to follow along!):</p><pre>&quot;&quot;&quot;<br>Infinite-Horizon Consumption–Saving Problem using Value Iteration<br>&quot;&quot;&quot;<br><br>import numpy as np<br>import matplotlib.pyplot as plt<br><br># 1. PROBLEM PARAMETERS<br><br># Economic parameters<br>beta = 0.95238   # Discount factor (patience): how much future utility matters<br>gamma = 1.5    # Risk aversion coefficient: higher = more risk averse  <br>r = 0.03       # Risk-free interest rate on savings<br><br># Income process<br>Y_vals = np.linspace(0, 2, 5)  # Possible income values<br>Y_prob = np.full_like(Y_vals, 1/len(Y_vals))  # Equal probabilities<br><br># 2. NUMERICAL SETUP<br><br># Wealth grid (state space)<br>W_min, W_max = 0.0, 20.0  # Wealth bounds<br>N_W = 300                  # Number of wealth grid points<br>W_grid = np.linspace(W_min, W_max, N_W)<br><br># Consumption grid (for optimization at each state)<br>N_C = 40  # Number of consumption choices to evaluate per wealth level<br><br># Convergence parameters<br>max_iter = 400    # Maximum iterations for value iteration<br>tol = 1e-6        # Convergence tolerance<br><br>print(f&quot;\nNumerical Setup:&quot;)<br>print(f&quot;  Wealth grid: [{W_min}, {W_max}] with {N_W} points&quot;)<br>print(f&quot;  Consumption choices per state: {N_C}&quot;)<br>print(f&quot;  Max iterations: {max_iter}, tolerance: {tol}&quot;)<br><br># 3. UTILITY FUNCTION<br><br>def utility(c):<br>    if gamma == 1.0:<br>        return np.log(c)<br>    else:<br>        return (c ** (1 - gamma)) / (1 - gamma)<br><br># 4. VALUE ITERATION ALGORITHM<br><br>print(f&quot;\nStarting Value Iteration...&quot;)<br><br># Initialize arrays<br>V_old = np.zeros(N_W)        # Current value function<br>V_new = np.zeros(N_W)        # Updated value function  <br>policy_c = np.zeros(N_W)     # Optimal consumption policy<br><br># Track convergence<br>error_history = []<br><br># Main value iteration loop<br>for iteration in range(max_iter):<br>    <br>    # For each wealth level, find optimal consumption<br>    for i, W in enumerate(W_grid):<br>        <br>        # Create consumption grid: can&#39;t consume more than current wealth<br>        C_grid = np.linspace(1e-8, W, N_C)  # Small lower bound avoids log(0)<br>        <br>        # Evaluate value for each consumption choice<br>        values = np.full(N_C, -np.inf)  # Initialize with very low values<br>        <br>        for j, c in enumerate(C_grid):<br>            # Current period utility<br>            current_utility = utility(c)<br>            <br>            # Next period wealth for each possible income<br>            W_next = (W - c) * (1 + r) + Y_vals<br>            <br>            # Expected continuation value (interpolate since W_next may not be on grid)<br>            expected_continuation = np.dot(Y_prob, np.interp(W_next, W_grid, V_old))<br>            <br>            # Total value = current utility + discounted expected future value<br>            values[j] = current_utility + beta * expected_continuation<br>        <br>        # Choose consumption that maximizes value<br>        best_choice = np.argmax(values)<br>        V_new[i] = values[best_choice]<br>        policy_c[i] = C_grid[best_choice]<br>    <br>    # Check convergence<br>    max_error = np.max(np.abs(V_new - V_old))<br>    error_history.append(max_error)<br>    <br>    if max_error &lt; tol:<br>        print(f&quot;✓ Converged after {iteration + 1} iterations&quot;)<br>        print(f&quot;  Final max error: {max_error:.2e}&quot;)<br>        break<br>    <br>    # Update for next iteration<br>    V_old = V_new.copy()<br>    <br>    # Progress indicator<br>    if (iteration + 1) % 50 == 0:<br>        print(f&quot;  Iteration {iteration + 1}: max error = {max_error:.2e}&quot;)<br><br>else:<br>    print(f&quot;⚠ Warning: Reached maximum iterations ({max_iter}) without full convergence&quot;)<br>    print(f&quot;  Final max error: {max_error:.2e}&quot;)<br><br># 5. RESULTS<br><br>print(f&quot;\nResults Summary:&quot;)<br>print(f&quot;  Converged: {&#39;Yes&#39; if max_error &lt; tol else &#39;No&#39;}&quot;)<br>print(f&quot;  Final iterations: {len(error_history)}&quot;)<br>print(f&quot;  Wealth range: [{W_grid[0]:.1f}, {W_grid[-1]:.1f}]&quot;)<br>print(f&quot;  Consumption range: [{policy_c.min():.3f}, {policy_c.max():.3f}]&quot;)</pre><p>Let’s break down that script by parts and then we’ll look at visualizations and convergence.</p><p>First (note the image below), we set the parameters for the model. These are exogenous (determined by us) and usually calculated empirically when attempting to simulate a world-world scenario. The <em>Y-vals</em> line sets up an array of 5 equally spaced values between 0 and 2, and <em>Y_prob</em> assigns equal probabilities of occuring to those values. Cumultively, they form the stochastic income shock and the corresponding “random” distribution (it is not that the <em>distribution</em> is random, but rather the choice is random <em>according to the distribution</em>):</p><pre># 1. PROBLEM PARAMETERS<br><br># Economic parameters<br>beta = 0.96    # Discount factor; how much future utility matters<br>gamma = 1.5    # Risk aversion coefficient; higher = more risk averse  <br>r = 0.03       # Risk-free interest rate on savings<br><br># Income process<br>Y_vals = np.linspace(0, 2, 5)  # Possible income values<br>Y_prob = np.full_like(Y_vals, 1/len(Y_vals))  # Equal probabilities</pre><p>Next, we set up the mentioned wealth grid. We establish the minimum, maximum, and number of points, and use <em>np.linspace</em> to create an array, evenly spacing those 300 grid points throughout the defined min/max interval. We then apply convergence bounds by setting the max number of possible iterations to 400 and the convergence threshold, or the maximum spread allowed prior to convergence, to be a very small number:</p><pre># 2. NUMERICAL SETUP<br><br># Wealth grid (state space)<br>W_min, W_max = 0.0, 20.0  # Wealth bounds<br>N_W = 300                  # Number of wealth grid points<br>W_grid = np.linspace(W_min, W_max, N_W)<br><br># Consumption grid (for optimization at each state)<br>N_C = 40  # Number of consumption choices to evaluate per wealth level<br><br># Convergence parameters<br>max_iter = 400    # Maximum iterations for value iteration<br>tol = 1e-6        # Convergence tolerance<br><br>print(f&quot;\nNumerical Setup:&quot;)<br>print(f&quot;  Wealth grid: [{W_min}, {W_max}] with {N_W} points&quot;)<br>print(f&quot;  Consumption choices per state: {N_C}&quot;)<br>print(f&quot;  Max iterations: {max_iter}, tolerance: {tol}&quot;)</pre><p>Third, we set up the utility function u(c) of the standard CRRA form. There’s a special case when gamma equals 1; if this happens, we have to log the variable (this is standard to the CRRA setup and reflects marginal diminshing returns). If not, we adhere to the utility function described: (c^1-gamma)/1 — gamma).</p><pre># 3. UTILITY FUNCTION<br><br>def utility(c):<br>    if gamma == 1.0:<br>        return np.log(c)<br>    else:<br>        return (c ** (1 - gamma)) / (1 - gamma)</pre><p>Fourth, we run through the core value iteration. We initialize<em> V_old</em> as the value function from the previous iteration, <em>V_new</em> the updated value function for the current iteration (computed based on <em>V_old</em>), and <em>policy_c</em> the optimal consumption policy that we’re trying to find.</p><p>Now’s the meat of the script: where we actually recursively solve the problem. Here’s the high-level setup:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*Z6IMs3vhla43hs7Bt3P7Yg.png" /></figure><p>Translating this process into code, we use a nested for loop, create another array (grid) for consumption, and calculate the utility for each consumption point given the wealth point in another nested for loop. Note that <em>argmax</em>, short for “argument of the maximum”, returns the input value that maximizes a given function, in this case the best consumption choice. That <em>best_choice</em> value then updates the value function.</p><pre># 4. VALUE ITERATION ALGORITHM<br><br>print(f&quot;\nStarting Value Iteration...&quot;)<br><br># Initialize arrays<br>V_old = np.zeros(N_W)        # Current value function<br>V_new = np.zeros(N_W)        # Updated value function  <br>policy_c = np.zeros(N_W)     # Optimal consumption policy<br><br># Main value iteration loop<br>for iteration in range(max_iter):<br>    <br>    # For each wealth level, find optimal consumption<br>    for i, W in enumerate(W_grid):<br>        <br>        # Create consumption grid: can&#39;t consume more than current wealth<br>        C_grid = np.linspace(1e-8, W, N_C)  # Small lower bound avoids log(0)<br>        <br>        # Evaluate value for each consumption choice<br>        values = np.full(N_C, -np.inf)  # Initialize with very low values<br>        <br>        for j, c in enumerate(C_grid):<br>            # Current period utility<br>            current_utility = utility(c)<br>            <br>            # Next period wealth for each possible income<br>            W_next = (W - c) * (1 + r) + Y_vals<br>            <br>            # Expected continuation value (interpolate since W_next may not be on grid)<br>            expected_continuation = np.dot(Y_prob, np.interp(W_next, W_grid, V_old))<br>            <br>            # Total value = current utility + discounted expected future value<br>            values[j] = current_utility + beta * expected_continuation<br>        <br>        # Choose consumption that maximizes value<br>        best_choice = np.argmax(values)<br>        V_new[i] = values[best_choice]<br>        policy_c[i] = C_grid[best_choice]<br>    </pre><p>We finally check for convergence by checking the spread between the new value and the old value and comparing it to the <em>tol</em> value. If it’s smaller than the <em>tol</em> threshold, then the value is considered converged and the script exits the loop.</p><pre># Check convergence<br>    max_error = np.max(np.abs(V_new - V_old))<br>    <br>    if max_error &lt; tol:<br>        print(f&quot;✓ Converged after {iteration + 1} iterations&quot;)<br>        print(f&quot;  Final max error: {max_error:.2e}&quot;)<br>        break<br>    <br>    # Update for next iteration<br>    V_old = V_new.copy()<br>  <br><br>else:<br>    print(f&quot;⚠ Warning: Reached maximum iterations ({max_iter}) without full convergence&quot;)<br>    print(f&quot;  Final max error: {max_error:.2e}&quot;)</pre><p><em>I won’t expand on section #5 — that’s just printing results.</em></p><p>So, when running the script, here’s the terminal output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/540/1*cEypu_lNgLv7hQCbTpPY1A.png" /><figcaption>Bellman-Method Iterative “Learning” Over Time</figcaption></figure><p>You can visually how the algorithm “improves” its guesses over time through the Bellman equation, with a max error (spread) of 0.0274 at iteration 50, then 0.00355 at iteration 100, and so on before converging with a spread of 0.0000000985 after 357 iterations.</p><p>Here some plots of the script over time and the insights it delivers (created using the matplotlib Python library):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6VQE0845J-cbxP_LOcMmjQ.png" /><figcaption>Bellman Value Iteration — Convergence of Value Iteration &amp; Converged Value Function</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S60f3iMUskTH_XvOLOynAA.png" /><figcaption>Bellman-Derived Savings Rate &amp; Consumption Policy</figcaption></figure><p>That concludes our two examples!</p><h4>Summary</h4><p>Here’s a quick rundown of everything we’ve covered to this point:</p><ul><li><strong>Foundational concepts: </strong>states, actions, rewards, dynamic programming, and reinforcement learning.</li><li><strong>Bellman Equation struture:</strong> immedaite rewards + discounted expected future values.</li><li><strong>Uncertainty in Bellman equations: </strong>probability distributions and expected values to model stochastic shocks.</li><li><strong>Policies versus actions: </strong>the difference between choices (actions) and strategies (decision rules/policies).</li><li><strong>Solution methods: </strong>value iteration as a computational approach using Bellman equations to solve optimization problems.</li><li><strong>Two examples using the Bellman equation</strong>: a 2-period machine maintenance example solved by hand, and an infinite-horizon consumption-savings model requiring numerical methods to solve.</li><li><strong>Implementation techniques: </strong>discretizing continious variables, grid-based optimization, and convergence criteria.</li></ul><p>There’s a lot more to the world of optimization methods that Bellman methods open up, but we’ve covered the essentials. If I missed anything, or you’re curious about a related subject, make sure to leave it in the comments. Thanks for reading!</p><p><strong>If you liked this article, you’ll probably like </strong><a href="https://jonwlaw.medium.com/monte-carlo-simulations-an-intuitive-introduction-python-example-dd559070275b"><strong>this article I just wrote about Monte Carlo simulations</strong></a><strong>, </strong>which rely on many similar concepts we’ve employed when exploring the Bellman equations and are a very useful applied means of simulating the real world in economics, finance, and beyond.</p><p>Or, if you’re into (or studying!) economics, you’re in the right place, and make sure to check out my other articles <a href="https://jonwlaw.medium.com/list/economics-36ae74971581">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5b812054949f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Monte Carlo Simulations - An Intuitive Introduction (& Python Example)]]></title>
            <link>https://jonwlaw.medium.com/monte-carlo-simulations-an-intuitive-introduction-python-example-dd559070275b?source=rss-3364945644f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/dd559070275b</guid>
            <category><![CDATA[monte-carlo]]></category>
            <category><![CDATA[dynamic-programming]]></category>
            <category><![CDATA[monte-carlo-simulation]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[monte-carlo-method]]></category>
            <dc:creator><![CDATA[Jon Law]]></dc:creator>
            <pubDate>Sat, 17 May 2025 08:51:35 GMT</pubDate>
            <atom:updated>2025-05-17T08:51:35.318Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GNCyO3UPQzvw4TNvVd-D4g.png" /><figcaption>A Monte Carlo Simulation — we’ll explain this in the article!</figcaption></figure><p><strong>Monte Carlo simulations </strong>seem to be on the recommended skills sheet of<strong> </strong>every big investment bank and consulting group. Yet, at least in my experience, it’s also near-entirely ignored by the degree curriculums supposed to prepare you for those jobs.</p><p>So, I’ll introduce some of the conceptual basics of Monte Carlo methods and simulations. By the end of the article, you’ll be able to explain what they are, why they’re important, and the core technical strategies and terms.</p><p>But first—if you heard of Monte Carlo simulations or methods and thought of the famed <strong>Monaco Casino</strong>, you wouldn’t be the first.</p><p>Monte Carlo methods were named by <strong>Stanisław Ulam, </strong>a Polish -American mathematician working on the Manhattan Project (yes, the nuclear one!). Ulam was analyzing the probability of winning <strong>solitaire</strong> and he nicknamed the method “Monte Carlo” because his uncle liked gambling in Monaco. He later applied Monte Carlo methods to neutron‑diffusion problems, so yes, this method was discovered in <strong>serving to create the first thermonuclear weapons.</strong></p><p>That said, Monte Carlo methods are a broad range of simulation techniques used to estimate quantities that are difficult or impossible to derive analytically (to “solve” for) — just like the scientist and his card problem. They’re <strong>inferential </strong>in nature, which means we’re not “solving” our problem or question like we would an algebra or calculus problem. Instead, we’re using data and estimations to inform our understanding of the whole problem and dataset.</p><p>More intuitively, we’re using <strong>randomness</strong> to identify truth. Say we wanted to find the average height of humans in the world. The “real” way to do this would be to measure every person on the planet and take the average. However, it’s practically impossible to do that. Instead, we can go find some random people, take their heights, and find the average.</p><p>Now, if we went and measured the heights of 10 people to make a guess about the average height in the world, we might be pretty far off—maybe there’s a few really tall people, only children, etc.</p><p>If we measured 100 people, our guess would likely be much closer to the real average. The same applies if we measured 1000 people, and 1,000,000, and so on. <strong>As we take more random samples, we get closer to the real value. </strong>This is what’s called the law of large numbers, and it forms the basis of the Monte Carlo methodology.</p><p>Similarly to our method for finding average heights, real-world Monte Carlo methods simply involve making lots of random guesses and observing the output. That output could be a height average, or something more sophisticated, like what will happen to a stock price or an economy.</p><p>That said, let’s get a little more technical, with a focus on Monte Carlo methods as you would use them when programming or in the workforce:</p><p>What’s great about Monte Carlo approaches is that we don’t need to know much about the world or environment we’re dealing with. This makes simulations much more available to solve a larger set of problems.</p><p>In the height example, as our number of random guesses increase and our average changes, our guess is probably getting closer to the real average. <strong>More specifically, our sample mean converges to the true population mean as our sample size grows. </strong>All we need for the simulation are probability distributions for <strong>inputs </strong>(say, people we can guess the height of), which we select exogenously.</p><p>For example, say one of our inputs is the number of points LeBron James will score in his next game. Using data from his last 100 games, we can plot a histogram and use that as the distribution.</p><p>Or, returning to our weather example, a probability distribution (or “stochastic” distribution) for an input could be as simple as rains half the time, and is sunny half the time (P(r)=.5, P(s)=0.5).</p><p>Advanced simulations can have a lot more than one or two variables—think hundreds or thousands. With these given distributions, a Monte Carlo simulation will begin to, well, <strong>simulate</strong> — to “play” the game, or make random guesses.</p><p>I like the game analogy because it infers an element of randomness—you rarely play a video game or sports game twice and get the same score. Similarly, in each “round” of our game, our agent (algorithm) will randomly select one value for each input. It will keep doing this for each round of the game, until it’s played thousands (or millions) of times.</p><p>Remember, while the input “choices” are random in each period, they each adhere to an underlying probability distribution. So, by iterating through the environment many, many times (by “playing the game” in many different ways), you get some <strong>results </strong>that happen more often than others.</p><p>Back to the game analogy: if you played the same round of golf 100 times, your scores aren’t the same every time — instead, you get a distribution. Maybe you took 23 shots 25% of the time, 18 shots 14% of the time, 15 shots 8% of the time, and so on.</p><p>Like so, Monte Carlo simulations model out the world by “experimenting” in that world through random guesses. These guesses have to be <strong>unbiased</strong>—even if we measured 1,000,000 people to make a guess about average height, if those were 1,000,000 tall people, our guess would be very far from the average.</p><p>These kinds of simulations are valuable for many reasons. If you’re an investment fund, you might want to perform a Monte Carlo simulation for risk analysis to determine how likely certain events are which severly impact asset performance. If you’re a hedge fund, you might simulate firm earnings to decide whether to hold or buy. The list goes on.</p><p>Let’s get back to the graph I put at the top of the article. We’re making this in Python (I’ll show you the code in a second), and we’re imagining a situation in which we have a $10 million stock portfolio. We want to simulate how much our portfolio will go up or down tomorrow.</p><p>As stated earlier, we’ll start by assuming a distribution for the input, that being daily returns (daily volatility). We’ll assume a normal distribution (known as a bell curve or Gaussian distribution).</p><p>Then, we’ll simulate <em>tomorrow</em> 10,000 times—this many guesses will get us close to the real average move in our portfolio when tomorrow actually rolls around.</p><p>In the graph, the<strong> histogram bars </strong>show<strong> </strong>how many times each outcome range showed up, and the <strong>dashed vertical line</strong> is a <strong>Value-at-Risk</strong> threshold, meaning everyone left of the line (and inversely on the right side of the distribution) happens only 5% of the time. You’ve likely heard of 95% confidence intervals—this is similar, except it’s a one-sided measure where we’re looking at the worst 5% of cases (this is common in applied simulations and risk analysis):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GNCyO3UPQzvw4TNvVd-D4g.png" /></figure><p>So, it’s most likely in our simulation that our portfolio won’t make or lose any money, since it’s centered around 0. However, we certainly could make money, or lose it, and occasionally (less than 5% of the time) we’ll make or lose an extreme amount, like $400,000.</p><p>Here’s the Python script for the simulation. I’ll show you the whole thing, and then let’s walk through it line by line:</p><pre><br>import numpy as np<br>import matplotlib.pyplot as plt<br><br>portfolio_value = 10_000_000    <br>mu = 0.0005                 <br>sigma = 0.015             <br>n_draws = 10_000               <br>confidence = 0.95          <br><br>log_returns = np.random.normal(mu, sigma, n_draws)<br><br>pl = portfolio_value * (np.exp(log_returns) - 1)<br><br>var_percentile = 100 * (1 - confidence)  <br>var_1d = -np.percentile(pl, var_percentile)<br>print(f&quot;95% one‑day VaR ≈ ${var_1d:,.0f}&quot;)<br><br>plt.hist(pl, bins=50, edgecolor=&#39;black&#39;, alpha=0.7)<br>plt.axvline(-var_1d, linestyle=&#39;--&#39;, linewidth=2, label=&#39;95% VaR&#39;)<br>plt.title(&#39;Simulated 1‑Day Profit/Loss&#39;)<br>plt.xlabel(&#39;Profit / Loss ($)&#39;)<br>plt.ylabel(&#39;Number of simulations&#39;)<br>plt.legend()<br>plt.tight_layout()<br>plt.show()</pre><p>First, we’re importing <strong>numpy</strong> and <strong>matplotlib</strong>, which are two Python libraries used for things like arrays, random draws, and graphing. You don’t have to use these, but they make our life easier:</p><pre>import numpy as np<br>import matplotlib.pyplot as plt</pre><p>Next, we create and define five variables. We’ll set the value of our portfolio to the mentioned $10 million. Then, to describe the distribution, we’ll create variables for average volatility and returns—the amount we usually make (mean), and the average deviation (standard deviation). We want to play out “tomorrow” 10,000 times, so we’ll call a variable n_draws and set it equal to 10,000. We’ll also want to plot a 95% confidence interval—this would be a standard metric you’d use in a report.</p><pre>portfolio_value = 10_000_000      # dollar notional<br>mu = 0.0005                       # avg daily log‑return  = +0.05 %<br>sigma = 0.015                     # daily volatility     = 1.5 %<br>n_draws = 10_000                  # how many futures to simulate<br>confidence = 0.95                 # target risk level    = 95 % VaR</pre><p>Then, we’ll draw 10,000 random numbers from a normal distribution with the mean and standard deviation we set. We’re using the numpy library to make things simple.</p><pre>log_returns = np.random.normal(mu, sigma, n_draws)</pre><p>Then, we want to translate each return into a percentage return. Returns were logged, so we’ll transform it into a percentage return with exp(x) — 1 and then into a dollar profit or loss by multiplying that percent change by our portfolio value (the $10m).</p><pre>pl = portfolio_value * (np.exp(log_returns) - 1)</pre><p>To find the 95% confidence interval, we’ll find the 5th percentile of the profit/loss array (the 5% of random samples with the largest losses), and print that value:</p><pre>var_percentile = 100 * (1 - confidence)   # 5  for 95 %<br>var_1d = -np.percentile(pl, var_percentile)<br>print(f&quot;95% one‑day VaR ≈ ${var_1d:,.0f}&quot;)</pre><p>Finally, we just want to visualize the simulation, so we’ll make our histogram:</p><pre>plt.hist(pl, bins=50, edgecolor=&#39;black&#39;, alpha=0.7)<br>plt.axvline(-var_1d, linestyle=&#39;--&#39;, linewidth=2, label=&#39;95% VaR&#39;)<br>plt.title(&#39;Simulated 1‑Day Profit/Loss&#39;)<br>plt.xlabel(&#39;Profit / Loss ($)&#39;)<br>plt.ylabel(&#39;Number of simulations&#39;)<br>plt.legend()<br>plt.tight_layout()<br>plt.show()</pre><p>Generally, the steps we took were as follows:</p><ol><li><strong>Specify inputs by creating variables.</strong></li><li><strong>Take our random samples.</strong></li><li><strong>Convert guesses into metrics we care about (optional).</strong></li><li><strong>Create a confidence interval (optional).</strong></li><li><strong>Visualize our simulation (optional).</strong></li></ol><p>Note that a <em>lot</em> of that was optional—while that was the practical way to do it, here’s an uber-simple simulation that still employs a Monte Carlo method (“random” import is practically a random number generator, or RNG):</p><pre>import random                                            # RNG<br>print(sum(random.random() &lt; 0.5 for _ in range(10_000))  # simulate flips<br>      / 10_000)                                          # average → P(Heads)</pre><p>Here, we’re still drawing random samples, applying a simple rule, and averaging the outcomes. We’re generating a random number in the interval [0,1) and counting it as “heads” if it’s below 0.5, like if we were flipping a coin. Then we’re repeating that “coin flip” 10,000 times, summing the number of “heads” results, dividing by the number of samples (10,000) to turn the count into a percentage (frequency). If 10,000 was 100,000, or 1,000,000 and so on, our result would get closer to 0.5.</p><p>So, there you have it! You should understand more about Monte Carlo methods and real-world simulations. If you’re into (or studying!) economics, you’re in the right place, and make sure to check out my other articles <a href="https://jonwlaw.medium.com/list/economics-36ae74971581?source=user_lists---------2-------36ae74971581------3364945644f7----------------------">here</a>. Thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dd559070275b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Finding AC, MC, and Natural Monopoly from a Nonlinear Cost Function | Econ Step-by-Step]]></title>
            <link>https://jonwlaw.medium.com/finding-ac-mc-and-natural-monopoly-from-a-nonlinear-cost-function-econ-step-by-step-bf1d124dbcf2?source=rss-3364945644f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/bf1d124dbcf2</guid>
            <category><![CDATA[economics]]></category>
            <category><![CDATA[economics-101]]></category>
            <category><![CDATA[economics-assignment-help]]></category>
            <category><![CDATA[economics-homework-helper]]></category>
            <category><![CDATA[economics-homework-help]]></category>
            <dc:creator><![CDATA[Jon Law]]></dc:creator>
            <pubDate>Tue, 06 May 2025 20:30:53 GMT</pubDate>
            <atom:updated>2025-05-06T20:30:53.543Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YB88f8up_IclyNDpyc8tFQ.png" /><figcaption>Finding AC, MC, and Natural Monopoly from a Nonlinear Cost Function</figcaption></figure><p>In this problem, we’ll analyze a cost function to explore how it affects a firm’s average and marginal costs.</p><p>We’ll <strong>derive</strong> these costs, <strong>interpret</strong> and <strong>graph</strong> their relationship, and assess whether this cost structure reflects a <strong>natural monopol</strong>y. If you’re looking for a full <strong>cost minimization problem</strong>, check out <a href="https://jonwlaw.medium.com/solving-a-cost-minimization-problem-using-isoquants-and-isocosts-with-a-cobb-douglas-production-881f6cfc1f37">this</a> article.</p><p>To start, for this problem, all we’re given is a <strong>nonlinear</strong> <strong>cost function </strong>(nonlinear due to the exponent):</p><ul><li><strong>C(Q) = 2Q^(1/2)</strong></li></ul><p>We know that is Q is raised to a smaller power (exponent) than 1, then it will feature<strong> diminishing marginal costs. </strong>Intuitively, this means that producing more lowers marginal cost, or the cost per additional unit of output produced. From this cost function, we also know that there’s no <strong>fixed cos</strong>t and only <strong>variable cost</strong>—cost that varies with output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-KnJDZa-6fT0CziaL6guJA.png" /><figcaption>Diminishing marginal costs</figcaption></figure><p>We’ll now derive average cost and marginal cost. Average cost is cost divided by quantity, or C/Q:</p><ul><li><strong>AC = C/Q = (2Q^(1/2))/Q = 2Q^(-1/2) = 2/(Q^(1/2))</strong></li></ul><p>Marginal cost is the derivative of the cost function:</p><p><strong>dC/dQ = d/dq(2Q^(1/2)) = (1/2)(2)(Q^(-1/2)) = 1(Q^(-1/2)) = Q^(-1/2)</strong></p><p>Hence, we can think of Q^(-1/2) as 1/Q^(1/2)) to easily see that <strong>AC</strong> is double <strong>MC. </strong>This means that <strong>AC</strong> will always be larger than <strong>MC</strong>—for any value we insert for Q, AC is simply a larger fraction of that value (2/x versus 1/x).</p><p>Let’s plot the marginal cost and average cost in relation to quantity:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YB88f8up_IclyNDpyc8tFQ.png" /><figcaption>Average cost and marginal cost plotted against output (quantity).</figcaption></figure><p>Note that average cost curves can only intersect marginal cost curves when the two costs are equal. Logically, if your marginal cost (your cost to produce one additional unit) is under your average cost, then your cost will go down, while if your marginal cost is greater than your average cost, then your verage cost will increase with that additional unit of output. This intuitively explains why the only point where average cost stops falling and starts rising is where MC = AC.</p><p>As the final linguistic portion of this question: do we know whether this cost function represents a <strong>natural monopoly</strong>?</p><p>Natural monopolies occur when the cheapest way to produce is one firm. Mathematically, this translates to three criteria:</p><ul><li>Does AC decrease at all levels of output?</li><li>Is MC always smaller than AC?</li><li>Are there economies of scale over the full range of output?</li></ul><p>If these are met, we know we’re dealing with a natural monopoly since average cost <em>always</em> falls as production increases, and therefore one firm producing <em>all</em> output in a market will maintain the lowest average cost.</p><p>In our problem, note that AC = 2/(Q^(1/2)) and MC = 1/(Q^(1/2). Therefore, 2/(Q^(1/2)) = 2(1/(Q^(1/2))), or AC = 2MC, so <strong>MC = 1/2AC.</strong> In this equation, we can plug in any value for Q and MC will always be 1/2 the value of AC. So, our three criteria are met.</p><p>Those are the five parts of this problem! <strong>If you’re a student in econ 101 or 201, you’ll find </strong><a href="https://jonwlaw.medium.com/list/economics-201202-721d3c89e989"><strong>these</strong></a><strong> articles useful</strong> (exact problem walkthroughs like the ones you’ll see on exams).</p><p>Else, hope you found this helpful, and leave any questions in the comments!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bf1d124dbcf2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Solving a Cost Minimization Problem Using Isoquants and Isocosts with a Cobb-Douglas Production…]]></title>
            <link>https://jonwlaw.medium.com/solving-a-cost-minimization-problem-using-isoquants-and-isocosts-with-a-cobb-douglas-production-881f6cfc1f37?source=rss-3364945644f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/881f6cfc1f37</guid>
            <category><![CDATA[economics]]></category>
            <category><![CDATA[econ]]></category>
            <category><![CDATA[economics-homework-help]]></category>
            <category><![CDATA[economics-homework-helper]]></category>
            <category><![CDATA[economics-assignment-help]]></category>
            <dc:creator><![CDATA[Jon Law]]></dc:creator>
            <pubDate>Tue, 06 May 2025 02:06:21 GMT</pubDate>
            <atom:updated>2025-05-06T20:21:06.065Z</atom:updated>
            <content:encoded><![CDATA[<h3>Solving a Cost Minimization Problem Using Isoquants and Isocosts with a Cobb-Douglas Production Function</h3><p>Continuing my recent streak of micro 101/201 walkthroughs, here we’ll walk through a classic<strong> cost minimization problem</strong> using a <strong>Cobb-Douglas</strong> production function with <strong>capital</strong> and <strong>labour</strong>.</p><p>We’ll see how firms can minimize costs for a given level of output, and graph this visually through <strong>isoquants</strong> and <strong>isocosts</strong> (the firm version of <strong>indifference curves</strong> and <strong>budget constraints</strong>, which we consumers get!).</p><p>We’ll be deriving the <strong>marginal rate of technical substitution (MRTS)</strong>, apply a tangency <strong>optimality condition</strong>, and finally solving for an equilibrium condition of labour and capital.</p><p>So, this article will certainly be useful if you’re a student in econ or 201 — or for anyone learning for fun. Leave any questions in the comments!</p><p>For this problem, we start with a firm production function featuring capital (K) and labour (L) as <strong>production inputs</strong>:</p><ul><li><strong>Q(K,L) = K*L</strong></li></ul><p>Notation-wise, note that “Q(K,L)” denotes <strong>Q</strong> as being a function of the inputs K and L, while K*L is the <em>form</em> of that function—5 units of capital and 5 units of labour as inputs, for example, yields Q = 25.</p><p>Note that this production function exhibits <strong>constant returns to scale (CRS)</strong> because doubling both inputs doubles output (you can test this by adding a lambda to the inputs and seeing if it equals lambda times the output).</p><p>We’re also given standard costs associated with capital and labour, such being <strong><em>wage</em></strong> (w) and <strong><em>interest rate</em></strong> (r). You can think of the interest rate generally as the <strong><em>price of capital</em></strong>. These are linear costs associated with the production inputs, producing a cost function as follows:</p><ul><li><strong>C = rK + wL</strong></li></ul><p>So, the cost to a given firm to produce <strong><em>Q</em></strong> units through<strong><em> K </em></strong>units of capital and<strong><em> L</em></strong> units of labour is <strong><em>K</em></strong> multiplied by the interest rate<em> </em><strong><em>r </em></strong>and <strong><em>L</em></strong> multiplied by the wage rate <strong><em>w</em></strong>. For example, a firm employing 5 people at $100 per day to work on 4 machines has a cost function 4r + (100)(5) = 4r + 500.</p><p>To graph this problem, we’ll set up a graph with<strong><em> L</em></strong> on the x-axis and<strong><em> K</em></strong> on the y-axis. To find the slope of the relationship between these inputs, we can adjust the cost function for KL</p><ul><li><strong>C = rK + wL</strong></li><li><strong>C — wL = rK</strong></li><li><strong>K = C/r — (w/r)L</strong></li></ul><p>This is an <strong>isocost, </strong>which shows all combinations of inputs at a fixed cost (all combinations of<em> K </em>and <em>L </em>that result in the same total cost <em>C</em>). There are an infinite number of isocosts, each for a different cost (5, 50, 500, 500.000001 and so on). You’ll note isocosts are similar to indifference curves, whereas an infinite number of indifference curves exist for different levels of utility. C/r is the Y-intercept, and -w/r is the slope. Let’s graph our K/L relationship:</p><p><em>(note that a bar over a variable represents a fixed quantity)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V3i51ZMasOWyNRXmbkppew.png" /><figcaption>Solving a Cost Minimization Problem Using Isoquants and Isocosts with a Cobb-Douglas Production Function</figcaption></figure><blockquote><strong>Hey! </strong>Quick note from Jon. <strong>If you’re a student in econ 101 or 201, you’ll find </strong><a href="https://jonwlaw.medium.com/list/economics-201202-721d3c89e989"><strong>these</strong></a><strong> articles useful</strong> (exact problem walkthroughs like the ones you’ll see on exams). Back to the problem!</blockquote><p>So, each of these lines represents a different cost level. Next, we’ll plot an <strong>isoquant</strong>, which is similar to an isocost, except a quantity (q-bar) is fixed instead of cost. Isoquants look like indifference curves.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aEwvjqYuYdpHTQdBK0HRQw.png" /><figcaption>Solving a Cost Minimization Problem Using Isoquants and Isocosts with a Cobb-Douglas Production Function</figcaption></figure><p>There are also infinite isoquants representing different quantity levels. To find an equilibrium, we’ll find the closest isocost to the origin (farthest “left”) touching an isoquant which intercepts the equilibrium values of K and L.</p><p>Let’s start by calculating the <strong>marginal rate of technical substitution</strong> (MRTS), which is the slope of the isoquant. It’s calculated as the<strong> marginal product of labour</strong> (the x-axis good) divided by the <strong>marginal product of capital</strong> (the y-axis good). These marginal products are found by taking the partial derivative of the production function <strong><em>Q</em></strong> with respect to the production input at hand (<strong><em>K</em></strong> or<strong><em> L</em></strong>), as follows:</p><p><strong>MRTS = MP<em>L/</em>MP<em>K = (</em>∂Q/∂L)/<em>(</em>∂Q/∂K) = (∂(K*L)/∂L)/(∂(K*L)/∂K) = K/L</strong></p><p>Since we know the isoquant is downward sloping, it’s slope is therefore -K/L. Now that we know the equations for the isocost and isoquant, let’s add them to the graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r3OhI--bz0nPKNwxEr1ABQ.png" /><figcaption>Isoquant and Isocost Equations</figcaption></figure><p>Next, we’ll find the cost-minimizing equilibrium bundle for the firm. We’ve already done most of the hard work—all we need to do is set the slopes of the isoquant and isocost equal to one another, as this point is where our <strong>firm gets the same additional output per dollar spent</strong> on labor and capital. To go deeper on that:</p><ul><li>MRTS <em>=</em> w/r</li><li>MP<em>L/</em>MP<em>K = </em>w/r</li><li><strong>MP<em>L/w = </em>MP<em>K/r</em></strong></li></ul><p>So, at the point where the isocost is tangential to the isoquant, firms get the same marginal value per dollar spent whether they invest in labour or capital. That must mean they’re at an equilibrium. That said, let’s set those values equal to each other:</p><ul><li><strong>-K/L = -w/r</strong></li><li><strong>K/L = w/r</strong></li></ul><p>If you’re given this problem in a course (let’s say Microeconomics 101 or 201), you’ll likely be given simply values for <em>w, r, </em>and<em> Q </em>to solve out the cost-minimizing allocation (basically, the question of “Given a required output level Q, what’s the cheapest way to produce it?”).</p><p>In our problem, here’s the values we’ll move ahead with, which produces the question <em>what’s the cheapest way to produce an output of 100?:</em></p><ul><li><strong>w = 1</strong></li><li><strong>r = 1</strong></li><li><strong>Q = 100</strong></li></ul><p>Let’s return to our optimality condition and plug in <em>w </em>and<em> r</em>:</p><ul><li><strong>K/L = w/r</strong></li><li><strong>K/L = 1/1</strong></li><li><strong>K/L = 1</strong></li><li><strong>K = L(1)</strong></li><li><strong>K = L</strong></li></ul><p>So, given <em>w</em> and <em>r</em> equal 1, K and L are equivelant. If our values were <em>w,r = 1,2, K </em>would equal <em>2L</em>, and so on. Knowing this, let’s plug in values for our production function to find what are called <strong><em>conditional demands</em></strong>:</p><ul><li><strong>Q = KL</strong></li><li><strong>Q = KK = LL</strong></li><li><strong>Q = K² = L²</strong></li><li><strong>K = L = square root of Q = 10</strong></li></ul><p>Now, as a last step, let’s plug our conditional demands into the cost function (as we already know the values for<em> w</em> and <em>r</em>):</p><ul><li><strong>C = rK + wL</strong></li><li><strong>C = (1)(sqrQ) + (1)(sqrL)</strong></li><li><strong>C = (1)(sqr100) + (1)(sqr100)</strong></li><li><strong>C = (1)(10) + (1)(10)</strong></li><li><strong>C = 10 + 10</strong></li><li><strong>C = 20</strong></li></ul><p>In summary, the cost-minimizing levels of inputs required to produce an output of 100 are 10 units of capital and 10 units of labour, which produce a total cost of 20.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3mIRTcnOKoGCoWtWf264Gw.png" /><figcaption>Cost Minimization Problem Isoquants and Isocosts w/ Equilibrium</figcaption></figure><p>Hope you found this helpful, and leave any questions in the comments! <strong>Check out my other articles on economics </strong><a href="https://jonwlaw.medium.com/list/economics-36ae74971581"><strong>here</strong></a><strong> (useful for studying, or just learning).</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=881f6cfc1f37" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Solving Equilibrium Price, Quantity, andWelfare in a Monopolistic and Competitive Market (Econ…]]></title>
            <link>https://jonwlaw.medium.com/solving-equilibrium-price-quantity-andwelfare-in-a-monopolistic-and-competitive-market-econ-2998895d3852?source=rss-3364945644f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/2998895d3852</guid>
            <category><![CDATA[economics-101]]></category>
            <category><![CDATA[economics]]></category>
            <category><![CDATA[economics-homework-helper]]></category>
            <category><![CDATA[economics-assignment-help]]></category>
            <category><![CDATA[economics-homework-help]]></category>
            <dc:creator><![CDATA[Jon Law]]></dc:creator>
            <pubDate>Sun, 04 May 2025 20:54:11 GMT</pubDate>
            <atom:updated>2025-05-06T20:21:36.324Z</atom:updated>
            <content:encoded><![CDATA[<h3>Solving Equilibrium Price, Quantity, and Welfare in a Monopolistic and Competitive Market (Econ Follow-Along)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m71ADQv_4_Fx3mqgB_Q5yw.png" /><figcaption>Solving Equilibrium Price and Quantity and Welfare in a Monopolistic and Competitive Market</figcaption></figure><p>Understanding and calculating how firms choose output and price, as well as how these choices affect net welfare, is a cornerstone of micro, and something you’ll come across regularly as a student or an enthusiast.</p><p>In this article, we’ll work our way through a classic monopoly problem with a linear demand curve and a quadratic cost function. We’ll derive the profit-maximizing price and quantity, as well as consumer surplus, producer surplus, and deadweight loss. We’ll then shift to a nearly identical problem (save for a modified firm-level cost function) in a competitive market to determine the long-run competitive equilibrium and welfare.</p><p>Graphing is very useful for problems like these, so graphs are present alongside the math and conceptual explanations.</p><p>We’ll start with a production function, a cost function, and the first question of our monopoly problem: what is the equilibrium price and quantity?</p><ul><li><strong>Q = 60 — P </strong><em>(hence P = 60 — Q)</em></li><li><strong>C = 36 + 2Q²</strong></li></ul><p>To do this, we’ll take the first order condition (FOC) by finding the equation for profit, and then take the partial derivative with respect to price.</p><p>When calculating an FOC, you take partial derivatives with respect to <strong><em>inputs</em></strong>, and in this model firms face in inverse demand curve whereas they set the price P to adjust quantity. So, price is the input and thus what we’ll take the partial derivative with respect to.</p><p>Intuitively, the reason we’re taking the FOC of profit at all is because we’re finding the point at which profit is “flat” on the graph with a slope of 0. At this local maximum, firms have no incentive to adjust their price, because doing so can only decrease profits (note that we’ll confirm this using a second-order concavity check in a second). Here’s that concept explained visually:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bi99opU3w3Qi9f1J5QKNnQ.png" /><figcaption>Taking the FOC of Profit with Respect to Price to Calculate Monopoly Equilibrium Quantity &amp; Price</figcaption></figure><p>Now, let’s set up our profit equation. We know that profit equals revenue minus costs:</p><ul><li><strong>π = R — C</strong></li></ul><p>And that revenue equals quantity times price</p><ul><li><strong>π = PQ — C</strong></li></ul><p>Now, let’s plug in our cost function, and we’ll swap out both Q’s for 60 — P since we know Q = 60 — P:</p><ul><li><strong>π = P(60-P) — 36 + 2(60 — P</strong>)<strong>²)</strong></li></ul><p>Now we’ll take the derivative with respect to price (this would be a partial derivative, but since we’ve swapped out Q, π is just a function of P, so this is a total first-order derivative). We’ll need to use the chain rule, but note that the value inside the parenthesis equates to 0 once derived:</p><ul><li><strong>dπ/dP = 60–2P + 2*2(60 — P</strong>)</li></ul><blockquote><strong>Hey! </strong>Quick note from Jon. <strong>If you’re a student in econ 101 or 201, you’ll find </strong><a href="https://jonwlaw.medium.com/list/economics-201202-721d3c89e989"><strong>these</strong></a><strong> articles useful</strong> (exact problem walkthroughs like the ones you’ll see on exams). Back to the problem!</blockquote><p>Now let’s expand this:</p><ul><li><strong>dπ/dP = 60–2P + 4(60 — P</strong>)</li><li><strong>dπ/dP = 60–2P + 240 — 4P</strong></li><li><strong>dπ/dP= 300 — 6P</strong></li></ul><p>Now, as part of the FOC, we’ll set this value equal to zero and solve:</p><ul><li><strong>300–6P = 0</strong></li><li><strong>6P = 300</strong></li><li><strong>P = 50</strong></li></ul><p>And we’ll plug that value into our production function to solve for Q:</p><ul><li><strong>Q = 60 — P</strong></li><li><strong>Q = 60–50</strong></li><li><strong>Q = 10</strong></li></ul><p>So, our monopoly equilibrium price is $50, and our quantity is 10. Let’s update our graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gGVVf5TI42aYB97eGH49Jw.png" /><figcaption>Taking the FOC of Profit with Respect to Price to Calculate Monopoly Equilibrium Quantity &amp; Price</figcaption></figure><p>As mentioned, we can take a second-order check to ensure that our point is a maximum and thus that the inverted U-shape in the graph holds (otherwise we wouldn’t be finding an equilibrium quantity, as firms would have an incentive to shifts prices to increase profits):</p><ul><li><strong>dπ/dP = 300–6P</strong></li><li><strong>d²π/dP² = -6</strong></li></ul><p>Since this value is negative, we’ve confirmed that profits curve downward around our point and that it holds as the equilibrium price and quantity.</p><p>Let’s also quickly calculate the marginal cost before moving on, as this is generally handy and will be useful shortly. We’ll take the first derivative of the cost function to do this:</p><ul><li><strong>C = 36 + 2Q²</strong></li><li><strong>MC = 4Q</strong></li></ul><p>Now, given our equilibrium quantity and price, we’ll calculate the consumer and producer surplus. It’s easiest to understand this graphically, so let’s first graph our equilibrium demand function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GlxWFP1c6yB_VZEjElSQBA.png" /><figcaption>Equilibrium Price &amp; Quantity, Consumer/Producer Surplus, and Deadweight Loss in a Monopolistic Market</figcaption></figure><p>We know this to be the case since Q = 60 — P means (0, 60) and (60, 0) are intercepts (and then you can just draw a line to connect those points).</p><p>Now, we’ll add both the <strong>marginal revenue</strong> and the <strong>marginal cost </strong>curves to this graph. We’ll do that because monopoly firms will not set price at the point where demand equals cost; instead they will set price equal to the intersection of their marginal cost and marginal revenue curves.</p><p>We know that marginal revenue is downward sloping since it decreases as the quantity produced increases, and that marginal cost is upward-sloping, since it increases as quantity produced increases.</p><p>Additionally, we previously calculated that marginal cost = 4Q. Let’s quickly calculate marginal revenue as well using the PQ component of our profit equation:</p><ul><li><strong>R = QP</strong></li><li><strong>R = Q(60 — Q)</strong></li><li><strong>R = 60Q -Q²</strong></li><li><strong>MR = dR = 60–2Q</strong></li></ul><p>So, let’s graph both the MC and MR curves using our equations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*REjcEnCudkuz0JioRvO3eQ.png" /><figcaption>Equilibrium Price &amp; Quantity, Consumer/Producer Surplus, and Deadweight Loss in a Monopolistic Market</figcaption></figure><p>Intuitively, note that P = 60 — Q (the D curve), while MR = 60–2Q, so so the MR curve is twice as steep as the Demand curve.</p><p>Next, as stated, the equilibrium price and quantity for a monopolist occurs when MC = MR and shifting vertically upward to the demand curve. We know that this equilibrium occurs at (10,50). Let’s add that point to the graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ff_sAsQkEYmobOZhW_edCg.png" /><figcaption>Equilibrium Price &amp; Quantity, Consumer/Producer Surplus, and Deadweight Loss in a Monopolistic Market</figcaption></figure><p>Now, consumer surplus is the area under the demand curve and above the horizontal price line at 50 over the interval (0,10), producer surplus is the area over the MC line and below the price line, and deadweight loss is the triangle between the monopoly equilibrium output and the competitive equilibrium output which would otherwise occur at the intersection of D and MC curves. We’ll also plot variable cost on the graph, which is the area under the MC curve to the left of the equilibrium quantity value:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ga6MVCw9t5zfArr21-aqdw.png" /><figcaption>Equilibrium Price &amp; Quantity, Consumer/Producer Surplus, and Deadweight Loss in a Monopolistic Market</figcaption></figure><p>Though graphing our consumer surplus, producer surplus, and deadweight loss, we can easily calculate the values by looking at CS and PS as rectangles and right triangles:</p><p>CS is ((60-50)*10)/2, or <strong>50</strong>. For producer surplus, we know the “rectangle” is 50*10, and that the part of the rectangle we want to calculate the area for is PS, not VC. Let’s write that out:</p><ul><li><strong>PS = (50*10) — VC</strong></li></ul><p>We know that C = 36 + 2Q², so variable cost (the cost that changes with quantity) is 2Q²:</p><ul><li><strong>PS = (50*10) — 2Q²</strong></li></ul><p>Since we have the dimensions of the triangle (really, the interval over which we’re solving PS), we can see that Q = 10. Let’s plug that in and solve:</p><ul><li><strong>PS = (50*10) — 2(10)²</strong></li><li><strong>PS = 500–2(100)</strong></li><li><strong>PS = 300</strong></li></ul><p>Finally, to calculate the deadweight loss, we need to figure out the competitive equilibrium. As stated, this occurs when P = MC. We already have both those values from previous calculations, so let’s just set them equal to each other and solve for Q and P:</p><ul><li><strong>P = 60 — Q, MC = 4Q</strong></li><li><strong>P = MC</strong></li><li><strong>60 — Q = 4Q</strong></li><li><strong>60 = 5Q</strong></li><li><strong>Q = 12</strong></li><li><strong>P = 60–12 = 48</strong></li></ul><p>Let’s update the graph with these competitive equilibrium values, and also with the price value of MC = MR:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m71ADQv_4_Fx3mqgB_Q5yw.png" /><figcaption>Equilibrium Price &amp; Quantity, Consumer/Producer Surplus, and Deadweight Loss in a Monopolistic Market</figcaption></figure><p>Now we can easily calculate the area of the deadweight loss (DL):</p><ul><li><strong>((50–40)*(12–10))/2 = (10*2)/2 = 20/2 = 10</strong></li></ul><p>So, we’ve calculated the consumer surplus as 50, the producer surplus as 300, and the deadweight loss as 10 in a monopolistic market.</p><p>Next, we’ll look at this market through competitive dynamics, starting with firm-level competitive supply and cost curves. We’ll use the same production function but change the cost function slightly to C = 36 + 4q².</p><p>In our production function, “Q” is market output (equal to firm output in a monopoly), so we’ll denote “q” as firm-level supply.</p><p>In a competitive market, we know that P = MC, and MC in our revised cost function is 8q (the derivative of 4Q²). So, P = 8q. This means our inverse supply curve (quantity as a function of price) is <strong>q = P/8.</strong></p><p>We’ll graph our MC cost curve, and our <strong>average cost</strong> curve. Average cost is cost divided by quantity:</p><ul><li><strong>AC = (36 + 4q²)/q = 36/q + 4q</strong></li></ul><p>To find the <strong>minumum of the average cost curve</strong>, which is where it intersects the marginal cost curve, we can take the partial derivative of AC with respect to quantity (another FOC, as firms now set quantity instead of price).</p><ul><li><strong>∂AC/∂q</strong> = <strong>-36/q² + 4</strong></li></ul><p>Let’s set this equal to 0:</p><ul><li><strong>-36/q² + 4 = 0</strong></li><li><strong>-36/q² = -4</strong></li><li><strong>-36 = -4q²</strong></li><li><strong>-36/-4 = q²</strong></li><li><strong>9 = q²</strong></li><li><strong>q = 3</strong></li></ul><p>As follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e1EW8Whmyxi6kSXhMFaq_w.png" /><figcaption>Marginal Cost and Average Cost Curves</figcaption></figure><p>A quick note on firm versus market-level supply: we know that inverse firm-level supply is <strong>q = P/8</strong>. To denote market-level supply (Q), we can use summation notation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/462/1*m8UZctWWwhX1HK-1C135mw.png" /><figcaption>Market Supply</figcaption></figure><p>To say that the market-level supply constitutes the summation of all the firm-level supply functions (Q(q)).</p><p>Next, to compute our new (given the revised cost function) long-run competitive equilibrium, we can note that price is equivalent to the minumum of the average cost curve as a function of average cost (basically, we plug 3 into AC).</p><ul><li><strong>P = AC(3) = 36/(3) + 4(3) = 12 + 12 = 24</strong></li></ul><p>So, our equilibrium price is 24. Given our production function P = 60 — Q, P = 60–24 = <strong>36</strong>. This makes our equilibrium price and quantity 24, 36. Let’s graph that out:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wh2ytXbf-WPwxhx352N6Wg.png" /><figcaption>Equilibrium Price &amp; Quantity, Consumer/Producer Surplus in a Competitive Market</figcaption></figure><p>Also note that we know firms produce a quantity of 3 at the minumum of the AC curve and we’ve calculated that the <em>market</em> quantity in the long-run equilibrium is 36. Thus, dividing 36 by 3 identifies <strong>12</strong> as the number of firms in the market.</p><p>We’ll now re-calculate producer and consumer surplus. Like previously, we’ll graph it first by adding the supply curve to the graph. Note that the supply curve, being a competitive market, is equivalent to the supply curve. We already derived each firm’s supply as q = P/8. Multiplying this by 12 (for 12 firms in the market), we get Q = (3/2)P, or <strong>P = (2/3)Q</strong>. This is the market supply, and the equation of the supply (marginal cost ) curve. Plugging the equilibrium quantity (36) into this function yields 24, or the equilibrium price.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/964/1*6SAvCEyn1J8Pc3XKlKeggA.png" /><figcaption>Equilibrium Price &amp; Quantity, Consumer/Producer Surplus in a Competitive Market</figcaption></figure><p>The area under the marginal cost curve is variable cost; we know this is 832 because of (24*36)/2, but for sake of understanding: variable cost per-firm is 4q², and given q = 3, (4)(3)² = 36. Market variable cost is thus 36*12 = 432. Total revenue is QP = 24*36 = <strong>864. </strong>So, R — VC = 864 – 432 = <strong>432 </strong>on the market level, and (3)(24) — 36 = 72–36 = <strong>36 </strong>on the firm level (also 432/12).</p><p>Note that there is no deadweight loss, as this is a perfectly competitive market.</p><p>So, there you go! We’ve run through equilibriums given similar production and cost functions in a monopolistic and competitive market, and calculated welfare in each market setup.</p><p>Hope you found this helpful, and leave any questions in the comments! <strong>Check out my other articles on economics </strong><a href="https://jonwlaw.medium.com/list/economics-36ae74971581"><strong>here</strong></a><strong> (useful for studying, or just learning).</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2998895d3852" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Solving a Simple Signaling Model in Labor Economics (Education & High/Low Productive Workers)]]></title>
            <link>https://jonwlaw.medium.com/solving-a-simple-signaling-model-in-labor-economics-education-high-low-productive-workers-3e3e75979532?source=rss-3364945644f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/3e3e75979532</guid>
            <category><![CDATA[economics]]></category>
            <category><![CDATA[economics-homework-help]]></category>
            <category><![CDATA[economics-homework-helper]]></category>
            <category><![CDATA[microeconomics]]></category>
            <category><![CDATA[economics-assignment-help]]></category>
            <dc:creator><![CDATA[Jon Law]]></dc:creator>
            <pubDate>Sun, 04 May 2025 17:09:49 GMT</pubDate>
            <atom:updated>2025-05-06T20:21:23.086Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nU9We1j5eg87ded3lzPN8Q.png" /><figcaption>Signaling Model in Labor Economics (Education &amp; High/Low Productive Workers)</figcaption></figure><p>As an economics student, or if learning for fun, you’ll often run into signaling problems. I’ll break down a straightforward labor-economics signaling model with education and two types of workers. We’ll work out:</p><ol><li>What wage firms will set without directly observing ability.</li><li>The threshhold share of high-productive workers above which education is not worthwhile.</li><li>How much a worker gains by signaling themselves.</li></ol><p>You’ll see how this model can generate both a pooling and a separating equilibria. Leave any questions in the comments, and let’s get into it:</p><p>Let’s say two types of workers exist in our (theoretical) world. There are <strong>high</strong> productive workers, and <strong>low</strong> productive workers. Each type of worker produces a different <strong>marginal productivity,</strong> which is value of output they produce for the firm they work for.</p><p>High productivity workers have a marginal productivity of 40,000, and low productivity workers have a marginal productivity of 20,000. Workers cannot change their own productivity, and firms cannot <strong>directly observe</strong> the productivity of workers.</p><p>Instead, firms must rely on <strong>signals</strong> about the worker. In our problem, workers can<strong> acquire education</strong> for a cost of 15,000, and education can be a signal to the firm about the quality of the worker.</p><p>The first question we’ll ask is what salary a firm will offer (remember, they can’t distinguish between low and high productivity workers). To solve this, we’ll need to find the expected value of marginal productivity to the firm; firms will pay this exact amount in salary.</p><p>Since expected marginal productivity depends on the number of high quality versus low quality workers, we’ll assign a variable theta (θ) to denote the percent of workers who are high-productivity. We could use another variable to denote low-productivity workers, but we know these proportions must sum to one, so therefore we can denote the proportion of low-productivity workers as 1 - θ.</p><blockquote><strong>Hey! </strong>Quick note from Jon. <strong>If you’re a student in econ 101 or 201, you’ll find </strong><a href="https://jonwlaw.medium.com/list/economics-201202-721d3c89e989"><strong>these</strong></a><strong> articles useful</strong> (exact problem walkthroughs like the ones you’ll see on exams). Back to the problem!</blockquote><p>With that in mind, let’s set up our expected value equation. We’ll multiply θ and 1 - θ by the marginal productivity of those worker types:</p><p><strong>(40,000)(θ) + 20,000(1 - θ)</strong></p><p>In this way, if 10% workers are high productive (θ = .1), then the expected marginal productivity to the firm is 4,000 + 18,000 = 22,000 and so firms will offer this amount as a salary. Let’s expand the equation:</p><p><strong>(40,000)(θ) + 20,000(1 — θ)</strong></p><p><strong>40,000θ + 20,000 - 20,000θ</strong></p><p><strong>20,000θ + 20,000</strong></p><p>So, given some percentage of high-value workers in the marketplace, we can easily calculate the salary paid by the firm.</p><p>Our next question concerns the values of the proportion<strong> </strong><em>θ</em> of highly productive workers for which workers are not incentivized to acquire education.</p><p>This is really asking what salary has to be offered to make getting an education and paying 15,000 worthwhile to a high-productive worker. To solve this, we’ll set the expected values from getting education and not getting education equal to one another, and then solve for theta.</p><p>Let’s start with the expected value (salary) if a worker goes to school. We know that if a worker signals herself as a high productive worker she gets a salary of 40,000, and so if she must pay 15,000 to get this, then she will get 40,000 – 15,000 = 25,000. For the expected salary without going to school, we already calculated that equation above—the 20,000θ — 20,000. Let’s set those equal to eachother and solve:</p><p><strong>20,000θ + 20,000 = 40,000 – 15,000</strong></p><p><strong>20,000θ = 25,000 — 20,000</strong></p><p><strong>20,000θ = 5,000</strong></p><p><strong>θ = 5,000/20,000</strong></p><p><strong>θ = 1/4 = 25%</strong></p><p>To visualize what this means, let’s plug .2 and .3 into the equation:</p><p><em>θ = .2</em></p><p><strong>20,000(.2) + 20,000 = 40,000–15,000</strong></p><p>24,000 &lt; 25,000</p><p><em>θ = .3</em></p><p><strong>20,000(.3) + 20,000 = 40,000–15,000</strong></p><p>26,000 &gt; 25,000</p><p>If theta is <em>under</em> 1/4 such as in our <em>θ = .2 </em>setup, then high productive workers will get an education because the 25,000 they get from signaling themselves as a high value worker even after paying 15,000 beats the 24,000 earned from not acquiring an education.</p><p>However, if theta is <em>over</em> 1/4 like in our <em>θ = .3</em> equation, then the cost of school isn’t justified for high-productive workers.</p><p>(Keep in mind that since education doesn’t change worker productivity, low productivity workers won’t pay for education because they will still get paid 20,000(.2) + 20,000.)</p><p>For the next question, let’s assume that θ = .1:<strong> </strong>what can a high productive worker gain by signaling herself as such? To solve this, let’s simply plug our theta value into the equation and find the salary spread:</p><p><strong>20,000(.1) + 20,000 = 40,000–15,000</strong></p><p><strong>22,000 &lt; 25,000</strong></p><p>So, by signaling as high productive, the high-productive workers will gain <strong>3,000.</strong></p><p>That’s the problem! Note that an equilibrium with signaling is a <strong>separating</strong> equilibrium, since it separates out the workers by type, and an equilibrium without signaling is a <strong>pooling</strong> equilibrium.</p><p>Else, hope you found this helpful! Leave any questions in the comments! <strong>Check out my other articles on economics </strong><a href="https://jonwlaw.medium.com/list/economics-36ae74971581"><strong>here</strong></a><strong> (useful for studying, or just learning).</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3e3e75979532" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>