<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Nanfei Jiang on Medium]]></title>
        <description><![CDATA[Stories by Nanfei Jiang on Medium]]></description>
        <link>https://medium.com/@figo.nf.jiang?source=rss-a93dc23da208------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*eMYlamnk11ueAFMT7bbwdQ.jpeg</url>
            <title>Stories by Nanfei Jiang on Medium</title>
            <link>https://medium.com/@figo.nf.jiang?source=rss-a93dc23da208------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 00:08:22 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@figo.nf.jiang/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Can One Geometric Idea Explain All Convex Duality?]]></title>
            <link>https://medium.com/@figo.nf.jiang/can-one-geometric-idea-explain-all-convex-duality-ff0932ae8ca2?source=rss-a93dc23da208------2</link>
            <guid isPermaLink="false">https://medium.com/p/ff0932ae8ca2</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[convex-optimization]]></category>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[duality]]></category>
            <dc:creator><![CDATA[Nanfei Jiang]]></dc:creator>
            <pubDate>Sat, 18 Jul 2026 03:16:32 GMT</pubDate>
            <atom:updated>2026-07-18T03:16:32.730Z</atom:updated>
            <content:encoded><![CDATA[<p>In the first article, we started from a simple geometric fact:</p><blockquote><em>A closed convex set can be represented as the intersection of all halfspaces that contain it.</em></blockquote><p>That idea already gives us a different way to think about duality. Instead of treating duality as a bag of formulas, we can treat it as a change of language. A convex object can be described internally, by its points, or externally, by the linear inequalities that support it.</p><p>This article pushes that viewpoint further.</p><p>Many people first meet convex duality through the Lagrange dual problem. But if we start there, several things feel arbitrary. Why do inequality multipliers have a sign restriction? Why do equality multipliers have no sign restriction? What is the relationship between Lagrange duality, dual cones, support functions, and Fenchel conjugates?</p><p>Textbooks answer these questions, but often only after introducing several definitions that initially seem unrelated. The result is elegant, but the route can feel opaque.</p><p>Here is the claim I want to make:</p><blockquote><em>In convex optimization, many dual objects are different expressions of the same geometric move: describe a convex object from the outside by its supporting halfspaces.</em></blockquote><p>We will start with convex sets, because that is where the picture is cleanest.</p><h3>Starting From Convex Sets</h3><p>Recall the definition of a convex set. A set C ⊆ ℝⁿ is convex if for any two points x, y ∈ C, the whole line segment between them remains in C:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*6FS38m2OQu2ycZeEblcn9Q.png" /></figure><p>This is the internal description of convexity. It talks about points inside the set.</p><p>But the supporting hyperplane theorem gives us another description. A closed convex set can also be represented by all the hyperplanes that support it. In other words, instead of listing points in C, we can list all the linear inequalities that are valid for C.</p><p>This external description is often more useful.</p><p>Imagine an ellipse in the plane. Choose a unit vector u. Now move a line perpendicular to u toward the ellipse from far away. Stop when the line just touches the ellipse. That line is a supporting line.</p><p>It has two pieces of information:</p><ul><li>its normal direction u;</li><li>its offset from the origin.</li></ul><p>If we repeat this for every direction u, the collection of supporting lines recovers the entire ellipse.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*hvXmnDckWB3LZuKsHXMY3g.gif" /></figure><p>There is a small technical issue: the contact point may not be unique. But the supporting line itself is still easy to describe. A hyperplane with normal vector u can be written as</p><p>uᵀx = a</p><p>For a fixed set C and direction u, the largest possible value of uᵀx over x ∈ C gives exactly the offset of the supporting hyperplane. This leads to the support function.</p><h3>The Support Function</h3><p>The support function of a set C is defined by</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*D-ErI7L6qw4tbKD4bhQV9g.png" /></figure><p>Geometrically, h_C(u) tells us how far C extends in the direction u. Once h_C(u) is known, the supporting inequality in direction u is</p><p>uᵀx ≤ h_C(u)</p><p>If we know this inequality for every direction u, then under the usual closedness and convexity assumptions, we can recover the set:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*hLNxekqYVVyK1HvDbq2OBg.png" /></figure><p>This is already a dual representation. The set C has been encoded by a function over directions. Instead of studying points x in the original space, we study linear probes u.</p><p>The support function also has useful algebraic properties.</p><p>First, it is positively homogeneous:</p><p>h_C(αu) = αh_C(u), for all α ≥ 0</p><p>Second, Minkowski sums turn into ordinary sums:</p><p>h_{C+D}(u) = h_C(u) + h_D(u)</p><p>where</p><p>C + D := {x + y : x ∈ C, y ∈ D}</p><p>Third, convex hulls of unions and intersections have simple support-function bounds:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*_JRPlSMYhM0VGLFvYJSvmg.png" /></figure><p>and</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*nJa-T-dbEKWhsmOmAFPy9w.png" /></figure><p>There is also a differential interpretation: if the supporting point in direction u is unique, then h_C is differentiable at u, and its gradient recovers that supporting point.</p><p>The main point is not the list of identities. The main point is that a geometric object has been converted into a functional object. This is one of the basic patterns of convex duality.</p><h3>Why This Is Useful</h3><p>The support-function view gives us a natural way to approximate complicated convex sets.</p><p>Suppose we want to solve</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*mJlFsjE0n4p-vcfpKncRKA.png" /></figure><p>If C is a closed convex set, then C can be described by all inequalities of the form</p><p>uᵀx ≤ h_C(u)</p><p>Of course, using all directions u is usually impossible. But we can sample finitely many directions uᵢ and replace C by an outer polyhedral approximation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*Pfff6vouTNIcW_6a4K6FyA.png" /></figure><blockquote>The feasible region is now a polytope described by finitely many affine inequalities. We have approximated a potentially complicated convex set from the outside by selecting finitely many supporting halfspaces.</blockquote><p>This is the same geometric mechanism again: points are hard, supporting inequalities are easier.</p><h3>Dual Cones</h3><p>Another common dual object in convex analysis is the dual cone.</p><p>Once we have the support-function viewpoint, the definition of a dual cone becomes much less mysterious.</p><p>A convex cone is a convex set that is also closed under nonnegative scaling. If K is a convex set and</p><p>αK ⊆ K for all α ≥ 0</p><p>then K is a convex cone.</p><p>Important examples include the nonnegative orthant ℝⁿ₊, the cone of positive semidefinite matrices, completely positive cones, and copositive cones.</p><p>Now ask the same question as before: what happens if we describe a cone using supporting hyperplanes?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/854/1*hMXQEtt20u2X14UbtZhg7Q.gif" /></figure><p>Because a cone extends to infinity, its support function has a special form. In a direction u, either u gives a valid supporting inequality for the cone, or the supremum shoots to infinity. In the convention used here, we define the dual cone as the directions that are nonpositive on K:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*pKSwa1kyrgj75o7-iJNM5g.png" /></figure><p>Then the support function of K is</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*3LKwX0LTNmpxxqD8qKlJhA.png" /></figure><p>Some books use the opposite sign convention and define the dual cone by uᵀx ≥ 0. Both conventions describe the same geometry with opposite normal directions. What matters is consistency.</p><p>Under this viewpoint, the dual cone is not an arbitrary definition. It is simply the set of normal directions whose supporting halfspaces contain the cone.</p><p>This is why the definition is useful: it records which linear functionals are valid certificates for the cone.</p><h3>Where Lagrange Multiplier Signs Come From</h3><p>This geometric picture also explains a question that often feels like a memorization rule:</p><blockquote><em>Why do inequality constraints impose sign restrictions on their multipliers, while equality constraints do not?</em></blockquote><p>Consider the usual constrained problem</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*uVUHPZ2Lcl5rMuCidVgEGQ.png" /></figure><blockquote>Depending on the sign convention for the Lagrangian, you may see the inequality multipliers written as λ ≥ 0 or λ ≤ 0. These are not two different theories. They are the same theory written with opposite signs.</blockquote><p>The conic view makes the rule systematic.</p><p>Suppose the constraint is written abstractly as</p><p>ℓ(x) ∈ K</p><p>Then the corresponding dual variable must live in the dual cone:</p><p>s ∈ K∗</p><p>That is the real rule. The familiar multiplier signs are just special cases.</p><p>For the inequality constraints fᵢ(x) ≤ 0, write the vector constraint as f(x) ∈ ℝᵐ₋</p><p>where ℝᵐ₋ is the nonpositive orthant. Under the nonpositive dual-cone convention above,</p><p>(ℝᵐ₋)∗ = ℝᵐ₋</p><p>So the multiplier satisfies λ ∈ ℝᵐ-, that is, λ ≤ 0</p><p>If instead your textbook defines the Lagrangian with the opposite sign, the same statement appears as λ ≥ 0. The geometry has not changed; only the orientation of the normal vector has changed.</p><p>For equality constraints, write</p><p>h(x) ∈ {0ₚ}</p><p>The set {0ₚ} is also a convex cone, and its dual cone is the entire space:</p><p>{0ₚ}∗ = ℝᵖ</p><p>Therefore the equality multiplier satisfies</p><p>μ ∈ ℝᵖ</p><p>which means μ is unrestricted.</p><p>So the rule is not:</p><blockquote><em>inequality multipliers have signs, equality multipliers do not.</em></blockquote><p>The better rule is:</p><blockquote><em>dual variables live in the dual cone of the constraint set.</em></blockquote><p>The familiar signs λ ≥ 0, λ ≤ 0, and μ free are just what this general statement becomes in simple cones.</p><h3>The Unifying Picture</h3><p>We can now summarize the story.</p><p>A convex set can be described from the inside by its points. But it can also be described from the outside by its supporting halfspaces. The support function stores those halfspaces as a function of direction. A dual cone records which directions define valid supporting inequalities for a cone. Lagrange multiplier sign constraints are the same idea applied to constraint cones.</p><p>This gives us a much cleaner view of why these definitions appear together:</p><ul><li>A support function describes a convex set by its supporting inequalities.</li><li>A dual cone describes a cone by the linear functionals that are valid on it.</li><li>Lagrange multipliers are linear certificates attached to constraint violations.</li><li>The sign or feasibility constraint on a multiplier comes from the dual cone.</li></ul><p>These are not disconnected definitions. They are different faces of the same external-description principle.</p><p>The next step is to apply the same idea not only to sets, but to convex functions. That is where Fenchel conjugates enter the picture. A convex function can be studied through its epigraph, and its conjugate describes the affine lower bounds that support it.</p><p>That will connect this geometric picture to Fenchel duality.</p><p>*<em>References</em>*</p><p>[1] Stephen Boyd and Lieven Vandenberghe. <em>Convex Optimization</em>. Cambridge University Press, 2004.</p><p>[2] LittleO. “Please explain the intuition behind the dual problem in optimization.” Mathematics Stack Exchange, 2012.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ff0932ae8ca2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Is Duality in Convex Optimization, Really?]]></title>
            <link>https://medium.com/@figo.nf.jiang/what-is-duality-in-convex-optimization-really-1820be5d6f6f?source=rss-a93dc23da208------2</link>
            <guid isPermaLink="false">https://medium.com/p/1820be5d6f6f</guid>
            <category><![CDATA[duality]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[convex-optimization]]></category>
            <category><![CDATA[mathematics]]></category>
            <dc:creator><![CDATA[Nanfei Jiang]]></dc:creator>
            <pubDate>Wed, 15 Jul 2026 17:02:36 GMT</pubDate>
            <atom:updated>2026-07-18T03:19:20.942Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KU4eZNaKqzDz2jYxj3W4Vg.png" /></figure><p>If you have taken a course in operations research, convex optimization, or machine learning, you have probably seen the word <em>duality</em> many times.</p><p>In linear programming, dual variables often come with an economic interpretation: shadow prices, market-clearing prices, or marginal values of resources. In constrained convex optimization, the first thing we often do is write down a Lagrangian and derive a dual problem. In machine learning, the support vector machine is one of the classic examples where the dual formulation is not just mathematically elegant, but computationally useful.</p><p>There are also game-theoretic interpretations: solving a primal-dual pair can be viewed, in certain settings, as finding the equilibrium of a zero-sum game.</p><p>So duality is everywhere. But this is exactly what makes it confusing.</p><p>Many students learn that duality is useful before they understand where it comes from. They know how to write down a Lagrangian. They know that the dual problem gives a lower bound. They may remember the KKT conditions. But the deeper question remains unclear:</p><p><em>Why do so many different mathematical objects have something called a “dual”?</em></p><p>Dual variables, dual cones, support functions, Fenchel conjugates, Lagrange duality, strong duality, weak duality. These ideas appear in different chapters of a textbook, often with different definitions and motivations. At first they look like a collection of clever tricks discovered independently.</p><p>The purpose of this series is to argue that they are not independent tricks. At least in convex optimization, they all point back to one geometric idea:</p><p><em>A closed convex set can be represented as the intersection of all halfspaces that support it.</em></p><p>This article is the first step toward that viewpoint.</p><h3>What Does “Dual” Mean?</h3><p>Before talking about convex optimization, it is worth asking what the word <em>dual</em> means in mathematics more broadly.</p><p>There is no single universal definition that covers every use of the word. It is often better to treat duality not as one specific theorem, but as a mathematical principle: a way of seeing the same object from two different perspectives.</p><p>Roughly speaking, a duality appears when a mathematical object can be represented in a different form, and that different form is useful enough to reveal structure that was hidden in the original representation.</p><p>One schematic version looks like this. Suppose we are studying an object X in some mathematical class V. The object might be a vector, a function, a set, a measure, or something else. If there is an operation D that maps V to a dual class V*, and D(X) captures the same underlying information in a structurally useful way, then we may think of D(X) as a dual representation of X.</p><p>In many classical examples, the transformation also has a strong reversibility property. Applying the dual operation twice brings us back to the original object, possibly after identifying the two spaces in a natural way:</p><p><em>applying duality twice gives us back X.</em></p><p>This is the familiar involutive flavor of many dualities. The condition is not the whole story, but it captures the feeling that a dual object is not merely a random transformation of the original. It is another representation of the same mathematical content.</p><p>Examples are everywhere:</p><ul><li>A vector space and its space of linear functionals.</li><li>A function and its Fourier transform.</li><li>A convex function and its Legendre-Fenchel conjugate.</li><li>A convex cone and its dual cone.</li><li>A primal optimization problem and its dual problem.</li></ul><p>These objects look very different, and it would be too ambitious to force all of them into one simple definition. But the shared pattern is clear:</p><p><em>Duality gives us a second language for describing the same object.</em></p><p>The important question in convex optimization is therefore not “What is the formula for the dual problem?” The better question is:</p><p><em>What is the object being described, and what are the two languages used to describe it?</em></p><h3>The Standard Story: Lagrange Duality</h3><p>In a typical convex optimization course, duality is introduced through the Lagrangian.</p><p>Consider a constrained convex optimization problem:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*tR0T5XLKnNy_t4DiG_-74g.png" /></figure><p>Here f and each fᵢ are convex functions, and each hᵢ is affine. We introduce Lagrange multipliers λ in ℝᵐ and μ in ℝᵖ, and define the Lagrangian:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*YJTPzoA78hukDPgCdGIYPA.png" /></figure><p>The usual dual function is obtained by minimizing the Lagrangian over x:</p><p><em>g(λ, μ) is the infimum over x of L(x, λ, μ).</em></p><p>Then the Lagrange dual problem is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*ufirNokJATWIP9aDTNyfHw.png" /></figure><p>The standard theory tells us several important facts.</p><p>First, every dual feasible point gives a lower bound on the primal optimum. This is weak duality: for every λ ≥ 0, the dual value is no larger than the primal optimum.</p><p>Second, under suitable regularity conditions, such as Slater’s condition, strong duality holds: the optimal dual value equals the optimal primal value.</p><p>Third, under appropriate assumptions, primal and dual optimality can be characterized by the KKT conditions.</p><p>This is a powerful framework. It is also the version of duality that many students encounter first.</p><p>But if this is where the story begins, several things feel mysterious.</p><p>Why does the Lagrangian have exactly this form? Why do we add the constraints through inner products with multipliers? Why must the multipliers for inequality constraints satisfy λ ≥ 0, while the multipliers for equality constraints are unrestricted? Why do we need a regularity condition such as Slater’s condition before strong duality can be guaranteed? What is the relationship between this dual problem and other “dual” objects, such as dual cones or Fenchel conjugates?</p><p>Textbooks answer these questions, but often not in the order in which intuition wants them answered. A definition appears, then a theorem, then a proof. Several chapters later, the definitions begin to connect. The final structure is elegant, but the scaffolding has been removed.</p><p>The goal of this series is to put some of that scaffolding back.</p><h3>A Different Starting Point</h3><p>The Lagrange dual problem is important, but I do not think it is the most primitive way to understand duality in convex optimization.</p><p>The more primitive object is not the optimization problem itself. It is the convex set.</p><p>Convex optimization is built on the geometry of convex sets. Feasible regions are convex sets. Epigraphs of convex functions are convex sets. Cones are convex sets with additional scaling structure. Even convex functions can be studied through their epigraphs.</p><p>So if we want to understand why duality appears so naturally in convex optimization, we should first ask:</p><p><em>What is the dual way to describe a convex set?</em></p><p>Here is the central geometric fact:</p><p><em>Every closed convex set can be represented as the intersection of all closed half-spaces that contain it.</em></p><p>Equivalently, a convex set can be recovered from its supporting hyperplanes.</p><p>This is the geometric source of many dual constructions in convex analysis. Lagrange multipliers, dual cones, support functions, and Fenchel conjugates all become less mysterious once this picture is in place.</p><h3>Convex Analysis as “Linear Algebra with Inequalities”</h3><p>Stephen Boyd has a useful way of describing convex analysis: it is like linear algebra, except that equality is replaced by inequality.</p><p>This is not just a slogan.</p><p>Recall the definition of a convex function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*2Th5tWQZguHuagTxrL3hmQ.png" /></figure><p>If we replace the inequality sign by equality, then the function becomes affine. In that sense, convexity is what happens when linearity is relaxed into an inequality.</p><p>The same idea appears for sets. In linear algebra, we often study solution sets of linear equations such as:</p><p><em>aᵢᵀ x = bᵢ.</em></p><p>The solution set is an affine subspace: a line, a plane, a hyperplane, or a higher-dimensional affine object. These objects are rigid. They can often be described using relatively few parameters.</p><p>Convex analysis replaces equalities by inequalities:</p><p><em>aᵢᵀ x ≤ bᵢ.</em></p><p>Each inequality no longer describes a hyperplane. It describes a halfspace. A halfspace is much less rigid than a hyperplane: it keeps one side and discards the other.</p><p>Now the intersection of many such halfspaces can produce rich geometric objects. The intersection of finitely many halfspaces gives a polyhedron. The intersection of infinitely many halfspaces can describe far more general convex sets.</p><p>This is the point where convex geometry begins to look like duality.</p><p>A convex set can be described internally, by listing its points. But it can also be described externally, by listing all the halfspaces that contain it.</p><p>Those are two different languages for the same object.</p><h3>Supporting Hyperplanes</h3><p>The key theorem is the supporting hyperplane theorem.</p><p>Let C be a nonempty closed convex set in ℝⁿ, and let x₀ be a boundary point of C. Then there exist a nonzero vector a and a scalar b such that:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*Sw2nzkm8OstnIuubXcz0MA.png" /></figure><p>The hyperplane defined by aᵀ x = b is called a supporting hyperplane of C at x₀.</p><p>Informally, a supporting hyperplane is a hyperplane that touches the convex set without cutting through it. The whole set lies on one side of the hyperplane, and at least one boundary point lies on the hyperplane.</p><p>This theorem says that at every boundary point of a closed convex set, there is at least one such supporting hyperplane.</p><p>Now comes the more important representation result:</p><p><em>A nonempty closed convex set is the intersection of all closed halfspaces that contain it.</em></p><p>In symbols, we can write the idea as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*W2VgAED5biAp2kYG3OpMBw.png" /></figure><p>Here H_C denotes the collection of supporting halfspaces associated with C.</p><p>This formula is the geometric heart of the story.</p><p>It says that a convex set has two equivalent descriptions:</p><ul><li>The internal description: all points x in C.</li><li>The external description: all linear inequalities that support or contain C.</li></ul><p>The second description is dual in spirit because it replaces points by linear functionals. Instead of asking “Which points are in the set?”, we ask “Which linear inequalities are valid for the set?”</p><p>That change of perspective is the root of duality in convex optimization.</p><h3>Why This Explains the Feeling of Duality</h3><p>The external description of a convex set is not just a different notation. It changes what becomes easy to see.</p><p>Suppose a set is complicated internally. Its boundary may be curved, and listing all of its points may be impossible. But if we describe it through supporting halfspaces, the geometry becomes linear: each supporting halfspace is defined by a vector a and a scalar b.</p><p>This is why linear functionals keep appearing in convex analysis. They are the measuring devices used to probe convex sets from the outside.</p><p>The support function makes this idea explicit. For a set C, define h_C(u) as the supremum of uᵀ x over all x in C. Equivalently, h_C(u) tells us how far the set extends in direction u:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*c1AVnItIjM-Bhc57dWcGdg.png" /></figure><p>For each direction u, this gives a halfspace containing C. If we know this inequality for every direction u, then under suitable closedness and convexity assumptions, we can recover C.</p><p>This is a dual representation: the set is encoded by a function over directions.</p><p>Once this perspective is in place, many later definitions become more natural:</p><ul><li>A dual cone collects all linear functionals that are nonnegative on a cone.</li><li>A Lagrange multiplier is a linear weight assigned to a constraint violation.</li><li>A Fenchel conjugate describes a convex function through its affine lower bounds.</li><li>Strong duality asks when the external supporting description is tight enough to recover the original optimum exactly.</li></ul><p>These are not identical constructions, but they share the same geometric DNA.</p><h3>Returning to the Lagrangian</h3><p>With this viewpoint, the Lagrangian looks less arbitrary.</p><p>A constrained convex optimization problem asks us to minimize a function over a feasible set. The feasible set is described by inequalities and equalities. When we form the Lagrangian, we are not merely adding artificial penalties. We are constructing linear certificates that separate feasible behavior from infeasible behavior.</p><p>The sign constraint λ ≥ 0 is also no longer a random rule. Inequality constraints define an order. To preserve valid lower bounds, the multipliers must respect that order. Later, when we discuss cones, this will become the statement that dual variables live in the dual cone.</p><p>Similarly, equality constraints have unrestricted multipliers because an equality can be viewed as two inequalities pointing in opposite directions, or as membership in a subspace whose dual has no positivity restriction.</p><p>This is why I prefer not to introduce convex duality as a bag of formulas. The formulas are correct, but they are downstream of the geometry.</p><p>The primitive fact is this:</p><p><em>Convex objects can be studied through the linear inequalities that support them.</em></p><p>The Lagrangian is one of the most powerful ways this fact enters optimization.</p><h3>What Comes Next</h3><p>This article only sets up the viewpoint. The main claim is that duality in convex optimization is not a collection of unrelated tricks. It is rooted in the external representation of convex sets by supporting halfspaces.</p><p>In the next part, we will make this viewpoint more concrete by starting with convex sets themselves. We will look at support functions, dual cones, and how the familiar sign constraints in Lagrange duality arise naturally from conic duality.</p><p>The larger goal is to build a unified picture:</p><p><em>Duality is the art of replacing a convex object by the family of linear measurements that support it.</em></p><p>Once that idea is clear, the many versions of duality in convex optimization start to feel like different expressions of the same geometry.</p><h3>References</h3><p>[1] Boyd, S., &amp; Vandenberghe, L. (2004). <em>Convex Optimization</em>. Cambridge University Press.</p><p>[2] LittleO. (2012, October 29). <a href="https://math.stackexchange.com/questions/223235/please-explain-the-intuition-behind-the-dual-problem-in-optimization">Please explain the intuition behind the dual problem in optimization</a>. Mathematics Stack Exchange.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1820be5d6f6f" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>