<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by AlphaNova on Medium]]></title>
        <description><![CDATA[Stories by AlphaNova on Medium]]></description>
        <link>https://medium.com/@alphanova?source=rss-6cf01f00ee8c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*HdY3NgpaeMiUTeTqd2jOsQ.png</url>
            <title>Stories by AlphaNova on Medium</title>
            <link>https://medium.com/@alphanova?source=rss-6cf01f00ee8c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 05:32:53 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@alphanova/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Aliens, Forecasts, and the Shape of Signal Space]]></title>
            <link>https://alphanova.medium.com/aliens-forecasts-and-the-shape-of-signal-space-2c0df6d867ff?source=rss-6cf01f00ee8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/2c0df6d867ff</guid>
            <dc:creator><![CDATA[AlphaNova]]></dc:creator>
            <pubDate>Thu, 04 Sep 2025 12:03:29 GMT</pubDate>
            <atom:updated>2025-09-04T12:12:36.896Z</atom:updated>
            <content:encoded><![CDATA[<p>I’m going to do a remix of <a href="https://alphanova.medium.com/aliens-cross-sectional-financial-forecasts-spherical-codes-and-spherical-cap-packing-188825d22f88?utm_source=chatgpt.com">this earlier article</a>. The feedback I got was: <em>“The preamble made sense, but the math lost me.”</em> That’s fair. This time there will be almost no math. You’ll just have to picture things in your head. My mission here is to educate highly intelligent potential stakeholders who may not be mathematicians.</p><h3>Our Thesis</h3><p>At AlphaNova, we believe there is room for possibly <strong>millions of predictive forecasts</strong>. That’s why the search for new signals — especially the way we’re doing it — is not only warranted but essential.</p><p>One critical point: you don’t want to just <em>average</em> millions of signals. That would be like piling noise on top of signal. A simple example will make this obvious. (And for the record, AlphaNova doesn’t average — we use deep learning to combine forecasts intelligently.)</p><p>Contrary to much of the consensus, we believe <strong>more signals are better</strong>, provided they are managed correctly. We’re in good company here: Renaissance Technologies is widely thought to discover signals with machines and data, not hand-built economic models. Humans simply cannot design millions of forecasts. The space is too vast.</p><h3>Relative Forecasts</h3><p>We focus on forecasting the <em>relative returns</em> of a universe of assets.</p><p>Relative means: how well will each asset perform compared to the average of the others. This is the backbone of long–short trading: long what’s predicted to outperform, short what’s predicted to underperform.</p><p>You can think of capacity in two dimensions:</p><ul><li><strong>Time</strong>: more observations = more degrees of freedom.</li><li><strong>Space</strong>: more assets = richer geometry, and vastly more degrees of freedom.</li></ul><h3>A Stylized Example: The Time Dimension</h3><p>Time is the easier dimension. More observations give you more signals, due to what we might loosely call “seasonality.” The growth is linear.</p><p>Imagine trading BTC daily:</p><ul><li>You hire one analyst who can perfectly predict the return on <strong>January 1</strong>. Every other day, they’re a coin flip. You’re forced to trade $100k nominal every day. One good day, 364 coin flips. Result: a dreadful Sharpe ratio.</li><li>Now suppose you hire <strong>365 analysts</strong>, one for each day. Analyst #1 is perfect on Jan 1, Analyst #2 on Jan 2, and so on. Each is random on the other 364 days.</li></ul><p>If you just averaged them, you’d drown in noise. But if you <strong>selectively listen</strong> to the right analyst on the right day, you’re right every day. In this toy world, your Sharpe ratio is infinite.</p><p>The lessons are clear:</p><ol><li><strong>Time adds linear degrees of freedom.</strong> 365 days = 365 potential signals. Double the days, double the signals.</li><li><strong>Naive averaging is fatal.</strong> What matters is smart, selective ensembling.</li></ol><p>So time is helpful, but the payoff is linear. To see where the exponential elbow room comes from, we need to look at <strong>space</strong>.</p><h3>The Space Dimension</h3><p>Here’s where things get interesting. Because of how relative returns are normalized, the geometry of forecast space depends on the number of assets — and it lives on a sphere. It may not be the everyday sphere you know, but for our purposes, it behaves the same way.</p><h3>The Alien One-Shot Landing Game</h3><p>The Alien One-Shot Landing Game is a thought experiment to explain the geometry of forecast space.</p><ul><li><strong>The setup:</strong> imagine aliens arriving from space. They can sense Earth’s magnetic field but can’t see. Their mothership tells them: “There’s something special about the North Pole,” but not which pole is which.</li><li><strong>The rule:</strong> each alien gets only <strong>one attempt</strong> to land somewhere on Earth.</li><li><strong>The goal:</strong> land as close as possible to the North Pole.</li><li><strong>The analogy:</strong> each alien represents a forecast, and the North Pole represents the true relative returns. An alien that lands close to the Pole is like a forecast that is close to being correct. Aliens landing on the same spot are redundant, and aliens landing on the South Pole are maximally wrong.</li></ul><p>This is more than just a story. The analogy is surprisingly precise from a mathematical perspective: forecasting relative returns really does reduce to a geometry problem on spheres.</p><h3>Playing the Game</h3><p><strong>Version 1: Two assets → two points</strong><br> With just BTC and ETH, forecast space collapses to two poles: North and South.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pxpQ7fNSU_vCGT8l0VICLQ.png" /></figure><ul><li>If one alien hits the North Pole, that’s the perfect forecast.</li><li>Another alien can only pile on top (redundant) or land at the South Pole (perfectly wrong).</li></ul><p>There’s no elbow room for more aliens to do something “different but still good.”</p><p><strong>Version 2: Three assets → a circle</strong><br> Add SOL, and the forecast space opens into a great circle: the union of the 77th Meridian East (running through Delhi, Colombo, Bangalore) and the 103rd Meridian West (passing through Oklahoma City, Monterrey, and into South America).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*axD9qIv4WwxUDTZ-GsALfg.png" /></figure><p>Now the aliens can land anywhere on this circle. Those north of the equator are “closer” to the Pole and thus useful forecasts.</p><ul><li>The distance from the Pole to the equator is about 6,200 miles.</li><li>If aliens must be at least <strong>1,000 miles apart</strong>, you can fit about <strong>12 aliens</strong> on the northern arc.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t5UOPu0huk8gUyg27b_FKg.png" /></figure><p>That’s already a big improvement over the two-asset case: 12 non-redundant forecasts instead of just 1.</p><p><strong>Version 3: Four assets → the whole sphere</strong><br> Add AAVE, and the forecast space becomes the entire globe.</p><p>Now aliens can spread across the northern hemisphere. With the same 1,000-mile spacing, you can fit on the order of <strong>100 distinct aliens</strong>, all reasonably close to the Pole.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8fjId6a8Ens7MaGfw7-X_A.png" /></figure><p>This is the geometric explosion: going from 2 → 3 → 4 assets increases the elbow room not linearly, but superlinearly.</p><h3>And Beyond: Higher Dimensions</h3><p>With 5, 10, or 100 assets, the geometry generalizes to higher-dimensional spheres. These are hard to visualize, but the principle is the same: the “surface area” of these spaces grows dramatically faster than linearly.</p><p>That’s why we say there’s room for millions of forecasts. The geometry itself guarantees it.</p><h3>Why This Matters</h3><p>Accuracy matters — being close to the North Pole is good. But redundancy kills value — multiple aliens landing on the same spot are wasted. The value comes from <strong>diversity</strong>: aliens spread out across the northern hemisphere, each one distinct, each one adding value.</p><p>Time gives you linear growth. Space gives you exponential elbow room. That’s why AlphaNova is building machines to search signal space — and deep learning to combine them intelligently.</p><p>That’s how we move from thousands of forecasts… to millions.</p><p>If you know how to blend these forecasts intelligently and not just take the average, the rewards are absolutely massive. If you’re an investor, let’s have a chat <a href="https://www.alphanovafund.com/">https://www.alphanovafund.com/</a>. If your an aspiring scientist, sign up on our platform <a href="https://www.alphanova.tech/">https://www.alphanova.tech</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2c0df6d867ff" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Aliens, Cross Sectional Financial Forecasts, Spherical Codes, and Spherical Cap Packing]]></title>
            <link>https://alphanova.medium.com/aliens-cross-sectional-financial-forecasts-spherical-codes-and-spherical-cap-packing-188825d22f88?source=rss-6cf01f00ee8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/188825d22f88</guid>
            <dc:creator><![CDATA[AlphaNova]]></dc:creator>
            <pubDate>Mon, 18 Aug 2025 11:51:16 GMT</pubDate>
            <atom:updated>2025-08-21T01:40:08.182Z</atom:updated>
            <content:encoded><![CDATA[<p>#AlphaNova #AI #QuantTrading #Crowdsourcing</p><h3>Preamble</h3><p>Picture this: an alien spacecraft drifts into orbit, filled with beings of astonishing intelligence but utterly without sight. They have no eyes, no images in their minds. Instead, their sensory endowment is tuned differently from ours. Like us, they perceive <strong>gravity</strong> and interact with the <strong>electromagnetic</strong> and <strong>nuclear forces</strong>, but their perception of electromagnetism is split. Unlike humans, they cannot see light; the visual spectrum is lost to them. Instead, they experience a <strong>noisy sense of magnetism</strong> — the other half of electromagnetism — as if a compass needle were buried deep within their nervous system. But this compass is imperfect: it jitters, drifts, and wavers, giving them only a blurred and probabilistic feeling for “true north.”. For them, “north” is not a direction on a map but a feeling in their bones — a compass needle humming faintly in their minds.</p><p>Their compass is terribly noisy. It jitters, wobbles, and drifts, offering only a shaky sense of where true north might be. Each alien interprets this signal differently, some better, some worse.</p><p>When they disembark, they are free to choose their drop point anywhere on the planet — a mountain ridge, a desert plain, a lonely coastline — based on what their magnetic sense suggests. Then their mission begins: take 1000 steps, each step guided by that noisy compass. The challenge is simple to state but devilishly hard to achieve: minimize the average angular deviation from the true north pole over the entire journey.</p><p>Some aliens march with uncanny steadiness, hugging the pole tightly. Others wobble, drifting in wide arcs that betray the noise of their compass. And of course, no two aliens follow exactly the same path: their noisy perceptions create endless diversity in the trajectories they carve across the sphere.</p><p>This whimsical experiment is more than just an alien thought-experiment. It is a parable for our forecasts. Replace aliens with models, the noisy compass with imperfect statistical correlation, and the north pole with ground truth. Each model specifies where to begin, then proceeds to take steps — time steps — always trying to stay “true” on average, though never perfectly. And just as we can cluster aliens by the similarity of their wandering paths, so too can we cluster forecasts by their average correlations.</p><p>In the geometry of this problem, the aliens’ mission and ours become the same: how do you arrange as many noisy, north-seeking paths as possible on the sphere without them collapsing into the same trajectory?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6j6eZWriZLBaiD4xBzWXgg.png" /></figure><h3>Motivation</h3><p>Back to Planet Earth, as it were.</p><p>For the last month or so, I’ve been musing over an interesting problem we at <a href="http://www.alphanova.tech/">AlphaNova</a> face. We run ML/Data Science <a href="https://www.alphanova.tech/competition/competition-august-2025">competitions</a> and want to onboard as many forecasts as possible, subject to three conditions:</p><ol><li>The forecasts aren’t overfit.</li><li>They are predictive with reasonable confidence.</li><li>They are not too correlated with each other.</li></ol><p>This leads to obvious questions: <strong>How many forecasts are even possible?</strong> What’s a good <strong>geometric/topological framework</strong> for representing this problem in a way that respects statistics (at least up to second moments)?</p><p>The setup below addresses these questions. A more formal math paper is in the works, but this note captures the main ideas.</p><h3>Cross Sectional Forecasts</h3><p>Suppose you want to build a cross-sectional forecast of a vector time series of <strong>one-step-ahead returns</strong> for N assets.</p><p><em>(In reality, trading often requires multi-step-ahead forecasts, which complicates things due to autocorrelation and “effective” sample size reductions. For clarity, we ignore that here.)</em></p><p>Why cross-sectional forecasts? They’re a natural way of creating signals for <strong>long/short, dollar-neutral or market-neutral strategies</strong>. Effectively, they forecast <em>relative returns versus an equally weighted index</em> of the assets.</p><p>Let’s fix a historical sample length M. Denote the realized returns as</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/454/1*u-7ZP9XiBcpQYMJv_CfeHQ.png" /></figure><p>where gij is the return of asset j over period [i,i+1].</p><p>Now, we <strong>cross-sectionally z-score</strong> each gij:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/1*9Vb-jCQfkXTWKLdWGcLOSQ.png" /></figure><p>Thus each gi becomes a standardized vector with mean 0 and variance 1 across assets.</p><p>A forecast sequence is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/1*1R31oGl76gxA241cvZ1OcQ.png" /></figure><p>with the same z-scoring</p><p>If we treat fi​ as weights in a long/short portfolio at time i, then the <strong>Sharpe ratio estimate</strong> is (as long as no autocorrelation exists):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*LVsRaDSl8RjHzE0o-PrRFw.png" /></figure><p>i.e. <strong>time-average cross-sectional correlation divided by the time-standard-deviation of correlation</strong>.</p><h3>Spheres and Spherical Codes</h3><p>Because of z-scoring, each fi and gi​ lies on a sphere:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*Bug-tkJWKIBoGigvPeDAEQ.png" /></figure><p>(That is, the vectors live in the hyperplane orthogonal to the all-ones vector intersected with a standard sphere of radius sqrt{N}. The radius sqrt{N} is a detail we can ignore for geometry: assume unit radius without loss of generality.)</p><p>Thus the sequence F is a kind of <a href="https://en.wikipedia.org/wiki/Spherical_code"><strong>spherical code</strong></a>, closely related to <a href="https://en.wikipedia.org/wiki/Spherical_cap"><strong>spherical cap packing</strong></a> in Information Theory and Error-Correcting Codes. The central problem in spherical codes is:</p><p><em>How can we arrange points on a sphere to maximize their minimum pairwise distance?</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/212/1*3EnDtdAfsa6w-IfIh3byCQ.png" /></figure><p>The distance between two unit vectors is related to their dot product:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/502/1*X3zmbYvut9KfscULIaWaFw.png" /></figure><p>And since</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/102/1*o2n02W8sqrpBGQOYerg1ng.png" /></figure><p>is just the cross-sectional correlation, the geometric language maps naturally to statistics.</p><h3>Poor Man’s Parallel Transport via Rotations</h3><p>We want to simplify things further by “centering” the target sequence G.</p><p>Without loss of generality, we can assume g0​ is at the <strong>North Pole</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/316/1*gRGoD6o24icenGkry2vJzg.png" /></figure><p>If g0 isn’t the South Pole, there’s a unique minimal rotation that takes it there, leaving all inner products invariant.</p><p>Then, iteratively:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/862/1*yDLFP8KnV47q-YRHGwFwiQ.png" /></figure><p>At the end, all gi​ map to NP. The forecast sequence is transformed into</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/80/1*H-8mQM8xplShTSc83HW0tA.png" /></figure><p>You can refer to our Alien preamble to see the mapping, no pun intended.</p><p>This preserves:</p><ul><li>Each correlation of fi with gi.</li><li>The time-averaged correlation and its variance.</li><li>Cross-forecast correlations if we had multiple forecasts.</li></ul><p>So we’ve reduced the ground truth to a <strong>constant North Pole reference</strong>, simplifying geometry.</p><h3>Why bother?</h3><p><strong>Visualisation</strong></p><p>In low dimensions N≤4, you can actually visualize forecasts on a 2 dimensional sphere:</p><ul><li>A “good” forecast stays close to NP. Just like the aliens that get it right on average.</li><li>An “excellent” one has <strong>low variance</strong> in radial distance (stable correlation). Just like aliens that march around a noisy circle around the north Pole.</li></ul><p>In higher dimensions, you can’t visualize directly — but this rotation framework still frames what “good” means: average correlation &gt; 0 (upper hemisphere). Adjusting for statistical confidence might shift the threshold northward (e.g. toward the Tropic of Cancer).</p><p><strong>Overfitting</strong></p><p>Even more importantly, this perspective <strong>simplifies the geometry of overfitting</strong>. In <a href="https://alphanova.medium.com/overfitting-part-3-higher-dimensions-06c7ea4bba7b">this earlier note</a>, overfitting was described as taking place within a “fattened-up neighborhood” of the ground truth path. Here, the problem becomes simpler: it is equivalent to solving for a <strong>spherical cap around the North Pole</strong> such that if a model has average correlation below cos⁡(r), then it lies outside the cap and is considered <strong>overfit</strong>.</p><p>This reframing turns overfitting from a fuzzy high-dimensional neighborhood problem of a manifold into a clean <strong>geometric thresholding problem on the sphere from the North Pole</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HLdB69bS_X9KMvAw5O8Qtw.png" /></figure><h3>Expanding to M Copies of the Sphere</h3><p>Finally, instead of thinking of F as M points on one sphere, we can equivalently view it as a <strong>point in the product space</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*a7ES7NFVS923eogHKOK_-Q.png" /></figure><p>Think of this as frames in a crude movie. Reality is in spacetime, we approximate spacetime by a sequence of pictures in pure space.</p><p>This perspective makes it clear that:</p><ul><li>Each forecast is one point in a huge product of spheres.</li><li>Average correlation corresponds to the <strong>inner product in this product space</strong>, normalized by M.</li><li>Clustering and packing questions naturally become <strong>sphere packing problems in high dimension</strong>.</li></ul><p>Here is an example of a cross-sectional forecast for N=4 assets, M=4 timesteps, rotated as in the previous section. We first plot the four observations on a <strong>single sphere</strong>, and then represent the forecast ensemble as a <strong>point in the 4-fold product of spheres</strong>.</p><p>One one sphere:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U63nsSuNFoVDysAdZlVlvw.png" /></figure><p>On a sequence of 4 copies of spheres.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xM2Cey_taAnl0zS4SivjPw.png" /></figure><p>The point of expanding a forecast over M timesteps into a point on an M-fold product of (N−2)-dimensional spheres is that the geometry of interest — the inner product (correlation) inherited from the ambient plain vanilla N-1 dimensional space — behaves very nicely under the product structure. Concretely, the inner product of two such forecast sequences in the product space is just the <strong>sum of the inner products of their components</strong>. Equivalently, the <strong>average correlation</strong> of a forecast with the North Pole (NP) sequence is, up to a factor of 1/M, simply the normalized inner product of the product-space point with NP.</p><p><em>Remark.</em> This M-fold product construction is exact when there is <strong>essentially zero autocorrelation across timesteps</strong>. If forecasts have temporal dependence, the effective dimension is lower (roughly scaled by an “effective sample size”), but the cleanest formulation is in the independent case.</p><h3>The Packing Problem</h3><p>With this reformulation, the problem becomes:</p><blockquote><strong><em>How many points can we pack on the M-fold product of spheres such that:</em></strong></blockquote><ol><li>Each has average correlation ≥r≥0 with the NP sequence, and</li><li>Pairwise average correlation between forecasts is at most H.</li></ol><p>This is a <strong>modified spherical cap packing problem</strong>, generalized to a product of spheres.</p><p>Mathematically, if we let</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/746/1*3vZ7k7sFinx2macOBo_oyw.png" /></figure><p>Then a forecast sequence is a point F:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/240/1*RUNYG6l_vOWiXV8n_Z6Xsw.png" /></figure><p>The similarity measure between two forecasts F and G is :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/260/1*l3jBgR-TDKE6aMTLxeagsw.png" /></figure><p>i.e. the average correlation across timesteps.</p><p>The NP sequence, ie, pure North Pole corresponds to (NP,…,NP)</p><p>Now the problem is: pack as many points in the M fold product of N-2 spheres, ie,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/160/1*oHNO5hr-qo57QbFivMYzRA.png" /></figure><p>as possible subject to (a) correlation with NP ≥r, and (b) pairwise correlation ≤H.</p><h3>Connection to the Classical Problem</h3><p>If M=1, this reduces to the <strong>classical spherical cap packing problem</strong>: arranging points on a single N-2 dimensional sphere subject to correlation constraints and each other.</p><p>If M≫1, the situation becomes more flexible because the ambient space dimension is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/486/1*wX9h-y4LuHOzYJNcdXoMTA.png" /></figure><p>This increase in dimension means there are a lot more degrees of freedom for constructing forecasts is M, or even N grow.</p><h3>Extremal Cases (with explicit N-dependence)</h3><p>Let:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/302/1*6CU2UZJQn2weI5493hDuuQ.png" /></figure><p>be the maximum number of forecasts that can be represented as points in the M-fold product of (N−2)-dimensional spheres,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/392/1*uEU6QvW82WGGUOj6MCfSOA.png" /></figure><p>such that:</p><ol><li>Each forecast has <strong>average correlation with the North Pole (NP)</strong> at least ρ.</li><li>For any two forecasts F1,F2​ in the set, their <strong>pairwise average correlation</strong> is at most H.</li></ol><p><strong>Note on Elbow room:</strong> As N increases, the per-time sphere of dimension N-2 has higher dimension, yielding more geometric “elbow room” to place points subject to the same constraints. (All else equal, larger N increases capacity.)</p><h4>Case M=1</h4><ul><li>If ρ=0 and H=0: then J(N,1,0,0) =N−1, realized by an orthonormal basis sitting in the upper hemisphere. I leave this as an exercist to the reader..</li><li>If ρ=0 and H=1: then J(N,1,0,1) = ∞, since identical forecasts are allowed.</li><li>For 0&lt;H&lt;1: we <strong>conjecture</strong> J(N,1,0,H) is at least twice differentiable in H, <strong>monotone increasing</strong> (J′ ⁣≥0), and <strong>convex</strong> (J′′ ⁣​&gt;0). Fairly sure is true, but we offer no proof.</li></ul><h4>Case M&gt;1</h4><ul><li>If ρ=0 and H=0: J(N,M,0,0) = M (N−1), since the dimension of the ambient linear space is M(N-1) (mutual orthogonality in the product embedding).</li><li>If ρ=0 and H=1: J(N,M,0,1) = ∞.</li><li>For 0&lt;H&lt;1: a <strong>simple linear lower bound</strong> is J(N,M,0,H) ≥ M*J(N,1,0,H)),by replicating a one-sphere construction across the M copies.<br>In practice, because <strong>correlations are averaged across copies</strong> and you are effectively packing in ambient dimension d=M(N−1) capacity typically grows <strong>much faster than linearly in M</strong> (closer to high-dimensional spherical code behavior)</li></ul><p>we will return to this later, in the technical paper, but keep in mind when N=20 and M=10000, the space of forecasts is up to 180000…..and, really, much much larger….</p><h3>📦 How Big Can it Be? — Greedy Packing Estimate</h3><p>To get an intuition for how large the ensemble of admissible forecast vectors can be under a correlation constraint, let’s consider the simplest meaningful case: <strong>20 assets and a single time step</strong> — that is, N=20 and M=1, i.e. just one time stamp observation.</p><p>We ask:</p><blockquote>How many forecast vectors can I construct such that every pair has average correlation no more than H?</blockquote><h3>Greedy Packing Heuristic</h3><p>A common heuristic for estimating how many well-spaced forecast vectors you can pack on a sphere is to treat each as occupying a “no‑go zone” — a spherical cap of a certain angular radius — and then approximate the maximum number by dividing the total sphere area by the exclusion‑zone area. This geometric approach is standard in the sphere‑packing and spherical‑code literature (e.g., see Cohn &amp; Zhao, <a href="https://arxiv.org/abs/1212.5966"><em>Sphere packing bounds via spherical codes</em></a>, 2012):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/1*c8A1nkEw7by4TwiaQ1NnnQ.png" /></figure><p>This works well in <strong>moderate to high dimensions</strong> and when the <strong>correlation threshold is not too tight</strong>.</p><h3>Example: N=20,ρ=0,M=1</h3><p>Here’s the greedy estimate of J(20,1,ρ=0,H) as a function of the allowed pairwise correlation H:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Cas0N0qzZiRmQj2LEh2-tg.png" /></figure><p>As you can see, the number of admissible vectors grows rapidly as H→1. Around H=0.3, the capacity starts ramping up — H=0.5, it’s possible to fit over 1 million models.</p><p>And that is is for just one observation. If the number of observations is large, then clearly the number of possible models is astonishgly large!</p><h3>H-Clustering and Practical Use</h3><p>Setting aside the technical challenge of deriving sharp bounds for J(N,M,ρ,H) when M&gt;&gt;1, one clear implication is that — when both N (number of assets) and M (sample size) are sufficiently large — the number of theoretically possible forecasts is <strong>immense</strong>, far larger than the small handful of component models typically used by practitioners. In this sense, the geometry strongly suggests there is far more “capacity” for diverse forecasts than most people exploit.</p><p>In our framework, the principle is simple: <strong>the more forecasts that are predictive and not overly correlated with each other, the better</strong>.</p><p>Suppose, for example, that in a competition we receive 500 model submissions that are statistically predictive at some confidence level. We then construct clusters of these forecasts (together with our existing library of models) according to the H-threshold condition. In other words, two models fall in the same cluster if their average correlation is at least H. This clustering is not just an abstract data-analytic exercise: it has a <strong>precise geometric meaning</strong> on the M-fold product of (N−2)-dimensional spheres. Each cluster corresponds to a <strong>connected component</strong> in this correlation-threshold geometry.</p><p>From each cluster, we would then select the representative with the <strong>highest Sharpe ratio</strong> as the “winning” forecast. That representative would then be onboarded into our production system.</p><p>This construction naturally raises two practical questions:</p><p><strong>Choosing the optimal H.¹</strong><br> A higher value of H produces more clusters, but also means forecasts across clusters are more correlated. A lower H produces fewer clusters, but with larger separations between them. In practice, one may want to tune H to minimize the variance of equally-weighted portfolio sums of forecasts — balancing diversity with reliability.</p><ol><li><strong>Detecting “desert” regions.</strong><br> Just as a night-time satellite image of the Earth reveals bright cities and dark, sparsely populated deserts, the geometry of our product space will exhibit regions densely filled with models and others left blank. An open problem is how best to detect and exploit these “forecast deserts” — regions of the sphere product far from any existing clusters. Such regions may represent opportunities for constructing entirely new, uncorrelated forecasts.</li></ol><p>Stay tuned for more mind-bending AI-quant stuff that is a bit left field from what you’re used to.</p><p>— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —</p><ol><li>As one AlphaNova scientist on our platform (who is also incidentally an investor) pointed out, picking best Sharpe Ratio representatives from each correlation cluster doesn’t necessarily yield an ensembled best Sharpe. Furthermore, AlphaNova uses a Deep Neural net whose features include several weak forecasts. Therefore, we will be using an algorithm to iteratively add forecasts from each cluster such that a) there is one forecast per cluster, b) the loop picks iteratively the best foreast that incrementally improves Sharpe ratio downstream from the Neural Net output. Through warm startup, parallelization, fine-tuning and <a href="https://arxiv.org/abs/1511.05641">Net2Net</a> methods, this can be made to be feasible from a computational perspective.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=188825d22f88" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Moonshot guess: Harnessing the Universal TOPOLOGY of embeddings.]]></title>
            <link>https://alphanova.medium.com/moonshot-guess-harnessing-the-universal-topology-of-embeddings-166e778c29e4?source=rss-6cf01f00ee8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/166e778c29e4</guid>
            <dc:creator><![CDATA[AlphaNova]]></dc:creator>
            <pubDate>Fri, 23 May 2025 13:15:17 GMT</pubDate>
            <atom:updated>2025-05-23T14:57:44.020Z</atom:updated>
            <content:encoded><![CDATA[<h3>Preserving Topology in Text Embedding Spaces: A Topological Perspective on the vec2vec Method</h3><p>Edit: a lot more needs to be thought about in respect of the “order” defined below. as it stands, the ordering leads to “trivial” topologies. More to follow.</p><p>In recent research on <strong>unsupervised text embedding translation (</strong><a href="https://arxiv.org/pdf/2505.12540">https://arxiv.org/pdf/2505.12540</a>), the <em>vec2vec</em> method has shown how we can translate embeddings between different models without needing paired data, encoders, or predefined matches. While this method was initially celebrated for aligning embeddings from models with different architectures, it also has a potential deeper connection to <strong>topology</strong> that I find particularly fascinating.</p><p>The <em>vec2vec</em> approach preserves the <strong>relative geometry</strong> of embeddings, but what does that mean from a <strong>topological</strong> perspective? It’s not made entirely clear in the paper, but I would *guess” that it would imply that the method preserves the <strong>partial order</strong> of embeddings across spaces, which is a fundamental concept in order theory. The <strong>partial order</strong> induced by the distances between embeddings translates naturally into topological structures, where the embeddings form a <strong>simplicial complex</strong> (a space built from simplices like points, edges, and triangles) based on their relative closeness.</p><h3>Partial Order and Topology: What’s the Connection?</h3><p>If I’m right, by maintaining the <strong>partial order</strong> between embeddings, <em>vec2vec</em> preserves the <strong>topological structure</strong> of the associated partially ordered sets of the embedding spaces. This is where things get interesting. In topology, a <strong>poset topology</strong> (or partial order topology) can be mapped to a <strong>simplicial complex</strong>. When the <em>vec2vec</em> method preserves the partial order between embeddings, it also preserves the underlying topological structure, creating a natural mapping between <strong>simplicial complexes</strong>.</p><h3>The Functor and Structure Preservation: A Deeper Topological Insight</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/917/1*9Fpihd4SrF0R9MUQ13-5FA.png" /></figure><p>Here’s where the fun begins: when applying the <strong>Classifying Space</strong> functor (a tool from algebraic topology and category theory), this partial order preservation translates into a <strong>structure-preserving</strong> mapping between the associated simplicial complexes. While the mapping may not be <strong>bijective</strong> (injective and surjective), it still ensures that the <strong>topological structure</strong> — the way embeddings are connected and related — is <strong>preserved</strong> in the translated space.</p><p>The mapping respects the <strong>relative geometry</strong> and <strong>nearness relationships</strong> between embeddings, which can be understood as preserving some elements of the <strong>topological structure</strong> in a weaker sense. This implies that the translated embeddings share the same <strong>topological properties</strong> as the original embeddings, though they are not exactly the same in a bijective sense.</p><p>It would be very interesting to understand if this mapping is actually a homological or homotopical equivalence. Might also be interesting to understand how universality of embeddings maps to universality of associated topological spaces via the classifying space functor.</p><h3>Implications: A New Lens on Embedding Spaces</h3><p>This topological perspective adds a new layer of depth to our understanding of text embeddings. It shows that the <em>vec2vec</em> method doesn’t just align vectors or preserve semantics; it preserves the <strong>fundamental topological structure</strong> of the embeddings in terms of their <strong>relative ordering</strong> and <strong>closeness</strong>.</p><p>By ensuring that the mapping between these spaces respects the partial order, it guarantees that the embeddings’ <strong>nearness relations</strong> and <strong>semantic structure</strong> are maintained — even if the mapping isn’t strictly one-to-one or onto. This provides a powerful tool for aligning embeddings across models and spaces while retaining their topological essence.</p><h3>Conclusion: A Powerful Intersection of Geometry, Topology, and AI</h3><p>At its core, <em>vec2vec</em> provides more than just a way to translate embeddings across different models. It offers insights into how <strong>geometry</strong> and the <strong>topology of an associated partially ordered set </strong>relate to <strong>preserving semantic relationships</strong>. This intersection between <strong>embedding spaces</strong>, <strong>partial order</strong>, and <strong>topology</strong> is an exciting frontier in AI and natural language processing.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=166e778c29e4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The GodBrain of trading.]]></title>
            <link>https://alphanova.medium.com/the-godbrain-of-trading-c58a6c5a0c82?source=rss-6cf01f00ee8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/c58a6c5a0c82</guid>
            <dc:creator><![CDATA[AlphaNova]]></dc:creator>
            <pubDate>Thu, 27 Mar 2025 11:27:33 GMT</pubDate>
            <atom:updated>2025-05-15T12:48:12.099Z</atom:updated>
            <content:encoded><![CDATA[<p>Let’s talk trading. Here’s the gripe — and we’ll start simple.</p><p>Picture a hedge fund. Two portfolio managers. One asset: BTC. One goes long, the other short. Classic standoff. Now, in theory, this is great — opposing views bring diversification, which hedge funds love. But here’s where it gets messy: if both PMs trade independently, aiming for 1 BTC in either direction, the net position is zero. Nada. Still, the fund eats costs on both sides. Lame.</p><p>Now, a smarter shop? They centralize execution. Net the trades, save on costs, allocate PnL back to each PM. Clean. Efficient. Sensible. That’s how the big boys roll. And yeah, it works.</p><p>But let’s zoom out for a sec.</p><p>Two PMs = two views. Two forecasts. Maybe even two armies of PhDs behind each view. So why are we netting trades — the <em>expression</em> of those views — when we could be aggregating the <em>forecasts</em> themselves?</p><p>Enter: the Eye in the Sky. Or let’s go full sci-fi — the <strong>GodBrain</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BHUFflOgMaMMKGodHxf5MQ.png" /></figure><p>Instead of squashing trades into some Frankenstein portfolio, why not fuse the actual beliefs about the future? Forecasts of BTC, ETH, S&amp;P, oil, rates, volatility — the whole buffet. Combine all that intel into one unified mega-model that dictates what to buy and sell. <em>That’s</em> the game we’re playing.</p><p>Forget the tree of PMs, analysts, and junior desk guys all branching out with their trades. Sure, a centralized trading engine saves some basis points. Cute. But a centralized <strong>intelligence engine</strong>? That’s next-level. Take the forecasts — the raw vision of what’s coming — and ensemble them into a single master prediction. A world model.</p><p>The further upstream you push prediction, the better. Why act downstream on positions when you can align at the forecast level? One massive brain, seeing across all markets, making all the calls. That’s <a href="https://www.alphanova.tech/?utm_source=medium&amp;utm_medium=social&amp;utm_campaign=profile">AlphaNova</a>’s core belief.</p><p>And for those who say trading isn’t about prediction — I love you, but you’re wrong. Every trade is a prediction. Might be explicit, might be buried in some complex options structure, but at the end of the day, you’re putting capital on the line because you believe you’ll make money. That’s a forecast, friend. Embrace it.</p><p>Prediction is the core act. Risk management flows from that. Full stop.</p><p>Now, let’s talk inspiration.</p><p>We love <a href="https://www.rentec.com/Home.action?index=true">RenTech</a>. No PMs. Just scientists, data, code, and one giant system. Beautiful. And we love <a href="https://www.worldquant.com/">WorldQuan</a>t too — they believe in harnessing talent across the planet. Also beautiful. They still use human PMs. We blend both philosophies. One global ensemble of forecasts, powered by human and machine intelligence.</p><p>Because here’s the deal: the future of asset management is machine-driven. No more humans in the loop for the sake of tradition.</p><p>Think about it. We trust 23-year-olds to fly fighter jets. But if you’ve got brain cancer, you’re probably not picking a 25-year-old surgeon over a seasoned 60-year-old. Some tasks require wisdom, repetition, depth. Others demand raw speed. But the sweet spot? That’s where machines come in. Relentless. Scalable. Consistent.</p><p>We’re not anti-human. We’re just pro-GodBrain.</p><p>So yeah, that’s the vibe. Happy to spar, debate, or dive deeper — this is the stuff we live for.</p><p><strong>If you’re a scientist with a keen interest in challenging forecasting problems, come join us at </strong><a href="https://www.alphanova.tech/?utm_source=medium&amp;utm_medium=social&amp;utm_campaign=profile"><strong>AlphaNova</strong></a><strong> and compete for cash and profit sharing rewards.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c58a6c5a0c82" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Death of Gut Feel: Why Machine Learning Crushes Human “Insights” in Finance]]></title>
            <link>https://alphanova.medium.com/the-death-of-gut-feel-why-machine-learning-crushes-human-insights-in-finance-a33e7f9f9b66?source=rss-6cf01f00ee8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/a33e7f9f9b66</guid>
            <dc:creator><![CDATA[AlphaNova]]></dc:creator>
            <pubDate>Fri, 14 Feb 2025 22:35:43 GMT</pubDate>
            <atom:updated>2025-05-15T11:51:37.211Z</atom:updated>
            <content:encoded><![CDATA[<p>For decades, financial pundits have clung to the belief that stock market forecasting requires human intuition — some mystical ability to “understand” the markets. They argue that data-driven machine learning (ML) approaches are doomed to fail because they don’t incorporate “insights” about what’s happening in the world. This argument isn’t just wrong; it’s embarrassingly outdated, the intellectual equivalent of a horse-and-buggy driver sneering at automobiles.</p><h4>The Renaissance Technologies Slap in the Face</h4><p>If data-driven approaches don’t work, someone should probably tell <strong>Renaissance Technologies</strong>, the hedge fund that built the <strong>Medallion Fund</strong>, one of the most successful investment vehicles in history. The Medallion Fund is so absurdly profitable that Renaissance had to close it to outside investors because its compounding wealth was getting out of hand. It has averaged <strong>returns north of 66% before fees and about 39% after fees</strong> over decades — a level of success that should make any finance professor who still preaches efficient market theory weep into their tenure paperwork.</p><p>What’s Medallion’s secret? <strong>Purely mathematical, data-driven forecasts, which, as far as we know, are fed into one master model.</strong> No CNBC talking heads. No gut-feel stock pickers scribbling notes on their napkins over whiskey. Just hardcore statistical analysis, signal detection, and machine learning. Renaissance didn’t succeed by hiring the loudest “market insight” blowhards — it hired physicists, mathematicians, and computer scientists who <strong>buried human intuition beneath an avalanche of data.</strong></p><h4>The Problem With “Human Insights”</h4><p>The biggest problem with the “human insights” approach is obvious: <strong>if your insight is so good, why does everyone already know it?</strong> The moment an idea becomes popular, the market prices it in.</p><p>Human-driven ideas spread like a <strong>virus</strong> in the financial industry. Analysts, traders, and portfolio managers constantly move from shop to shop, taking their insights and models with them. The information doesn’t just spread linearly — it spreads exponentially, as people share it with their networks, teach it to junior analysts, and inevitably publish some variant of it in research notes.</p><p>The simplest mathematical signals — things like moving averages, momentum factors, and basic mean reversion strategies — get devoured by the market almost instantly because they’re easy to understand and easy to replicate. If a strategy can be <strong>written on a napkin over lunch</strong>, it won’t last long in the real world.</p><p>But here’s the key difference: <strong>subtler, more complex signals don’t spread like this.</strong> Deeply nonlinear relationships, intricate combinations of variables, and patterns that require vast datasets and sophisticated processing to detect <strong>don’t travel from firm to firm as easily as a simple moving average crossover strategy.</strong> These complex patterns are exactly what machine learning excels at uncovering — because they don’t fit neatly into human intuition or back-of-the-envelope calculations.</p><p>And that’s why ML-driven strategies continue to dominate. They exploit relationships that aren’t obvious, that don’t go viral among traders, and that can’t be casually passed along at industry conferences. Renaissance Technologies didn’t get rich off of strategies that some Goldman analyst could figure out in an afternoon; they found signals buried under layers of noise — signals that required computational horsepower and statistical rigor to extract.</p><h4>Machine Learning’s Edge: Finding What Humans Can’t</h4><p>Machine learning doesn’t care about what’s obvious. It <strong>finds patterns that human brains are too limited to see.</strong> It identifies anomalies, inefficiencies, and fleeting signals that disappear before anyone on Bloomberg can talk about them. It doesn’t need to listen to the Fed chair’s speech — it models the reaction of markets to previous speeches and <strong>adjusts before you even finish reading the transcript.</strong></p><p>Data-driven approaches have crushed the old ways in:<br>✅ <strong>High-frequency trading</strong> (firms like Citadel and Virtu dominate using ML)<br>✅ <strong>Statistical arbitrage</strong> (where human traders are simply too slow)<br>✅ <strong>Algorithmic portfolio optimization</strong> (far better than seat-of-the-pants strategies)<br>✅ <strong>Sentiment analysis</strong> (because ML processes millions of articles and tweets in seconds)</p><p>Meanwhile, the insight-driven approach has given us <strong>Jim Cramer screaming into a camera</strong>, mutual funds that can’t beat the S&amp;P 500, and billion-dollar hedge funds that collapse under their own bad bets.</p><h4>The Future: Adapt or Die</h4><p>The finance world has already seen this shift: <strong>human traders are dinosaurs, and ML-driven strategies are the meteor.</strong> The most successful funds aren’t run by people making big market calls; they’re run by data scientists and engineers <strong>who let the numbers speak for themselves.</strong></p><p>So the next time some old-school market “expert” scoffs at ML-based trading, ask them: <strong>if human intuition is so great, why are the best funds all run by quants?</strong> Then sit back and watch them scramble to defend their rapidly shrinking relevance.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a33e7f9f9b66" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Hedge Fund Hierarchy and the Optionality Paradox]]></title>
            <link>https://alphanova.medium.com/the-hedge-fund-hierarchy-and-the-optionality-paradox-c738a11faa7e?source=rss-6cf01f00ee8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/c738a11faa7e</guid>
            <category><![CDATA[ai-finance]]></category>
            <dc:creator><![CDATA[AlphaNova]]></dc:creator>
            <pubDate>Thu, 06 Feb 2025 13:26:12 GMT</pubDate>
            <atom:updated>2025-02-06T13:26:12.446Z</atom:updated>
            <content:encoded><![CDATA[<h3>Breaking It Down: The Hedge Fund Machine</h3><p>Hedge funds are these towering, complex, and somewhat absurd structures where money flows in a carefully (or not-so-carefully) engineered system of incentives. There’s a clear hierarchy, but at the top of this food chain is an interesting paradox that keeps me scratching my head. The CIO — the general. The PMs — the colonels. The analysts — the soldiers grinding away, hunting for an edge in a market that is itself a massive, unpredictable beast.</p><p>But here’s the kicker: optionality.</p><p>In a traditional financial setup, you expect some form of counterparty risk. One side wins, the other loses. Someone always bears the downside. But hedge funds? Nope. Somehow, the CIO has the option to nuke a PM out of existence when things go south, and the PMs have an equally glamorous option to rake in a ridiculous windfall when things go right. When they lose? They get cut, but who cares? They’ve already made enough to secure a golden parachute for the next fund they join. It’s an insane equilibrium where both parties somehow walk away with asymmetric upside and someone else (LPs, pension funds, retail exposure through alternative strategies) bears the real brunt of the drawdowns. <strong>This is peak capitalism.</strong></p><h3>The Typical Hedge Fund Setup: A Power Pyramid</h3><p>A hedge fund usually operates under a star system: a big-name CIO, a brand built around their persona, and a cadre of PMs who run different strategies, employing brilliant analyst teams who are basically trying to level up into colonels themselves. It’s a relentless meritocracy, but one built on shifting sands.</p><p>Everyone in this ecosystem wants one thing: to move up. Analysts want to become PMs, PMs want to become mini-CIOs, and CIOs want to become hedge fund gods, running multi-billion-dollar vehicles with absolute control. It’s a system of hierarchy and power consolidation that mirrors the military — except instead of fighting wars, it’s a war on inefficiencies, opportunities, and the ever-fleeting alpha.</p><p>And yet, despite all the intelligence, the PhDs, and the data, a shocking amount of inefficiency remains. The execution layers in many hedge funds are redundant.</p><p>Take this classic example: PM X is long Tech Stock Y. PM Z is short Tech Stock Y. Both are executing trades independently, running up execution costs and market friction that eats into overall returns. Any hedge fund that doesn’t actively resolve these inefficiencies is <strong>burning</strong> money. Some of them do address this by implementing centralization strategies — centralizing execution, margin, and risk management — but many funds still operate under the old model of fragmented decision-making.</p><p>And this is where I think the whole model is fundamentally flawed.</p><h3>The Case for AI Over Humans in Hedge Funds</h3><p>Here’s my take: <strong>PMs are obsolete.</strong></p><p>PMs are highly paid, high-maintenance, high-risk, and often short-lived. If they’re good, they jump ship for more money. If they’re bad, they get blown up. Either way, the hedge fund is caught in a perpetual talent churn.</p><p>Why aggregate and net out PMs’ competing views when you can just eliminate them entirely? <strong>Replace PMs with AI.</strong></p><p>Instead of having human brains making emotional, bias-ridden decisions, use machine intelligence to aggregate forecasts, risk models, and trading strategies. AI doesn’t care about bonuses, isn’t shopping around for better pay, and won’t jump to a rival fund after a good year. It just executes. Relentlessly.</p><p>Here’s what this future hedge fund would look like:</p><ol><li><strong>AI-Driven Asset Allocation</strong> — No more conflicting human PMs making fragmented decisions. A centralized AI aggregates, nets out, and optimizes trades based on real-time data, historical patterns, and predictive analytics.</li><li><strong>Automated Research &amp; Forecasting</strong> — Analysts are still useful (for now) in curating and structuring data, but AI will increasingly handle sentiment analysis, market pattern recognition, and alpha discovery.</li><li><strong>Centralized Execution &amp; Risk Management</strong> — No wasted costs on redundant trades and competing bets. Execution is lean, efficient, and adjusted dynamically based on aggregated risk models.</li></ol><h3>The End of the Star PM Model</h3><p>The traditional star PM model is dying. It’s just too inefficient. The hedge funds that embrace AI and centralization will dominate, and those that cling to the old ways will bleed capital until they’re irrelevant.</p><p>We’re at an inflection point. The future of hedge funds isn’t more glorified bankers playing capitalistic musical chairs — it’s <strong>an intelligence-driven, optimized machine that doesn’t need egos, politics, or excessive compensation to function.</strong></p><p>So yeah, hedge funds are cool. But they could be way cooler if we stopped pretending humans are the best decision-makers in an increasingly complex and data-driven market. Let’s get the colonels out of the way and let the machines take over.</p><h3>Welcome to the future of hedge funds. Adapt or die.</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c738a11faa7e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Overfitting Part 3: Higher dimensions]]></title>
            <link>https://alphanova.medium.com/overfitting-part-3-higher-dimensions-06c7ea4bba7b?source=rss-6cf01f00ee8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/06c7ea4bba7b</guid>
            <dc:creator><![CDATA[AlphaNova]]></dc:creator>
            <pubDate>Tue, 04 Feb 2025 09:16:05 GMT</pubDate>
            <atom:updated>2025-02-05T02:45:40.316Z</atom:updated>
            <content:encoded><![CDATA[<p>Before we begin: join us on <a href="https://discord.gg/s2BGPet9S6">Discord</a>!</p><p>Kicking it up another notch on Overfitting! I’ll keep the math light and make things as intuitive as possible, relying on geometric reasoning. I’ll throw in some mathematical terms here and there, but I’ll skip the heavy details.</p><p>Let’s start with a simple alpha (and you can refer to <a href="https://alphanova.medium.com/overfitting-part-2-how-i-learned-to-stop-worrying-and-love-the-sphere-6790aa01ee8e">Overfitting Part 2</a> for this), which is a forecast that takes the following form:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/246/1*dxKv2nrpjxFvlWbRzgCQKg.png" /></figure><p>The set of zero-residual points for this alpha is the set of points on the sphere such that:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/190/1*HLbsz3IZRkIxcu6U7AXhAw.png" /></figure><p>This means these are the points on the sphere where the model “perfectly predicts” s.</p><p>What I want to highlight is how overfitting depends on the “dimension” of the space occupied by all zero-residual points of a model on a sphere. I’ll wrap up with some interesting potential connections to Kolmogorov Complexity, a way of expressing algorithmic complexity.</p><h3>Recap from Overfitting — Part 2</h3><p>In the previous article, we looked at a simple AR(1) process and graphed its zero-residual points constrained to a 2D sphere (think of a beach ball). We saw that these points formed a pair of circles with the north and south poles removed. Then, we “fattened up” those circles, covering 5% of the sphere’s surface. Here’s a graph for reference:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6yLth-aZ1FQrkt-JP9Fgxg.png" /></figure><p>The key takeaway: Overfit models take up too much space on the sphere.</p><p>If your in-sample data falls outside that fattened-up region, your model is overfit because expanding the covered region further would mean it covers more than 5% of the sphere. That implies your model would perform just as well (in terms of correlation) on more than 5% of randomly generated paths.</p><h3>A Quick Thought Experiment on Dimension and Fattening</h3><p>Let’s visualize this on a 2D sphere (the beach ball). Consider these cases:</p><ul><li>The <strong>North and South Poles</strong> — a 0-dimensional set.</li><li>The <strong>Equator</strong> — a 1-dimensional set.</li><li>If we <strong>fatten up</strong> the poles, we get two <strong>2-dimensional disks</strong> (think polar ice caps).</li><li>If we <strong>fatten up</strong> the equator by including all points within a distance d, we get a <strong>ring</strong> (like the region between the Tropics of Capricorn and Cancer).</li></ul><p>Now, here’s the key insight:</p><ul><li>For a fixed d that isn’t too large, the fattened equator covers <strong>more</strong> of the sphere than the fattened poles.</li><li>Flipping that around: To cover 5% of the sphere, you’d need a <strong>larger d</strong> for the poles than for the equator.</li></ul><p>Imagine two models:</p><ol><li><strong>Pole Model</strong> — Zero residual points at the poles.</li><li><strong>Equator Model</strong> — Zero residual points along the equator.</li></ol><p>It’s easier to overfit with the Equator Model because an in-sample point must land in a much narrower range than with the Pole Model. The safe zone is smaller, so the probability of being outside it (and thus overfitting) is higher.</p><p>Here’s a visualisation fattening of the poles vs fattening the equator covering 5% of sphere:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VA_ScR1mGmkYwINPx_JTnQ.png" /></figure><h3>Overfitting and Dimension</h3><p>This leads to an intuitive claim (though not a formal proof):</p><blockquote><em>The higher the dimension of a model’s zero-residual set, the easier it is to overfit.</em></blockquote><p>Even though both the poles and the equator have zero probability of being randomly selected, their <strong>fattened-up versions</strong> do not. This is the key to understanding overfitting.</p><h3>Extending to AR(k) Models</h3><p>Now, let’s move beyond spheres. Consider a general AR(k) model:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/652/1*S6EO30RGnaDi5tfhTWAObw.png" /></figure><p>For now, let’s not project onto a sphere — we’ll get to that in a bit. Assume we have an in-sample size of m. The zero-residual set is just given by recursion on the above equation and setting the residual to 0.</p><p>This set is determined entirely by:</p><ol><li>The <strong>k-dimensional coefficient vector</strong></li><li>The <strong>k-dimensional initial conditions vector</strong></li></ol><p>For fun, let’s take k=2 and m=5:</p><p>Then, given x1 and x2, we have:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/254/1*4-jmzPNfveeKycMZoRkwhw.png" /></figure><p>Using a mathematical tool called the <a href="https://en.wikipedia.org/wiki/Companion_matrix"><strong>companion matrix</strong></a><strong> </strong>or just brute force algebra, we can express this as a matrix (whose entries are polynomials in the coefficients) times a vector of initial conditions:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*wpTDHKeYaIL8eEZQRMZb3Q.png" /></figure><p>This structure is <strong>linear</strong> in the initial conditions and <strong>polynomial</strong> in the coefficients. Now here’s where things get cool.</p><h3>Algebraic Geometry and Zero Residual Sets</h3><p>Either by appealing to some theorems in <a href="https://en.wikipedia.org/wiki/Real_algebraic_geometry"><strong>Algebraic Geometry</strong></a><strong> </strong>or by some clever first principle methods (which I can explain by pm)<strong>, </strong>we can express all AR(2) zero-residual points as the solution set of polynomial equations in 5-dimensional space.</p><p>This is the equation you get:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/564/1*RGjNLA551OPp8U2I5VTaGw.png" /></figure><p>In our special case of AR(2) in 5D, this is just a determinant=0 equation. In other cases, you have to be a bit more clever to extract the polyomials.</p><p>So a point on the zero residual set should satisfy this equation. Interesting how we magically got rid of expressing this in terms of coefficients?</p><p>Specifically:</p><ul><li>The zero-residual set is an <a href="https://en.wikipedia.org/wiki/Algebraic_variety"><strong>Algebraic Variety</strong></a> (a solution set of polynomial equations).</li><li>It’s kind of like a smooth space (think of a sphere or a torus aka donut), but it can also contain singular (non-smooth) points and self-intersections with lower-dimensional structures.</li></ul><p>We have a set of points in 5D that is defined by one equation. That would make this a 4D space. Very hard to visualise.</p><p>We can, however, project to some lower dimensions to get some insights, or not. Here’s how the AR(2) zero residual set, which is an Algebraic Variety, looks when projected onto a few 3D subspaces (some are indeed quite bizarre):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ww0bITT-ZIBAMPEd8WjfNQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*02_6dDhIajYXdpQnVWdkqA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wj43mXGr6BIE3PLbNadPIw.png" /></figure><p>Notice that it has a geometric structure but also intersections and some singularities, unlike a simple sphere or torus. but at least you can discern that these projections are in fact 2D surfaces, as bizarre as they might look.</p><p>By the way, here are some graphs of well knonw 2D algebraic varieties in 3D space:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CBygii-ptvEM7WKTb4EZeA.png" /></figure><p>Even though there are irregularities and/or intersections we can still talk about dimension. The largest connected piece of our AR(2) variety behaves like a 4D space sitting inside a 5D space. This aligns with our intuition — our coefficient and initial condition vectors span a 4D space, so the solution set should be 4-dimensional.</p><p>If we now <strong>intersect this variety with the unit sphere in 5D</strong>, we reduce the dimension by 1. That means the zero-residual set for in-sample points on the unit 4D sphere in 5D space has <strong>dimension 3</strong>.</p><h3>The General Case</h3><p>A general result holds (which I will not prove but if anyone wants to reach out, can show how its done):</p><ul><li>For an <strong>AR(k) model</strong> on an in-sample of size <strong>m</strong>, the algebraic variety defining the zero-residual set has <strong>2k dimensions if k &lt; m/2, and m otherwise</strong>. This means for k≥m/2, you have basically the TotalEvil predictor (See Overfitting Part 2; it’s the worst case of overfitting).</li><li>When restricted to the <strong>unit sphere of size m-1</strong>, the dimension becomes <strong>2k-1 if k &lt; m/2, and m-1 otherwise, again TotalEvil.</strong></li></ul><h3>The Bigger Picture: Dimension and Complexity</h3><p>Now we tie it back to our earlier argument:</p><ul><li>The <strong>higher k</strong> in AR(k), the <strong>larger the dimension</strong>.</li><li>A higher-dimensional zero-residual set takes up <strong>more space</strong> on the sphere. And by the way, while you have to take care of nasty irregularities, you can still fatten up an Algebraic Variety.</li><li>This mirrors the fattening-up argument from earlier, reinforcing the idea that <strong>higher-dimensional (in our geometric sense) models overfit more easily.</strong></li><li>Lots of parameters doesn’t, prima faci , imply higher dimensional zero residual sets; there could be a lot of collapsing/compressing, so you can’t just count the number of parameters. Conceptually, if you did have some compression it might suggest you have some visceral fat in your code in the way of parameters.</li></ul><p>Interestingly, this reasoning is reminiscent of <a href="https://en.wikipedia.org/wiki/Kolmogorov_complexity"><strong>Kolmogorov Complexity</strong></a>, which measures the shortest length algorithm needed to generate an output. Here’s a table comparing computing AR(k) zero residual set in terms of Kolmogorov Complexity and dimension of Zero Residual Set on the m-1 sphere. KC is dominated by the cost of storing the initial conditions and coefficients. And that is also clearly what drives the dimension of the Zero Residual set.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/958/1*RecwiyvqHTlfj7VXuAFxOg.png" /></figure><p>There might be a deeper equivalence between these ideas — perhaps overfitting is, in some sense, a function of the <strong>compressibility of a model’s zero-residual set</strong>. Perhaps Entropy can be brought in somehow.</p><p>And that’s a wrap! Hope this gives you a fresh perspective on overfitting, geometry, and complexity.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=06c7ea4bba7b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Overfitting Part 2: How I Learned to Stop Worrying and Love the Sphere]]></title>
            <link>https://alphanova.medium.com/overfitting-part-2-how-i-learned-to-stop-worrying-and-love-the-sphere-6790aa01ee8e?source=rss-6cf01f00ee8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/6790aa01ee8e</guid>
            <dc:creator><![CDATA[AlphaNova]]></dc:creator>
            <pubDate>Mon, 27 Jan 2025 03:37:50 GMT</pubDate>
            <atom:updated>2025-01-27T03:37:50.935Z</atom:updated>
            <content:encoded><![CDATA[<p>Ok, we’re going to kick up my ramblings a notch or two. I’ll try to keep the math side as light as possible so that an intelligent reader with some geometry and stats chops can follow along.</p><p>You’re given a time series of length MM.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/270/1*nEzOCOKU1OaQNPikvarTCw.png" /></figure><p><strong>Your mission:</strong> Build a simple alpha where you optimize some utility funciton in-sample. Using any or all information up to time i, your alpha predicts the next value in the time series at time i+1. By “simple alpha,” I mean the only information you can use is the time series itself — no other time series allowed. I’m sticking to this restriction for clarity; everything here generalizes to more complex alphas.</p><p>So here’s what you’ll do: chop the time series into at least two (but possibly three) sections — a training (in-sample) section, and either a testing section or both a validation and testing section. For what follows, we don’t care whether you’ve got one or two extra sections. What’s crucial is that you <strong>don’t peek</strong> at these sections during the process. Ideally, a gatekeeper should stop you from looking at the testing section entirely, and you definitely shouldn’t cheat by peeking at the validation section to tweak your model or its parameters for the training set. Let’s assume your in-sample section has length m:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/270/1*VuaaX94EHQsrtX6E1H6vOg.png" /></figure><h3>Utility Function U</h3><p>You’re utility function U measures goodness of fit. I don’t want to right down some complex mathematical expression for the domain of this function, but we can slop around and say it’s U(p,x) where p is your prediction time series and x is your in_sample. We’ll assume U is “scale invariant.” This means if you multiply your predictions or time series by any constant, U remains unchanged. (Lead-lag correlation has this property, as does Sharpe Ratio, the latter of which you can either prove this from first principles or just note that Sharpe Ratio is a function of correlation, and functions of scale-invariant metrics are also scale-invariant.) For simplicity, let’s roll with correlation. I note that in practise what one would do is minimize the variance of the residual, but in our case that will have the nice property of optimizing correlation.</p><h3>Detrending and Normalizing</h3><p>A common practice is to detrend and normalize your in-sample time series. If you’re worried about “peeking into the future” in-sample, don’t be — as long as you don’t apply the same treatment to the validation or testing sets. Sure, you could detrend and normalize on an expanding window basis, but that complicates things unnecessarily for this exposition.</p><p>After detrending and normalizing your in-sample​, its length is now 1:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/270/1*EoVIkGAWDJuVCGjqiOBfNA.png" /></figure><p>This means in-sample​ now “lives” on a high-dimensional unit sphere of dimension m−1. Here’s the definition of a unit n-sphere:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/1*AtGfWS0cgMFIBF4_cciEBQ.png" /></figure><p>The 0-sphere consists of two points, which you can think of as the north and south poles. The 1-sphere is the familiar circle. The 2-sphere is a beach ball. And it keeps going: spheres all the way down (or up). They’re beautiful, symmetrical objects. You can rotate from any point to any other point on a sphere. Other than dimension 0, the n-sphere is connected, and for dimensions &gt;2, the complement of an n-1 sphere sitting inside an n-sphere is also connected (this follows by Alexander Duality: <a href="https://en.wikipedia.org/wiki/Alexander_duality">https://en.wikipedia.org/wiki/Alexander_duality</a>). For dimension 2, though, there are two connected components (think north and south of the equator). The list of cool facts about spheres goes on and on — Hopf fibrations, exotic 7-spheres, you name it.</p><h3>The Simple Alpha</h3><p>Since you’ve detrended and normalized your xinxin​, your simple alpha is now a function of the form:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/318/1*Uq9fOuR2XwJja-qR_6Lg4g.png" /></figure><p>Here E is some parameter space and we do note that the range isn’t necessarily on the m-1 sphere, but we can always project it to this m-1 sphere (there is some math involved in this as it relates to some of the ideas below).</p><p>Here’s the catch: the i+1-th component of F can only depend on components up to i. This restriction makes things interesting. If you simulate trillions of Monte Carlo samples of i.i.d. standard Gaussian normals of size m, then normalize each sample to have unit length, you’ll get points uniformly distributed on the m−1-sphere. From both a geometric and measure-theoretical perspective, this distribution is perfectly “democratic”: every point is as good as any other. The probabilty measure is just the standard uniform measure on the sphere.</p><p>But your alpha? It imposes restrictions. It’s carving out a subset of the sphere where your in-sample, and alpha, is “special,” hopefully allowing you to predict the next value in the series with minimal error. Now imagine you’re given this magical point:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/1*lRWqopvlNpbCpCSKLEe2eQ.png" /></figure><p>This point is special — we’ll call it AwesomeX. If you’re lucky enough to have AwesomeX, you wouldn’t need to think hard to build a perfect alpha. But odds are, your in_sample​ isn’t AwesomeX — it’s noisy, random, and crummy. And because of the functional restrictions on F, you can’t just rotate the alpha for AwesomeX to fit your crummy in-sample​.</p><h3>TotalEvil: The Overfitting Monster</h3><p>Let’s now build the most ridiculously overfitted simple alpha model, which I’ll call TotalEvil:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/674/1*OLeYFlHhdSsOnsQN5zLLqg.png" /></figure><p>Note that the parameter space of TotalEvil IS just the entire m-1 sphere, looked at as parameters.</p><p>TotalEvil has some wildly evil properties:</p><ol><li>It can be calibrated to perfectly fit in_sample​.</li><li>It can also perfectly fit <strong>any</strong> point on the sphere.</li><li>The range of possible correlations or Sharpe Ratios it produces by varying parameters is massive — correlations from -1 to 1, Sharpe Ratios from −∞ to ∞.</li></ol><p>Point 2 is the textbook definition of overfitting. TotalEvil isn’t predicting anything; it’s just contorting itself to fit whatever data you throw at it. It’s not even really using information from the in-sample​; it’s just fitting a meaningless “past.”</p><p>Point 3 (and I’m simplyfying), is a linked to a method used by many practitioners and is framed as a hypothesis testing, whose null is the model’s sharpe is really just bad, say, zero. It mitigates signal monkeys from just changing parameters till they get a good Sharpe.</p><p>Our approach is more aligned with Point 2, as we shall see.</p><h3>Back to semi Reality: A Toy Example</h3><p>To make this concrete, suppose your in-sample size is 3. This means your data lives on a 2-sphere. Consider an AR(1) model:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/216/1*0cLz5Ic1vPUpZLC2NdrpXw.png" /></figure><p>The points that perfectly fit an AR(1) form two circles (1-spheres) on this 2-sphere, excluding the poles (apologies for it showing some gap near the poles).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/328/1*QyPZXB2IYbKSyg0kWjLliA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/554/1*6cj-EPXe8Rv1e01R_9KsNA.png" /></figure><p>Minimizing residual variance here is equivalent to finding the closest point on these circles to your in-sample. Now let’s fatten these circles so their total area is 5% of the sphere.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/504/1*zNoj-w1tnSA99KvpCXm8RQ.png" /></figure><p>If your in_sample​ lies outside this fattened region, we’d say your model is <strong>overfit</strong>. Why? Because covering your in-sample​ would require fattening the circles even more, taking up too much space on the sphere, ie, more than 5% of the sphere’s area. Now, notwithstanding the fact that you probably wouldn’t worry about AR(1) models overfitting and would use other methods to check the suitability of AR, overfitting is a statement about models taking up too much space on the sphere <em>and about the structure of the in_sample.</em></p><h3>Final Thoughts</h3><p>Here’s the gist: <strong>overfit models</strong> take up too much space on the sphere. If your in-sample point lies outside a reasonably fattened “perfect” point set, the model is overfit. This geometric approach to overfitting, while toy-like in this example, generalizes beautifully. Once you compute the fattened region for a given model, it’s reusable, and beyond the beauty of the math, it’s got practical value too.</p><p>That’s it!</p><p>P.S. Much thanks to Guolong Li for the original concept, from which I went down a deep rabbit hole!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6790aa01ee8e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Backtesting]]></title>
            <link>https://alphanova.medium.com/backtesting-b541e01e309f?source=rss-6cf01f00ee8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/b541e01e309f</guid>
            <category><![CDATA[backtesting]]></category>
            <category><![CDATA[quant-finance]]></category>
            <category><![CDATA[quantitative-finance]]></category>
            <dc:creator><![CDATA[AlphaNova]]></dc:creator>
            <pubDate>Sun, 26 Jan 2025 03:24:46 GMT</pubDate>
            <atom:updated>2025-01-26T03:24:46.849Z</atom:updated>
            <content:encoded><![CDATA[<p>So, not sure if I ever mentioned this, but I hold a Ph.D. in backtesting — with a focus on overfitting. This illustrious degree was unofficially granted during the years 2011 to 2014 when I invested an absurd amount of time tinkering on platforms like Quantopian and QuantConnect. I tested thousands of ideas, optimized parameters, and watched pnl curves morph into something aesthetically pleasing. And let me tell you, I wasn’t alone. I’ve seen brilliant individuals, with actual Ph.D.s from top-tier universities, do the exact same thing right in front of me. It’s mesmerizing — staring at a screen, waiting for backtests to finish, only to scream at the heavens when the results disappoint, and then starting over. It’s eerily reminiscent of those casino regulars glued to slot machines for hours on end.</p><p>That said, there seems to be a growing chorus of voices criticizing backtesting itself lately. Ironically, many of these detractors have likely never run a backtest in their lives. It’s a dangerous trend, akin to the proverbial taxi driver waxing poetic about Bitcoin or the inexplicable hatred children once had for Barney the dinosaur. Remember that? “We hate Barney!” — a bizarrely viral phenomenon.</p><p>Let’s be real: if you’re developing a trading strategy or signal, you <em>have</em> to backtest. There’s no way around it. You can’t spend countless hours refining a model, mulling over its philosophical implications — “Is it causality? Does it capture market dynamics?” — and then skip straight to production without ever running a backtest. Even the legendary Jim Simons, in his characteristically understated way, once said, “You try different things and see how they worked on historical data.” Translation? You backtest. Call it research or call it something else — semantics don’t change the necessity. Imagine a scientist proclaiming, “STOP! DO NOT TEST YOUR THEORY — testing isn’t research!” Absurd, right? (Well, except for geology or palaeontology, where live testing isn’t exactly feasible.)</p><p>Here’s the nuance: there’s a critical distinction between an alpha/signal/prediction and a trading strategy — and the way you backtest each is different.</p><p>An alpha is a model or function that predicts some future outcome. For instance, it might forecast the next hour’s average temperature at a weather station. A trading strategy, on the other hand, is a more complex construct. It combines one or more alphas with additional rules like thresholds, position sizing, and other parameters. My advice? <strong>Do not try to optimize everything at once</strong> — alphas, thresholds, sizing, and the whole strategy — right out of the gate. That’s like taking an untested car to the racetrack without first validating its engine or tires. Start with the alphas. Test their predictive power and robustness against overfitting. Treat them as modular components — refined jewels you can reuse in various contexts. Only once you’ve nailed that should you evaluate the broader strategy.</p><p>Let’s keep it simple and focus on a univariate alpha — a prediction of a single asset’s return (relative, not absolute). In finance, the goal isn’t necessarily to match the prediction to the outcome perfectly. What matters is alignment — how closely the prediction vector correlates with the outcome vector. In other words, you want a small angle between those vectors, or high correlation (cosine similarity). This is especially true if your utility function is the Sharpe Ratio, which on an unscaled basis is just the ratio of expected value to standard deviation (where s is the alpha and r is the return):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/324/1*xFDMjyoSqlQaGZKKvgwWcw.png" /></figure><p>Most machine learning models minimize the residual variance (the squared distance between predictions and targets). Conveniently, that also maximizes the correlation between predictions and outcomes — boosting the Sharpe Ratio. For zero-mean alphas and returns (assuming Gaussian distributions — cue the groans and yes, you’re right this won’t be valid for nasty asset classes), the link between Sharpe Ratio and lead-lag correlation can be expressed as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/324/1*dL5vcoIvUJsclYNxAWdonQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/658/1*2rfodVHKc0psp3i8UhZC5Q.png" /></figure><p>Backtesting an alpha, in essence, involves pretending you’re trading nominally proportional to the alpha and computing its Sharpe Ratio, potentially visualizing cumulative pnl. It’s just a way to measure the signal’s lead-lag correlation with the outcome. This principle holds across domains — whether you’re trading asset prices or forecasting hourly temperatures.</p><p>And yes, visualization matters. Consider pnl curves with the same Sharpe Ratio. They might share identical statistical properties (Sharpe ~0.173, scaled to root 250 ~2.7), yet look wildly different.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/676/1*AUr7rV7dDyYSzQurZq2Ndw.png" /></figure><p>Clearly, Sharpe alone doesn’t tell the whole story.</p><p>But that’s a discussion for another day.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b541e01e309f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cointegration]]></title>
            <link>https://alphanova.medium.com/cointegration-d1eb1371014d?source=rss-6cf01f00ee8c------2</link>
            <guid isPermaLink="false">https://medium.com/p/d1eb1371014d</guid>
            <category><![CDATA[quant-finance]]></category>
            <category><![CDATA[machine-learing]]></category>
            <category><![CDATA[statistical-arbitrage]]></category>
            <dc:creator><![CDATA[AlphaNova]]></dc:creator>
            <pubDate>Fri, 24 Jan 2025 12:20:25 GMT</pubDate>
            <atom:updated>2025-01-24T12:20:25.088Z</atom:updated>
            <content:encoded><![CDATA[<p>Let’s chat about this arcane beast called <strong>cointegration</strong>. Back in the day, it was all the rage — still makes appearances on the CVs of noobs bragging about their “cointegration strategy” on stock pairs or baskets.</p><p>Confession: I used to be one of those noobs. Let me break it down in plain English, though, because this thing is an academic’s fever dream of complexity.</p><p>I once explained it to a super sharp hedge fund manager — guy loved it. Can’t say the metaphor is 100% mine; I cobbled it together from random blogs, but it clicks.</p><p>Here’s the gist: Picture a strategy where you trade one asset long and another short, banking on the idea that the pair will eventually snap back to some mean.</p><p>Now, that sounds like basic correlation, right? Nope. Not even close — at least not if you’re the kind of math-obsessed nerd who takes stats seriously. Some pairs might be super correlated mathematically but not cointegrated, and vice versa. The math isn’t hard, but the nuance? That’s the real trick.</p><p>So here’s the metaphor: Imagine two nerds in a park, magically connected like quantum-entangled weirdos. Every time one makes a move, the other “senses” it and adjusts, even if they’re 100 meters apart. Over time, no matter where they wander, you can kinda predict their next steps relative to each other. Call it “correlation of moves.” It’s how some market relationships work — like how the S&amp;P 500 is oddly correlated with AUD/JPY. Risk-on, risk-off vibes.</p><p>Now flip the script: Two drunks, same park, but they’re chained at the ankles with some elastic rope. Their moves are a mess — zero correlation. Pure randomness. But here’s the kicker: After hours of staggering around, they’re still close to each other. That’s <strong>cointegration</strong>.</p><p>In finance, the purest examples of cointegration are dual-listed assets (same stock on different exchanges) or cash vs. futures. Everything else? Cointegrated until it’s not. The moment that chain snaps — watch out. But calling something “temporarily cointegrated”? Yeah, no.</p><p>For truly cointegrated pairs, like NKY futures across exchanges, it’s less about fancy math and more about optimizing servers and maybe dabbling in physics. Fun fact: Some dude (Harvard guy, I think) even got a patent for applying <strong>Special Relativity</strong> to trading arbitrage on dual listings. Wild.</p><p>On my end, I got swept up in the hype — built pipelines to scan for pairs, test for temporary cointegration, calibrate Ornstein-Uhlenbeck processes, all that jazz. Got geeky with Dickey-Fuller tests, even tried flipping Bayes’ Theorem to invert p-values and estimate the probability a pair <em>isn’t</em> cointegrated. Cool in theory. In practice? A rabbit hole of overfitting. And yep, there’s that word again: overfitting.</p><p>Look, going long some assets and shorting others isn’t a bad trading style. It’s my vibe. But whether you’re chasing correlation, cointegration, or some ranking scheme, beware: these strategies are <strong>negatively skewed</strong>. That means you’ll rake in small wins most of the time, then get clobbered by rare, massive losses. Facts.</p><p>It’s funny how nerds get tribal over this stuff. Correlation folks vs. cointegration folks vs. ranking folks — it’s like analog vs. digital, geometry vs. computer science. At the end of the day? All of it works. Until it doesn’t.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d1eb1371014d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>