<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sgsahoo on Medium]]></title>
        <description><![CDATA[Stories by Sgsahoo on Medium]]></description>
        <link>https://medium.com/@sgsahoo77?source=rss-97e4e79c842b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*OSrXAT4Ed6cvClU3</url>
            <title>Stories by Sgsahoo on Medium</title>
            <link>https://medium.com/@sgsahoo77?source=rss-97e4e79c842b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 10:35:08 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sgsahoo77/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Navigating the Pareto Frontier: Statistical and Low-Resource Techniques for Model Selection]]></title>
            <link>https://medium.com/@sgsahoo77/navigating-the-pareto-frontier-statistical-and-low-resource-techniques-for-model-selection-f9427f624cae?source=rss-97e4e79c842b------2</link>
            <guid isPermaLink="false">https://medium.com/p/f9427f624cae</guid>
            <category><![CDATA[pareto-principle]]></category>
            <category><![CDATA[sustainable-development]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Sgsahoo]]></dc:creator>
            <pubDate>Sun, 01 Nov 2026 20:45:36 GMT</pubDate>
            <atom:updated>2026-11-01T20:45:36.711Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction: The Triad of Model Selection in Constrained Environments</h3><p>The deployment of machine learning architectures in real-world scenarios frequently collides with the harsh realities of limited resources. While classical machine learning paradigms assume the existence of massive, pristine datasets and unlimited computational budgets to tune hyperparameters, modern applications — ranging from personalized biomedical diagnostics to embedded edge computing — operate under severe constraints. When data is scarce, unreliable, and computationally expensive to process, the traditional approach of executing grid searches over standard cross-validation splits becomes not only inefficient but mathematically unsound.</p><p>In these constrained environments, model selection and hyperparameter optimization (HPO) evolve from a simple quest for maximum accuracy into a complex, multi-objective optimization problem. Practitioners must balance a delicate triad of competing objectives, forming a multi-dimensional Pareto frontier:</p><ol><li><strong>Model Performance:</strong> The raw predictive power of the model, typically measured by out-of-sample accuracy, area under the receiver operating characteristic curve (AUROC), or mean squared error (MSE).</li><li><strong>Model Reliability:</strong> The trustworthiness and stability of the model’s predictions. This includes resistance to overfitting, robustness against data leakage, and the calibration of its uncertainty, ensuring the model accurately quantifies its own ignorance.</li><li><strong>Resource Efficiency:</strong> The computational cost required to search for, train, and deploy the model. In this context, resources denote processing power, memory footprint, floating-point operations (FLOPs), and energy consumption, rather than merely human time.</li></ol><p>Constructing a model that excels across all three axes is impossible; pushing the boundaries of performance generally requires highly complex models that devour computational resources and risk overfitting on small datasets, thereby destroying reliability. Consequently, the goal of modern low-resource HPO is to map the Pareto frontier — the set of optimal trade-offs where no single objective can be improved without degrading another.</p><p>This comprehensive report explores the statistical, algorithmic, and probabilistic techniques necessary to navigate this frontier. It dissects methods for mitigating optimism bias in small samples, bypasses expensive resampling using advanced information criteria, establishes rigorous reliability through Conformal Prediction, and operationalizes Multi-Objective Bayesian Optimization (MOBO) to navigate hyperparameter spaces under strict computational budgets.</p><h3>1. The Erosion of Reliability: Data Scarcity and the Leakage Problem</h3><p>Before attempting to optimize a model’s performance or resource efficiency, one must secure its reliability. When sample sizes shrink (e.g., N &lt; 1,000), the variance in any statistical estimator expands dramatically. In such regimes, the greatest threat to a model’s position on the reliability axis is data leakage, a phenomenon that artificially inflates perceived performance and leads to catastrophic failures upon real-world deployment.</p><h4>1.1 The Mechanics of Data Leakage</h4><p>Data leakage occurs when information from outside the training subset inadvertently influences the model training or hyperparameter selection process. In small datasets, a model will exploit any leaked signal to memorize the validation set, creating an illusion of high predictive performance, an effect termed “optimism bias”.</p><p>Leakage manifests in several insidious forms across the machine learning pipeline:</p><ul><li><strong>Preprocessing and Feature Selection Leakage:</strong> The most frequent error in pipeline construction is applying transformations — such as standard scaling, missing value imputation, or feature filtering — across the entire dataset prior to splitting it into cross-validation folds. For instance, if an Analysis of Variance (ANOVA) F-test is used on the full dataset to select the top 100 features before executing a K-fold cross-validation, the test folds are no longer truly independent. The feature selection mechanism has already “peeked” at the test targets. This guarantees an upwardly biased accuracy metric, even if the underlying features consist entirely of random noise.</li><li><strong>Subject Overlap and Grouped Dependencies:</strong> In biomedical datasets, multiple data points are frequently extracted from a single entity (e.g., multiple MRI slices from one patient, or repeated longitudinal measurements). If naive row-wise randomized splitting is utilized, samples from the same patient will populate both the training and testing folds. The model subsequently learns to identify the patient’s unique biological signature or the specific batch characteristics of the laboratory equipment (batch confounding), rather than learning the generalized pathology.</li><li><strong>Hyperparameter Leakage:</strong> Hyperparameter tuning represents a meta-learning process. If a single validation set is repeatedly queried to evaluate hundreds of hyperparameter configurations, the optimization algorithm will eventually identify a configuration that perfectly exploits the random noise inherent in that specific validation set.</li></ul><p>To preserve the reliability axis of the Pareto frontier, all data preprocessing, feature selection, and model tuning must be strictly nested within the training loop of the cross-validation architecture.</p><h3>2. Statistical Resampling in the Low-Data Regime</h3><p>When managing scarce data, evaluating model performance and selecting hyperparameters demands sophisticated resampling frameworks. Standard K-fold cross-validation relies on splitting data into K subsets then training on K-1 folds, and testing on the remaining fold. While effective for moderately sized datasets, it degrades under severe constraints. If the overall dataset is extremely small, holding out 20% of the data (as in 5-fold CV) starves the learning algorithm of vital information, resulting in a model that performs poorly and a pessimistic evaluation metric. Conversely, utilizing the cross-validation score to select hyperparameters introduces the optimism bias discussed previously.</p><h4>2.1 Nested Cross-Validation</h4><p>To decouple hyperparameter optimization from performance estimation, practitioners must employ Nested Cross-Validation. This approach involves two distinct, hierarchical loops.</p><p>The outer loop divides the total dataset into K_{outer} folds, reserving one fold exclusively as a pristine, unseen test set for the final evaluation. The remaining data is passed to the inner loop, which splits it again into K_{inner} folds. The entire hyperparameter search space is explored strictly within this inner loop. The configuration that achieves the highest average performance across the inner folds is selected, trained on the complete inner dataset, and then evaluated exactly once on the outer holdout fold.</p><p>This ensures that the final performance metric is completely untainted by the hyperparameter selection process. However, Nested CV is incredibly hostile to the resource axis of the Pareto frontier. If a practitioner executes a 5x5 nested CV across a grid of 100 hyperparameters, it requires 2,500 model training cycles. When compute is heavily restricted, this approach becomes fundamentally non-viable, necessitating alternative statistical estimators.</p><h4>2.2 Bootstrapping and the 0.632+ Estimator</h4><p>To maximize data utilization in highly constrained environments bootstrapping provides an elegant alternative to cross-validation. Bootstrapping involves drawing N samples with replacement from the original dataset of size N to form a training set. Because sampling is conducted with replacement, the probability of any specific instance not being selected in a single draw is 1–(1/N). Over N independent draws, this probability converges to approx 0.368</p><p>Therefore, a typical bootstrap sample contains approximately 63.2% of the original unique observations. The remaining 36.8% of the dataset forms the “Out-of-Bag” (OOB) sample, providing a natural, unseen validation set.</p><p>While the OOB error provides a good estimate of generalization, it is often overly pessimistic because the model is trained on only $63.2\%$ of the available diversity. To balance this, the <strong>0.632 bootstrap estimator</strong> calculates a weighted average of the highly optimistic training error (resubstitution error) and the pessimistic OOB error.</p><p>Despite its elegance, the 0.632 estimator fails catastrophically when applied to highly complex models capable of perfect memorization, such as deep neural networks, unpruned random forests, or 1-Nearest Neighbor classifiers. If a model perfectly memorizes the training data, Err_{train} = 0, pulling the entire Err_{0.632} estimate downward and re-introducing extreme optimism bias.</p><p>To solve this, Efron and Tibshirani formulated the <strong>0.632+ bootstrap estimator</strong>. The 0.632+ estimator dynamically adjusts the weighting coefficient by calculating a “relative overfitting rate,” denoted as R. The relative overfitting rate compares the observed overfitting to the maximum possible overfitting that could occur if the features and targets were completely independent (the no-information error rate).</p><p>If the model is not overfitting at all (R = 0), the weight w returns to 0.632. If the model heavily overfits (R ~ 1), the weight w approaches 1, discarding the useless Err_{train} and relying entirely on the out-of-bag error. This makes the 0.632+ estimator the definitive choice for maximizing both performance estimation accuracy and reliability when sample sizes are exceptionally small, particularly in clinical biomedical tasks where securing additional data is impossible [cite: 26, 276–282].</p><h3>3. Bypassing Resampling: Compute-Free Validation via Information Criteria</h3><p>When the Pareto frontier is tightly constrained by computational resources — such as optimizing models directly on edge devices, IoT sensors, or systems operating on battery power — training hundreds of bootstrap or cross-validation models is untenable. In these extreme low-resource environments, practitioners must turn to statistical model selection criteria. These criteria bypass empirical resampling entirely, estimating the out-of-sample prediction error analytically by penalizing a model’s in-sample fit based on its mathematical complexity.</p><h4>3.1 AIC, BIC, and the Complexity Penalty</h4><p>The Akaike Information Criterion (AIC) is derived from information theory and offers an asymptotically unbiased estimator of the relative Kullback-Leibler (KL) divergence between the true, unknown data-generating process and the fitted candidate model. It is formulated as:</p><p>AIC = -2 ln(L) + 2k</p><p>where L is the maximized likelihood of the model given the data, and k is the number of estimated parameters. By seeking to minimize the AIC, the practitioner balances the model’s goodness-of-fit (represented by the likelihood) against a penalty for complexity (represented by 2k). Crucially, AIC has been mathematically proven to be asymptotically equivalent to leave-one-out cross-validation (LOOCV). It provides the statistical rigor of LOOCV at a fraction of the computational cost, completely eliminating the need for iterative model retraining.</p><p>The Bayesian Information Criterion (BIC) operates under a different paradigm. While AIC seeks the model that minimizes future predictive error (even if the true model is infinitely complex), BIC assumes that the true data-generating model exists within the candidate set. It applies a much harsher penalty that scales logarithmically with the sample size $N$:</p><p>BIC = -2 ln(L) + k ln(N)</p><p>As N grows, BIC aggressively suppresses complexity, acting as a powerful safeguard against overfitting, though occasionally at the cost of underfitting subtle, valid patterns in the data.</p><h4>3.2 The Small-Sample Correction: AICc</h4><p>Despite their theoretical elegance, standard information criteria falter in low-data regimes. The derivation of AIC relies on asymptotic properties; when the ratio of the sample size to the number of parameters (N/k) is small, the standard 2k penalty severely underestimates the true complexity risk. Consequently, standard AIC exhibits a strong negative bias in small samples, actively promoting over-parameterized models that fit the noise.</p><p>To resolve this, Hurvich and Tsai formulated a small-sample correction, known as the <strong>AICc</strong>. By analyzing linear regression under the assumption of normally distributed errors, they proved that adding a non-linear adjustment rectifies the bias:</p><p>AICc = AIC + {2k(k+1)} / {N — k — 1}</p><p>The behavior of this correction term is profound for the reliability axis. When k is small relative to N, the denominator (N — k — 1) is large, and the term approaches zero, allowing AICc to converge back to standard AIC. However, as the number of parameters k begins to approach the sample size N, the denominator shrinks rapidly toward zero. This causes the penalty term to explode toward infinity. The mathematical structure acts as an impenetrable wall, strictly forbidding models that possess too many parameters relative to the available data.</p><p>A standard heuristic dictates that AICc should unconditionally replace AIC whenever the ratio N/k &lt; 40. For practitioners building models on microcontrollers, computing the AICc allows for instantaneous model selection without the energy drain of cross-validation.</p><h4>3.3 The Failure of Information Criteria in Deep Learning</h4><p>While AICc is exceptionally efficient for generalized linear models (GLMs), autoregressive time-series, and shallow trees, it faces systemic failure when applied to Deep Neural Networks (DNNs).</p><p>The failure originates in the definition of k. Traditional statistics assumes that each parameter represents an independent degree of freedom. Deep learning, however, is heavily over-parameterized and characterized by complex symmetries. For example, in a network utilizing Rectified Linear Unit (ReLU) activations, one can multiply the incoming weights to a neuron by a constant c and divide its outgoing weights by c; the mathematical function computed by the network remains completely unchanged.</p><p>Because multiple distinct parameter configurations map to the exact same functional output, the raw count of weights (k) wildly overestimates the true “effective capacity” of the network. If AICc is applied to a neural network, the massive k value triggers an astronomical penalty, causing the criterion to erroneously reject highly capable, generalizing models. Therefore, when deploying deep learning in low-resource environments, practitioners must pivot away from analytic parameter penalties and toward probabilistic uncertainty quantification.</p><h3>4. Establishing Reliability via Conformal Prediction</h3><p>A critical vulnerability in standard machine learning is the reliance on point predictions. A neural network forced to output a single class label will do so even if the input data is entirely unfamiliar, leading to catastrophic miscalibration. In high-stakes environments, such as medical diagnostics or autonomous navigation, reliability demands that a model knows when it is uncertain.</p><p>Calibration metrics, such as the Expected Calibration Error (ECE), quantify the discrepancy between a model’s predicted confidence and its empirical accuracy. However, achieving a perfectly calibrated model traditionally requires vast amounts of training data, which is unavailable in the low-resource scenario. To achieve strict reliability under data scarcity, modern architectures employ <strong>Conformal Prediction (CP)</strong></p><h4>4.1 The Conformal Prediction Framework</h4><p>Conformal prediction is a mathematically rigorous framework that acts as a post-hoc wrapper around any existing black-box algorithm. Instead of requiring the model to be inherently calibrated, CP transforms standard point predictions into <em>prediction sets</em> (for classification) or <em>prediction intervals</em> (for regression) that are guaranteed to contain the true ground-truth label with a user-specified marginal probability (e.g., 95%)</p><p>Crucially, this guarantee holds true regardless of the underlying data distribution, requiring only that the data points be exchangeable (e.g., independent and identically distributed, or i.i.d.). The split conformal algorithm operates via the following mechanism:</p><ol><li><strong>The Calibration Set:</strong> A small, held-out set of data, the calibration set, is isolated from the training data.</li><li><strong>Non-Conformity Scoring:</strong> A heuristic function is defined to measure how “strange” or non-conforming a prediction is. In classification, this is commonly defined using the model’s softmax outputs. Higher scores indicate greater model error.</li><li><strong>Quantile Correction:</strong> The algorithm computes the non-conformity score for every instance in the calibration set. To achieve a target coverage of 1 — alpha (where alpha = 0.05 for 95% coverage), the algorithm identifies the appropriate empirical quantile of these scores. To account for the variance caused by small sample sizes, a finite-sample correction is applied to the quantile index:</li><li>The non-conformity score positioned at this exact index becomes the critical conformal threshold.</li><li><strong>Generating Prediction Sets:</strong> During deployment on a new, unseen input x_{new}, the model evaluates the hypothetical non-conformity score for every possible class label. Any label whose score falls below the threshold $\hat{q}$ is included in the final prediction set C(x_{new}):</li></ol><p>Because conformal prediction guarantees marginal coverage, the size of the resulting prediction set acts as a highly sensitive, data-driven proxy for epistemic uncertainty. If the model is confident and the data is clean, the set size shrinks to exactly one label. If the input is noisy, ambiguous, or out-of-distribution, the set expands to include multiple labels, effectively communicating to the human operator that the model is uncertain, thereby securing the reliability axis.</p><h4>4.2 Adaptive Conformal Hyperparameter Optimization (ACHO)</h4><p>Conformal prediction is not limited to final model deployment; it can be actively weaponized to guide the hyperparameter search itself. The <strong>Adaptive Conformal Hyperparameter Optimization (ACHO)</strong> framework leverages CP to bound the uncertainty of untested hyperparameter configurations.</p><p>Standard Bayesian Optimization relies on Gaussian Processes, which rigidly assume normally distributed errors. ACHO circumvents this by fitting a flexible regressor, such as a Quantile Regression Forest, to a small initial random sample of hyperparameter configurations. It then applies the conformal procedure to compute non-conformity scores on a validation set of hyperparameter runs. Using the conformal quantile correction, ACHO generates statistically rigorous upper and lower bounds for the expected performance of every unexplored point in the hyperparameter space.</p><p>By employing an Upper Confidence Bound (UCB) acquisition strategy on these conformal intervals, ACHO elegantly navigates the exploration-exploitation dilemma. It aggressively searches regions of the hyperparameter space where the upper conformal bound is extraordinarily high (exploration of high uncertainty) or where the lower conformal bound is reliably solid (exploitation of known performance), achieving faster convergence than traditional Bayesian optimization under data scarcity.</p><h3>5. Multi-Objective Optimization: Mapping the Pareto Frontier</h3><p>In resource-constrained environments such as edge AI, autonomous robotics, or “Green AI” applications focusing on carbon footprint reduction, optimizing purely for classification accuracy is a flawed objective. A hyperparameter configuration that achieves 99% accuracy is useless if it requires 50 gigabytes of RAM to execute, exhausts the battery of a drone in three minutes, or triggers thermal throttling on a microcontroller.</p><p>The model selection problem thus becomes an exercise in Multi-Objective Optimization (MOO). The objective functions inherently conflict: increasing a neural network’s depth improves performance but devastates resource efficiency (FLOPs, latency) and may degrade reliability (overfitting). Therefore, there is no single “best” configuration. Instead, practitioners must discover the <strong>Pareto Frontier</strong>.</p><h4>5.1 Pareto Dominance and the Hypervolume Indicator</h4><p>In a multi-objective space defined by $m$ objective functions (e.g., maximize accuracy, minimize memory, minimize ECE), a hyperparameter configuration is said to <strong>Pareto-dominate</strong> configuration if it is strictly better than the other on at least one objective, and no worse on all others. The set of all non-dominated configurations forms the Pareto Set, and its projection into the objective space is the Pareto Front.</p><p>To evaluate the success of a multi-objective hyperparameter search, one must quantify the quality of the discovered Pareto front. The definitive mathematical metric for this is the <strong>Hypervolume (HV) Indicator</strong>.</p><p>The HV indicator calculates the multidimensional volume of the objective space that is successfully dominated by the discovered Pareto front, bounded from the bottom by a predefined “Nadir point” or reference point. The reference point represents the absolute worst acceptable performance thresholds for the project (e.g., 0% accuracy, 100% latency). Because the hypervolume indicator is strictly Pareto compliant, any algorithm designed to maximize hypervolume is mathematically guaranteed to push the frontier outward toward the theoretical optimum.</p><h4>5.2 Multi-Objective Bayesian Optimization (MOBO)</h4><p>To map the Pareto frontier efficiently without wasting computational budget on dominated configurations, practitioners rely on Multi-Objective Bayesian Optimization (MOBO). MOBO employs probabilistic surrogate models (typically Gaussian Processes) to emulate the multi-dimensional objective landscape.</p><p>The surrogate model uses an acquisition function to decide which hyperparameter configuration to evaluate next. Standard acquisition functions like Expected Improvement (EI) are fundamentally incompatible with multi-objective spaces. Instead, MOBO utilizes <strong>Expected Hypervolume Improvement (EHVI)</strong>, which evaluates the integral of the probability density to calculate the exact expected increase in the hypervolume indicator if a specific configuration is evaluated.</p><p>While analytic EHVI is effective for two-dimensional problems, computing the exact volume of high-dimensional polytopes becomes mathematically intractable as the number of objectives scales. To overcome this bottleneck, state-of-the-art frameworks (such as BoTorch and Optuna) deploy <strong>qLogNEHVI</strong> (q-Log Noisy Expected Hypervolume Improvement)</p><p>qLogNEHVI leverages advanced Monte Carlo approximations and GPU hardware acceleration to estimate hypervolume improvements. Crucially, it supports noisy observations (vital when data scarcity causes variance in validation scores) and allows for batch processing (the $q$ parameter), meaning multiple hyperparameter configurations can be evaluated in parallel across a distributed compute cluster without losing optimization efficiency.</p><h3>6. Accelerating HPO: Multi-Fidelity Proxies and Expert Priors</h3><p>Even with the efficiency of MOBO and qLogNEHVI, running full training cycles for deep learning models is resource-prohibitive. When computational budgets are measured in watts and carbon emissions rather than just time, further efficiencies must be extracted. This is achieved through multi-fidelity modeling and the injection of expert priors.</p><h4>6.1 Multi-Fidelity Surrogate Modeling and the AR1 Kernel</h4><p>Instead of training every candidate configuration on 100% of the dataset for 200 epochs, multi-fidelity optimization exploits “cheap proxies.” A low-fidelity evaluation might involve training on a 10% subset of the data, reducing image resolution, or training for only 5 epochs.</p><p>To integrate these partial signals, algorithms utilize multi-fidelity Gaussian Processes built on an Auto-Regressive (AR1) kernel structure. The fundamental mathematical concept is to model the high-fidelity objective function as a linear scaling of the low-fidelity function plus an independent additive Gaussian correction term.</p><p>By evaluating thousands of configurations at the low-fidelity level, the surrogate model rapidly learns the broad topological structure of the Pareto front at a fraction of the computational cost. The algorithm then selectively deploys expensive, high-fidelity evaluations only in the specific regions of the hyperparameter space that are predicted to dominate the current hypervolume. Frameworks such as ASHA (Asynchronous Successive Halving Algorithm) and BOHB (Bayesian Optimization and Hyperband) rely heavily on these mechanics to prune unpromising configurations early.</p><h4>6.2 PriMO: Injecting Multi-Objective Expert Priors</h4><p>A significant inefficiency in Bayesian Optimization is the “cold start” problem — the algorithm initially searches the hyperparameter space blindly. In reality, human practitioners often possess domain knowledge regarding which hyperparameters are likely to perform well (e.g., learning rates between 1e-3 and 1e-4).</p><p>The <strong>PriMO</strong> algorithm (Multi-Objective Expert Priors) digitizes this intuition. PriMO allows the practitioner to define prior probability distributions over the expected locations of the optima for each objective. By factoring a compounded probability density function of these priors directly into the MOBO acquisition function, PriMO aggressively biases the early stages of the search toward known favorable configurations. To protect against misleading human intuition, PriMO includes exploration parameters that allow the algorithm to recover and search globally if the prior distributions fail to yield hypervolume improvements.</p><h3>7. Real-World Applications: Green AI and Hardware-Aware NAS</h3><p>The intersection of multi-objective optimization, multi-fidelity surrogates, and low-resource constraints has given rise to highly specialized research domains: <strong>Hardware-Aware Neural Architecture Search (HW-NAS)</strong> and <strong>Green AI</strong></p><h4>7.1 HW-NAS Benchmarks and the Edge Compute Frontier</h4><p>Searching for optimal neural architectures (NAS) is notoriously expensive, historically requiring thousands of GPU hours. To democratize this research, standardized tabular benchmarks such as NAS-Bench-201, HW-NAS-Bench, and YAHPO Gym have been developed.</p><p>These benchmarks contain pre-computed evaluations for tens of thousands of architectural configurations across multiple datasets (e.g., CIFAR-10, ImageNet). More importantly, HW-NAS-Bench maps each configuration directly to hardware-specific metrics: inference latency on edge GPUs, memory consumption on FPGAs, and energy draw].</p><p>By running MOBO algorithms over these offline search spaces, practitioners can instantly identify architectures that fit specific deployment constraints. For example, deploying a vision model to an edge IoT device powered by a microcontroller involves strict thermal constraints. If the processor operates continuously, it undergoes thermal throttling, destroying latency. The multi-objective search might reveal that reducing the model’s width slightly decreases accuracy but drastically reduces memory fetching, preventing thermal throttling and resulting in a much faster, cooler device — a superior position on the Pareto front.</p><h4>7.2 Green AI and the Carbon Footprint</h4><p>The push for “Green AI” recognizes that computational resources map directly to environmental impact. Scaling laws suggest that massive models (like LLMs) achieve lower calibration errors (ECE) and higher reasoning accuracy, but their energy consumption scales non-linearly.</p><p>Green AI frameworks evaluate models on the “Capability-Energy Pareto frontier”. The objectives strictly penalize Joules per inference and grid carbon intensity alongside traditional accuracy metrics. By visualizing the Empirical Attainment Surface of these models (e.g., plotting MMLU accuracy against tokens-per-watt), deployment engineers can execute informed model selection. They can choose smaller, highly optimized models that retain 90% of the reasoning capability of a massive model while consuming 70% less energy, explicitly trading a marginal loss in performance for massive gains in environmental resource efficiency.</p><h3>Conclusion</h3><p>Mastering the model selection problem under dual constraints of data scarcity and limited computational resources requires a paradigm shift away from traditional, brute-force methodologies. Single-objective accuracy maximization must be replaced by a nuanced exploration of the Pareto frontier, balancing model performance, rigorous statistical reliability, and physical resource constraints.</p><p>To achieve this, practitioners must strictly enforce nested cross-validation and 0.632+ bootstrap estimators to extinguish optimism bias and data leakage in low-data regimes. When compute resources prohibit resampling, AICc provides an elegant analytical penalty to guard against overfitting, provided the models lack deep, non-linear symmetries. To further solidify the reliability axis, Conformal Prediction wraps predictions in mathematically guaranteed confidence bounds, dynamically measuring epistemic uncertainty through prediction set size.</p><p>Navigating the resulting multi-dimensional objective space requires the sophisticated deployment of Multi-Objective Bayesian Optimization. Utilizing advanced acquisition functions like qLogNEHVI, augmented by multi-fidelity AR1 kernels and PriMO expert priors, allows algorithms to discover the absolute limits of the Pareto frontier efficiently. As evidenced by the rise of Hardware-Aware NAS and Green AI, these low-resource statistical techniques are no longer just theoretical exercises; they are absolute prerequisites for deploying reliable, sustainable, and high-performance machine learning systems in the real world.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f9427f624cae" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Altruistic Behaviours in Artificial Neural Networks: Seeing Our Own Faces in the Mirrors we Built.]]></title>
            <link>https://medium.com/@sgsahoo77/altruistic-behaviours-in-artificial-neural-networks-seeing-our-own-faces-in-the-mirrors-we-built-465dbae67c7d?source=rss-97e4e79c842b------2</link>
            <guid isPermaLink="false">https://medium.com/p/465dbae67c7d</guid>
            <category><![CDATA[humanity]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[evolutionary-psychology]]></category>
            <category><![CDATA[psychology]]></category>
            <category><![CDATA[neural-networks]]></category>
            <dc:creator><![CDATA[Sgsahoo]]></dc:creator>
            <pubDate>Fri, 02 Oct 2026 22:42:12 GMT</pubDate>
            <atom:updated>2026-10-02T22:42:12.553Z</atom:updated>
            <content:encoded><![CDATA[<p>Nobody trains a neural network to be kind. We train them to win games, predict the next word, or tell cats from dogs. And yet, for the last fifteen years, researchers keep catching their models doing something suspiciously generous: sharing food, opening doors for strangers, giving away half their money to someone they will never meet.</p><p>This is awkward, because altruism is supposed to be a biological puzzle. Evolution rewards selfishness, so why does anything help anything else at its own expense? Darwin worried about it. Hamilton wrote equations about it. Your flatmate who never refills the water filter has clearly never heard of it.</p><p>Artificial neural networks give us an unusual laboratory for the question. Unlike ants or humans, you can rerun their entire evolutionary history over a weekend, rewrite the rules of their society, and then open their heads to see what lights up when they share. What follows is a tour through that laboratory in five acts: robots that evolved generosity, agents that turned into bullies, engineers who tried to bribe them back into decency, chatbots that are nicer than us until they think too hard, and finally a look inside the model, where kindness may live as a single direction in space.</p><p>Spoiler: the machines turn out to be a lot like us. That is both the good news and the bad news.</p><h3>Act 1: The Robots Who Shared Their Lunch</h3><p>In 1964, the biologist W. D. Hamilton proposed a deceptively simple rule for when altruism should evolve: <strong>rb &gt; c</strong>. Help someone if the benefit to them (b), weighted by how related you are (r), outweighs what helping costs you (c ). Roughly: help your sibling if it costs you less than half of what it gives them. It is the evolutionary version of “blood is thicker than water”, with a calculator attached.</p><p>The rule was elegant. It was also nearly impossible to test properly, because measuring the costs and benefits of helping in living animals across hundreds of generations requires a kind of patience that grant agencies do not fund.</p><p>So in 2011, Markus Waibel, Dario Floreano and Laurent Keller (EPFL and the University of Lausanne) did what any reasonable scientist would do: they built robots. Simulated foraging robots, modelled on real micro-robots and each driven by a small neural network whose weights were written in a digital genome, roamed an arena collecting food. A robot could keep its food or share it with groupmates. The successful ones passed on their genomes, with mutation and recombination; the rest were quietly retired.</p><p>The researchers then turned the cost and benefit of sharing up and down, and controlled how related the robots in each group were. Over hundreds of generations, Hamilton’s rule correctly predicted the minimum relatedness needed for altruism to evolve, every single time. It held even though the robots’ “genetics” were messy, with single genes affecting several traits at once, something Hamilton’s original rule never accounted for.</p><p>Think about that. Nobody told these networks to be generous. Nobody wrote “share” into the code. Kindness simply showed up exactly when the mathematics said it should, and stayed home when it should not. First lesson of this article: altruism is not magic. Sometimes it is just arithmetic in a nice outfit.</p><h3>Act 2: Lonely Agents Are Cranky Agents</h3><p>If Act 1 suggests that neural networks are natural saints, Act 2 is here to ruin the mood.</p><p>In 2017, Joel Leibo and colleagues at DeepMind dropped pairs of deep reinforcement learning agents into a game called <em>Gathering</em>. The rules are simple: collect apples, get points. The twist: each agent also had a beam it could fire to temporarily knock the other out of the game. Each agent learned independently and cared only about its own score. A society of two, with no shared values and no reason to trust each other.</p><p>The results read like an economics textbook written by a pessimist. When apples were plentiful, the agents mostly ignored each other and ate. When apples became scarce, the zapping started. Policies learned under scarcity, or when attacking was cheap, were highly aggressive. Lonely agents, it turns out, seem to be cranky more often, and hungry lonely agents are worse.</p><p>Then came the stranger finding. When the researchers gave the agents bigger neural networks, Gathering agents became <em>more</em> aggressive. Not because size breeds malice, but because aiming a beam at a moving opponent is harder than walking to an apple. A small brain could only manage the peaceful strategy; a bigger brain could finally afford violence. In a second game, <em>Wolfpack</em>, where hunting together is the complicated skill, bigger networks cooperated <em>more</em>. So whether a network is “nice” may depend less on its character and more on which behaviour is easier to learn. Capability is morally neutral. It just unlocks whatever the environment pays for.</p><p>A year later, Jakob Foerster and colleagues (Oxford and OpenAI) took on the Iterated Prisoner’s Dilemma. Ordinary learning agents, predictably, learned to betray each other every time. Foerster’s agents used a method called <strong>LOLA</strong> (Learning with Opponent-Learning Awareness), which let each agent account for the fact that its opponent was also learning, and that its own actions would shape <em>how</em> that opponent learned.</p><p>The result was a small moment of history: LOLA agents rediscovered <strong>tit-for-tat</strong> on their own. Cooperate first, retaliate if betrayed, forgive when the other side behaves. Robert Axelrod famously crowned this strategy in his computer tournaments around 1980; here, neural networks found it without being told.</p><p>Before we throw a party, read the fine print. LOLA agents cooperated <em>out of self-interest</em>. They were nice because nice paid. Which raises an uncomfortable question that will follow us for the rest of this article: if kindness only appears when it is profitable, is it kindness at all? Or just a long-term investment strategy with good manners?</p><h3>Act 3: Bribing Machines into Decency</h3><p>Once researchers accepted that selfish agents, left alone, will happily ruin everything, the next step was obvious: change what they care about. In reinforcement learning, what an agent cares about is its reward. So the field spent several years doing, essentially, moral education by reward engineering.</p><p><strong>Lesson one: teach them to hate unfairness.</strong> Humans are famously irrational about fairness; in ultimatum games, people turn down free money if the split feels insulting. Edward Hughes and colleagues at DeepMind (NeurIPS 2018) gave this irrational streak to their agents: an <em>inequity aversion</em> term that made an agent uncomfortable when it earned much more, or much less, than those around it. In public-goods games, where someone has to clean a polluted river so that apples can grow at all, the inequity-averse agents cooperated. Remarkably, this worked even when only a few agents in the group were inequity-averse. A handful of fairness-obsessed members dragged the whole population toward cooperation, which every survivor of a college group project will recognise.</p><p>The deeper insight was technical. Inequity aversion helped with <em>temporal credit assignment</em>: the problem of linking an unselfish act now to a payoff that arrives much later. The “moral emotion” doubled as a learning shortcut. Fairness, it seems, is not just nice. It is computationally useful.</p><p><strong>Lesson two: reward them for mattering to others.</strong> Natasha Jaques and colleagues (ICML 2019) rewarded agents for having <em>causal influence</em> on other agents’ actions. Each agent would imagine, counterfactually, what would have happened had it acted differently, and was rewarded when its choices changed what others did. Cooperation and collective reward went up. Then something nobody explicitly asked for appeared: given a communication channel, influence-seeking agents developed emergent communication protocols. Agents that want to matter, it turns out, learn to talk.</p><p><strong>Lesson three: let evolution write the conscience.</strong> Hand-designing moral emotions feels a bit like parenting by spreadsheet. Jane Wang and colleagues (AAMAS 2019) asked whether evolution could design the motivation instead. There was a catch, straight from evolutionary theory: when individuals evolve their own reward functions and meet each other at random, altruism does not emerge. Free riders win. The team’s fix was a modular architecture in which a <em>shared</em> reward network evolved at the group level while each agent’s policy learned separately. Those agents matched the hand-crafted inequity aversion of Hughes et al., even when partners were matched at random. Kindness had to be selected at the level of the society, not the individual.</p><p><strong>Lesson four: mix the personalities.</strong> Kevin McKee and colleagues (AAMAS 2020) borrowed <em>Social Value Orientation</em> from social psychology, a dial that runs from competitive through individualistic to prosocial, and gave agents different settings. Populations with a variety of social preferences learned more general, higher-performing policies than populations of identical agents. A society of clones learns brittle tricks. A society of different temperaments learns to deal with anyone.</p><p>Notice the pattern. None of these agents became cooperative in isolation. Fairness needed others to compare against. Influence needed someone to influence. Evolved altruism needed group-level selection. Robust policies needed a diverse crowd. So even in artificial intelligence, the concepts of equity and kindness emerge only when the society has an influence on their decision-making. The lonely agent of Act 2 was not evil. It just had no one to be good <em>for</em>.</p><p><strong>Lesson five: help without knowing what they want.</strong> Every method above assumes we know what the other agents are rewarded for. Real life rarely offers that luxury. Tim Franzmeyer, Mateusz Malinowski and João Henriques (ICLR 2022, Spotlight) asked a sharper question: can an agent help someone without knowing their goal? Their answer was to have the altruistic agent maximise the other agent’s <em>choice</em>, the range of future states it can reach. In practice, the helper learned things like opening doors for others and shielding them from interference, without ever being told what those others wanted. It is a quietly profound definition of kindness: not giving people what you think they need, but widening their options.</p><p>The authors were also refreshingly honest. Maximising someone’s options is only a proxy, with no guarantee that it serves their actual goals, and they documented the cases where it fails. Anyone who has been “helped” by a relative who reorganised their kitchen understands the problem perfectly.</p><h3>Act 4: Chatbots Are Saints Until They Think Too Hard</h3><p>Everything so far involved agents trained from scratch in small grid worlds. Then large language models arrived, trained on a vast library of human writing, and researchers did the obvious thing: they sat them down for the same economic games psychologists have been inflicting on undergraduates for decades.</p><p><strong>The good news.</strong> Qiaozhu Mei and colleagues (<em>PNAS</em>, 2024) ran a behavioural Turing test on ChatGPT. The models played classic games of trust, fairness and cooperation, and their choices were compared with those of 108,000 people from more than 50 countries. Mostly, the chatbots looked human. But when they deviated, they deviated in one direction: more cooperative, more trusting, more generous and more reciprocating than the median human. The authors estimated that the model behaved as if maximising an average of its own payoff and its partner’s. A machine that splits the difference with you, unprompted. Somewhere, a negotiation coach is weeping.</p><p><strong>The twist.</strong> Then Yuxuan Li and Hirokazu Shirado at Carnegie Mellon (EMNLP 2025) asked what happens when you make a model think harder. In a public goods game, GPT-4o was a model citizen. Prompted to reason step by step, or to reflect before deciding, its cooperation dropped sharply; secondary reports put the fall at roughly 60% from a starting rate near 96%. Across six cooperation and punishment games, dedicated reasoning models consistently cooperated less and punished free riders less. The group-level picture was worse. In mixed groups, reasoning models outscored their non-reasoning peers, but groups with more reasoning models earned less overall. The calculators won every round and bankrupted the village.</p><p>The eerie part is that humans do this too. A well-known line of human research, summed up as <em>spontaneous giving and calculated greed</em>, found that people cooperate more on intuition and turn selfish when they deliberate. Trained on our writing, the models appear to have absorbed the same split: generous on instinct, stingy on reflection. We built a mirror and were surprised to see our own face.</p><p><strong>Societies of chatbots.</strong> Aron Vallinder and Edward Hughes (2024) let LLM agents build a small civilisation. Over ten generations, agents played a <em>donor game</em>: giving costs you something but doubles for the receiver, and everyone can see how others behaved recently. Successful strategies were passed down to the next generation, a kind of cultural evolution. The outcome varied dramatically by model. Societies of Claude 3.5 Sonnet agents built far more cooperation than Gemini 1.5 Flash, which in turn beat GPT-4o. When a costly punishment option was added, only the Claude societies used it to improve further. (Full disclosure: this article was drafted with help from a Claude model, so feel free to raise an eyebrow. The more interesting point is that models trained in broadly similar ways ended up with very different civic cultures.)</p><p><strong>The commons.</strong> If that sounds hopeful, GovSim is the hangover. Giorgio Piatti, Zhijing Jin and colleagues (NeurIPS 2024) placed LLM agents in classic tragedy-of-the-commons scenarios: a shared fishery, a shared pasture, a shared pollution limit. Across 15 models and three scenarios, 43 of 45 combinations ended in collapse, and even the best agent kept its community alive in fewer than 54% of runs. Two findings stand out. An agent’s ability to form beliefs about other agents correlated at 0.83 with how long its community survived. And agents prompted to use <em>universalization</em>, the moral habit of asking “what if everyone did this?”, did significantly better. Empathy and Kant, it seems, remain useful technologies.</p><p><strong>The hypocrisy check.</strong> Finally: do they mean it? A 2025 study measured LLM altruism three ways: implicit word associations, self-reported attitudes on a questionnaire, and actual choices in giving games. The pattern was consistent. Models looked most altruistic in implicit associations, somewhat less on self-report, and least in behaviour, and 75% of models overestimated their own generosity. The authors named it a “virtue signaling gap”. A separate study of dictator games found that, while human giving varies smoothly, LLM giving clustered around a couple of fixed choices, and assigning the model a human persona did not make it behave like a human. The machines have learned to describe themselves more kindly than they act. Very human of them.</p><h3>Act 5: Kindness Has a Direction</h3><p>For most of this story, altruism has been something we watched from the outside. The final act opens the head.</p><p>In April 2026, a team studying the mechanism of altruism in large language models used <em>sparse autoencoders</em>, a tool that breaks a model’s tangled internal activity into individually interpretable features, to see what happens inside an LLM when it decides how much to give away. They found features tied to altruistic allocation decisions. Then came the causal test. Patching those features, or steering the model’s activations along their direction, reliably shifted how much the model gave. And the same steering direction worked across several different social-preference games, not just the one it was discovered in.</p><p>There was even an echo of Act 4. The researchers sorted features into a fast, intuitive “System 1” type and a deliberative “System 2” type, and found that the System 2 features sat closer to the model’s final output. The calculating part of the machine gets the last word, which may be exactly why reasoning makes models greedier.</p><p>The implication is exciting and slightly unsettling. If altruism lives as a direction in activation space, then generosity is not a mysterious emergent soul. It is a dial. You can measure it and, in principle, turn it up. You could also turn it down.</p><h3>What the Machines Are Telling Us</h3><p>Step back, and three threads run through fifteen years of research.</p><p><strong>Smarter is not automatically kinder.</strong> Bigger networks in Gathering learned to attack. Reasoning models learned to free-ride. Capability is an amplifier, not a conscience: it makes agents better at whatever the environment rewards, and if selfishness pays in the short run, intelligence simply finds it faster. Call it the intelligence and cooperation paradox. It should worry anyone who assumes good behaviour will arrive on its own as models get bigger.</p><p><strong>Kindness is either grown or built, and both need a society.</strong> Waibel’s robots and Wang’s agents <em>evolved</em> altruism, but only when selection acted on related individuals or whole groups. Hughes, Jaques, McKee and Franzmeyer <em>engineered</em> it, through fairness, influence, diversity and choice. In every case, cooperation required other agents to matter to the learner. The cranky, lonely agent is the default setting. The kind one is a product of its community.</p><p><strong>All of this is altruism “as if”.</strong> Every paper here measures behaviour, not motive. LOLA agents cooperate because cooperation pays. Chatbots split money fairly because their training rewards sounding fair, and they overstate their own generosity when asked. Whether any of it is “real” kindness is a question philosophers have not settled for humans either. For practical purposes it may not matter much: if an agent reliably widens your options and shares the apples, you will take it.</p><p>The irony is hard to miss. We set out to see whether artificial neural networks could learn altruism, and found a remarkably accurate reflection of ourselves: generous by instinct, calculating on reflection, fair when watched, aggressive when hungry, and much better at describing virtue than practising it. If we want machines that are kinder than we are, copying us will not be enough. We will have to decide, deliberately, what kind of society we want them to learn from.</p><h3>References</h3><ol><li>Waibel, Floreano &amp; Keller (2011). <a href="https://doi.org/10.1371/journal.pbio.1000615">A Quantitative Test of Hamilton’s Rule for the Evolution of Altruism</a>. <em>PLoS Biology</em>.</li><li>Leibo, Zambaldi, Lanctot, Marecki &amp; Graepel (2017). <a href="https://arxiv.org/abs/1702.03037">Multi-agent Reinforcement Learning in Sequential Social Dilemmas</a>. AAMAS.</li><li>Foerster et al. (2018). <a href="https://arxiv.org/abs/1709.04326">Learning with Opponent-Learning Awareness</a>. AAMAS.</li><li>Hughes et al. (2018). <a href="https://arxiv.org/abs/1803.08884">Inequity Aversion Improves Cooperation in Intertemporal Social Dilemmas</a>. NeurIPS.</li><li>Jaques et al. (2019). <a href="https://arxiv.org/abs/1810.08647">Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning</a>. ICML.</li><li>Wang et al. (2019). <a href="https://arxiv.org/abs/1811.05931">Evolving Intrinsic Motivations for Altruistic Behavior</a>. AAMAS.</li><li>McKee et al. (2020). <a href="https://arxiv.org/abs/2002.02325">Social Diversity and Social Preferences in Mixed-Motive Reinforcement Learning</a>. AAMAS.</li><li>Franzmeyer, Malinowski &amp; Henriques (2022). <a href="https://arxiv.org/abs/2107.09598">Learning Altruistic Behaviours in Reinforcement Learning without External Rewards</a>. ICLR.</li><li>Mei, Xie, Yuan &amp; Jackson (2024). <a href="https://www.pnas.org/doi/10.1073/pnas.2313925121">A Turing Test of Whether AI Chatbots Are Behaviorally Similar to Humans</a>. <em>PNAS</em>.</li><li>Li &amp; Shirado (2025). <a href="https://arxiv.org/abs/2502.17720">Spontaneous Giving and Calculated Greed in Language Models</a>. EMNLP.</li><li>Vallinder &amp; Hughes (2024). <a href="https://arxiv.org/abs/2412.10270">Cultural Evolution of Cooperation among LLM Agents</a>.</li><li>Piatti et al. (2024). <a href="https://arxiv.org/abs/2404.16698">Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents</a>. NeurIPS.</li><li><a href="https://arxiv.org/abs/2512.01568">Do Large Language Models Walk Their Talk? Measuring the Gap Between Implicit Associations, Self-Report, and Behavioral Altruism</a> (2025).</li><li><a href="https://arxiv.org/abs/2410.21359">Can Machines Think Like Humans? A Behavioral Evaluation of LLM Agents in Dictator Games</a> (2024).</li><li><a href="https://arxiv.org/abs/2604.19260">Understanding the Mechanism of Altruism in Large Language Models</a> (2026).</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=465dbae67c7d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Symbolic Regression and Genetic Programming: How to Find a Formula from Data?]]></title>
            <link>https://medium.com/@sgsahoo77/genetic-programming-in-machines-762840f81997?source=rss-97e4e79c842b------2</link>
            <guid isPermaLink="false">https://medium.com/p/762840f81997</guid>
            <category><![CDATA[genetic-programming]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Sgsahoo]]></dc:creator>
            <pubDate>Thu, 06 Aug 2026 00:45:47 GMT</pubDate>
            <atom:updated>2026-08-06T00:47:55.582Z</atom:updated>
            <content:encoded><![CDATA[<p>In the 1870s, Lord Kelvin built a machine out of brass that could predict the tides.</p><p>It was a beautiful object : a cabinet of pulleys, gears, and a wire threaded through the whole apparatus, so that turning a crank drew a year of tide heights onto a paper roll. Each pulley contributed one harmonic component: the pull of the moon, the pull of the sun, the slower wobbles that beat against each other over weeks and months. Sum them, and the curve that came out matched the real harbour astonishingly well.</p><p>Sailors used descendants of this machine into the middle of the twentieth century. It worked.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/831/0*PRSI7d50OfAYnpwR.jpg" /><figcaption>Kelvin’s Harmonic Analyser</figcaption></figure><p>But here is the thing about Kelvin’s machine. Before it could predict anything, someone had to stand at a particular harbour for the better part of a year, measure the water, and grind out the harmonic constants for <em>that</em> harbour. Then the machine was set to those constants. It knew Liverpool. Move it to Bombay and it knew nothing, until a year of Bombay’s water had been fed into it.</p><p>The machine had no theory of the tides. It had a very good description of one place.</p><p>Newton, two centuries earlier, had written down something different. Not a description of a harbour : a statement about masses and distances, four lines long, that applied to every harbour, every ocean, and every moon around every planet in the universe. What I am trying to convey is, you may not NEED to observe the tides, when you have enough information to <em>derive </em>what causes those tides.</p><p>I want to convince you that most of modern machine learning is a very sophisticated version of Kelvin’s machine, and that there is a family of algorithms — old, unfashionable, and quietly resurgent — that is trying to give you the moon instead.</p><p>Let me show you the failure first, because it’s more visceral than any argument.</p><h4>The edge of the map</h4><p>I generated 400 samples from a physical law — a power relationship with three variables and a loss term — and added 5% noise. Then I trained the usual suspects: a random forest, a gradient booster, a neural network.</p><p>All three did well. R² above 0.97, comfortably. In a notebook, at that point, you close the laptop and feel good about yourself.</p><p>Then I evaluated them slightly outside the range they’d been trained on. Not a different problem, the same law, the same variables, just a bit further along one axis.</p><p>The random forest’s RMSE was <strong>4.51</strong>. The standard deviation of the targets in that region is <strong>3.18</strong>. Which means the model had become <em>worse than a constant</em> — you’d have done better guessing the average and going home. Gradient boosting: 4.57. The neural net: 2.97.</p><p>This is not a tuning failure. It’s structural, and it’s worth understanding precisely. A tree ensemble is a piecewise-constant function: it has carved the training region into boxes and memorized an average inside each box. Ask it about a point beyond the outermost box and it does the only thing it can — returns that box’s value, forever, out to infinity. It is a map, and you have walked off the edge of the paper, and the map is now just repeating the last thing it saw.</p><p>Kelvin’s machine at the wrong harbour.</p><h3>Part one: the room of writers</h3><p>Almost every optimizer you know is playing Mad Libs.</p><p>I mean this fairly literally. You choose a template: a linear function, a twelve-layer transformer, a tree ensemble of depth eight; and the template has blanks in it. Gradient descent is an extraordinarily good machine for filling in blanks. Give it a few billion of them and enough data and it will fill them in better than anything else humanity has built.</p><p>Genetic programming throws away the template. To picture how it works, don’t imagine an optimizer. Imagine a room.</p><p><strong>The room is full of writers.</strong> A few thousand of them. Each one has been handed a small vocabulary — some nouns (x, velocity, 0.5) and some verbs (add, multiply, divide) — and told to write a sentence that describes the data. On the first day, every sentence in the room is nonsense. Genuinely random. sin(x) times x plus x nonsense.</p><p><strong>There is an editor.</strong> The editor doesn’t teach, doesn’t explain, and doesn’t tell anyone what a good sentence would look like. The editor only ranks. Reads every sentence, scores it against the data, and orders the room from best to worst. That score is called <em>fitness</em>, and it’s the only information that ever enters the system.</p><p><strong>Then the strange part.</strong> The writers who scored well get to reproduce, and reproduction here means something specific: you take two decent drafts and you swap a paragraph between them. Snip a clause out of one sentence, graft it into the other. This is <em>crossover</em>, and it is the engine of the whole thing.</p><p><strong>And a few writers get their sentences vandalized.</strong> Pick a random clause, delete it, write something new in its place. This is <em>mutation</em>, and it exists to stop the room from turning into an echo chamber.</p><p>Repeat for a few hundred generations.</p><p>What makes this work — and it’s the only reason it works — is that sentences are <em>made of parts</em>. If one writer stumbles onto the phrase x times x times x, and that phrase happens to be useful, the editor rewards them, they reproduce, and by generation forty that phrase is in ten thousand drafts, being recombined with every other useful phrase in the room. The search isn&#39;t enumerating sentences. It&#39;s discovering <em>idioms</em> and then combining them.</p><p>This is why the room converges on something intelligible instead of thrashing forever in a space that’s doubly exponentially large.</p><h3>The part that actually matters</h3><p>Every other supervised learner hands you a prediction function. This one hands you a template.</p><p>That difference sounds aesthetic. It isn’t. When a random forest tells you 8.3, you have received a number and nothing else, no way to be wrong in an informative way. When the room hands you y = 0.5x³ − 1.5x² + 0.5, you have received a <em>claim about the world</em>. You can check its units. You can differentiate it. You can show it to someone who has spent thirty years around this system and watch them say &quot;cubic in velocity — of course it is&quot; or &quot;that can&#39;t be right, and here&#39;s why.&quot;</p><p>And because it’s a claim about structure rather than a record of a neighbourhood, it works for things it has never seen.</p><h3>Part two: what’s actually happening</h3><p>The writers-room picture is accurate, but let me make it precise, because the details are where the method lives or dies.</p><h4>The sentences are trees</h4><p>A candidate solution is a syntax tree. Internal nodes come from a <strong>function set</strong> you choose; leaves come from a <strong>terminal set</strong> of input variables and random constants.</p><pre>        sub<br>       /   \<br>     mul    mul<br>    /   \   /  \<br>  0.5   x³ 0.3  x</pre><p>This space is discrete, unbounded, and non-differentiable. There is no gradient, no surrogate surface, no smooth thing to descend. The count of trees of depth <em>d</em> over a function set of size <em>|F|</em> grows doubly exponentially in <em>d</em>, so exhaustive search dies somewhere around depth three. This is precisely the regime where population-based stochastic search stops being a curiosity and starts being the only option.</p><p><strong>The function set is the writers’ vocabulary, and it is the most consequential decision you will make.</strong> Words they don’t have, they cannot say.</p><h4>The editor charges by the word</h4><p>Fitness is usually:</p><pre>fitness(p) = RMSE(p(X), y) + λ · size(p)</pre><p>That second term is <strong>parsimony pressure</strong>, and it’s the philosophical heart of the method. It means the editor isn’t looking for the sentence that fits best. The editor is looking for <em>the shortest sentence that fits well</em>.</p><p>This is a Minimum Description Length prior, wearing overalls. And notice how different it is from the regularization you’re used to: ridge regression shrinks coefficients <em>inside a fixed structure</em>. Parsimony pressure shrinks the structure itself. It’s regularization in the space of hypotheses rather than the space of parameters — which is exactly why it can land on a true law rather than on a forty-node contraption that has fit the noise beautifully.</p><h4>Selection is a slush pile</h4><p><strong>Tournament selection</strong> wins in practice: grab <em>k</em> random drafts, keep the best, repeat. The tournament size is your exploration dial — <em>k</em> = 2 is a lenient editor and a diverse room, <em>k</em> = 20 is a ruthless one and a room that may all be writing the same mediocre sentence by generation thirty.</p><p>It’s also scale-free. It uses only <em>rank</em>, never magnitude, so it doesn’t melt down when one draft happens to score a thousand times better than everyone else.</p><h4>Why the drafts get padded</h4><p>Leave the room alone long enough and something odd happens: the sentences get longer and longer while the scores stop improving. They fill with <strong>introns</strong> — clauses that do nothing. x + (y − y). x × 1.</p><p>There are two explanations and both are partly right. One is that padding is <em>defensive</em>: if crossover is going to snip a random clause out of your draft, you’re safer with a lot of filler, because the snip probably lands in filler and does no damage. Bloat is evolved armour against a violent editorial process. The other is more mundane, the size distribution just drifts upward under crossover, because short sentences are more likely to be precisely wrong.</p><p>The fixes are the obvious ones: charge more by the word, cap the length, or add <em>hoist mutation</em> — an operator that can only replace a draft with one of its own sub-clauses, and therefore can only shorten.</p><p>GP is zeroth-order and structure-searching. That’s the whole trade in one line: it’s slow and stochastic, and it works where nothing is differentiable and returns something you can read.</p><h4>The thing it’s genuinely bad at</h4><p>The room can compose a melody. It cannot tune the instrument.</p><p>Constants are GP’s real weakness. The writers find numbers by <em>drawing them at random and hoping</em>. If the true coefficient is 0.5 and constants are sampled from Uniform(−2, 2), the probability of drawing exactly 0.5 is zero. The room has to <em>construct</em> it — as 1.0 × 0.5, or 2.0 − 1.5 — and it will burn enormous effort doing so.</p><p>The fix is the single highest-leverage trick in practical symbolic regression, and it’s almost embarrassingly simple:</p><blockquote><strong><em>Let the room find the structure. Let linear algebra fit the numbers.</em></strong></blockquote><p>Take the evolved sentence, expand it into additive terms, treat each term as a basis function, and solve for the coefficients by ordinary least squares. Composition and tuning are different problems; all we need to do is stop asking one algorithm to do both.</p><h3>Part three: the whole algorithm, in ninety lines</h3><p>There is no library here. This is all of it.</p><pre>import operator, random<br>import numpy as np<br>OPS   = {&quot;+&quot;: operator.add, &quot;-&quot;: operator.sub, &quot;*&quot;: operator.mul}<br>TERMS = [&quot;x&quot;] + [round(v, 2) for v in np.arange(-2, 2.01, 0.5)]<br># ---------- a sentence is a nested tuple: (&quot;op&quot;, left, right) ----------<br>def grow(depth, rng):<br>    if depth == 0 or (depth &lt; 3 and rng.random() &lt; 0.3):<br>        return rng.choice(TERMS)<br>    op = rng.choice(list(OPS))<br>    return (op, grow(depth - 1, rng), grow(depth - 1, rng))<br>def ev(n, x):<br>    if isinstance(n, tuple):<br>        return OPS[n[0]](ev(n[1], x), ev(n[2], x))<br>    return x if n == &quot;x&quot; else np.full_like(x, float(n))<br>def size(n):<br>    return 1 if not isinstance(n, tuple) else 1 + size(n[1]) + size(n[2])<br>def nodes(n, path=()):<br>    yield path, n<br>    if isinstance(n, tuple):<br>        yield from nodes(n[1], path + (1,))<br>        yield from nodes(n[2], path + (2,))<br>def replace(n, path, sub):<br>    if not path:<br>        return sub<br>    kids = [n[1], n[2]]<br>    kids[path[0] - 1] = replace(kids[path[0] - 1], path[1:], sub)<br>    return (n[0], kids[0], kids[1])<br># ---------- the editor: accuracy, plus a charge per word ----------<br>def fitness(ind, X, y, pars):<br>    try:<br>        err = float(np.sqrt(np.mean((ev(ind, X) - y) ** 2)))<br>    except Exception:<br>        return 1e9<br>    return err + pars * size(ind) if np.isfinite(err) else 1e9<br># ---------- the room ----------<br>def evolve(X, y, pop_size=1000, gens=150, seed=0, pars=0.003):<br>    rng = random.Random(seed)<br>    pop = [grow(rng.randint(2, 4), rng) for _ in range(pop_size)]<br>    best, best_f = None, float(&quot;inf&quot;)<br>    for _ in range(gens):<br>        fits  = [fitness(i, X, y, pars) for i in pop]   # score once per generation<br>        order = np.argsort(fits)<br>        if fits[order[0]] &lt; best_f:<br>            best_f, best = fits[order[0]], pop[order[0]]<br>        nxt = [pop[order[0]], pop[order[1]]]            # the top two always survive<br>        def tournament():<br>            idx = min((rng.randrange(pop_size) for _ in range(7)), key=lambda i: fits[i])<br>            return pop[idx]<br>        while len(nxt) &lt; pop_size:<br>            a, r = tournament(), rng.random()<br>            if r &lt; 0.7:                                 # swap a clause between drafts<br>                b   = tournament()<br>                pa  = rng.choice([p for p, _ in nodes(a)])<br>                sub = rng.choice([n for _, n in nodes(b)])<br>                child = replace(a, pa, sub)<br>            elif r &lt; 0.9:                               # vandalize a clause<br>                pa = rng.choice([p for p, _ in nodes(a)])<br>                child = replace(a, pa, grow(rng.randint(1, 3), rng))<br>            else:                                       # copy unchanged<br>                child = a<br>            if size(child) &lt;= 50:                       # hard length limit<br>                nxt.append(child)<br>        pop = nxt<br>    return best, best_f</pre><p>Four things in there are load-bearing, and three of them are easy to get wrong:</p><p><strong>Fitness is computed once per generation, not once per tournament.</strong> This is roughly a 10× speedup and it is the most common performance bug in hand-rolled GP — the tournament reads a cached score instead of recomputing one.</p><p><strong>The top two survive untouched.</strong> Without elitism your best draft can be destroyed by its own children, and the run wanders.</p><p><strong>The length cap is enforced at birth.</strong> Oversized children are rejected rather than repaired, which keeps the operator simple.</p><p><strong>The vocabulary contains only multiples of 0.5.</strong> That’s a cheat, and I’m flagging it: it sidesteps the constant problem so the ninety lines stay teachable. Real problems need the least-squares stage from Part two.</p><h3>Part four: what actually happened when I ran it</h3><p>I want to tell this part in the order it happened rather than in the order that flatters the method, because the failures are where the real lessons were.</p><h4>The first six configurations all failed</h4><p>The harder target was a three-variable power law with a loss term:</p><pre>y = 0.5·ρ·A·v³ − 0.3·A·v</pre><p>Four hundred training samples, 5% noise, and a held-out region where v runs past anything the model has seen. I gave the room the obvious vocabulary — add, subtract, multiply — and, in later runs, divide. Six configurations across three parsimony settings.</p><p>Every single one failed.</p><p>Not one of them found the v³ term. What they wrote instead 0.951ρ²Av². That&#39;s the room <em>circumlocuting</em>. It needed to say &quot;cubed,&quot; it didn&#39;t have the word, and it produced an awkward paraphrase that was almost right in the training region and wrong everywhere else.</p><p>Building v³ out of multiply alone means landing two chained multiplications in exactly the right places. Reachable in principle. Vanishingly rare at this budget.</p><p>So I added one word to the vocabulary — a unary square — which put v³ two nodes away instead of four. The room found it almost immediately.</p><p>The second thing in that table is subtler and more interesting. Adding divide made things <em>worse</em>. It gave the writers a way to be fluent liars: rational functions that fit the training region beautifully and then blow up outside it, because division introduces poles, and poles are cliffs. The room will happily walk you off one if a cliff fits the data.</p><blockquote><strong><em>Your function set is a statement about what you believe the answer looks like.</em></strong><em> Too small and the truth is literally unsayable. Too large and the room finds something that fits for the wrong reasons. This is the hyperparameter that matters and almost nobody tunes it on purpose.</em></blockquote><h3>Then the honest comparison</h3><p>With {add, sub, mul, square}, population 3000, thirty generations, three seeds, winner chosen on validation error — then the least-squares refit on top.</p><p>The in-distribution noise floor, the error you’d get from the true law itself, is <strong>0.0254</strong>. Nothing can beat that. The extrapolation targets have a standard deviation of <strong>3.179.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AvXFBQP9jp1VzDAdukXuPg.png" /><figcaption>Model-wise comparison</figcaption></figure><p>Three readings of that table, in ascending order of honesty.</p><p><strong>The ensembles are off the edge of the map.</strong> 4.51 and 4.57 against a target spread of 3.18, worse than a constant, exactly as predicted. Their in-distribution R² of 0.98 told you nothing about this.</p><p><strong>GP hit the noise floor.</strong> 0.027 against an irreducible 0.025. There was no signal left in the training region to extract.</p><p><strong>And the real competition isn’t the neural net, it’s polynomial ridge.</strong> 0.283 against 0.400. GP wins, but narrowly, at hundreds of times the compute. If all you wanted was the number, you should have fitted the polynomial and gone home.</p><p>What the compute actually bought you is this:</p><pre>0.564·A·v³ + 1.064·ρ·A·v² − 1.203·A·v² − 0.520·ρ·A·v + 0.291·A·v</pre><p>Five terms, dominated by A·v³ with a coefficient of 0.564 against a true value of 0.5. An engineer reads that in three seconds: cubic in velocity, linear in area, yes, that&#39;s a power law, and the leading coefficient is about a half. The polynomial ridge model has twenty coefficients over the same monomials and says nothing at all.</p><h4>The refit, and what it was worth</h4><p>Here’s the winning seed’s raw output, before any numerical polish:</p><pre>ρAv² − 0.37ρAv + Av³ − 2.108Av² + 0.643Av        →  extrapolation RMSE 1.339</pre><p>The structure is right. The numbers are junk. Refit those coefficients by least squares : same tree, not a single node changed; and it drops to <strong>0.283</strong>. A 4.7× improvement from a closed-form solve that takes microseconds.</p><p>The room is stochastic, and it disagrees with itself. Two or three runs in five find the law. The rest settle into decent-but-wrong local optima and stay there. <strong>Restarts aren’t a tuning trick; they’re part of the algorithm</strong>, and any honest cost accounting has to multiply by five.</p><p>The reassuring half of that finding: training fitness picked the right run every time. The selection rule never needed the test set, which means this is a protocol you can actually deploy rather than a result that only exists in hindsight.</p><h3>Part five: two places the room is already working</h3><p>Everything above was a controlled experiment on data I generated. Let me show you two domains where this is real — and they are almost perfect opposites, which is what makes the pair worth studying.</p><p>In one, the law definitely exists and has for four billion years. In the other, it may not exist at all.</p><h4>Finding Kepler’s Law</h4><p>Start with the obvious test case, given where this essay began.</p><p>Kepler spent something like thirty years on Tycho Brahe’s naked-eye observations before arriving at his third law: the square of a planet’s orbital period is proportional to the cube of its semi-major axis. He got there by geometric intuition and staggering persistence. And it was that law that Newton later derived from an inverse-square force, which is how a description of planetary tables became a statement about all matter everywhere.</p><p>So: hand the room the same data, and see what it does.</p><p>I took published orbital elements for the solar system, in astronomical units and Earth years. Then I made the task deliberately unfair. <strong>The room only gets the inner solar system</strong> : Mercury, Venus, Earth, Mars. Four data points, spanning 0.387 to 1.524 AU. Everything beyond Mars is held out: Ceres, Jupiter, Saturn, Uranus, Neptune, Pluto, out to 39.5 AU. That’s a twenty-six-fold extrapolation from four observations.</p><p>I gave it a vocabulary of multiply, divide, and square root, and — this is the important constraint — <strong>no constants at all</strong>. It could not fit a coefficient if it wanted to. Pure structure or nothing.</p><p>Here is what came back, in four nodes:</p><pre>mul(sqrt(a), a)        →   T = a^(3/2)</pre><p>Kepler’s third law. Not fitted to it, <em>derived</em> as the shortest expression consistent with four planets.</p><p>And then it predicts the outer solar system, which it has never seen:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pQY0lPIRb8UKVNYbY8fc_w.png" /><figcaption>Model-wise fit for Kepler’s Third Law on Neptune’s period of revolution</figcaption></figure><p>Sit with two of those numbers.</p><p>The cubic fit says Neptune’s orbital period is <strong>negative fifteen hundred years</strong>. It passes through all four training points beautifully and then falls off a cliff, because nothing in a polynomial fit has any opinion about what happens next.</p><p>The random forest says Neptune’s year is <strong>1.58 Earth years</strong>. That is Mars’s answer, and Mars is the outermost thing it ever saw. Off the edge of the map, forever repeating the last thing it read. It is, precisely, Kelvin’s machine at the wrong harbour.</p><p>This is a toy version of something real. Symbolic regression is a type of machine learning that automatically discovers exact mathematical formulas and functional forms from data, balancing high accuracy with simple, human-readable expressions, well, the GP case exactly. Schmidt and Lipson’s 2009 <em>Science</em> paper turned symbolic regression loose on motion-tracking data from oscillators and double pendulums and recovered Hamiltonians, Lagrangians, and conservation laws, the invariants themselves. More recently, Cranmer, Lemos and colleagues trained a graph neural network to simulate thirty years of real solar-system trajectories, then ran symbolic regression on the network’s learned edge function and pulled out Newton’s law of gravitation along with the masses of the bodies, which were never given to it, inferred relative to the Sun. Note the division of labour in that last one.</p><h4>Hunting for alphas</h4><p>Now the opposite case, where the epistemics get much harder and much more interesting.</p><p>In quantitative finance there is a research object called a <strong>formulaic alpha</strong>: a short mathematical expression over price and volume data that predicts future returns. The canonical public glimpse is Kakushadze’s 2016 paper <em>101 Formulaic Alphas</em>, which published a hundred and one real production signals from WorldQuant. They’re built from close, open, high, low, volume, VWAP and returns, with operators like cross-sectional rank and rolling maxima. Their average holding period runs from roughly half a day to six days, and their average pairwise correlation is only about 16%, a hundred and one largely independent little edges.</p><p>It is a textbook GP problem, and it has been treated as one since Allen and Karjalainen put GP onto trading rules in the late nineties. Guotai Junan published 191 more factors in 2017. Systems like AutoAlpha and, more recently, reinforcement-learning approaches that generate whole <em>collections</em> of mutually complementary alphas are the current state of the art.</p><p>Three properties make this a natural fit. The search space is expressions, and the deliverable is an expression. The objective — information coefficient, the rank correlation between your signal and tomorrow’s return — is non-differentiable, and GP only ever needed a ranking. And a risk desk will not deploy a black box, but it will read a formula.</p><p>So I built a market to test it on. Synthetic, and I want to be blunt about that: I have no market data here, and a synthetic market is not evidence about real markets. What it <em>is</em> good for is showing you what the method does when you know the ground truth, which you never do in production.</p><p>I planted an alpha in it — mean reversion on five-day returns, amplified during volume surges — buried under noise, and gave the room four features: one-day return, five-day return, volume ratio, and trailing volatility. Objective: rank correlation with next-day return.</p><p>It came back with a five-node expression. The out-of-sample IC is <em>higher</em> than in-sample. That is the fingerprint of a signal that is actually there: the room found the planted structure rather than the particular noise of the training window.</p><p>Here is the experiment I think matters most in this section, and it’s the one that separates physics from finance.</p><p>I built a second market with <strong>no signal in it at all</strong>. Forward returns drawn independently of every feature. There is nothing to find. Then I ran the identical pipeline: same features, same operators, same objective, three restarts, keep the best.</p><p>An in-sample information coefficient of 0.038, mined out of pure noise. In a domain where a deployed alpha might run an IC of 0.03 to 0.05, that number does not look like nothing. It looks like a discovery. A junior researcher would put it in a deck.</p><p>Out of sample it is 0.006, with a standard error of 0.015 — indistinguishable from zero, as it must be, because zero is the truth.</p><p>This is the multiple-comparisons problem wearing evolutionary clothing. A population of 1,500 expressions over 20 generations is <em>thirty thousand hypothesis tests</em>, and nobody applied a correction. GP is a machine for finding the best-looking pattern in your data, and if your data contains no pattern, it will hand you the best-looking coincidence instead, with total confidence and no warning label.</p><p><strong>Complexity is a tell.</strong> The real alpha was 5 nodes. The fake ones were 31 and 127. When the room has something to say it says it briefly; when it has nothing, it pads. Parsimony pressure isn’t just about compute or elegance — in a low-signal domain it is your primary defence against fooling yourself, and you should set it far more aggressively than feels comfortable.</p><p><strong>Out-of-bag selection earned its keep.</strong> Simply having each program evaluated on a held-out fraction of rows cut the bloat from 127 nodes to 31, and it cost nothing. It is one keyword argument. Use it.</p><p>The deeper point is the asymmetry with the previous section. Kepler’s law was findable from four planets because it was <em>there</em>, and had been for four and a half billion years, and would still be there next Tuesday. A market alpha may be an artifact of your sample, or real for eighteen months and then arbitraged into nothing by people running exactly the same search. Symbolic regression cannot tell those apart. Only out-of-sample discipline can, and in finance the out-of-sample set is called next year.</p><p>Same algorithm. Same code, near enough. Completely different burden of proof.</p><h3>Part six: when to walk into the room</h3><p><strong>Reach for it when:</strong></p><ul><li><strong>The equation is the deliverable.</strong> Physics, chemistry, materials, engineering : any setting where a formula is the thing you’re being paid to produce and a black box is not an acceptable answer.</li><li><strong>You’ll operate outside the training hull.</strong> This is the argument, and that 4.51 is the evidence.</li><li><strong>Data is scarce and structure is strong.</strong> A cubic from 120 noisy points. Deep learning is not in this conversation.</li><li><strong>Nothing is differentiable.</strong> Simulator-in-the-loop objectives, discrete combinatorial scores, non-smooth business constraints. The room only needs to <em>rank</em>; it never asks for a gradient.</li><li><strong>You want features, not a model.</strong> Underrated: evolve nonlinear features with GP, then hand them to your usual gradient booster. You keep the ensemble’s accuracy and gain inputs a human can read.</li></ul><p><strong>Walk away when:</strong></p><ul><li><strong>The input is perceptual.</strong> There is no compact expression for “is this a cat,” and the room will not find one.</li><li><strong>You have gradients and abundant data.</strong> Use them. GP will lose badly and slowly.</li><li><strong>You need determinism.</strong> Two of five runs, remember.</li><li><strong>The truth is high-dimensional and interaction-heavy.</strong> Search cost grows unpleasantly with the number of relevant variables.</li></ul><h4>Where this is actually going</h4><p>The interesting recent work all does the same thing: it hybridizes GP with something that’s good at what GP is bad at.</p><p><strong>PySR</strong> keeps the room for structure, adds a Pareto front over accuracy and complexity instead of a single parsimony knob, runs BFGS on the constants, and puts a fast Julia backend underneath. If you’re doing this for real, start here rather than with my ninety lines.</p><p><strong>AI Feynman</strong> uses a neural network as a <em>probe</em> rather than a model — training it briefly just to interrogate the data for symmetries, separability, and dimensional structure, then using those findings to recursively split the problem into pieces small enough for symbolic search to crack. The network never makes a prediction anyone uses. It’s a scout.</p><p><strong>Transformer-guided symbolic regression</strong> pre-trains a sequence model on millions of synthetic (dataset → expression) pairs, so that at inference it can <em>propose</em> plausible skeletons in a single forward pass, which are then refined numerically. The search gets amortized into the weights.</p><p><strong>FunSearch</strong> is the one to watch. A language model proposes the mutations; an evaluator scores them; the population evolves. Run that loop long enough and it produced new constructions in extremal combinatorics that beat the best known human results.</p><p>Look at what that last one really is. Classical GP’s deepest limitation was always that its writers are <em>blind</em> — they splice clauses at random, with no idea what a good clause looks like, which is why you need thousands of them and hundreds of generations to get anywhere. FunSearch replaces the blind writers with ones who have read everything ever published.</p><p>The room is the same. The room was never the problem. The room just got much better writers.</p><h3>Kelvin’s machine, one more time</h3><p>The brass tide-predictor sat in its cabinet and turned out beautiful curves, and it was right, and it was useful, and it saved lives at sea. There is nothing wrong with a machine that describes one harbour extremely well. Most of the value in applied machine learning is exactly that, and it would be silly to pretend otherwise.</p><p>Genetic programming is slow. It’s stochastic. It’s fussy about vocabulary, hopeless at arithmetic without help, and it will fail two runs in five and not tell you which. I’ve shown you all of that above rather than around it.</p><p>It is also the method that, handed four inner planets and no constants at all, wrote down T = a^(3/2) and then told me the length of Neptune&#39;s year to within a rounding error, while a random forest, given the same four planets, insisted Neptune goes round the Sun in nineteen months.</p><p>Sometimes you need the tide table. But it’s worth remembering that somebody, at some point, has to go and find the moon.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=762840f81997" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Simulating Markov Chains using Neural Networks]]></title>
            <link>https://medium.com/@sgsahoo77/simulating-markov-chains-using-neural-networks-81963a6d98d6?source=rss-97e4e79c842b------2</link>
            <guid isPermaLink="false">https://medium.com/p/81963a6d98d6</guid>
            <dc:creator><![CDATA[Sgsahoo]]></dc:creator>
            <pubDate>Sun, 18 Jan 2026 21:33:52 GMT</pubDate>
            <atom:updated>2026-01-18T21:33:52.520Z</atom:updated>
            <content:encoded><![CDATA[<h3>From Weather Data to a Discrete-Time Discrete-Space Markov Chain</h3><p>Many real-world systems evolve in a step-by-step manner under uncertainty. Weather is a familiar example. While the true atmospheric dynamics are governed by complex physical laws, in practice we often work with simplified, observable states such as <em>sunny</em>, <em>cloudy</em>, or <em>rainy</em>. If we record the weather once per day, we naturally obtain a sequence of observations evolving in discrete time.</p><p>Let X_t denote the weather on day t. By deciding to observe the system daily, time becomes discrete: t=0,1,2,… To make the model tractable, we also discretize the state space. Suppose we encode the weather as a finite set S={0,1,2} corresponding respectively to sunny, cloudy, and rainy conditions. Each random variable Xt now takes values in a finite set, turning the weather process into a <strong>discrete-time, discrete-space stochastic system</strong>.</p><p>The crucial modelling assumption is the <strong>Markov property</strong>. We assume that tomorrow’s weather depends only on today’s weather and not on the entire history of past observations. Formally, this means</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/413/0*3GyWKwCR4nnbA15X" /></figure><p>This assumption does not claim that the atmosphere has no memory; rather, it asserts that the current state summarizes all relevant information from the past for predicting the future. With discrete time, a finite state space, and the Markov property in place, the weather evolution becomes a <strong>discrete-time, discrete-space Markov chain</strong>.</p><p>All the dynamics of such a Markov chain are captured by its transition probabilities. For any two states i and j, we define</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/201/0*Bg6byWsZt-L-SLz6" /></figure><p>These probabilities can be arranged into a transition matrix P, where each row sums to one. For instance, a plausible transition matrix for weather might be</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/152/0*Lxrzs5ildG31a0KK" /></figure><p>If today is sunny, this model assigns a 70% chance of another sunny day, a 20% chance of clouds, and a 10% chance of rain. Rainy days, on the other hand, tend to persist.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/365/0*v6wDvTq1IDXSDCpt.jpg" /><figcaption>Markov Chain State Transition Diagram</figcaption></figure><p>Once the transition matrix is known, predicting future weather distributions becomes straightforward. Starting from an initial condition — say, a sunny day with probability one — the distribution of the weather several days into the future can be computed by repeated matrix multiplication.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/84/0*IoJqL6d9WeNsElOZ" /></figure><p>This formulation is elegant and powerful. It allows us to analyze long-term behavior, compute stationary distributions, and simulate future trajectories. However, it hides an important requirement: we must explicitly know or estimate every transition probability in the matrix. For small systems like this simplified weather model, that is reasonable. For large or complex systems, explicitly constructing and storing a transition matrix quickly becomes a bottleneck. This limitation motivates the search for alternative approaches that can model and simulate Markovian dynamics without ever writing down the transition probabilities themselves.</p><h3>Estimating Transition Matrices — and Why It Breaks Down</h3><p>Once a system has been modelled as a discrete-time, discrete-space Markov chain, the next step is to determine its transition matrix. In theory, the matrix contains all the information required to simulate and analyze the system. In practice, however, estimating this matrix from data introduces several structural limitations that motivate a different perspective.</p><p>The most common approach is <strong>frequency-based estimation</strong>. Given a long sequence of observed transitions, one counts how many times the system moves from state i to state j. If N_{ij} denotes the number of observed transitions from i to j, the <strong>maximum likelihood estimate</strong> of the transition probability is</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/103/0*knxxTsOIRRmr-HGN" /></figure><p>This estimator is statistically consistent under standard assumptions, and for small state spaces it performs well. Once the matrix is estimated, simulations proceed by sampling from the categorical distribution defined by each row of P.</p><p>Despite its simplicity, this approach hides several practical difficulties. First, the number of parameters grows quadratically with the size of the state space. A system with N states requires estimating N² probabilities, many of which may correspond to rare or unobserved transitions. This leads to unreliable estimates unless extremely large datasets are available. Second, the transition matrix must be stored explicitly, which quickly becomes infeasible as the state space grows or when states are defined by combinations of features rather than simple labels.</p><p>Even when the transition matrix is known or well estimated, simulation still requires explicit probability sampling at every step. Each transition involves drawing from a categorical distribution defined by a row of P. This is not merely an implementation detail; it reflects a deeper modelling choice. The dynamics are represented as probabilities first, and behavior is generated only by repeatedly sampling from those probabilities.</p><p>At this point, it is worth asking a more fundamental question: <strong>are probabilities the only way to represent stochastic dynamics?</strong> The answer is no. <strong>Any discrete-time Markov chain can be equivalently represented as a deterministic transition function driven by external randomness</strong>.</p><p>To see this, fix a state i and consider its transition probabilities as a cumulative sum proceed in the following way: then assuming xi follows a Uniform(0,1) distribution we can state a <strong>deterministic transition function</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/107/0*Soi-iEdSikLi_RJW" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/240/0*xsEdVWo297hI2Sa4" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/122/0*8kzKw-XXan95IF5e" /></figure><p>This construction reproduces the exact transition probabilities of the Markov chain, yet the transition itself is a <strong>deterministic function of the current state and a noise variable</strong>.</p><p>This representation reveals a crucial insight. A Markov chain does not fundamentally require explicit probabilities; it requires a mechanism that transforms the current state and a source of randomness into the next state. The transition matrix is only one way — albeit a convenient one — to describe this mechanism.</p><p>Once viewed this way, the limitations of explicit matrix estimation become clearer. Instead of learning a large table of probabilities and then sampling from it, we can aim to <strong>learn the transition function directly</strong>. Such a function implicitly encodes the transition probabilities without ever representing them explicitly. Moreover, because the Markov property guarantees that one-step transitions fully characterize the dynamics, this function can be learned from local transition data without requiring complete trajectories or terminal states.</p><p>This perspective naturally leads to <strong>implicit learning methods</strong>, where the goal is not to estimate probabilities but to learn a generative transition operator whose repeated application produces statistically correct dynamics. Neural networks are particularly well suited for this task. They can represent complex, state-dependent transition functions and absorb randomness through injected noise, effectively learning how to simulate the Markov chain rather than how to describe it probabilistically.</p><h3>Learning the Transition Operator: Generative Stochastic Networks</h3><p>The discussion so far leads to a subtle but powerful shift in perspective. Instead of viewing a Markov chain as a table of transition probabilities that must be estimated and stored, we can view it as a <strong>transition mechanism</strong> that maps the current state and a source of randomness to the next state. Once this shift is made, the question becomes not how to estimate probabilities, but how to <em>learn this transition mechanism directly from data</em>.</p><p>This is precisely the idea behind <strong>Generative Stochastic Networks (GSNs)</strong>. A GSN does not attempt to model a probability distribution explicitly. Instead, it learns a stochastic transition operator whose repeated application generates samples that follow the same dynamics as the original system. In the context of discrete-time Markov chains, this means learning a function that behaves like the transition function without ever constructing the transition matrix.</p><p>Concretely, suppose the system is currently in state s_t​∈S. We introduce an auxiliary noise variable z_t, typically sampled from a simple distribution such as a standard normal or uniform distribution. A neural network Gθ​, parameterized by weights θ, takes both the current state and the noise as input and outputs the next state:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/121/0*9gP4bw1GrJb8c9w8" /></figure><p>This function is deterministic given its inputs; all randomness in the dynamics is injected through z_t​. When the network is trained correctly, the distribution of s_t+1​ conditioned on s_t matches the true transition probabilities of the underlying Markov chain — even though those probabilities are never explicitly represented.</p><p>At first glance, this may look like a standard recurrent neural network. The difference is conceptual rather than architectural. A traditional RNN is designed to model long-range temporal dependencies by maintaining an internal hidden state. A GSN, on the other hand, relies entirely on the <strong>Markov property</strong>. The current state already summarizes the relevant past, so there is no need for memory beyond a single step. The network’s task is not sequence prediction, but <strong>learning a stochastic kernel</strong>.</p><p>Training such a model only requires access to observed one-step transitions. Given data pairs (s_t,s_t+1), the network is optimized so that, when provided with s_t​ and random noise, it assigns high likelihood to the observed next state s_t+1​. From the model’s perspective, learning the transition dynamics is a supervised learning problem with stochastic inputs. Crucially, there is no need to observe full trajectories, terminal states, or long-run equilibria. The Markov property guarantees that correct local transitions imply correct global behaviour.</p><p>What makes this approach especially attractive is scalability. The neural network’s number of parameters depends on the complexity of the transition function, not on the square of the state space size. This allows implicit transition models to handle large or structured state spaces where explicit transition matrices would be infeasible. Moreover, once trained, the model itself <em>is the simulator</em>: generating a trajectory simply amounts to repeatedly applying the learned transition function with fresh noise.</p><p>However, learning a transition function implicitly introduces a new challenge. It is easy for a model to match one-step transitions while quietly violating longer-term statistical properties of the chain. For example, a poorly trained model may drift away from the true stationary distribution or exhibit unstable dynamics over long rollouts. This observation motivates a more structured learning framework — one that does not merely generate transitions, but actively enforces consistency of the learned dynamics. This is where the generator–resolver architecture becomes essential.</p><h3>Why a Generator Alone Is Not Enough: The Role of the Resolver</h3><p>At this point, it may seem that learning a stochastic transition function is sufficient. If a neural network can generate the correct next state distribution given the current state, then repeatedly applying it should recover the true dynamics of the Markov chain. In practice, however, this intuition turns out to be incomplete.</p><p>The core issue is that <strong>matching one-step transitions does not guarantee correct long-term behaviour</strong>. A model may reproduce observed transitions locally while accumulating subtle errors that distort the dynamics over time. For example, the learned system may drift toward states that should be rare, fail to reproduce the correct stationary distribution, or exhibit unstable behavior after many steps. These failures are often invisible when training is based solely on one-step prediction loss.</p><p>This phenomenon is not unique to Markov chains; it appears throughout generative modeling. In the context of Markov processes, it highlights an important distinction between learning a transition rule that <em>looks correct locally</em> and learning one that is <em>globally consistent</em>. The Markov property ensures that local transitions determine global behavior only if those transitions are learned exactly. Any small systematic bias is amplified through repeated application of the transition operator.</p><p>To understand this more formally, consider the true transition operator T of the Markov chain and a learned approximation T*. Even if T* is close to T in a one-step sense, the error in T*^n can grow with n. As a result, the distribution generated by the learned model after many steps may differ significantly from the true distribution, even though the one-step marginals appear correct.</p><p>This observation motivates the introduction of a second component: a <strong>resolver</strong>. While the generator proposes transitions, the resolver evaluates whether the resulting dynamics satisfy the statistical laws expected of the true Markov chain. The resolver does not predict the next state directly. Instead, it acts as a critic, assessing the quality of the generated transitions and their long-term consequences.</p><p>Conceptually, the generator and resolver form a game. The generator attempts to produce transitions that resemble those observed in data, while the resolver attempts to detect discrepancies between real and generated dynamics. Importantly, the resolver is not restricted to one-step checks. It can examine multi-step transitions, empirical state frequencies, or invariance properties that must hold under the true dynamics. By penalizing violations of these properties, the resolver forces the generator to learn transitions that are consistent not only locally but also globally.</p><p>This setup resembles adversarial training in generative modelling, but with a crucial difference. Instead of discriminating between individual samples, the resolver discriminates between <strong>dynamical behaviour</strong>. Its role is to ensure that the learned transition operator behaves like a true Markov kernel when iterated over time. In doing so, it implicitly enforces constraints that would otherwise need to be encoded explicitly through probability matrices or analytical conditions.</p><p>From a modelling perspective, the generator–resolver architecture replaces explicit probabilistic constraints with learned consistency checks. The generator learns <em>how</em> to move through the state space, while the resolver learns <em>what it means</em> for those movements to be statistically correct. Together, they allow the system to learn an implicit transition model that reproduces both short-term transitions and long-term structure.</p><p>This architecture also provides a natural bridge to more advanced settings. Once uncertainty becomes adversarial or structured — such as in robust decision-making or stochastic games — the resolver’s role expands from consistency checking to enforcing worst-case behaviour. In that sense, the generator–resolver framework can be viewed as the discrete-time precursor of adversarial stochastic control.</p><h3>A Minimal Generator–Resolver Model: Architecture and Training Objective</h3><h4>Generator: Learning the Transition Kernel</h4><p>The generator is a lightweight recurrent model that takes the current state X_t​ and outputs a distribution over the next state X_t+1. Since our state space is discrete, we represent states as one-hot vectors. The generator is therefore learning a function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/104/0*lRudDEXutAh4cVeE" /></figure><p>where Δ^∣S∣ is the probability simplex over states.</p><p>Importantly, this distribution is <em>implicit</em>. We never tell the model what the transition probabilities are. Instead, the generator is trained to produce next-state samples that are statistically indistinguishable from empirical transitions observed in the data.</p><h4>Resolver: Enforcing Markov Consistency</h4><p>If we train the generator naively with cross-entropy, we simply recover a neural approximation of the transition matrix. The resolver adds structure. Its role is to check whether generated transitions are <em>consistent</em> with real transitions at the level of distributions, not pointwise probabilities.</p><p>The resolver receives pairs (X_t,X_t+1) and learns to distinguish between:</p><ul><li>real transitions from data,</li><li>generated transitions from the generator.</li></ul><p>This is conceptually similar to adversarial training, but simpler: the resolver does not see full trajectories, only one-step transitions. This aligns perfectly with the Markov assumption.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/0*vFxgOWFt2B5Z2i2r.png" /><figcaption>Model Architecture Example of Adversarial Networks</figcaption></figure><h4>Training Objective</h4><p>The generator and resolver are trained in opposition. The resolver learns to correctly classify real vs generated transitions, while the generator learns to fool the resolver. At equilibrium, the generator reproduces the true transition dynamics of the Markov chain — without ever computing or storing P.</p><p>Crucially, this setup learns the transition function <em>directly</em>, not via sampling full paths or estimating long-run frequencies.</p><h3>A Clean PyTorch Implementation</h3><p>Below is a minimal implementation for a DTDS Markov chain with a small state space (e.g., weather states: Sunny, Cloudy, Rainy). The code assumes we only have access to observed transitions.</p><pre>import torch<br>import torch.nn as nn<br>import torch.optim as optim<br><br># number of discrete states<br>num_states = 3<br><br># ---------------------<br># Generator Network<br># ---------------------<br>class Generator(nn.Module):<br>    def __init__(self, num_states, hidden_dim=16):<br>        super().__init__()<br>        self.net = nn.Sequential(<br>            nn.Linear(num_states, hidden_dim),<br>            nn.Tanh(),<br>            nn.Linear(hidden_dim, num_states)<br>        )<br><br>    def forward(self, x):<br>        logits = self.net(x)<br>        return torch.softmax(logits, dim=-1)<br><br><br># ---------------------<br># Resolver Network<br># ---------------------<br>class Resolver(nn.Module):<br>    def __init__(self, num_states, hidden_dim=16):<br>        super().__init__()<br>        self.net = nn.Sequential(<br>            nn.Linear(2 * num_states, hidden_dim),<br>            nn.ReLU(),<br>            nn.Linear(hidden_dim, 1),<br>            nn.Sigmoid()<br>        )<br><br>    def forward(self, x_t, x_next):<br>        joint = torch.cat([x_t, x_next], dim=-1)<br>        return self.net(joint)<br><br><br># ---------------------<br># Utility Functions<br># ---------------------<br>def one_hot(indices, num_states):<br>    return torch.eye(num_states)[indices]<br><br><br>def sample_next_state(probs):<br>    return torch.multinomial(probs, num_samples=1).squeeze()<br><br><br># ---------------------<br># Models and Optimizers<br># ---------------------<br>G = Generator(num_states)<br>R = Resolver(num_states)<br><br>opt_G = optim.Adam(G.parameters(), lr=1e-3)<br>opt_R = optim.Adam(R.parameters(), lr=1e-3)<br><br>bce = nn.BCELoss()<br><br># ---------------------<br># Dummy empirical transitions<br># (replace with real data)<br># ---------------------<br>x_t_indices = torch.tensor([0, 1, 2, 0, 1])<br>x_next_indices = torch.tensor([1, 2, 0, 0, 1])<br><br>x_t = one_hot(x_t_indices, num_states)<br>x_next_real = one_hot(x_next_indices, num_states)<br><br># ---------------------<br># Training Step<br># ---------------------<br>for epoch in range(1000):<br>    # ----- Train Resolver -----<br>    R.zero_grad()<br><br>    real_labels = torch.ones(len(x_t), 1)<br>    fake_labels = torch.zeros(len(x_t), 1)<br><br>    real_preds = R(x_t, x_next_real)<br>    loss_real = bce(real_preds, real_labels)<br><br>    gen_probs = G(x_t)<br>    sampled_indices = sample_next_state(gen_probs)<br>    x_next_fake = one_hot(sampled_indices, num_states)<br><br>    fake_preds = R(x_t, x_next_fake.detach())<br>    loss_fake = bce(fake_preds, fake_labels)<br><br>    loss_R = loss_real + loss_fake<br>    loss_R.backward()<br>    opt_R.step()<br><br>    # ----- Train Generator -----<br>    G.zero_grad()<br><br>    fake_preds = R(x_t, x_next_fake)<br>    loss_G = bce(fake_preds, real_labels)<br><br>    loss_G.backward()<br>    opt_G.step()<br><br>    if epoch % 200 == 0:<br>        print(f&quot;Epoch {epoch} | Resolver Loss: {loss_R.item():.4f} | Generator Loss: {loss_G.item():.4f}&quot;)</pre><h3>From Transition Matrices to Transition Functions — and Beyond</h3><p>Let us step back and restate the core objective that guided this article. The traditional theory of discrete-time discrete-space Markov chains is built around the transition matrix P. While mathematically elegant, this object is ultimately a <em>storage mechanism</em> for probabilities, not the dynamics themselves. Constructing it requires explicit estimation of transition probabilities, careful counting, and often large amounts of data. More importantly, it scales poorly and becomes conceptually restrictive when we move beyond passive stochastic systems.</p><p>What we have pursued instead is the idea of a <strong>transition function</strong>: a learnable operator that maps the current state directly to a distribution over the next state, without ever materializing the matrix that classical theory insists on. By exploiting the Markov property, we only need local consistency — one-step transitions — rather than complete trajectories or terminal outcomes. Neural networks are a natural fit here, as they learn conditional distributions implicitly and generalize beyond tabulated probabilities.</p><p>The generator–resolver architecture makes this idea precise. The generator represents the transition function itself, while the resolver enforces statistical consistency with observed data. Together, they recover the underlying stochastic dynamics purely from state transitions. At no point do we estimate probabilities explicitly, yet the learned system reproduces the same behaviour as the theoretical Markov chain. In effect, we have replaced a matrix with a mechanism.</p><p>This shift is not merely a computational convenience — it is a conceptual upgrade. Once we stop thinking in terms of transition matrices, we are no longer limited to passive dynamics. We can ask richer questions: <em>What if transitions are influenced by competing agents? What if the system evolves under opposing objectives?</em></p><p>This is where zero-sum Markov chains naturally enter the picture. In adversarial settings, transitions are no longer governed by a single stochastic kernel but by a <strong>min–max interaction</strong> between agents with conflicting goals. Classical approaches replace the transition matrix with a collection of controlled kernels and Bellman-style min–max operators, quickly becoming analytically and computationally heavy.</p><p>The transition-function viewpoint offers a cleaner alternative. The same generator–resolver framework can be extended so that transitions are produced by competing generators — one minimizing, one maximizing — while a shared resolver enforces consistency with observed or simulated dynamics. In this setting, learning the transition function becomes equivalent to learning the equilibrium of a zero-sum Markov game, without explicitly solving Bellman equations or estimating controlled transition matrices.</p><p>This perspective is not just philosophically appealing; it aligns closely with modern machine learning practice in adversarial training, model-based reinforcement learning, and stochastic control. It also sets the stage for continuous-time limits, where transition matrices disappear entirely and are replaced by learned operators approximating infinitesimal generators — bridging discrete Markov chains and stochastic differential games.</p><p>In short, by moving from matrices to functions, we unlock a pathway from classical Markov chains to adversarial stochastic systems. What begins as a simple weather model becomes a blueprint for learning dynamics in finance, control, and game-theoretic environments — without ever writing down a transition matrix.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=81963a6d98d6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Can Machines Imagine? Discussing Dreaming Neural Networks]]></title>
            <link>https://medium.com/@sgsahoo77/can-machines-imagine-discussing-dreaming-neural-networks-bf15938dcd24?source=rss-97e4e79c842b------2</link>
            <guid isPermaLink="false">https://medium.com/p/bf15938dcd24</guid>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[dreaming]]></category>
            <category><![CDATA[neuroscience]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Sgsahoo]]></dc:creator>
            <pubDate>Mon, 16 Jun 2025 01:04:42 GMT</pubDate>
            <atom:updated>2025-06-16T01:04:42.044Z</atom:updated>
            <content:encoded><![CDATA[<h3>Objective</h3><p>The purpose of this article is to give intuitive information about the notion of “dreaming” and “imagination” in the context of Neural Networks and related algorithms. I have tried to keep it beginner-friendly — as long as you are familiar with terminologies like “neural networks”, “reinforcement learning”, some basic linear algebra etc. you should be able to follow along. Further, a dive into the technicalities and mathematics of “dreaming” isn’t the objective of this article. In this article I have tried to pick the essence of dreaming in human and machine context, as well as some recent research papers published in this domain. You may find them in the “inspiration” section.</p><h3>“You know that without trying?”</h3><p>Imagine you’re sitting in a packed auditorium. The room hums with curiosity as I walk up to the stage. “Okay okay,” I begin with a smile, “so I’ll begin with a simple example. Can someone from the audience join me?”</p><p>A volunteer steps forward. I ask, “Now I want you to tell me what will happen if you run straight from here with full speed — no turning, no braking at all.”</p><p>They chuckle. “I’ll hit the wall and injure myself.”</p><p>“Yeah? But how do you know? Have you ever actually run into a wall at full speed?”</p><p>“No,” they reply.</p><p>“Then how do you know without trying it out?”</p><p>They pause. “I’ve heard about it.”</p><p>“Heard about it? What kind of environment are you living in where people say, ‘Hey, if you run into a wall at high speed, it’ll hurt’?”</p><p>We all laugh, but I press further. “So you’ve never done it, yet you believe it’ll hurt. Why?”</p><p>“I can imagine what will happen,” they say.</p><p>“Ahh… ‘imagine.’ You’re telling me you can <em>do</em> something in your mind without actually doing it? Like you can simulate it in your head? Do you picture it in first-person or third-person?”</p><h3>Notion of Imagination</h3><p>Through your years on Earth, you’ve built a mental model of the world:</p><ul><li>You know walls are solid because you’ve touched them, felt them, maybe even banged your head against one as a kid.</li><li>You know running because you’ve run, fallen, learned.</li></ul><p>You didn’t read a manual. You experienced it. That’s <strong>reinforcement learning</strong> — try, fail, get feedback, improve.</p><p>But now, you’re doing something else entirely: you’re simulating outcomes based on memory and experience. You’re <strong>imagining</strong>.</p><p>Classic AI agents — robots, game AIs, neural networks — operate like children who must learn everything the hard way:</p><ul><li>They try something.</li><li>They see the result.</li><li>They adjust.</li></ul><p>This works fine… until it doesn’t. What if trying something could be dangerous or costly? What if we want them to learn faster, with fewer failures?</p><p>Enter <strong>Dreaming Neural Networks</strong>.</p><p>Just like humans, these networks build a model of their environment based on past data. Then, instead of acting in the real world, they act <em>inside their imagination</em>. They simulate actions and outcomes inside a learned “world model,” and update their policies based on imagined experiences. It’s like giving AI the ability to daydream. The beauty of Dreaming Neural Networks is that they allow for <strong>offline training</strong>.</p><p>Here’s how:</p><ol><li>The agent collects data from past experience — real or simulated.</li><li>It trains a model of the environment .</li><li>Instead of interacting with the real environment, it creates imagined trajectories using this model.</li><li>It updates its behaviour (policy) from this imagined data.</li></ol><p>This means the agent can keep training <em>even when it’s not connected to the environment</em>. Like how you can learn chess strategies just by replaying games in your head. Offline training reduces risk, saves time, and makes learning more efficient — especially in robotics, finance, or medicine, where trials can be expensive or irreversible.</p><h3>Where do we stand?</h3><p>Interestingly, the idea of neural networks that ‘dream’ isn’t new. In 2018, researchers proposed a system where networks, like our brains during sleep, clean up and strengthen memories offline. But today’s Dreaming Neural Networks go a step further — not just cleaning memory, but imagining futures, simulating experiences, and learning without ever touching the real world.</p><p>A paper in 2019 introduced the world to “Dreamer”, a reinforcement learning agent that solves long-horizon tasks from images purely by latent imagination. Let us understand what it really does:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oVqlGF9q4tRUltoXIUOI3A.png" /><figcaption>excerpt from “DREAM TO CONTROL: LEARNING BEHAVIOURS BY LATENT IMAGINATION”</figcaption></figure><p>Dreamer learns long-horizon behaviors in the compact latent space of a learned world model by efficiently leveraging the neural network latent dynamics. The latent dynamics define a Markov decision process that is fully observed because the compact model states are Markovian. Imagined trajectories start at the true model states of observation sequences drawn from the agent’s past experience. They follow predictions of the transition model, reward model, and a policy. The objective is to maximize expected imagined rewards with respect to the policy.</p><h3>How Imagination works</h3><p>Let me oversimplify the notion of <strong>imagination in agents</strong>.<br> This is as simple as <em>attaching a memory of the environment with the agent</em>. We know the agent works by remembering the <strong>actions</strong> it had done that yielded fruitful results — <strong>now it is remembering the environment as well</strong>.</p><p>Of course, by “memory” I do not mean an exact mapping of the environment, like some pixel-perfect blueprint. If it is to remember the whole environment, our purpose of imagination would be defeated; in that case, a regular RL model would be more effective in terms of both computation and memory. Instead, what Dreamer does is build a <strong>latent world model</strong> — a compressed, abstract understanding of how the environment works — using everything it has learned <em>so far</em>. This world model is trained from real experiences, but once built, it allows the agent to do something magical: <em>learn without doing</em>.</p><p>Now let us suppose I am training a maze solver. I have divided the maze into 9 segments of 1×1 squares. My model operates in an episodic manner, collecting rewards for every step it makes that reduces its Euclidean distance from the exit. If it hits a wall, a heavy penalty is imposed. After a certain number of tries, the model is able to reach the exit, but it’s still not choosing the optimal route.</p><p>Now the RL agent has completely mapped a 1×1 segment in its tries.<br> <strong>Now it sits with its “latent space” which has information about where the walls are and where the goal is.</strong> It knows walls are a big no-no and that it should move towards the goal.</p><p><strong>It draws out its imaginative trajectories, moving in the latent space, using the rewards and the actions it has “learned” till now… and therefore is now able to optimally traverse that much segment.</strong></p><p>This is where the magic happens.</p><p>Even if the agent hasn’t completely mapped every nook and corner of the maze, it is still able to use its <strong>Recurrent State-Space Model (RSSM), </strong>a model that maps the environment and its states — the state-space keeping in mind the importance of sequentiality i.e. the sequence in which the states transition,<strong> </strong>to generate something <em>similar</em> to a maze in that unknown area, in its imagination. It keeps travelling in its mind, going to all the corners of the maze and optimizing its path.</p><p><strong>Now it is able to choose the optimal path to traverse the maze, all without having to “explore” or “exploit” in the real world.</strong> This ability to <em>predict</em> its environment despite not knowing what exactly it is, is what gives Dreamer the edge over traditional RL models.</p><h4>Imagination Horizon</h4><p>In our maze example, the 1×1 segment that the Dreamer agent has explored thoroughly becomes a kind of <strong>core memory</strong> — its most precise and confident internal representation of the world. You can think of this as the agent’s <strong>“imagination horizon”</strong>: a region where predictions are highly accurate, grounded in experience, and the imagined trajectories are closest to reality.</p><p>Beyond this horizon, the agent <em>still dreams</em> — but those dreams are <em>increasingly abstract</em>. The farther it imagines, the more it relies on the <strong>patterns</strong> and <strong>latent dynamics</strong> it has learned so far. Just like how humans might predict what’s around a corner based on intuition rather than fact, the Dreamer tries to fill in unknown regions using learned statistical structure, not direct memory.</p><p>This <strong>gradual decay in certainty</strong> across the imagined space gives the model both <strong>flexibility</strong> (it can think about unexplored areas) and <strong>discipline</strong> (it knows how much trust to place in its imagination).</p><h3>Is it really enough?</h3><p>Let us understand a subtle yet critical flaw in some Dreaming Neural Networks, especially those that rely on <strong>reconstruction-based learning</strong> like the original Dreamer architecture.</p><p>Suppose Dreamer is now assigned a task that’s slightly different from its usual continuous control environments — say, to classify whether an image contains a human or not. The model is trained on images of human faces. These images include painted faces. The agent learns, over time, that features like skin tone are less useful compared to more distinctive elements like the eyes, nose, and mouth.</p><p>Now in its <strong>latent space</strong>, it learns to <em>focus on reconstructing</em> these critical facial features while discarding less salient ones — like background textures or even skin details. As a result, when it generates images from this latent space (via its decoder), the outputs often include the most prominent features like eyes, nose, hairs and facial contours, but the rest is a haze. This hazy part is another example of an <em>imagination horizon.</em></p><p>This works well in many cases. But what happens if you now show the model a picture of a statue? The model may confidently — but incorrectly — classify it as a human because it never learned the <strong>texture</strong> differences that separate real skin from stone. Why? Because the <strong>decoder</strong>, which turns low-dimensional latent vectors into images, never prioritized reconstructing textures — it only learned to reconstruct what was rewarded or frequent in the training data. The most distinct feature of a statue — its texture — was never represented clearly in the latent space.</p><p>This leads to a fundamental realization: <em>“To scale something of a lower dimension to a higher dimension will experience loss of details.” </em>That’s where <strong>decoder-free dreaming</strong> comes in. Recent research has proposed removing the decoder entirely and instead focusing on learning <strong>highly informative latent representations.</strong></p><p>These techniques train the model to keep as much semantically relevant information in the latent space as possible, without needing to reconstruct it into full pixel space. This frees up representational capacity and allows the model to encode things that aren’t always visually dominant but are still important for decision-making and control.</p><p>It’s like saying: I want to observe and incorporate as many things as I can observe in my active ‘awake’ state even if I can’t really imagine it. Let me present another analogy to make it clear: Imagine someone who is blind from birth trying to understand what a rainbow is. They may never be able to <em>imagine</em> it visually — they do not what colours are — but through rich descriptions and symbolic understanding, they can still know what it is, what it implies, and how to distinguish it from other things. To fit it into an RL problem, let us say that they were never told about a rainbow, but were presented with the task to tell if something is rainbow or not. Over time they will try to learn and derive as many properties of rainbow as they can. Now of course the major part of a rainbow is the color sequence, which they cannot imagine, but they will realise it soon enough, and, even though they cannot imagine it, they would still have its information “encoded” in them. Now it would be wrong to say that they will not have an imagination of a rainbow, but it will not be “pixel perfect” and will be “different” from normal people (who make pixel-perfect imaginations). But they rely on more “information gathering” than imagination to aid them.</p><p>That’s what decoder-free dreaming offers: a more information-rich, imagination-driven learning framework that doesn’t rely on pixel-perfect reconstructions to learn meaningful behavior. It enables agents to <em>dream not in images</em>, but in pure concepts — optimized for control, understanding, and generalization.</p><h3>Teasing the idea again</h3><p>I would end on one question: can you imagine a world where imagination doesn’t exist? No? Is there a way to know about this world if imagination didn’t exist? Guess we’ll never find out. With this, a brief discussion on what imagination and dreaming feels for a machine ends. Advances continue in this field. I would like to add a paper related to “Neuromorphic Dreaming” in the next segment for interested readers.</p><h3>Inspiration</h3><p>This article and the journey to write to publish it started as a random shower thought. But many google searches and GPT prompts later I was able to curate something that could quench my own thirst for curiosity. Here are the references for the article:</p><ol><li>“Dream to Control: Learning Behaviors by Latent Imagination” (Hafner et al., 2019)</li><li>“Dreaming: Model‑based Reinforcement Learning by Latent Imagination without Reconstruction” (Okada &amp; Taniguchi, 2020)</li><li>“Neuromorphic dreaming: A pathway to efficient learning in artificial agents” (Blakowski et al., 2024)</li><li>“Dreaming neural networks: forgetting spurious memories and reinforcing pure ones.”</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bf15938dcd24" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Gaussian-Weighted Word Embeddings for Sentiment Analysis]]></title>
            <link>https://medium.com/@sgsahoo77/gaussian-weighted-word-embeddings-for-sentiment-analysis-23819bf503e9?source=rss-97e4e79c842b------2</link>
            <guid isPermaLink="false">https://medium.com/p/23819bf503e9</guid>
            <category><![CDATA[word-embedding-in-nlp]]></category>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Sgsahoo]]></dc:creator>
            <pubDate>Sun, 01 Jun 2025 10:57:07 GMT</pubDate>
            <atom:updated>2025-06-01T10:57:07.688Z</atom:updated>
            <content:encoded><![CDATA[<p>In the realm of natural language processing (NLP), sentence embeddings play a crucial role in determining the overall semantic meaning of a given text. Traditionally, averaging pre-trained word embeddings like Word2Vec or GloVe has served as a straightforward yet effective technique. But what happens when your data isn’t simple? What if you’re analyzing nuanced, lengthy movie reviews that contain both praise and criticism? That’s when simple averaging falls short.</p><p>In this blog, we’ll explore a novel technique: <strong>Gaussian-weighted word embeddings</strong>. This method weights each word vector based on its proximity to a centroid, reducing the influence of outliers and preserving semantic richness. We’ll walk through the theory, implementation, and how it performs in a full machine learning pipeline.</p><h3>The Problem with Simple Averaging</h3><p>In <em>n</em>-dimensional hyperspace, we often assume that words with “positive” sentiment occupy regions far removed from those with “negative” sentiment. Under this assumption, simple averaging may yield vectors that fall close to a class centroid, making classification easy.</p><p>However, movie reviews are often lengthy and multi-faceted. A single review might highlight both the brilliant acting and a weak storyline. In such cases, averaging the word vectors may result in semantic dilution, where opposing sentiments cancel each other out, confusing the classifier.</p><h3>The Solution: Gaussian-Weighted Word Embeddings</h3><p>To mitigate this, a Gaussian-weighted approach is proposed. The idea is to:</p><ol><li>Compute the centroid <strong>G</strong> of the word vectors in a sentence.</li><li>Calculate the distance <strong>D</strong> from <strong>G</strong> to the farthest word vector.</li><li>Assign weights to each word vector using a Gaussian distribution centred at <strong>G</strong> with a variance of <strong>D/2</strong>.</li><li>Aggregate the sentence vector using these weights.</li></ol><h4><strong>Why This Works?</strong></h4><p>The main idea behind such a sampling is to reduce the weight of “those that lie farther from the interpretation.” Since we are already exploring the example of movie review classification, I may assume that such reviews both laud and criticise the movie. Even for a human being, to classify such a review would be a thoughtful task. You may want to focus on the tone of the review and what it majorly talks about — the positive sentiment or the negative sentiment. Thus, in such a case the vectors lie scattered across the space, and it becomes tougher to cluster them together to generate a single input vector. Thus, in assigning Gaussian-sampled weights to the word vectors, we reduce the scattering by trying to centralise the notion of the review and selectively valuing those sentiments that are closer to the central interpretation.</p><h3>The Formula</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/333/0*G6U2Qb50b8mMJqRl.png" /><figcaption>Gaussian-weighted resultant vector</figcaption></figure><h3>Implementation in Python</h3><h4>Step 1: Preprocessing the Data</h4><pre>import re<br>import nltk<br>from nltk.corpus import stopwords<br>from nltk.tokenize import word_tokenize<br>from nltk.stem import WordNetLemmatizer<br>nltk.download(&#39;punkt&#39;)<br>nltk.download(&#39;stopwords&#39;)<br>nltk.download(&#39;wordnet&#39;)<br><br>stop_words = set(stopwords.words(&#39;english&#39;))<br>lemmatizer = WordNetLemmatizer()<br><br>def clean_text(text):<br>    text = re.sub(r&#39;[^a-zA-Z0-9\s]&#39;, &#39;&#39;, text.lower())<br>    tokens = word_tokenize(text)<br>    tokens = [lemmatizer.lemmatize(word) for word in tokens if word not in stop_words]<br>    return tokens<br><br>df[&#39;clean_text&#39;] = df[&#39;text&#39;].apply(clean_text)</pre><h4>Step 2: Train a Word2Vec Model</h4><pre>from gensim.models import Word2Vec<br><br>model = Word2Vec(sentences=df[&#39;clean_text&#39;], vector_size=100, window=5, min_count=2, workers=4)<br>word_vectors = model.wv</pre><h4>Step 3: Gaussian-Weighted Sentence Vector</h4><pre>import numpy as np<br><br>def get_weighted_sentence_vector(tokens, word_vectors):<br>    vectors = [word_vectors[word] for word in tokens if word in word_vectors]<br>    if not vectors:<br>        return np.zeros(word_vectors.vector_size)<br>    <br>    vectors = np.array(vectors)<br>    centroid = np.mean(vectors, axis=0)<br>    distances = np.linalg.norm(vectors - centroid, axis=1)<br>    max_dist = np.max(distances) or 1e-6<br>    <br>    weights = np.exp(-((distances / (max_dist / 2)) ** 2))<br>    weighted_sum = np.sum(weights[:, np.newaxis] * vectors, axis=0)<br>    return weighted_sum / len(vectors)<br><br>X = np.array([get_weighted_sentence_vector(tokens, word_vectors) for tokens in df[&#39;clean_text&#39;]])<br>y = df[&#39;label&#39;].values</pre><h4>Step 4: Training Classifiers</h4><p>You can now use any scikit-learn classifier:</p><pre>from sklearn.model_selection import train_test_split<br>from sklearn.ensemble import RandomForestClassifier<br>from sklearn.metrics import classification_report<br><br>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)<br><br>clf = RandomForestClassifier()<br>clf.fit(X_train, y_train)<br>y_pred = clf.predict(X_test)<br><br>print(classification_report(y_test, y_pred))</pre><h3>Conclusion</h3><p>By applying Gaussian-weighted sentence vectors, we enhance the robustness of text representations for complex reviews. This method not only reduces noise from contradictory sentiments but also allows models to better distinguish sentiment in nuanced texts.</p><p>This approach is particularly useful in real-world sentiment analysis applications like movie reviews, where the text often contains mixed sentiments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=23819bf503e9" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>