<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Florian Kozikowski on Medium]]></title>
        <description><![CDATA[Stories by Florian Kozikowski on Medium]]></description>
        <link>https://medium.com/@florian.kozikowski?source=rss-8fa5f1e9b3cd------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*JvdlwXO5RGJHgs1yr66kGA.jpeg</url>
            <title>Stories by Florian Kozikowski on Medium</title>
            <link>https://medium.com/@florian.kozikowski?source=rss-8fa5f1e9b3cd------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 00:59:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@florian.kozikowski/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building custom Generalized Linear Models with skglm: Tailored Datafits, Penalties and Solvers]]></title>
            <link>https://medium.com/@florian.kozikowski/building-custom-generalized-linear-models-with-skglm-tailored-datafits-penalties-and-solvers-ca468490571a?source=rss-8fa5f1e9b3cd------2</link>
            <guid isPermaLink="false">https://medium.com/p/ca468490571a</guid>
            <category><![CDATA[skglm]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[generalized-linear-model]]></category>
            <category><![CDATA[scikit-learn]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Florian Kozikowski]]></dc:creator>
            <pubDate>Thu, 31 Jul 2025 15:06:48 GMT</pubDate>
            <atom:updated>2025-07-31T15:06:48.211Z</atom:updated>
            <content:encoded><![CDATA[<h3>Custom GLMs in Python with skglm: Build your own Datafit</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ac8BvAHh3bAGlcsZ.png" /></figure><p>Generalized Linear Models (GLMs) are widely used in machine learning because they’re easy to interpret and backed by strong theoretical guarantees. But real-world datasets often don’t fit standard assumptions. Data can be skewed (like Poisson counts), contain outliers (requiring robust losses such as the Huber loss), or demand structured sparsity beyond standard ℓ₁ or ℓ₂ penalties. Customizing the loss function and penalty can significantly improve your model by aligning it more closely with the specific characteristics of your data.</p><p>The Python library <a href="https://contrib.scikit-learn.org/skglm/index.html"><strong>skglm</strong></a> was created exactly for this flexibility (<a href="https://arxiv.org/abs/2204.07826">Bertrand et al., 2022</a>). It splits a GLM into three modular components — data-fit, penalty, and solver — each easily customizable through straightforward Python interfaces. This modular design allows you to quickly prototype new objectives without rewriting optimization routines, all while maintaining compatibility with scikit-learn and excellent performance.</p><p>In this tutorial, you’ll learn step-by-step how to create your own custom loss function (datafit) and integrate it into a working estimator. For custom penalties and solvers, stay tuned for upcoming articles in this series.</p><p><strong>The Modular Architecture of skglm</strong></p><p>As already mentioned, the core strength of <strong>skglm</strong> is its modular architecture: every estimator is built from three distinct and interchangeable components — <strong>Datafit</strong>, <strong>Penalty</strong>, and <strong>Solver:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ohKYz05sE9_LDPI3gae_rA.png" /></figure><p>This modular design has several advantages:</p><ul><li><strong>Flexibility</strong>: Mix-and-match existing components (e.g., combining a Cox loss with a fused Lasso penalty).</li><li><strong>Fast prototyping</strong>: Quickly develop and test new ideas by writing concise component classes without rewriting optimization logic.</li><li><strong>Fair benchmarking</strong>: Keep two components constant and vary the third to clearly isolate its impact on performance.</li></ul><p>Before we start off creating our custom datafit, make sure you have your environment set up correctly (pip install skglm, etc.).</p><h4>Rationale for Customizing Data-fit and Regularization</h4><p>Standard GLMs, while broadly useful, often make simplifying assumptions — such as normally distributed errors — that don’t always hold in practice. Real-world datasets can exhibit unique structures or complexities, such as:</p><ul><li><strong>Heavy-tailed distributions</strong>: Outliers or unusual observations that are better handled with robust loss functions (e.g., Huber loss).</li><li><strong>Asymmetric or domain-specific losses</strong>: Count data (e.g., Poisson-distributed events), survival analysis (Cox regression), or bespoke prediction metrics.</li></ul><p>By customizing the data-fit, you incorporate domain knowledge directly into the model. This leads to more accurate, interpretable results and improved predictive performance, effectively tailoring the model to your specific problem. skglm’s modular architecture makes such customization straightforward, enabling practitioners and researchers to quickly test and validate novel modeling approaches.</p><h4>Writing a Custom Data-Fit</h4><p>Before you start making your own datafit, first check if it is not already available on skglm. As of June 2025, skglm ships already a lot of useful loss functions:</p><ol><li><strong>Regression losses:</strong> <em>Quadratic</em> (MSE) <em>loss</em> (standard regression with normally distributed errors), <em>Weighted Quadratic loss</em> (when samples have different importance), <em>Huber loss</em> (robust to outliers), <em>Poisson loss</em> (count data),<em> Gamma loss</em> (continuous positive values), <em>Cox loss </em>(for survival analysis), <em>Pinball loss</em> (for quantile regression), <em>QuantileHuber </em>(quantile regression, easier to solve than Pinball; in development)</li><li><strong>Classification losses: </strong><em>Logistic loss</em> (binary cross entropy for binary classification), <em>QuadraticSVC loss</em> (for SVM classification)</li><li><strong>Multi-task losses:<em> </em></strong><em>QuadraticMultiTask loss</em> (multiple related regression tasks simultaneously)</li><li><strong>Group-based losses: </strong><em>QuadraticGroup loss</em> (regression with grouped features), <em>LogisticGroup loss</em> (binary classification with grouped features)</li></ol><p>Okay, so let’s say none of those datafits are perfectly suitable for our problem.</p><p>Every skglm datafit needs at least three components.</p><ol><li><strong>The loss function (</strong>value<strong>): </strong>This tells us “how wrong we are.” Think of it as a score that gets smaller when our predictions improve.</li><li><strong>The gradient (</strong>grad<strong>): </strong>This is our compass, pointing “which way to go.” It’s the derivative of the loss with respect to our predictions.</li><li><strong>The Lipschitz constant (</strong>L<strong>): </strong>This is our “speed limit.” It tells the optimizer the maximum rate at which the gradient can change. Without it, the optimizer might take steps that are too large and overshoot the minimum.</li></ol><h4>Example: Tweedie Data-fit</h4><p>Let’s pick an example to better showcase how to create such custom data-fit:</p><p>Imagine you are pricing motor-insurance policies.</p><p>Most drivers submit no claim in a given year, yet a few unlucky ones trigger a large, right-skewed payout. Classic GLMs struggle here: Gaussian distributions can’t handle the heavy right tail of claim amounts, Poisson models insist every policyholder files a claim, and Gamma distributions refuse to acknowledge that most drivers file zero claims.</p><p>Between those extremes lives the <a href="https://en.wikipedia.org/wiki/Tweedie_distribution">Tweedie distribution</a>. The Tweedie loss is designed for data that combines two patterns:</p><ol><li><strong>Exact zeros</strong>: Many observations are exactly zero (e.g., no insurance claims)</li><li><strong>Continuous positive values</strong>: When non-zero, values are continuous and often right-skewed</li></ol><p>Think of it as modeling a two-stage process:</p><ul><li><strong>Frequency</strong>: Does an event occur? (Poisson process)</li><li><strong>Severity</strong>: If yes, what’s the magnitude? (Gamma distribution)</li></ul><h4>The Loss Function (value)</h4><p>The unit deviance of a reproductive Tweedie distribution is given by:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/403/1*clfeyPR5MztUMJudWWfQ2g.png" /><figcaption>Source: <a href="https://en.wikipedia.org/wiki/Tweedie_distribution">https://en.wikipedia.org/wiki/Tweedie_distribution</a></figcaption></figure><p>Where:</p><ul><li>y = observed value</li><li>\mu = predicted value</li><li>p = power parameter (for insurance, typically p \in (1,2))</li></ul><p>For insurance applications, we focus on p ∈ (1,2), using the general formula with the crucial max(y,0) term that ensures numerical stability when y = 0 and handles the mathematical constraints properly (<a href="https://link.springer.com/article/10.1007/s11222-005-4070-y">Dunn &amp; Smyth, 2005</a>).</p><h4>The Gradient (grad)</h4><p>The gradient with respect to the predicted mean μ is essential for first-order optimization methods. Given the original loss function, I have arrived at these gradients (simply taking partial derivatives of the function above with respect to μ, feel free to verify).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/283/1*6TF9f2vo-sMC7WYJDgZFEQ.png" /></figure><p>The last one is the general case. This gradient has an intuitive interpretation: when μ &gt; y (overestimation), the gradient is positive, encouraging a decrease in μ. Conversely, when μ &lt; y (underestimation), the gradient is negative, encouraging an increase in μ. The factor μ^(-p) weights this correction according to the variance structure of the Tweedie distribution.</p><p><strong>Note on the max(y,0) formulation:</strong> While our derivation assumes y ≥ 0 (appropriate for insurance claims), the general Tweedie formula uses max(y,0)^(2-p) to handle edge cases and ensure mathematical well-definedness across all parameter ranges. For y ≥ 0, this reduces to our expression above.</p><h4>The Lipschitz constant (L):</h4><p>Think of the Lipschitz constant L as the “speed limit” for your optimization algorithm. The gradient tells us: “What’s the slope right here?”. The Lipschitz tells us: “What’s the steepest this slope could possibly get anywhere in our search space?”.</p><p>More formally, the Lipschitz constant bounds the rate of change of the gradient function. If you move a small distance δ in your prediction μ, the gradient can change by at most L × δ. This bound ensures that optimization algorithms don’t overshoot and miss the minimum.</p><p>Depending on the solver used, you need to define different Lipschitz functions:</p><p><strong>Coordinate-wise Lipschitz (for Coordinate Descent methods like AndersonCD):</strong> Each coordinate j gets its own “speed limit” L_j, allowing the algorithm to take different step sizes for different features. This is more expensive per iteration than global lipschitz but allows larger, adaptive steps.</p><p>For coordinate descent, the Lipschitz constant for coordinate j is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/214/1*XSqQoaLpG6WVDE86NFG93Q.png" /></figure><p>where Hessian_i is the second derivative of the loss with respect to the prediction for sample i, evaluated at a typical value (often at Xw = y for safety).</p><p><strong>Global Lipschitz (for methods like FISTA):</strong> Here, a single conservative “speed limit” L_global is used for all coordinates, ensuring safe steps for the entire parameter vector.</p><p>For global methods, the Lipschitz constant is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/213/1*X8DDJ_m3CiO2_fsRgvsBNA.png" /></figure><p>where ||X||₂ is the spectral norm (largest singular value) of the design matrix X.</p><p><strong>How do we compute the Hessian for Tweedie?</strong></p><p>The per-sample Hessian (second derivative with respect to μ) is derived from the loss and depends on the Tweedie power parameter p.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/354/1*BIUOzpveuIRPYHBC8pJJbg.png" /></figure><h4>Bringing It All Together: From Math to Code</h4><p>With these three mathematical components clearly defined, we now have everything needed to implement our custom datafit. Following skglm’s conventions, our Tweedie datafit needs several key components:</p><p><strong>Core mathematical methods:</strong></p><ul><li>value(): Computes the Tweedie unit deviance</li><li>gradient_scalar() and gradient(): Compute gradients using our derived formula (raw_grad() is needed to calculate these functions)</li><li>get_lipschitz() and get_global_lipschitz(): Provide coordinate-wise and global Lipschitz constants (raw_hessian() is needed to calculate the Lipschitz constant)</li></ul><p>Additionally we need some infrastructure methods to make it work.</p><ul><li>init(): Initializes the datafit with its parameters. For our Tweedie datafit, this stores the power parameter p that controls the distribution shape.</li><li>get_spec(): Defines the data types for Numba just-in-time compilation (performance optimization)</li><li>params_to_dict(): Serializes parameters for saving/loading models</li></ul><p>Rather than implementing everything at once, let’s build our Tweedie datafit incrementally. This approach helps catch errors early and ensures each component works correctly.</p><blockquote><strong>Note: </strong>With good AI coding tools out there (Cursor, ChatGPT, Claude), its really easy to prompt it all at once and get a quick datafit. <br>However, these tools still make mistakes especially when overwhelmed with a larger chunk of code. You end up with a huge file and many lines of code but it does not work and you are not able to explain the individual functions. <br><strong>SO ITS REALLY IMPORTANT TO START SMALL AND BUILD INCREMENTALLY! IT WILL SAVE YOU A LOT OF TIME!</strong></blockquote><blockquote>Trust me, I enjoy vibecoding too, but I have already made the mistake of not adhering to this principle too often.</blockquote><blockquote>Even though you will have a lot assumptions in your first tests, it pays off. Only when you are sure of each step, move on.</blockquote><p><strong>Step 1: Start with the Core Math</strong> Begin with just the value(), raw_grad(), and raw_hessian() methods. Test these on simple data to verify the mathematical formulas are correct:</p><pre>class TweedieDatafit(BaseDatafit):<br>    def __init__(self, power=1.5):<br>        self.power = power<br>    <br>    def value(self, y, w, Xw):<br>        p = self.power<br>        mu = np.maximum(Xw, 0.1)  # ensure positivity for mu<br><br>        if p == 0:<br>            # Normal<br>            return np.mean((y - mu) ** 2)<br>        elif p == 1:<br>            # Poisson<br>            return 2 * np.mean(y * np.log(y / mu) + mu - y)<br>        elif p == 2:<br>            # Gamma<br>            return 2 * np.mean(np.log(mu / y) + y / mu - 1)<br>        else:<br>            # General Tweedie<br>            return 2 * np.mean(<br>                np.maximum(y, 0) ** (2 - p) / ((1 - p) * (2 - p))<br>                - y * mu ** (1 - p) / (1 - p)<br>                + mu ** (2 - p) / (2 - p)<br>            )<br>        <br>     def raw_grad(self, y, Xw):<br>        p = self.power<br>        mu = np.maximum(Xw, 0.1)<br>        if p == 0:<br>            grad = 2 * (mu - y)<br>        elif p == 1:<br>            grad = 2 * (mu - y) / mu<br>        elif p == 2:<br>            grad = 2 * (mu - y) / mu**2<br>        else:<br>            grad = 2 * mu**(-p) * (mu - y)<br>        return grad / len(y)  # average over samples<br><br>    def raw_hessian(self, y, Xw):<br>        p = self.power<br>        mu = np.maximum(Xw, 0.1)<br>        if p == 0:<br>            return np.ones_like(mu) * 2<br>        elif p == 1:<br>            return 2 * y / (mu ** 2)<br>        elif p == 2:<br>            return 2 * (2*y - mu) / (mu ** 3)<br>        else:<br>            return 2 * mu**(-p-1) * (mu*(1-p) + p*y)</pre><blockquote>Note: <strong>Identity Link vs. Log Link</strong> We use an identity link (μ = Xw) with a simple constraint <em>mu = np.maximum(Xw, 0.1)</em> to ensure positivity. This is the simplest approach, but comes with trade-offs:</blockquote><blockquote><strong>- Identity Link Pros</strong>: Simple derivatives, direct interpretation, easier debugging<br>- <strong>Identity Link Cons</strong>: Requires careful data generation/preprocessing to prevent negative predictions during optimization (see below when applying the datafit)</blockquote><blockquote>For production systems, consider a log link (μ = exp(Xw)) which automatically ensures μ &gt; 0 but requires chain rule derivatives.</blockquote><p><strong>Step 2: Add Gradient Interface</strong> Once your core math is solid, add the gradient methods that skglm’s solvers expect:</p><pre>def gradient(self, X, y, Xw):<br>    return X.T @ self.raw_grad(y, Xw)<br>def gradient_scalar(self, X, y, w, Xw, j):<br>    return X[:, j] @ self.raw_grad(y, Xw)</pre><p><strong>Step 3: Implement Lipschitz Constants</strong> Add the “speed limits” for optimization. Start simple and optimize later:</p><pre>def get_lipschitz(self, X, y):<br>    hess = np.abs(self.raw_hessian(y, y))  # Use y as safe initial estimate<br>    max_hess = np.max(hess)<br>    return np.array([(X[:, j] ** 2).sum() * max_hess / len(y) <br>                    for j in range(X.shape[1])])</pre><p><strong>Step 4: Add skglm Infrastructure</strong> Add the boilerplate methods that skglm needs for serialization and performance. Depending on how you test, you can of course add this also at an earlier step.</p><pre>def __init__(self, power=1.5):<br>        self.power = power<br>def get_spec(self):<br>    from numba import float64<br>    return ((&#39;power&#39;, float64),)<br>def params_to_dict(self):<br>    return dict(power=self.power)</pre><p><strong>Step 5:</strong> <strong>Intercept Handling</strong></p><p>When fit_intercept=True, skglm needs to know how to optimally update the intercept term.</p><blockquote><strong>Note:</strong> Intercept fitting is not really a key element of the datafit, you should test first with centered data. Also some solvers in skglm(AndersonCD) support intercept fitting by default, others don’t and need some adjustments (FISTA).</blockquote><p>Add this method to enable intercept fitting:</p><pre>def intercept_update_step(self, y, Xw):<br>    return np.mean(self.raw_grad(y, Xw))</pre><p>This tells skglm: “To update the intercept, take the average of all residuals.” For most loss functions, this simple formula gives the optimal intercept update in each iteration.</p><h4>Applying our Data-Fit</h4><p>Now let’s see how to apply our TweedieDatafit.</p><p>We’ll use skglm’s <a href="https://contrib.scikit-learn.org/skglm/generated/skglm.GeneralizedLinearEstimator.html">GeneralizedLinearEstimator</a> to combine our custom datafit with built-in penalties and solvers (you can freely choose whatever fits well for your purpose):</p><pre># Generate sample data<br>np.random.seed(42)<br>X = np.random.randn(500, 10)<br>y = np.maximum(X @ np.random.randn(10) * 0.5 + 3 + np.random.randn(500), 0.1) # due to Identity Link<br><br># Fit Tweedie regression<br>estimator = GeneralizedLinearEstimator(<br>    datafit=TweedieDatafit(power=1.5),<br>    penalty=L1(alpha=0.01),<br>    solver=AndersonCD()<br>)<br>estimator.fit(X, y)<br># Evaluate<br>y_pred = estimator.predict(X)<br>print(f&quot;R² score: {r2_score(y, y_pred):.3f}&quot;)<br>print(f&quot;MSE: {mean_squared_error(y, y_pred):.4f}&quot;)<br>print(f&quot;Selected features: {np.sum(estimator.coef_ != 0)}/10&quot;)</pre><h4>Bonus: Production-Ready Features</h4><p>Only when everything above works, you shall start expanding your datafit and making it more sophisticated. In our case here, the basic TweedieDatafit works great, but skglm supports additional features for production use:</p><p><strong>Sparse Matrix Support</strong> For large datasets with many features, dense matrices become memory-intensive. skglm can work with sparse (CSC/CSR) matrices if you implement the corresponding sparse methods:</p><pre>def gradient_scalar_sparse(self, X_data, X_indptr, X_indices, y, Xw, j):<br>    # Compute gradient for feature j using sparse matrix format<br>    <br>def get_lipschitz_sparse(self, X_data, X_indptr, X_indices, y):<br>    # Compute Lipschitz constants for sparse matrices</pre><p>These methods work directly with the compressed sparse format, avoiding the memory overhead of converting to dense arrays.</p><p><strong>Advanced Solvers</strong> Different solvers have different benefits and also different requirements. E.g. coordinate descent methods (like AndersonCD) benefit from feature-wise Lipschitz constants, while proximal methods (like FISTA) prefer global constants. Choose an appropriate solver for your problem and add the necessary functions for your datafit.</p><p><strong>Testing and Validation</strong> Before deploying custom datafits, validate them against known solutions:</p><ul><li>Compare special cases (p=1, p=2) against skglm’s built-in Poisson and Gamma losses</li><li>Test gradient correctness using finite differences</li><li>Verify Lipschitz bounds ensure convergence</li><li>In our Tweedie example: For comparison with real insurance data, see the <a href="https://scikit-learn.org/stable/auto_examples/linear_model/plot_tweedie_regression_insurance_claims.html">scikit-learn Tweedie regression tutorial</a> which demonstrates similar techniques on French motor insurance claims.</li></ul><h4>Conclusion</h4><p>To summarize, we’ve now built a custom Tweedie datafit and integrated it into skglm’s ecosystem. As you can see, the mathematical foundation, in particular understanding the Gradient and the Lipschitz constant, was the most important part to ensure that the custom datafit works correctly with skglm’s optimizers. The coding part was relatively straightforward and you can now experiment with different power parameters, combining different penalties or exploring advanced solvers for larger datasets.</p><h4>References</h4><ul><li>Bertrand, Q., Klopfenstein, Q., Bannier, P. A., Gidel, G., &amp; Massias, M. (2022). Beyond L1: Faster and Better Sparse Models with skglm. <em>Advances in Neural Information Processing Systems</em>, 35. <a href="https://arxiv.org/abs/2204.07826">https://arxiv.org/abs/2204.07826</a></li><li>Dunn, P. K., &amp; Smyth, G. K. (2005). Series evaluation of Tweedie exponential dispersion model densities. <em>Statistics and Computing</em>, 15(4), 267–280. <a href="https://link.springer.com/article/10.1007/s11222-005-4070-y">https://link.springer.com/article/10.1007/s11222-005-4070-y</a></li><li>skglm documentation: <a href="https://contrib.scikit-learn.org/skglm/">https://contrib.scikit-learn.org/skglm/</a></li><li>Tweedie distribution — Wikipedia: <a href="https://en.wikipedia.org/wiki/Tweedie_distribution">https://en.wikipedia.org/wiki/Tweedie_distribution</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ca468490571a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[skglm tutorial — A Beginner’s guide to fast estimators for regularized GLMs in Python]]></title>
            <link>https://medium.com/@florian.kozikowski/skglm-tutorial-a-beginners-guide-to-fast-estimators-for-regularized-glms-in-python-978b8816c6a8?source=rss-8fa5f1e9b3cd------2</link>
            <guid isPermaLink="false">https://medium.com/p/978b8816c6a8</guid>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[skglm]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Florian Kozikowski]]></dc:creator>
            <pubDate>Mon, 28 Apr 2025 09:19:04 GMT</pubDate>
            <atom:updated>2025-04-28T09:19:04.879Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>skglm tutorial — A Beginner’s guide to fast estimators for regularized GLMs in Python</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p1hx7TODqFyVNhGZRhygqA.png" /></figure><p>Regularized models like Lasso and ElasticNet are powerful tools for feature selection and interpretability in machine learning. But they can get slow or chunky — especially on large or high dimensional datasets.</p><p><strong>Enter </strong><a href="https://contrib.scikit-learn.org/skglm/stable/index.html"><strong>skglm</strong></a><strong>: </strong>It’s a high performance, modular python library designed to make sparse <a href="https://medium.com/@sahin.samia/a-comprehensive-introduction-to-generalized-linear-models-fd773d460c1d">generalized linear models</a> faster and more flexible. And it&#39;s 100% compatible with sci-kit learn making it very easy to learn.</p><p>Whether you’re working on genomics, risk assessment, or you just want to build a better Lasso, this guide will help you get started with skglm in minutes — no prior experience required.</p><p><strong>Why use Sparse GLMs?</strong></p><blockquote>Feature selection:<em> Identify the most relevant predictors automatically.</em></blockquote><blockquote>Interpretability: <em>Easier to understand the model.</em></blockquote><blockquote>Scalability:<em> Works well with high-dimensional data (many features, few samples).</em></blockquote><p><strong>What is skglm?</strong></p><p>skglm (scikit-learn Generalized Linear Models) is an open-source Python library designed for efficient and scalable estimation of penalized generalized linear models. Key features include:</p><ul><li>Support for <strong>various sparse regression models</strong> such as Lasso, Elastic Net, and Group Lasso and many more</li><li>Ability to handle both <strong>convex and non-convex </strong>regularization penalties</li><li>able to tackle large datasets, either dense or sparse, with millions of features <strong>up to 100 times faster</strong> than scikit-learn</li><li>Access to <strong>advanced optimization algorithms</strong> like Anderson Acceleration and ProxNewton</li><li>Follows <strong>scikit-learn’s fit/predict API</strong>, making it easy to plug into existing machine learning workflows</li></ul><p><strong>Installation</strong></p><p>You can install skglm with pip</p><pre>pip install skglm </pre><p><strong>Your first skglm model: Lasso Regression</strong></p><p>If you are a beginner — like I was not too long ago — you are probably wondering what even is a Lasso.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Ftenor.com%2Fembed%2F18127245&amp;display_name=Tenor&amp;url=https%3A%2F%2Ftenor.com%2Fview%2Fpink-panther-lasso-rope-roping-cattle-in-the-pink-cowboy-hat-gif-18127245&amp;image=https%3A%2F%2Fmedia.tenor.com%2Fx2eMxT8lPxEAAAAN%2Fpink-panther-lasso-rope.png&amp;type=text%2Fhtml&amp;schema=tenor" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/ad91e16135aa7157203d31407b5ddd71/href">https://medium.com/media/ad91e16135aa7157203d31407b5ddd71/href</a></iframe><blockquote><strong>Lasso = Linear regression + automatic feature selection</strong></blockquote><blockquote><em>In it’s basic form, Lasso is a type of linear regression that adds a penalty to reduce the number of features used in the model. It automatically sets some coefficients to zero, effectively selecting only the most important variables.</em></blockquote><p>Okay, so then let’s fit a basic lasso regression model with skglm.</p><p>We’ll use the Diabetes dataset as a toy example.</p><pre>from sklearn.datasets import load_diabetes<br>from sklearn.model_selection import train_test_split<br>from sklearn.preprocessing import StandardScaler<br>from skglm import Lasso<br><br># Load dataset<br>X, y = load_diabetes(return_X_y=True)<br>X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)<br><br># Standardize features<br>scaler = StandardScaler()<br>X_train = scaler.fit_transform(X_train)<br>X_test = scaler.transform(X_test)<br><br># Fit skglm Lasso<br>model = Lasso(alpha=0.1)<br>model.fit(X_train, y_train)<br><br># Predict<br>y_pred = model.predict(X_test)</pre><p>That’s it!!! <br>You’ve just trained a sparse model with skglm.</p><p>Of course, this here is just for demonstration purposes to show how easy skglm works within the scikit-learn environment. It does not really reveal any relevant insights for now. If you are interested in a more comprehensive and applicable example of skglm, I recommend you check out this <a href="https://medium.com/@florian.kozikowski/survival-analysis-made-simple-predicting-patient-outcomes-with-skglm-7ba06df0668d">article</a>.</p><p><strong>To sum it up: Why should you use skglm over glmnet, scikit-learn and other glm libraries?</strong></p><ul><li>extremely fast on large data</li><li>highly modular to plug in different penalties and datafits</li><li>non-convex support to push beyond classic L1 or L2 constraints</li><li>acceleration methods, like Anderson acc., for faster convergence.</li></ul><p>Further readings:</p><ul><li><a href="https://contrib.scikit-learn.org/skglm/stable/index.html">skglm-documentation</a></li><li><a href="https://github.com/scikit-learn-contrib/skglm">skglm-github</a></li><li><a href="https://arxiv.org/abs/2204.07826">Beyond L1: Faster and Better Sparse Models with skglm</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=978b8816c6a8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Survival Analysis Made Simple: Predicting Patient Outcomes with skglm]]></title>
            <link>https://medium.com/@florian.kozikowski/survival-analysis-made-simple-predicting-patient-outcomes-with-skglm-7ba06df0668d?source=rss-8fa5f1e9b3cd------2</link>
            <guid isPermaLink="false">https://medium.com/p/7ba06df0668d</guid>
            <category><![CDATA[survival-analysis]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Florian Kozikowski]]></dc:creator>
            <pubDate>Tue, 22 Apr 2025 08:56:04 GMT</pubDate>
            <atom:updated>2025-04-22T08:56:04.942Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cPaJSfJVMPOJy8zW" /><figcaption>Photo by <a href="https://unsplash.com/@dawson2406?utm_source=medium&amp;utm_medium=referral">Stephen Dawson</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Imagine you’re a healthcare researcher or a clinician trying to understand how long patients might survive after a treatment. Rather than just getting a black-box prediction from a deep learning model, you need to know <em>why</em> some patients do better than others. That’s where <a href="https://contrib.scikit-learn.org/skglm/stable/index.html">skglm</a> comes in — offering a fast, interpretable, and flexible approach to survival analysis in Python.</p><p>In this article, we’ll walk through a practical example using skglm’s tools to build a sparse Cox model for survival data. The goal is not only to predict patient outcomes but also to clearly highlight which factors drive those predictions.</p><h4><strong>Why use skglm for survival analysis?</strong></h4><ul><li><strong><em>Interpretability:</em> </strong>skglm produces models with clear, sparse coefficients. That means, you can see which features (treatment type, age, blood pressure) are affecting survival outcomes.</li><li><strong><em>Speed:</em></strong> able to tackle any size of data, either dense or sparse, up to 100 times faster than<em> lifelines</em></li><li><strong><em>Flexibility: </em></strong>Allows custom estimators with any combination of its existing datafits and penalties</li><li><strong><em>Simplicity : </em></strong>Built with scikit-learn-like API (fit/predict makes it easy to understand)</li></ul><p>Okay, so let’s start with the example.</p><h4><strong>Part 1: Generating Synthetic Survival Data</strong></h4><p>Let’s say you’re analyzing data from a clinical study to predict patient survival times. This dataset might include several patient characteristics along with the time until an event (such as relapse or death) and an indicator showing whether the event was observed (or if the patient was censored). I don’t have a dataset right now, so I’ll just generate some data for demonstration purposes.</p><p>We’ll use skglm’s built-in utility to simulate a dataset with 500 patients and 100 features. Only a few of these features will actually influence survival — this mirrors real-world clinical settings where many variables are measured, but only a few are meaningful.</p><p>(For a full example, check out the <a href="https://contrib.scikit-learn.org/skglm/stable/auto_examples/plot_survival_analysis.html#sphx-glr-auto-examples-plot-survival-analysis-py">official skglm survival analysis guide</a>.)</p><pre>import numpy as np<br>import matplotlib.pyplot as plt<br>from skglm.utils.data import make_dummy_survival_data<br><br># Number of samples and features<br>n_samples, n_features = 500, 100<br><br># Generate synthetic survival data (normalized)<br>X, y = make_dummy_survival_data(<br>    n_samples, n_features,<br>    normalize=True,<br>    random_state=0<br>)<br><br># Extract survival times (tm) and censorship (s)<br>tm, s = y[:, 0], y[:, 1]<br><br># Quick look at the data distribution<br>fig, axes = plt.subplots(1, 3, figsize=(6, 2), tight_layout=True)<br>dists = (tm, s, X[:, 5])<br>titles = (&quot;Times&quot;, &quot;Censorship&quot;, &quot;5th Predictor&quot;)<br><br>for idx, (dist, title) in enumerate(zip(dists, titles)):<br>    axes[idx].hist(dist, bins=&quot;auto&quot;)<br>    axes[idx].set_title(title)<br><br>axes[0].set_ylabel(&quot;Count&quot;)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*tACGjfpClbEvZ1rlaBCTUg.png" /><figcaption>Figure 1. Histograms of the synthetic survival times (left), the censorship indicators (middle), and a randomly chosen predictor (right).</figcaption></figure><h4><strong>Part 2: Fitting a Sparse Cox Model with skglm</strong></h4><p>Next, we’ll set up a <a href="https://www.sthda.com/english/wiki/cox-proportional-hazards-model"><strong>Cox model</strong> </a>— a common method in survival analysis — by combining:</p><ul><li>A <strong>Cox datafit: </strong>This datafit calculates how well our model predicts the observed survival times.</li><li>An <strong>L1 penalty</strong> for sparsity: Enforces sparsity by “zeroing out” less important features — its essentially feature selection and helps with interpretability.</li><li>The <strong>ProxNewton</strong> solver: An advanced solver that converges quickly, even with large datasets.</li></ul><pre>from skglm.datafits import Cox<br>from skglm.penalties import L1<br>from skglm.solvers import ProxNewton<br>from skglm.utils.jit_compilation import compiled_clone<br><br># Regularization intensity<br>alpha = 1e-2<br><br># Initialize datafit and penalty<br>datafit = compiled_clone(Cox())<br>penalty = compiled_clone(L1(alpha))<br>datafit.initialize(X, y)<br><br># Create a solver (no intercept)<br>solver = ProxNewton(fit_intercept=False, max_iter=50)<br><br># Solve the optimization problem<br>w_sk = solver.solve(X, y, datafit, penalty)[0]<br><br># Report sparsity<br>non_zero = (w_sk != 0).sum()<br>print(f&quot;Non-zero coefficients: {non_zero} out of {len(w_sk)}&quot;)<br><br>&quot;&quot;&quot;Non-zero coefficients: 66 out of 100&quot;&quot;&quot;</pre><p>Any coefficient that remains <strong>non-zero</strong> indicates an important predictor. By focusing on these key drivers, clinicians can understand <em>which</em> factors truly impact patient survival.</p><pre># View the learned coefficients<br>print(&quot;Model coefficients (rounded):&quot;)<br>print(np.round(w_sk, 3))<br><br>&quot;&quot;&quot;Model coefficients (rounded):<br>[ 0.002  0.006  0.005  0.     0.    -0.097  0.047  0.    -0.035  0.001<br>  0.108  0.     0.     0.007 -0.067  0.    -0.072  0.     0.035 -0.017<br>  0.    -0.094 -0.012  0.     0.     0.    -0.02   0.094 -0.067 -0.143<br>  0.025 -0.116  0.088 -0.018  0.079  0.     0.    -0.083 -0.012  0.<br> -0.046  0.005  0.    -0.074 -0.019 -0.031 -0.067  0.001  0.095  0.007<br>  0.055  0.078  0.     0.031  0.     0.061 -0.058 -0.04   0.    -0.081<br> -0.051  0.074  0.024  0.    -0.07   0.     0.099  0.     0.    -0.089<br> -0.067  0.     0.119  0.    -0.013  0.068  0.044  0.    -0.062  0.014<br>  0.001  0.028  0.026 -0.002  0.     0.     0.    -0.045  0.     0.051<br>  0.     0.    -0.102  0.     0.045 -0.039  0.    -0.018 -0.022  0.   ]<br>&quot;&quot;&quot;<br></pre><p>And here we are already done. We’ve build a sparse Cox model for survival data. Now, once you’ve learned this you can explore more topics like group penalties, non-convex penalties, or combining datafits.</p><h4><strong>Comparison to another popular library: lifelines</strong></h4><p>You may already know the popular lifelines library for survival analysis. skglm achieves the same Cox model objective value but with a significant speed advantage — often <em>100x faster or more</em>. You can recreate the comparison by reading up comparison done in the<a href="https://contrib.scikit-learn.org/skglm/auto_examples/plot_survival_analysis.html#sphx-glr-auto-examples-plot-survival-analysis-py"> skglm documentation</a> for the full code and more detailed benchmarks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/655/1*ffg0z5rfnR7w0S3X6kTIMw.png" /><figcaption>Figure 2: skglm converges to the same objective in a fraction of the time of lifelines</figcaption></figure><h4><strong>Add-on: Plotting a Survival Curve for a Patient</strong></h4><p>When you are doing survival analysis, you’re probably interested in visualizing a survival curve.</p><p>While skglm doesn’t (yet) include a built-in .predict_survival() method, we can compute this ourselves using the estimated coefficients from our fitted Cox model, combined with the Breslow estimator for the baseline cumulative hazard function.</p><p>We can obtain the survival function using the Cox model’s fundamental equation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*-E_dAWHezG745AggJ-k9IQ.png" /></figure><p><strong>How the </strong><a href="https://dlin.web.unc.edu/wp-content/uploads/sites/1568/2013/04/Lin07.pdf"><strong>Breslow</strong></a><strong> Estimator Works</strong></p><p>For every time when an event happens, say at time t, you calculate a small increase in risk:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/944/1*yido0p8EryNp_R2bZGMuBw.png" /></figure><p>Adding these little steps gives you <em>H_0(t)</em>.</p><p>Here’s a complete code snippet that computes the survival curve for a specific patient (we’ll use Patient 0 as an example) and then plots it:</p><pre># Using X, y, and w_sk from above:<br>tm, s = y[:, 0], y[:, 1]<br><br># 1. Calculate risk scores for all patients<br>risk_scores = np.exp(np.dot(X, w_sk))<br><br># 2. Estimate the baseline cumulative hazard H0(t) using the Breslow estimator.<br>event_times = np.sort(np.unique(tm[s == 1]))<br>H0 = np.zeros_like(event_times)<br><br>for i, t in enumerate(event_times):<br>    # Select patients who are still at risk at time t (tm &gt;= t)<br>    risk_set = tm &gt;= t<br>    # Count how many events happened at time t (account for tied events)<br>    d_t = np.sum((tm == t) &amp; (s == 1))<br>    # Sum up the risk scores for patients in the risk set<br>    sum_risk = np.sum(risk_scores[risk_set])<br>    # Update the cumulative hazard<br>    if i == 0:<br>        H0[i] = d_t / sum_risk<br>    else:<br>        H0[i] = H0[i - 1] + d_t / sum_risk<br><br># 3. Pick a patient for which to plot the survival curve (e.g., Patient 0)<br>patient_idx = 0<br>patient_risk = np.exp(np.dot(X[patient_idx], w_sk))<br><br># 4. Calculate the survival function<br>patient_survival = np.exp(-H0 * patient_risk)<br><br># 5. Plot <br>plt.step(event_times, patient_survival, where=&quot;post&quot;, label=&quot;Patient 0&quot;)<br>plt.xlabel(&quot;Time&quot;)<br>plt.ylabel(&quot;Survival Probability&quot;)<br>plt.title(&quot;Estimated Survival Curve for Patient 0&quot;)<br>plt.ylim(0, 1.05)<br>plt.legend()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/630/1*mPYrRk9Oyswda4cTL4E_Bg.png" /><figcaption>Figure 3: A survival curve for a sample patient, plotted using the Cox model with the Breslow estimator to compute the baseline hazard. The curve shows how the patient’s probability of survival changes over time.</figcaption></figure><p>This code snippet calculates the risk score for every patient and it estimates the cumulative hazard H_0(t) using the Breslow method. It then computes the survival probability for Patient 0 over time and finally creates a step plot that shows how Patient 0’s chance of surviving changes over time.</p><p><strong>To sum it up</strong>, you now know how to:</p><ul><li>Fit a sparse Cox model with skglm</li><li>Interpret which features truly impact survival</li><li>Confirm model quality by comparing with lifelines</li><li>Compute and plot survival curves for individual patients</li></ul><p>Now, once you’ve learned this you can explore more topics like group penalties, non-convex penalties, or combining datafits.</p><p>If you want to dive deeper, check out the <a href="https://contrib.scikit-learn.org/skglm/stable/auto_examples/plot_survival_analysis.html#sphx-glr-auto-examples-plot_survival_analysis-py">official skglm survival analysis documentation</a> for the full code and more detailed benchmarks (except the survival curve). <a href="https://contrib.scikit-learn.org/skglm/stable/tutorials/cox_datafit.html#maths-cox-datafit">Here</a> you can read more about the mathematics behind the Cox datafit in skglm.</p><p>Feel free to reach out if you have any questions — or if you want to share stories of how you worked with skglm !</p><p><strong>References:</strong></p><p><em>“Comparison of lifelines with skglm for survival analysis” </em>skglm 0.3.1 documentation<em>, n.d., Web. 10 Apr. 2025, </em><a href="https://contrib.scikit-learn.org/skglm/stable/auto_examples/plot_survival_analysis.html#sphx-glr-auto-examples-plot-survival-analysis-py">https://contrib.scikit-learn.org/skglm/stable/auto_examples/plot_survival_analysis.html#sphx-glr-auto-examples-plot-survival-analysis-py</a>.</p><p><em>“Mathematic behind Cox datafit” </em>skglm 0.3.1 documentation<em>, n.d., Web. 10 Apr. 2025, </em><a href="https://contrib.scikit-learn.org/skglm/stable/tutorials/cox_datafit.html#maths-cox-datafit">https://contrib.scikit-learn.org/skglm/stable/auto_examples/plot_survival_analysis.html#sphx-glr-auto-examples-plot-survival-analysis-py.</a></p><p>Lin, D. Y. (2007). <em>On the Breslow estimator. </em>Lifetime data analysis, 13:471–480, 2007 Retrieved from: <a href="https://dlin.web.unc.edu/wp-content/uploads/sites/1568/2013/04/Lin07.pdf">https://dlin.web.unc.edu/wp-content/uploads/sites/1568/2013/04/Lin07.pdf</a></p><p>Davidson-Pilon, C. (2019). <em>lifelines: survival analysis in Python</em>. <a href="https://lifelines.readthedocs.io/en/latest/Survival%20Regression.html">https://lifelines.readthedocs.io/</a></p><p>Pölsterl, S. (2020). <em>scikit-survival: A library for time-to-event analysis</em>. <a href="https://scikit-survival.readthedocs.io/en/stable/user_guide/00-introduction.html">https://scikit-survival.readthedocs.io/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7ba06df0668d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>