<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by difandeng on Medium]]></title>
        <description><![CDATA[Stories by difandeng on Medium]]></description>
        <link>https://medium.com/@difandeng?source=rss-748442a9fa69------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*kCD96VAkMhlREpo-.jpg</url>
            <title>Stories by difandeng on Medium</title>
            <link>https://medium.com/@difandeng?source=rss-748442a9fa69------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 02:47:35 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@difandeng/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[SMAC3 Registered on OptunaHub]]></title>
            <link>https://medium.com/optuna/smac3-registered-on-optunahub-4fb9e90855cb?source=rss-748442a9fa69------2</link>
            <guid isPermaLink="false">https://medium.com/p/4fb9e90855cb</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[science]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[difandeng]]></dc:creator>
            <pubDate>Mon, 17 Feb 2025 03:01:03 GMT</pubDate>
            <atom:updated>2025-02-17T03:01:03.052Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_vfTRv_8UDazqwYVzI1-4w.png" /></figure><p><a href="https://github.com/automl/SMAC3">SMAC3</a> is developed by AutoML.org, mainly by <a href="https://github.com/automl/SMAC3?tab=readme-ov-file#contacting-us">the SMAC team</a>, in Germany and the article below is written by one of the core developers of SMAC3, <a href="https://scholar.google.com/citations?hl=en&amp;user=mvciDDAAAAAJ">Difan Deng</a>. AutoML.org is one of the world-leading AutoML research groups and the Optuna team collaborated with AutoML.org this time, having led to his contribution of a SMAC3-based sampler to OptunaHub. This article is about the newly introduced SMACSampler.</p><h3>Introduction</h3><p><a href="https://github.com/automl/SMAC3">SMAC3</a> [1] offers a robust and flexible framework for Bayesian Optimization to support users in determining well-performing hyperparameter configurations for their (machine learning) algorithms, datasets, and applications at hand. SMAC3 provides several default hyperparameter optimization pipelines for different use cases to make life easy. These default settings are summarized as <a href="https://automl.github.io/SMAC3/main/3_getting_started.html#facade">Facades</a> to provide users with a simple interface. However, users could also customize their optimization pipelines with SMAC3 components. For example, thanks to the modular design, researchers can easily benchmark different types of Bayesian optimization algorithms.</p><p>We introduced SMAC3 to OptunaHub this time and SMACSampler in OptunaHub also provides an interface for users to configure their Bayesian optimization pipeline, including but not limited to different types of initial designs, surrogate models, acquisition functions, and acquisition function optimizers. It is also possible to switch facades from the SMACSampler interface as well. In this article, we show some simple examples of SMACSampler.</p><p>To run the codes in this article, please install the dependencies as follows:</p><pre># Installation of Dependencies.<br>$ pip install optuna optunahub cmaes smac torch matplotlib</pre><h3>SMAC3 Registered on OptunaHub</h3><p>SMACSampler in OptunaHub is, by default, based on <a href="https://automl.github.io/SMAC3/v2.2.0/api/smac.facade.hyperparameter_optimization_facade.html#module-smac.facade.hyperparameter_optimization_facade">HyperparameterOptimizationFacade</a>, which uses Random Forest (RF) as a surrogate model and expects improvement (EI) with log transformation [2] as an acquisition function. To verify the efficiency of SMACSampler, we evaluate it on the <a href="https://www.sfu.ca/~ssurjano/branin.html">Branin function</a>. We optimize the difference between the evaluated Branin function value and the global minima, which we name regret. As a baseline, we used CMA-ES in Optuna. The result is shown in Figure 1. The horizontal axis shows the number of trials, and the vertical axis shows the regret (lower is better). The solid lines (blue: CmaEsSampler (CMA-ES), orange: SMACSampler) show each optimizer’s minimum objective function value. As demonstrated, we can now easily compare samplers in Optuna with SMAC3.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/898/0*FoRIawASr9KO9gas" /><figcaption>Figure 1. Comparison of SMAC (orange) and CMA-ES (blue) using Branin function.</figcaption></figure><p>The code used for benchmarking is as follows:</p><pre>from __future__ import annotations<br><br>import matplotlib.pyplot as plt<br>import numpy as np<br>import optuna<br>from optuna.samplers import CmaEsSampler<br>import optunahub<br><br><br>n_trials = 100<br>SMACSampler = optunahub.load_module(&quot;samplers/smac_sampler&quot;).SMACSampler<br><br><br>def branin(x1: float, x2: float) -&gt; float:<br>   # Branin function: https://www.sfu.ca/~ssurjano/branin.html<br>   a = 1<br>   b = 5.1 / (4*np.pi**2)<br>   c = 5 / np.pi<br>   r = 6<br>   s = 10<br>   t = 1 / (8 * np.pi)<br>   return a * (x2 - b * x1 ** 2 + c * x1 - r) ** 2 + s * (1-t) * np.cos(x1) + s<br><br><br>def objective(trial: optuna.Trial) -&gt; float:<br>   x1 = trial.suggest_float(&quot;x1&quot;, -5.0, 10.0)<br>   x2 = trial.suggest_float(&quot;x2&quot;, 0.0, 15.0)<br>   # (-pi, 12.275) is one of the global minima for Branin function<br>   return branin(x1, x2) - branin(-np.pi, 12.275)<br><br><br>def plot_results(*samplers: optuna.samplers.BaseSampler) -&gt; None:<br>   _, ax = plt.subplots(figsize=(10, 5))<br>   ax.set_yscale(&quot;log&quot;)<br>   for sampler in samplers:<br>       study = optuna.create_study(sampler=sampler)<br>       study.optimize(objective, n_trials=n_trials)<br>       values = np.minimum.accumulate([float(t.value) for t in study.get_trials()])<br>       ax.step(np.arange(n_trials) + 1, values, label=sampler.__class__.__name__)<br><br>   ax.set_xlim(0, n_trials)<br>   ax.set_xlabel(&quot;Number of Trials&quot;)<br>   ax.set_ylabel(&quot;Regret&quot;)<br>   ax.legend()<br>   ax.grid(which=&quot;minor&quot;, color=&quot;gray&quot;, linestyle=&quot;:&quot;)<br>   ax.grid(which=&quot;major&quot;, color=&quot;black&quot;)<br>   plt.show()<br><br><br>search_space = {<br>   &quot;x1&quot;: optuna.distributions.FloatDistribution(-5., 10.),<br>   &quot;x2&quot;: optuna.distributions.FloatDistribution(0., 15.),<br>}<br>seed = 42<br>plot_results(<br>   CmaEsSampler(seed=seed), SMACSampler(search_space, n_trials=n_trials, seed=seed)<br>)</pre><h3>Customize Your SMACSampler</h3><p>Like SMAC3, users can customize their own SMACSampler in OptunaHub by passing the corresponding arguments. Currently, we allow the users to customize:</p><ul><li>Surrogate model: gp for Gaussian process, gp_mcmc for Gaussian process with parameters sampled by MCMC, and rf for random forest,</li><li>Acquisition function: ei for expected improvement, ei_log for expected improvement with log transformation [2], pi for probability of improvement, and lcb for lower confidence bound, and</li><li>Initialization design: sobol for Sobol sequence, lhd for Latin hypercube, and random for uniform random.</li></ul><p>We defer to <a href="https://hub.optuna.org/samplers/smac_sampler/">SMAC3 | OptunaHub</a> for further details.</p><p>Here is another example using SMACSampler, but with Gaussian process and expected improvement as the surrogate model and acquisition function. We compare this SMACSampler with Optuna GPSampler:</p><pre>from optuna.samplers import GPSampler <br><br><br>plot_results(<br>   GPSampler(seed=seed),<br>   SMACSampler(search_space, surrogate_model_type=&quot;gp&quot;, acq_func_type=&quot;ei&quot;, n_trials=n_trials, seed=seed),<br>)</pre><p>The result is shown in Figure 2:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/897/0*1AWEne0-vMSMdjpR" /><figcaption>Figure 2. Comparison of SMAC GP (orange) and GP sampler (blue) using Branin function</figcaption></figure><h3>Conclusion</h3><p>This article introduced <a href="https://hub.optuna.org/samplers/smac_sampler/">SMACSampler</a>, a versatile Bayesian optimization for hyperparameter optimization, available on OptunaHub.</p><h3>References</h3><p>[1] Lindauer et al. “SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization”, Journal of Machine Learning Research,<a href="http://jmlr.org/papers/v23/21-0888.html"> http://jmlr.org/papers/v23/21-0888.html</a></p><p>[2] Watanabe, “Derivation of Closed Form of Expected Improvement for Gaussian Process Trained on Log-Transformed Objective”, <em>arXiv preprint arXiv:2411.18095</em>, <a href="https://arxiv.org/abs/2411.18095">https://arxiv.org/abs/2411.18095</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4fb9e90855cb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/optuna/smac3-registered-on-optunahub-4fb9e90855cb">SMAC3 Registered on OptunaHub</a> was originally published in <a href="https://medium.com/optuna">Optuna</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>