<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Dario Coscia on Medium]]></title>
        <description><![CDATA[Stories by Dario Coscia on Medium]]></description>
        <link>https://medium.com/@dariocos99?source=rss-fa5d05f8fc26------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*EUGLT3d_snvdcmP5DagukA.png</url>
            <title>Stories by Dario Coscia on Medium</title>
            <link>https://medium.com/@dariocos99?source=rss-fa5d05f8fc26------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 22:27:57 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@dariocos99/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Neural Operators and Where to Find Them]]></title>
            <link>https://medium.com/sissa-mathlab/neural-operators-and-where-to-find-them-19af4aa9da3e?source=rss-fa5d05f8fc26------2</link>
            <guid isPermaLink="false">https://medium.com/p/19af4aa9da3e</guid>
            <dc:creator><![CDATA[Dario Coscia]]></dc:creator>
            <pubDate>Thu, 09 Jan 2025 19:36:15 GMT</pubDate>
            <atom:updated>2026-02-03T10:54:17.326Z</atom:updated>
            <content:encoded><![CDATA[<p>Artificial Intelligence for the Natural Sciences progress</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4EhJEsVcqOlliERAMpv0nQ.png" /></figure><p>This is the third article of the series <strong>Deep Learning 4 Natural Sciences</strong>. In this article, we will explore the concept of Neural Operators and examine why they are rapidly gaining attention as a promising new approach to fast numerical solvers within the scientific community.</p><h4>Deep Learning 4 Natural Sciences series</h4><blockquote><a href="https://medium.com/@dariocos99/8634cda1022a">How will deep learning be used to speed up physical simulations?</a></blockquote><blockquote><a href="https://medium.com/sissa-mathlab/blending-neural-networks-with-physics-the-physics-informed-neural-network-d681b6b44eb8">Blending Neural Networks with Physics: the Physics Informed Neural Network</a></blockquote><blockquote><a href="https://medium.com/sissa-mathlab/neural-operators-and-where-to-find-them-19af4aa9da3e">Neural Operators and Where to Find Them</a></blockquote><blockquote>PINA, a Python Software for Scientific Machine Learning</blockquote><h4>What is a Neural Operator and Why Do We Need It?</h4><p>In recent years, the volume of available data simulations has increased, particularly in fields like climate science, engineering, and material design opening new frontiers for Deep Learning (DL). Traditional modeling techniques, and even advanced DL methods like PINNs (for a deeper discussion, see “<a href="https://medium.com/sissa-mathlab/blending-neural-networks-with-physics-the-physics-informed-neural-network-d681b6b44eb8">Blending Neural Networks with Physics: the Physics Informed Neural Network</a>”), have struggled to keep pace with the high variability and scale of modern simulations. These challenges have created a pressing need for a new approach — one that can efficiently handle diverse scenarios and does not require prohibitive computational resources. This is where Neural Operators emerge as a game-changer.</p><p>Neural Operators offer a flexible and scalable solution by learning generalized mappings across function spaces (hence don’t require a fixed discretization), allowing them to address new and unseen problems without the need for retraining. For example, in fluid dynamics, where simulations involve complex and highly variable flow patterns, Neural Operators can generalize across different flow configurations, providing accurate results without the time-consuming process of model adjustment for each new situation.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.canva.com%2Fdesign%2FDAGOehQLgdw%2FsY_RP34DheSF-yCD0dHTFQ%2Fview%3Fembed%26meta&amp;display_name=Canva&amp;url=https%3A%2F%2Fwww.canva.com%2Fdesign%2FDAGOehQLgdw%2FsY_RP34DheSF-yCD0dHTFQ%2Fedit%3Futm_content%3DDAGOehQLgdw%26utm_campaign%3Ddesignshare%26utm_medium%3Dlink2%26utm_source%3Dsharebutton&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=canva" width="1920" height="1080" frameborder="0" scrolling="no"><a href="https://medium.com/media/38129db91d40f9997c01c0fbebc29265/href">https://medium.com/media/38129db91d40f9997c01c0fbebc29265/href</a></iframe><p>Neural Operators are designed to capture the underlying physics of the systems they model, ensuring that simulations remain reliable and precise, even as they scale to more complex and varied datasets. This balance of efficiency and accuracy positions Neural Operators as a transformative tool in scientific computing, capable of meeting the growing demands of data-intensive fields.</p><h4>The Zoo of Neural Operator Architectures</h4><p>Neural Operators encompass a variety of architectures, each with its unique strengths and applications. Unlike traditional methods that depend on specific grid resolutions, neural operators operate independently of discretization, making them adaptable to various resolutions and problem scales.</p><p>At their core Neural Operators transform an input function <em>a </em>to an output function <em>u. </em>It is important to notice that, differently from classical Neural Networks, Neural operators work directly in the functional space, meaning their inputs and outputs are functions, not just vectors. This allows them to handle complex, continuous data more naturally.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*67VXKf1Hg5N2qNbTDqcmpA.png" /><figcaption>The general structure of a neural operator: The input function a is first encoded into an embedding space, and then processed through multiple layers of integral operators and pointwise non-linearities. Finally, a pointwise decoder generates the output function u.</figcaption></figure><p>The general structure of a Neural Operator comprehends three components:</p><ol><li><strong>Encoder: </strong>mapping the input into a specific embedding space. This step is crucial for transforming the input into a format suitable for the neural operator’s kernel.</li><li><strong>Processor:</strong> multiple layers performing function convolution, that is the core computational unit. They process the encoded input to learn convolutional kernels that capture the necessary features and patterns for the task.</li><li><strong>Decoder:</strong> mapping the processors’s output back into the desired output space, translating the processed embedding into the final functional output.</li></ol><p>By varying the design and implementation of these three components — encoder, kernel, and decoder — different neural operators are created, each tailored for specific applications or types of data. For instance: <strong>Fourier Neural Operator (FNO) [1] </strong>utilizes the Fourier transform in the processor to perform global convolutions, capturing long-range dependencies efficiently; <strong>Message Passing Neural Operator (MPNO) [2] </strong>leverages graph neural networks to exchange information between nodes, to perform convolution on unstructured domain; <strong>Universal Transformer Neural Operator (UTNO) [3] </strong>use transformers processors on a fixed embedded space obtained with a graph encoder, offering flexibility and scalability in handling a variety of problem domains.</p><h4>Training Neural Operator Architectures</h4><p>Despite their complex architecture, neural operators are trained using a straightforward, data-driven, supervised approach. This training involves learning from pairs of input and output functions derived from high-fidelity simulations — similar to how traditional classification models are trained on labeled images for tasks like classifying cats and dogs.</p><p>For time-independent systems, a basic loss function, defined as the error between the network’s prediction and the true output function, is typically sufficient. However, for time-dependent systems, more sophisticated methods are needed to accurately predict future states [2].</p><p>In conclusion, as the volume of data generated by simulations continues to grow exponentially, neural operators provide a scalable approach to developing surrogate solvers. These advanced models not only enable the generalization of complex systems but also significantly accelerate the process of obtaining solutions, paving the way for more efficient scientific discoveries. By harnessing the power of neural operators, researchers can tackle previously intractable problems, optimize simulations across various domains, and explore new frontiers in computational science with unprecedented speed and precision.</p><h4>References</h4><ol><li>Li, Zongyi, et al. “Fourier neural operator for parametric partial differential equations.” <em>arXiv preprint arXiv:2010.08895</em> (2020).</li><li>Brandstetter, Johannes, Daniel Worrall, and Max Welling. “Message passing neural PDE solvers.” <em>arXiv preprint arXiv:2202.03376</em> (2022).</li><li>Alkin, Benedikt, et al. “Universal physics transformers.” <em>arXiv preprint arXiv:2402.12365</em> (2024).</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=19af4aa9da3e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/sissa-mathlab/neural-operators-and-where-to-find-them-19af4aa9da3e">Neural Operators and Where to Find Them</a> was originally published in <a href="https://medium.com/sissa-mathlab">SISSA mathLab</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Blending Neural Networks with Physics: the Physics-Informed Neural Network]]></title>
            <link>https://medium.com/sissa-mathlab/blending-neural-networks-with-physics-the-physics-informed-neural-network-d681b6b44eb8?source=rss-fa5d05f8fc26------2</link>
            <guid isPermaLink="false">https://medium.com/p/d681b6b44eb8</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Dario Coscia]]></dc:creator>
            <pubDate>Sun, 18 Feb 2024 17:21:22 GMT</pubDate>
            <atom:updated>2026-02-03T10:52:27.130Z</atom:updated>
            <content:encoded><![CDATA[<p>Artificial Intelligence for the Natural Sciences progress</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s39u-qTWlJYb7i7IxYIJkw.png" /></figure><p>This is the second article of the series <strong>Deep Learning 4 Natural Sciences</strong>. In this article, we will present physics-informed neural networks for solving differential equations, and how inductive bias, i.e. extra knowledge of the system, can be used to regularize and faster the training.</p><h3>Deep Learning 4 Natural Sciences series</h3><blockquote><a href="https://medium.com/@dariocos99/8634cda1022a"><em>How will deep learning be used to speed up physical simulations?</em></a></blockquote><blockquote><a href="https://medium.com/sissa-mathlab/blending-neural-networks-with-physics-the-physics-informed-neural-network-d681b6b44eb8"><em>Blending Neural Networks with Physics: the Physics Informed Neural Network</em></a></blockquote><blockquote><a href="https://medium.com/sissa-mathlab/neural-operators-and-where-to-find-them-19af4aa9da3e"><em>Neural Operators and Where to Find Them</em></a></blockquote><blockquote><em>PINA, a Python Software for Scientific Machine Learning</em></blockquote><h4>The Basics of Physics Informed Machine Learning</h4><p>In the first article of the series, we saw how deep learning plays a pivotal role in accelerating numerical simulations. However, the amount of available data required to solve complex simulations is often insufficient to make AI predictions reliable and robust. Physics-informed neural networks [1] (PINNs) have been formulated to overcome the issues of missing data, by incorporating the physical knowledge into the neural network training. PINNs aim to approximate any differential equation solution by a neural network. The network is trained by solving a minimization problem in a supervised learning setting, where the physical constraints are used to obtain the network loss function.</p><p>Consider the simple example of solving an ordinary differential equation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/384/0*am5ERiaXRC2HhooZ" /></figure><p>with the analytical solution 𝓊(x) which is the exponential function. Suppose to have a neural network depending on some parameters θ which approximate the solution, let’s call it 𝓊(x; θ). Then if we plug the neural network into the above equation, we would obtain a right-hand side equal to zero only if 𝓊(x; θ) = 𝓊(x). That’s intuitive! Then we can use the right-hand side (residual) as a loss function to train the model:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/567/0*GTXO_clKE2x7EmsK" /></figure><p>which is zero when the network learns the solution. Implementing the PINN framework, for this simple problem is easy thanks to automatic differentiation, which computes the differential operator exactly. In the picture below you can see the result of training a PINN on this very simple problem, optimizing a shallow neural network with only 30 parameters using the Adam optimizer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LI9iubMI1Hm-eNcXq2xdhA.png" /><figcaption>Physics Informed Neural Network solution compared to the real solution and its absolute error on a simple ordinary differential equation.</figcaption></figure><h4>Inductive Bias in Physics Informed Machine Learning</h4><p>Since the original formulation of Physics-Informed Neural Networks (PINNs) by Raissi and his colleagues [1], researchers have made several advancements and improvements to enhance the performance and versatility of PINNs. Indeed, it is quite common to have some external knowledge about the solution, such as symmetry properties, periodicity, or boundness, which can be added to the PINN methodology. Introducing <em>inductive bias,</em><strong><em> </em></strong>i.e. external knowledge of the system properties, into the learning strategy eases the training and delivers better results.</p><p>As before, to deliver the idea of inductive bias we consider a specific problem, namely the Helmholtz equation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/527/0*Mw4aMUNZ8ir5d1Ym" /></figure><p>This equation forces the solution to be infinitely differentiable, and periodic on the domain x ∈ (- ∞, ∞). Due to the infinite differentiability condition on the boundaries, we cannot rewrite the loss function as before, since we would need infinite terms… A possible solution, diverging from the original PINN formulation, is to use <em>coordinates augmentation</em>. In coordinates, augmentation a coordinates transformation x → Φ(x) is used to ensure the infinite differentiability condition. It turns out (see [2]) that this can be easily achieved by writing Φ(x) = [1, cos(π x), sin(π x)], and training using these extra augmented coordinates only for the interior points. Above we can see the plot of the Helmholtz problem, where the solution is analytical u(x)=sin(πx)cos(3πx).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ujbIF4UVnNplJBfTwtifgQ.png" /><figcaption>Physics Informed Neural Network solution compared to the real solution and its absolute error on the Helmholtz equation.</figcaption></figure><p>A nice property of this technique is the fact that it generalizes by construction to all the real line since the function is periodic by construction. Indeed, if we test the model trained on x ∈ (0,2) on a bigger interval, for example, x ∈(-4, 4), we can expect zero loss in performance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TOEQTMXhnd6x8JKJdIblxQ.png" /><figcaption>Introducing the coordinates augmentation which forces the periodicity of the function ensures that the PINN solution is periodic on the real axis line.</figcaption></figure><p>From the picture is evident that not only the neural network solution but also the errors associated with it are periodic. There are other methods to do coordinate augmentations, such as the one based on the Fourier feature [3] for multi-scale differential equations, or learnable features [4] for augmenting the original coordinates.</p><p>Inductive bias can be exploited not only by coordinates augmentation but also in other ways. For example, in [5, 6] the authors developed different ways to ensure that the neural network solution satisfied specific boundary terms (hard constraints). Another possibility is to work by incorporating physical extra constraints in the loss, such as orthonormality [7]. Eventually, it is also possible [8] to add the underlying PDEs’ symmetries in the PINN’s training objective, which can reduce drastically the number of scattered domain data and speed up the training. Interestingly, it seems that all of these techniques which might be different at first glance, can be combined in a unique preconditioning operator framework [9]. Indeed adding inductive bias eases the training, and the authors in [9] show that this is because the conditioning of a specific differential operator is reduced when the above techniques are implemented!</p><p>In conclusion, we have seen that blending physics and more in general inductive bias of the underlying differential equation can be used to solve it efficiently using Neural Networks. The source code associated with this blog post can be found in the <a href="https://github.com/dario-coscia/deeplearning4science/blob/main/tutorials/pinns.ipynb">GitHub repository </a>of the series. The code based on PyTorch is a very simple start-off with PINNs, for more complex tasks we suggest the software <a href="https://mathlab.github.io/PINA/">PINA</a> [10], an open-source Python library providing an intuitive interface for solving differential equations by Neural Networks.</p><h3>References</h3><ol><li>Raissi, Maziar, Paris Perdikaris, and George E. Karniadakis. “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.” <em>Journal of Computational physics</em> 378 (2019): 686–707.</li><li>Dong, Suchuan, and Naxian Ni. “A method for representing periodic functions and enforcing exactly periodic boundary conditions with deep neural networks.” Journal of Computational Physics 435 (2021): 110242.</li><li>Wang, Sifan, Hanwen Wang, and Paris Perdikaris. “On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks.” Computer Methods in Applied Mechanics and Engineering 384 (2021): 113938.</li><li>Demo, Nicola, Maria Strazzullo, and Gianluigi Rozza. “An extended physics informed neural network for preliminary analysis of parametric optimal control problems.” Computers &amp; Mathematics with Applications 143 (2023): 383–396.</li><li>Moseley, Ben, Andrew Markham, and Tarje Nissen-Meyer. “Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations.” <em>Advances in Computational Mathematics</em> 49.4 (2023): 62.</li><li>Lu, Lu, et al. “Physics-informed neural networks with hard constraints for inverse design.” <em>SIAM Journal on Scientific Computing</em> 43.6 (2021): B1105-B1132.</li><li>Kim, T. and Yun, S.-Y. Revisiting orthogonality regularization: A study for convolutional neural networks in image classification. IEEE Access, 10:69741–69749, 2022. doi: 10.1109/ACCESS.2022.3185621.</li><li>Akhound-Sadegh, T., Perreault-Levasseur, L., Brandstetter, J., Welling, M., &amp; Ravanbakhsh, S. “Lie Point Symmetry and Physics Informed Networks”, accepted at NeurIPS 2023.</li><li>De Ryck, Tim, et al. “An operator preconditioning perspective on training in physics-informed machine learning.” <em>arXiv preprint arXiv:2310.05801</em> (2023).</li><li>Coscia, Dario, et al. “Physics-Informed Neural networks for Advanced modeling.” <em>Journal of Open Source Software</em> 8.87 (2023): 5352.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d681b6b44eb8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/sissa-mathlab/blending-neural-networks-with-physics-the-physics-informed-neural-network-d681b6b44eb8">Blending Neural Networks with Physics: the Physics-Informed Neural Network</a> was originally published in <a href="https://medium.com/sissa-mathlab">SISSA mathLab</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How will deep learning be used to speed up physical simulations?]]></title>
            <link>https://medium.com/sissa-mathlab/how-will-deep-learning-be-used-to-speed-up-physical-simulations-8634cda1022a?source=rss-fa5d05f8fc26------2</link>
            <guid isPermaLink="false">https://medium.com/p/8634cda1022a</guid>
            <category><![CDATA[science]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[differential-equations]]></category>
            <category><![CDATA[mathematical-modeling]]></category>
            <dc:creator><![CDATA[Dario Coscia]]></dc:creator>
            <pubDate>Mon, 07 Aug 2023 16:36:49 GMT</pubDate>
            <atom:updated>2026-02-03T10:52:43.470Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>How will Deep Learning be used to speed up Physical Simulations?</strong></h3><p>Artificial Intelligence for the Natural Sciences progress</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2CuzuYEt6wVmmzNT" /><figcaption>Photo by <a href="https://unsplash.com/@deepmind?utm_source=medium&amp;utm_medium=referral">Google DeepMind</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>This is the first article of the series <strong>Deep Learning 4 Natural Sciences</strong>. In the series we will dive into deep learning methods for solving differential equations, present open source software for the problem, and report latest advancements in the field. We will cover various models such as Physics Informed Neural Network, Neural Operar Learning, or recent reduced order models based on generative modelling [10, 11].</p><h4>Deep Learning 4 Natural Sciences series</h4><blockquote><a href="https://medium.com/@dariocos99/8634cda1022a">How will deep learning be used to speed up physical simulations?</a></blockquote><blockquote><a href="https://medium.com/sissa-mathlab/blending-neural-networks-with-physics-the-physics-informed-neural-network-d681b6b44eb8">Blending Neural Networks with Physics: the Physics Informed Neural Network</a></blockquote><blockquote><a href="https://medium.com/sissa-mathlab/neural-operators-and-where-to-find-them-19af4aa9da3e">Neural Operators and Where to Find Them</a></blockquote><blockquote>PINA, a Python Software for Scientific Machine Learning</blockquote><p>In the last decades, Deep Learning has seen increasing attention for solving complex tasks, in academia or industry. Computer vision, speech recognition, autonomous driving, and voice assistance are just a few of the incredible applications of deep learning. Alongside these advancements, a pressing question has emerged: can we use deep learning for natural sciences problems?</p><p>Natural sciences phenomena can be explained using differential equations. This mathematical object is so powerful that it can be used to model very complex and different systems. From predicting the weather to understanding the heart dynamics to model the molecules’ motion. Differential equations are fundamental tools used to understand the world around us and to be able to make predictions. In fields like sports car design or tumor expansion analysis, differential equations are crucial for making accurate predictions and developing efficient solutions. However, solving these equations by hand is extremely challenging, and expensive numerical solvers need to be used to accurately obtain a solution. Besides the computational complexity, numerical solvers can sometimes be so slow that the computation of a solution might be unfeasible, becoming challenging the use for engineering or industry.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YHey1cvZ7pZkVnAMzbXcyA.png" /><figcaption>Different problems can be solved by differential equations. The images show potential applications in academia and industry. Image sources from left to right in [1][2][3][4].</figcaption></figure><p>To address these challenges, Reduced Order Models (ROMs) have emerged as a promising field in computational science, offering efficient tools for real-time simulations. The underlying “<em>mantra</em>” of ROMs is to make differential equation simulations efficient and fast, at the cost of small errors compared to numerical solvers.</p><p>In recent years, deep learning techniques have played a pivotal role in advancing efficient ROM methods with exceptional accuracy and reduced computational costs, as also evidenced by the vast literature on it. Similarly to the human learning process, where the assimilation of knowledge is related to the experience gained from different examples, deep learning reduced-order models are trained by giving them little data from which the model generalises. These data come from differential equation solutions with different system parameters (e.g. time evolution, the shape of geometry, or differential equation-free variables). Nowadays two main frameworks are driving the deep learning community for learning differential equations: Physics Informed Neural Networks, and Neural Operator Learning. Both of them can be considered reduced-order models, enhanced by deep learning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EJI-UjaATh2uXwYjPcZIHg.png" /><figcaption>Reduced order models are smaller, efficient and faster with respect to a full complex model. Starting from few full complex model data, a reduced model is constructed approximating the full order solutions.</figcaption></figure><p>Physics Informed Neural Networks (PINNs) [5] were first introduced to include physical properties in deep learning models. Specifically, in addition to the little data coming from expensive simulations, the differential equation itself, which explains the phenomenon, can be inserted into the learning paradigm. With Physics Informed Neural Networks only one specific differential equation is learned, i.e. it can not generalise across multiple system configurations . More generally, Neural Operator Learning [6, 7, 8] has been introduced for generalising across different instances of the same kind of differential equation. Indeed, as also the name suggests, Neural Operator Learning approximates the differential operator that defines the differential equation. Once the differential operator is learned by the model, it can generalise across different system configurations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/409/1*EohCHKOVsCNoovEtUTNUOQ.jpeg" /><figcaption>Symmetry in Nature. The butterfly is symmetric with respect to the axis passing through its center (grey in the picture). Image source adapted from [9].</figcaption></figure><p>Nowadays, there are many challenges related to solving differential equations with the help of artificial intelligence. First, there are no mathematical guarantees about the errors the model will make. Quantifying the uncertainty of deep learning model predictions is very important to make the methodology reliable and trustful, allowing safe use in industry. Second, including symmetries in deep learning models. Symmetries can be thought of as repetitions of the same pattern, e.g. a butterfly is symmetric with respect to its center, see the picture above. This means that only half of the butterfly is needed to reconstruct the whole one. Intuitively, the same idea can be extended to differential equations: symmetries provide a way to reduce the number of possible solutions. Including these symmetries is fundamental to speeding up the model training, obtaining solutions that guarantee mathematical properties. Furthermore, symmetries can be used to augment data to obtain a more rich dataset. Finally, interpreting and understanding the learned models in the context of solving differential equations using deep learning is essential for responsible and trustworthy applications in industry. When making predictions about a system, having the ability to interpret the learned models fosters trust and confidence in the technology.</p><p>Deep learning will play a fundamental role in speeding up simulations due to the exceptional accuracy and reduced computational cost of the algorithms. Physics Informed Neural Network or Neural Operator are the first steps taken to tackle this challenging problem. However, there is still a long way to go and many challenges to face: uncertainty quantification, symmetry inclusion, interpretability and many more.</p><h4><strong>References</strong></h4><ol><li>Image source website: <a href="https://xenonhealth.com/nonlinear-dynamics-of-heart-rate-variability/">https://xenonhealth.com/nonlinear-dynamics-of-heart-rate-variability/</a></li><li>Image source website: <a href="https://redirect.cs.umbc.edu/2021/02/talk-modeling-and-simulation-for-reducing-risks-associated-with-extreme-weather-11-12-2-10/">https://redirect.cs.umbc.edu/2021/02/talk-modeling-and-simulation-for-reducing-risks-associated-with-extreme-weather-11-12-2-10/</a></li><li>Image source website: <a href="https://motorsportengineer.net/jobs-in-f1-how-to-become-a-formula-1-aerodynamicist/">https://motorsportengineer.net/jobs-in-f1-how-to-become-a-formula-1-aerodynamicist/</a></li><li>Image source website: <a href="https://en.wikipedia.org/wiki/Atomic_orbital">https://en.wikipedia.org/wiki/Atomic_orbital</a></li><li>Raissi, Maziar, Paris Perdikaris, and George E. Karniadakis. “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.” <em>Journal of Computational physics</em> 378 (2019): 686–707.</li><li>Lu, Lu, Pengzhan Jin, and George Em Karniadakis. “Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators.” <em>arXiv preprint arXiv:1910.03193</em> (2019).</li><li>Li, Zongyi, et al. “Fourier neural operator for parametric partial differential equations.” <em>arXiv preprint arXiv:2010.08895</em> (2020).</li><li>Brandstetter, Johannes, Daniel Worrall, and Max Welling. “Message passing neural PDE solvers.” <em>arXiv preprint arXiv:2202.03376</em> (2022).</li><li>Image source website: <a href="https://easydrawingguides.com/how-to-draw-a-monarch-butterfly/">https://easydrawingguides.com/how-to-draw-a-monarch-butterfly/</a></li><li>Vinuesa, Ricardo, et al. “β-Variational autoencoders and transformers for reduced-order modelling of fluid flows.” (2023).</li><li>Coscia, Dario, Nicola Demo, and Gianluigi Rozza. “Generative Adversarial Reduced Order Modelling.” <em>arXiv preprint arXiv:2305.15881</em> (2023).</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8634cda1022a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/sissa-mathlab/how-will-deep-learning-be-used-to-speed-up-physical-simulations-8634cda1022a">How will deep learning be used to speed up physical simulations?</a> was originally published in <a href="https://medium.com/sissa-mathlab">SISSA mathLab</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>