<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Shiyu Li on Medium]]></title>
        <description><![CDATA[Stories by Shiyu Li on Medium]]></description>
        <link>https://medium.com/@shiyu.syl?source=rss-165a42729b7a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*hrggoejclLRD07Wq</url>
            <title>Stories by Shiyu Li on Medium</title>
            <link>https://medium.com/@shiyu.syl?source=rss-165a42729b7a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 23:34:40 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@shiyu.syl/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Simulating Complex Physics with Graph Networks: Step by Step]]></title>
            <link>https://medium.com/stanford-cs224w/simulating-complex-physics-with-graph-networks-step-by-step-177354cb9b05?source=rss-165a42729b7a------2</link>
            <guid isPermaLink="false">https://medium.com/p/177354cb9b05</guid>
            <category><![CDATA[simulator]]></category>
            <category><![CDATA[cs224w]]></category>
            <dc:creator><![CDATA[Shiyu Li]]></dc:creator>
            <pubDate>Wed, 26 Jan 2022 02:24:19 GMT</pubDate>
            <atom:updated>2022-01-26T02:24:19.833Z</atom:updated>
            <content:encoded><![CDATA[<p>By Haochen Shi, Peng Chen, Shiyu Li as part of the Stanford CS224W course project</p><h3>Overview</h3><p>Many important tasks require high-resolution simulations of complex physics. For instance, scientists working on model-based planning would like to know how objects with complex physics such as deformable objects will transform after an environment change (e.g., pinched by a robot arm’s gripper, as shown in <a href="http://dpi.csail.mit.edu/">this work</a>[3]). Also, the movie industry is eager to generate realistic special effects such as massive explosions to produce more eye-catching scenes.</p><p>A promising method to generate simulations of complex physics is to train a GNN (Graph Neural Network) that can learn how to simulate. In this tutorial, we will introduce why this method is suitable to handle this task, and a step-by-step guide on how to build a GNN that can simulate complex physics. This tutorial is based on the paper <em>Learning to Simulate Complex Physics with Graph Networks </em>by DeepMind[1].</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fh7h9zF8OO7E%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dh7h9zF8OO7E&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fh7h9zF8OO7E%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e9d4b2a1c828e4af62a1996e74ea6710/href">https://medium.com/media/e9d4b2a1c828e4af62a1996e74ea6710/href</a></iframe><h3><strong>Why not traditional physics simulator?</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wfwybY8WSi6EnJla" /><figcaption>100 million particles simulated in 8 hours on a V100 GPU in Taichi physics engine. Credit:<a href="https://github.com/taichi-dev/taichi_elements">https://github.com/taichi-dev/taichi_elements</a></figcaption></figure><p>Although traditional physics simulators are powerful, there are some important drawbacks with them: 1) it is expensive and time-consuming to get high-quality results of large-scale simulations for traditional physical simulators; 2) to set up physical simulators, we need to have full knowledge of the physics parameters of the object and the environment, which are extremely hard to know in some cases.</p><p>To address these problems, many computer scientists are devoted to using machine learning methods to simulate complex physics. These machine learning models can learn the complex physics of objects directly from visual input without additional information from humans and make accurate and efficient predictions on the objects’ future states.</p><h3>Why is GNN a good choice for this task?</h3><h4><em>Input to a GNN</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iOjzATNMaKJ07mbOvSOPqA.png" /><figcaption>Less structured and structured data. Credit:<a href="http://web.stanford.edu/class/cs224w/slides/06-GNN1.pdf">http://web.stanford.edu/class/cs224w/slides/06-GNN1.pdf</a></figcaption></figure><p>Graph Neural Network (GNN) is a special subset of neural networks that take less structured data, such as a graph, as input, while other neural networks like Convolutional Neural Network (CNN) and Transformer, can only accept more structured data (e.g., grid and sequence). By “less structured”, it means that the input can have arbitrary shapes and sizes and can have complex topological relations.</p><p>In particle-based physics simulation, we have the unstructured position information of all the particles as the input, which inspires the idea of using a GNN.</p><h4><strong>Permutation equivariance</strong></h4><p>One key characteristic of GNN which distinguishes it from other neural networks is permutation equivalence. That is to say, the nodes in a graph do not have a canonical order, so how we “order” the nodes in a graph does not impact the results produced by GNNs.</p><p>Since particles of an object are “identical” in the particle-based simulation, they are permutation-equivariant when applying physics laws on them. Therefore, a permutation-equivariant model such as a GNN is suitable to simulate the interactions between particles.</p><h4>The Structure of GNN</h4><p>The structure of a GNN layer also contributes to our decision of modeling the physics with a GNN. The key steps in each layer of GNN are:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C2yvnmrf0XLqqeGmEFyKvw.png" /></figure><p>Intuitively, a particle-based physics model maps to the above structure of GNN perfectly: 1) nodes represent physical states of particles; 2) message-passing through edges resembles pairwise interactions between particles; 3) neighbor aggregation simulates the total effects of all the interactions on a particle from its neighbors; 4) the update step updates each particle’s position after calculating all the interactions.</p><h3><strong>Dataset</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/360/1*NbtOA4SvC0AuAAawoKlcvw.gif" /><figcaption>A trajectory of water drop in dataset.</figcaption></figure><p>The dataset can be found <a href="https://github.com/deepmind/deepmind-research/tree/master/learning_to_simulate">here</a>. It contains exhaustive scenarios, including the trajectories of particles with different physical properties (sand, goop, and water) in different scenarios (shaking and dropping). Our task is to predict trajectories of particles given initial conditions of particles and global properties of the system.</p><p>We select the WaterDrop dataset for the purpose of this tutorial.</p><h3><strong>Task Settings</strong></h3><p>The input vector of each particle consists of the current position of the particle, velocity of the particle in the past 5 steps, the features that represent the physical parameters of the particle, and, if applicable, the global property of the system (e.g., external force). The target of the GNN is to predict the acceleration of each particle. With the accelerations predicted by the GNN, an Euler integrator updates velocities and positions of particles.</p><h3><strong>Loss and Metric</strong></h3><p>Both the loss function and the final metric are defined by Mean Squared Error (MSE). The GNN outputs accelerations of particles, so the loss function is the MSE of accelerations, i.e.,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XVC5wDZupVcLKLDd0OO6uQ.png" /></figure><p>However, the learned simulator, which is composed of the GNN and the Euler integrator, outputs the positions of particles instead of accelerations. Thus, the metric of the learned simulator is defined as MSE of positions as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eUZAOH0j9UmayY096MZwRQ.png" /></figure><h3><strong><em>The Structure of GNS</em></strong></h3><p>The model in this tutorial is Graph Network-based Simulators(GNS) proposed by DeepMind[1]. In GNS, nodes are particles and edges correspond to interactions between particles. GNS consists of three parts, an encoder, a processor, and a decoder. To illustrate the structure of GNS in a more clear way, we refactored the open-source TensorFlow implementation by DeepMind into a more concise PyG version.</p><h4><strong><em>The Encoder</em></strong></h4><p>The encoder preprocesses data for the GNN. It adds an edge between any two particles close enough below a threshold R. Edge feature is the relative displacement between two particles. Before feeding these features into GNN, it encodes node features and edge features respectively through MLP.</p><p>The node encoder masks out the absolute position information of particles and encodes the velocity and physical parameters of particles into node features. The edge encoder encodes the relative position displacement between particles and their neighbors into edge features.</p><h4><strong><em>The Processor</em></strong></h4><p>The processor is a GNN with a stack of identical Interaction Network (IN) [2] layers. In each IN layer, both nodes and edges have hidden representations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*AusVkcRwY1nV7s5K" /><figcaption>Ilustration of messaging passing, aggregation and update in a layer.</figcaption></figure><p>GNN layers such as a IN layer can be easily implemented in PyTorch Geometric (PyG). In PyG, a GNN layer is generally implemented as a subclass of the MessagePassing class. We follow this convention and define the InteractionNetwork Class as follows</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ff7e337e19bcfc5e9411818fc1bc05dc/href">https://medium.com/media/ff7e337e19bcfc5e9411818fc1bc05dc/href</a></iframe><p>1) Construct a message for each edge of the graph. The message is generated by concatenating the features of the edge’s two nodes and the feature of the edge itself, and transforming the concatenated vector with an MLP.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/711a00e4db79fb910d0cdfbc0e4c0ac6/href">https://medium.com/media/711a00e4db79fb910d0cdfbc0e4c0ac6/href</a></iframe><p>2) Aggregate (sum up) the messages of all the incoming edges for each node.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7ff7a71e2a8ec54ae62ab68c43655561/href">https://medium.com/media/7ff7a71e2a8ec54ae62ab68c43655561/href</a></iframe><p>3) Update node features and edge features. Each edge’s new feature is the sum of its old feature and the message on the edge. Each node’s new feature is determined by its old feature and the aggregation of messages.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4eea8a89ee2dc092c5088d3126d21568/href">https://medium.com/media/4eea8a89ee2dc092c5088d3126d21568/href</a></iframe><h4><strong><em>The Decoder</em></strong></h4><p>The decoder uses an MLP to decode the accelerations of particles with node embeddings of the processor. It can also be regarded as a postprocessor of the GNN. With the decoded accelerations, an Euler integrator updates velocities and displacements of particles.</p><h4><strong>Putting Everything Together</strong></h4><p>Let’s include the encoder, the processor and the decoder together! Before GNN layers, input features are transformed by MLP so that the expressiveness of GNN is improved without increasing GNN layers. After GNN layers, final outputs (accelerations of particles in our case) are extracted from features generated by GNN layers to meet the requirement of the task.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ad4dc95ad4a5fe7d09e5914e1f6b7638/href">https://medium.com/media/ad4dc95ad4a5fe7d09e5914e1f6b7638/href</a></iframe><h3>Operation Modes of GNS</h3><p>The GNS works in two modes: one-step mode and rollout mode. In one-step mode, the GNS always makes predictions with ground-truth inputs. In rollout mode, the GNS predicts positions of particles in the next step based on its own predictions in the previous step. As a result, errors accumulate over time for rollout mode.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tmxNqxbJgLot_CWrJa0nyA.png" /><figcaption>Operations Modes of GNS: one-step mode and rollout mode. Each arrow points from an input of the model to the corresponding output of the model.</figcaption></figure><h3><strong><em>Key feature of the Model</em></strong></h3><h4><strong><em>Inductive Biases for Spatial Invariance</em></strong></h4><p>The physics interactions between particles are invariant to their spatial positions. To be consistent with this invariance, the encoder masks out absolute position information of particles and instead encodes the relative displacement between particles into features. This empowers the model to have strong inductive biases for spatial invariance.</p><h4><strong><em>Robustness</em></strong></h4><p>In simulations of complex physics of chaotic systems, the accumulation of errors over time frames could make huge negative impacts to the results. When training the model, researchers corrupt the input ground-truth data with random walk noise to represent errors generated by previous rollouts. This makes the model robust enough for such error accumulation.</p><h4><strong><em>Different Materials, One Model</em></strong></h4><p>The physical attributes of different materials are diverse. Most simulators are only applicable to one type of material[1]. GNS proposes a general approach to simulate the physics of different materials. It can produce accurate predictions for fluid, rigid and deformable material.</p><h3><strong>Results</strong></h3><p>Here is the result after training the model on the entire WaterDrop dataset for 5 epochs, which takes about 14 hours with a GeForce RTX 3080 Ti.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*T9rDvV_nZgA3ugXLwPOxOQ.gif" /><figcaption>Comparison between ground truth(left) and our result(right)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/398/0*DC-HtXn3lAnogg3C" /><figcaption>Training loss and validation loss.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/386/1*G7PsUho-UMlhDPfbbZ0zUw.png" /><figcaption>Mean Squared Error Metric when GNS works in one-step mode.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/392/1*ZEdfi8rr0bTW3bRMB8nxIw.png" /><figcaption>Mean Squared Error Metric when GNS works in rollout mode.</figcaption></figure><p>We plot the evolution loss and metric over the training process. The curve of loss and the curve of one-step MSE show that the model converges at the end of the training. But the curve of rollout MSE metric shows that the model is not so stable over a long time. One explanation is that the simulated system is chaotic in nature, such that one small numerical error will lead the system to behave totally differently.</p><p>Finally, we measure the metric on the test set. Our reproduction reaches a one-step MSE of 3.04e-9 and a rollout MSE of 1.42e-2, which are reasonable compared to the original paper as we have less computation power.</p><h3>Google Colab</h3><p>We made a Colab Notebook with all the code for this tutorial and a cropped version of the WaterDrop dataset. Here is the <a href="https://colab.research.google.com/drive/1hirUfPgLU35QCSQSZ7T2lZMyFMOaK_OF?usp=sharing">link</a>. Feel free to make a copy to your own drive and play with it!</p><h3><strong>Conclusion</strong></h3><p>In this tutorial, we walked through a step-by-step guide on applying GNN in a real-world application such as simulating complex physics! If you’re interested in the technical details, please read the <a href="https://arxiv.org/abs/2002.09405">original paper</a> by DeepMind. Thanks for spending the time with us!</p><h3>Reference</h3><p>[1] A. Sanchez-Gonzalez, J. Godwin, T. Pfaff, R. Ying, J. Leskovec, and P. W. Battaglia. Learning to simulate complex physics with graph networks. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13–18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 8459–8468. PMLR, 2020. URL <a href="http://proceedings.mlr.press/v119/sanchez-gonzalez20a.html">http://proceedings.mlr.press/v119/sanchez-gonzalez20a.html</a>.</p><p>[2] P. W. Battaglia, R. Pascanu, M. Lai, D. J. Rezende, and K. Kavukcuoglu. Interaction networks for learning about objects, relations and physics. In D. D. Lee, M. Sugiyama, U. von Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5–10, 2016, Barcelona, Spain, pages 4502–4510, 2016. URL <a href="https://proceedings.neurips.cc/paper/2016/hash/3147da8ab4a0437c15ef51a5cc7f2dc4-Abstract.html">https://proceedings.neurips.cc/paper/2016/hash/3147da8ab4a0437c15ef51a5cc7f2dc4-Abstract.html</a>.</p><p>[3] Y. Li, J. Wu, R. Tedrake, J. B. Tenenbaum, and A. Torralba. Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids. In ICLR, 2019.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=177354cb9b05" width="1" height="1" alt=""><hr><p><a href="https://medium.com/stanford-cs224w/simulating-complex-physics-with-graph-networks-step-by-step-177354cb9b05">Simulating Complex Physics with Graph Networks: Step by Step</a> was originally published in <a href="https://medium.com/stanford-cs224w">Stanford CS224W: Machine Learning with Graphs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>