<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Improbable on Medium]]></title>
        <description><![CDATA[Stories by Improbable on Medium]]></description>
        <link>https://medium.com/@Improbableio?source=rss-a1c0470dc7a4------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*Ws5zE3NXdqsn2u5S.png</url>
            <title>Stories by Improbable on Medium</title>
            <link>https://medium.com/@Improbableio?source=rss-a1c0470dc7a4------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 10:27:32 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@Improbableio/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Linda and a virtual world in a shoebox]]></title>
            <link>https://medium.com/an-improbable-future/linda-and-a-virtual-world-in-a-shoebox-929be1b8294b?source=rss-a1c0470dc7a4------2</link>
            <guid isPermaLink="false">https://medium.com/p/929be1b8294b</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[simulation]]></category>
            <category><![CDATA[virtual-reality]]></category>
            <dc:creator><![CDATA[Improbable]]></dc:creator>
            <pubDate>Wed, 18 May 2016 09:59:33 GMT</pubDate>
            <atom:updated>2016-05-19T15:50:55.800Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A vision of building mirror worlds</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*luw1HsX60h7VodASvDn6yw.jpeg" /></figure><p>“You will look into a computer screen and see reality.” More than two decades ago, David Gelernter, a professor of computer science at Yale University, outlined a bold, yet clear vision. <em>Mirror Worlds: Or The Day Software Puts the Universe in a Shoebox</em> claimed that the computer would become a digital edifice of the real world: its movements, its complexity.</p><p>“Some part of your world — the town you live in, the company you work for, your school system, the city hospital — will hang there in a sharp colour image, abstract but recognisable, moving subtly in a thousand places.”</p><p>On many fronts, the internet today is its realisation. He not only accurately described websites, but blogging, ebooks, search engines, internet telephony and tablets.</p><p>In <em>Mirror Worlds</em>, computers move from useful tools, or simple filing cabinets for information, to an integrated actor in human life. They become a means by which you can explore the reality which surrounds you, not as a passive watcher, but as an active explorer, “… for most purposes, you don’t merely sit and stare. You zoom in and poke around, like an explorer in a miniature sub.”</p><h3>More than a mirror</h3><p>Not your typical computer scientist, Gelernter’s vision was born not of his love, but of his frustration for computers, “<a href="http://www.economist.com/node/21540383">From its beginnings, the software industry has suffered from having too many engineers. There are too many people who love computers and too few who are impatient with them</a>”, he told one publication.</p><p>A philosopher and a writer too, he saw the potential for computers beyond a digital repository, a mere reflection, or even, in spite of the book’s title, a mirror. For Gelernter, the screen becomes flesh.</p><p>The worlds he describes model the way the world works in all its complexity with multiple interacting programs. Data fuels the world from a variety of sources (digital and non), yet the world relies not just on the patterns from the data it ingests, but from the rules which determine the behaviour of the agents. Concerned with both the overall view of the interaction and emergent behaviour, the “top sight”, and the detailed behaviour of particular agents, these agents are designed to follow particular rules of behaviour which mimic the real world. The result is two-fold: worlds which mimic the systems of the real-world (through inputting data) and yet worlds which constitute their own reality (by creating their own data).</p><h3>Hello Linda</h3><p>The sheer difficulty of computing large worlds with intricate detail, meant that a single desktop computer would not be enough.</p><p>As a graduate, he had already developed Linda, a parallel-programming language which allows programs running on different machines, to co-ordinate their actions. The concept behind this was simple: interconnectivity between computers can create a more powerful machine to work on a single problem.</p><p>In 1991, he and some colleagues at Yale demonstrated how this could work by linking up 14 workstation computers to create a network as powerful as a supercomputer, yet much cheaper. He emerged as a seminal thinker in the field of parallel, or distributed, computing. A forerunner of modern “cloud computing”, he is also attributed by many to have coined the term “cloud” to describe information “afloat in a virtual space”.</p><h3>Swarming software</h3><p>Beyond his own invention, Gelernter also saw the need for a new approach in designing software which could create virtual worlds. Quoting contemporary scientist Bernardo Huberman, he describes a “community of concurrent processes which, in their interactions, strategies, and competition for resources, behave like whole ecologies”.</p><p>Computers could only mirror ecologies and complex networks if they themselves learned to behave in way which mimics an ecological, rather than mechanical, system. Distributed computation would have to cope with the complicated business of rendering non embarrassingly parallel problems: problems which cannot be easily distributed across many machines but emerge in space and time. Phenomena which are both born of, and create, new complex relationships cannot be simplified into linear computational methods. It needs a new way.</p><p>At Improbable we have been working on exactly this problem. Creating a platform which can distribute compute across servers to create simulated worlds at scale. Simulating entire cities, for example, (with all their processes, infrastructure, and networks) requires huge compute power, distributed across many servers in the cloud.</p><p>SpatialOS, our platform, enables this distribution of work despite the fact that the entities (or the things that make up the simulation- cars, houses, buildings, pedestrians, for example) are not embarrassingly parallelizable; computing their relationship to each other cannot be done easily across a network of servers. This is because, unlike similar technologies which compute easily parallelizable problems, they exist and interact within space. Computing their complex relationship, the inter-machine messaging (think how the effect of one traffic jam, could affect the traffic flow of a completely separate part of town, computed on a different server), is the problem we have worked tirelessly on to solve.</p><p>SpatialOS then organically brings up as many servers the simulation requires: enabling developers to harness the power of these engines across vast spaces and complex behaviours. This smart structure responds to faults instantaneously: bringing up and replacing servers to recreate the state of the world within milliseconds. The system works like an organism, responding and reacting to the demands of the simulation in real-time.</p><h3>“The technology exists, but our Mirror World is uninhabited. It’s like a forest with nothing in it”.</h3><p>For the Economist, writing in 2013, the convergence of the digital and actual world is already happening. Smartphones and smart systems do just this. But the current applications are only a partial realisation of Gelernter’s vision. These applications are built on an ability to distribute embarrassingly parallelizable problems: they use big data in an effective way to create new forms of efficiency.</p><p>Creating simulated worlds which have as much detail as the real world, where you can interact with thousands of people in the same online space simultaneously, has not yet been realised.</p><p>The effects of these worlds, he saw, were not to confound or overpower human thinking but to empower the users, “the thick, dense, busy sub-world that encompasses you is also, now, an object in your hands. A brand new equilibrium is born.”</p><p><a href="https://improbable.io/">At Improbable</a>, we want to put this technology in your hands to create and populate simulated worlds.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=929be1b8294b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/an-improbable-future/linda-and-a-virtual-world-in-a-shoebox-929be1b8294b">Linda and a virtual world in a shoebox</a> was originally published in <a href="https://medium.com/an-improbable-future">An Improbable Future</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[When virtual worlds feel more alive]]></title>
            <link>https://medium.com/an-improbable-future/when-virtual-worlds-feel-more-alive-356bc64ecc8d?source=rss-a1c0470dc7a4------2</link>
            <guid isPermaLink="false">https://medium.com/p/356bc64ecc8d</guid>
            <category><![CDATA[games]]></category>
            <category><![CDATA[complexity]]></category>
            <category><![CDATA[game-development]]></category>
            <dc:creator><![CDATA[Improbable]]></dc:creator>
            <pubDate>Wed, 11 May 2016 15:35:24 GMT</pubDate>
            <atom:updated>2016-05-26T13:40:26.352Z</atom:updated>
            <content:encoded><![CDATA[<p><strong><em>Or why complexity science is relevant to gamers</em></strong></p><p>In 1984 The Santa Fe Institute, then a nascent, interdisciplinary research body, held one of its first workshops. “Emergent Syntheses in Science” was an event which gathered together scientists, mathematicians, and philosophers. They were interested in one thing: emergent behaviour. Why, they wondered, is the collective behaviour of a bee hive not easy to understand by just looking at a single bee?</p><p>Emergent behaviour is everywhere. Bees making honey, ants colonising, diseases spreading, people forming neighbourhoods, the assembly of a cancer cell, an embryo forming in the womb. These are the self-organising systems we see everywhere around us. Forming, spreading, determining outcomes in the world; these are systems with no leaders, hierarchical structures or central control. These are dynamic systems which adapt in and evolve with their surrounding environments. Neither are they static; this behaviour is always surprising.</p><h3>So what does this mean for game designers?</h3><p>This could seem like a very distant discussion from the world of games. But, in fact, emergent behaviour is the stuff of stories. Because while this has huge implications for our understanding of the world around us, it also confirms a very personal, human instinct: we don’t always get to control what happens next. Within our day to day lives, actions are not scripted, outcomes are unpredictable, behaviour emerges.</p><blockquote>Within our day to day lives, actions are not scripted, outcomes are unpredictable, behaviour emerges.</blockquote><p>You couldn’t have predicted the things that happened to you today. You might have known you would read an article today, but you probably didn’t know that it would be this one. And this is, arguably, what makes life engaging, what makes it immersive. This behaviour is not predetermined, it is based on rules. Physics, weather patterns, social norms. But combined, and interacting with the myriad of other systems and variables, it creates complex behaviour. Replacing scripted design for simulated- based design has created a very new kind of game experience. Minecraft, Half-Life, Second Life, these are all games which have transformed the player’s experience. How does it work?</p><h3>Let’s look at some birds.</h3><p>Let’s take a deeper look at this idea and start with a very simple flock of birds. This is actually a classic study used by complexity scientists.</p><p>Here we have simulated thousands of birds, each performing a role within a much larger body. They have goals: to find a nest, find food, find their next destination to which they are migrating. Even though there’s no leader, they move in patterns that are clearly not random. Let’s digitally recreate this real-world behaviour, using a simulation based on simple rules.</p><p><strong>Rule 1: All birds must travel at the same speed.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*ahVZZ1K-adxf0Ytnt6WWBQ.gif" /><figcaption>Rule 1: All birds must fly at the same speed.</figcaption></figure><p>Clearly, they are not flocking as we know they would in the wild. But each and every bird here is flying at the same speed.</p><p><strong>Rule 2: All birds must stay close to their neighbour.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*BOZPPDZR9ds6No6gUk6pDg.gif" /><figcaption>Rule 2: All birds must stay close to their neighbour.</figcaption></figure><p>Now we are a little closer to the shape of a flock. But we can see immediately that these birds have no concept of danger. They would not move out of a predator’s way or even each other’s; here we can see that they are crashing into each other. We clearly need to add in another rule…</p><p><strong>Rule 3: All birds must avoid danger</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*dYgRdFmcd58jMioB5jDTsw.gif" /><figcaption>Rule 3: All birds must avoid danger</figcaption></figure><p>Finally, we can see that there is a rule that the birds crash into each other or, indeed, other external parts of their environment.</p><p>Ta-da! Now the birds not only move at the same speed, and in the same direction, they now also form patterns as we see in nature; we have simulated the movement of birds flocking. Because this hasn’t been faked (or scripted) the birds would respond to their environment based on those rules; just as they would in the wild. This means that how you could interact with them would feel more real; you can’t fully predict their behaviour in an environment where multiple variables could affect their movement in a variety of ways. Simple rules, such as we showed in the flocking example, gives rise to complex behaviour.</p><h3>Simulation based design: the key to a more engaging world?</h3><p>This concept is at the basis of any simulation based design. Minecraft and other survival sandbox games are examples of how engaging these worlds can be to play in. But these all constrained by the size of their servers and so the dynamic nature of the things you can interact with are limited by the compute power they use. The idea of creating entire complex ecosystems within the worlds has only just begun. What if we could build worlds with hundreds of complex systems all interlocking and shaping how the player reacts? The world would have its own internal logic, its own expectations. But every time you play in this world you would experience something different, something surprising, something unexpected.</p><p>To see how you could integrate a flocking worker in your own game, check out our blog <a href="https://improbable.io/2016/04/21/create-custom-flocking-worker-spatialos">How to Create a Custom Flocking Worker in SpatialOS</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=356bc64ecc8d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/an-improbable-future/when-virtual-worlds-feel-more-alive-356bc64ecc8d">When virtual worlds feel more alive</a> was originally published in <a href="https://medium.com/an-improbable-future">An Improbable Future</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The future of the game engine]]></title>
            <link>https://medium.com/an-improbable-future/the-future-of-the-game-engine-564c7ec07e92?source=rss-a1c0470dc7a4------2</link>
            <guid isPermaLink="false">https://medium.com/p/564c7ec07e92</guid>
            <category><![CDATA[virtual-reality]]></category>
            <category><![CDATA[gaming]]></category>
            <category><![CDATA[game-development]]></category>
            <dc:creator><![CDATA[Improbable]]></dc:creator>
            <pubDate>Tue, 26 Apr 2016 10:03:20 GMT</pubDate>
            <atom:updated>2016-04-27T09:39:07.750Z</atom:updated>
            <content:encoded><![CDATA[<p><em>To read the original post, </em><a href="https://improbable.io/2016/04/21/future-game-engine"><em>see here</em></a><em>.</em></p><p>The dream of virtual worlds has been upon us for decades. Science fiction is obsessed with it; where will they take us? What will our interactions look like in them? Will we care about the things that happen within them? Will the virtual world replace the real world? While the promise of artificial intelligence is augmenting the way we think, virtual worlds promise a new landscape for the imagination to roam.</p><p>In living worlds, we will be able to connect with thousands of other adventurers, all exploring the same world with us; our actions will matter, the world won’t be scripted. But while we focus on the headsets and on the first games to come out, what, we wonder, will be the future of the technology used to build them? And what does the future of the game engine look like?</p><h3>The game engine today</h3><p>The current generation of games is pushing the boundaries of what’s possible with today’s technology, both in terms of art and gameplay. Game engines such as Unity and Unreal do a fantastic job of rendering game worlds for the player, yet they are also doing all the work under the hood to make the game itself happen — running physics, AI, pathfinding, and game logic.</p><p>For a single player game, the engine will be doing all this work on the player’s device. The concept of the game loop hasn’t changed much since its invention decades ago.</p><p>Multiplayer games aren’t very different. The game loop is split between client and server, with the client dealing with inputs and rendering, and the server performing the world simulation work.</p><p>But there is a limit to what can be achieved building games this way. You can only fit so much physics and AI on a single server, and this inevitably leads to compromises in terms of the size of the world, the number of players, or the complexity of the physics or the AI.</p><p>This, in turn, imposes limits on the creative vision. When game logic, physics and AI are already competing for a share of the available resources, there’s not much room left to experiment with truly innovative ideas.This is a problem we’ve been doing a lot of thinking about.</p><h3>The Entity-Component-System pattern</h3><p>Most games follow the <a href="https://en.wikipedia.org/wiki/Entity_component_system">Entity-Component-System</a> (ECS) pattern to some degree. Every “thing” in the game world is represented by <em>entities</em>. These can be characters, trees, parts of cars or entire spaceships, depending on your game.</p><p>Entities are defined by their <em>components</em>, which group the properties that describe their state. Components can be shared between different entities, which leads to a modular design. For example, both characters and trees can share a Physical component that defines their position and rotation, whereas an Inventory component will be used only by characters.</p><p>These entities and components are brought to life by <em>systems</em>, typically threads that deal with the different aspects of the game. The physics system updates the Physical component, the AI system will observe the world and make decisions, the game logic system will make the game world behave in the way its designers intended, and so on.</p><p>The limitation of this approach comes from the fact that all these systems run on a single server with limited capacity; each system is competing for cycles with every other system.</p><p><strong>But what if each system could be a <em>distributed system</em>?</strong></p><p>This is the same epiphany Google had in the early 2000s: it is cheaper, simpler and infinitely more scalable and reliable to develop applications that run in clusters of commodity hardware instead of running on monolithic mainframes.</p><p>Of course, building distributed systems is hard. Insanely hard, even; you need to debug race conditions, write netcode, think about concurrency and consistency, try to understand Paxos and implement Raft, build fault tolerance into your design, have a strategy to deal with network latency and packet loss, put metrics and monitoring systems in place, and a myriad other issues which are not directly related to the fun and joy of making games.</p><p>It seems like a hopeless situation. The solution exists in theory, but in practise it seems impossibly complex for game developers, who are already stretched to their limits with everything involved in building a game!</p><h3>Introducing workers: a new approach.</h3><p>Fortunately, it is possible to formulate this problem in an elegant way, which preserves the simplicity of the ECS pattern but makes it possible for a platform to abstract away all of the unpleasantness of building distributed systems.</p><p>The key innovation is introducing the concept of <em>workers</em>. We take each system in ECS and replace it with a distributed system; each distributed system is built out of many workers, each of them being a program that can simulate some components of a subset of the entities in the world. The most obvious general- purpose examples of workers are game engines, logic workers or even game clients (desktop, mobile or even VR headset).</p><p>But say, for example, you would like to build a world with a fully functional ecosystem. Perhaps, you want to have realistic flocking behaviour. You could introduce this into this same approach. In this way of developing, a model of flocking behaviour could also be integrated. By giving each instance of a worker authority to simulate a component for a set of entities, it enables you to scale your game world. For these entities, it sends periodic updates reflecting their state. The worker then performs the simulation it is designed to do, and sends updates back to SpatialOS.</p><p>We call this the <strong>Entity-Component-Worker</strong> architecture:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*-wSakyHzqedZGVmZPlC2Dw.png" /><figcaption>The Entity-Component- Worker architecture</figcaption></figure><p>This new architecture means the size of your game world is no longer limited by, for example, how much the physics engine on a single server can handle, because you run hundreds of instances of the physics engine to simulate different parts of the world.</p><p>This is rendered even more complicated when the number of workers you have are completely dynamic: you might need a physics engine for part of the game world, but not for another. Working out how to distribute the load across many, potentially hundreds of physics engines, is therefore a massive challenge. This is the problem we solved here at Improbable. How to manage these workers and run them in the cloud.</p><h3>Conclusion</h3><p>We envision the next generation of games will be built on a set of workers that cooperate to simulate a game world far bigger and far more complex than it is possible with a traditional architecture or in a single server.</p><p>Game engines leveraging cloud infrastructure in intelligent ways, will help us realise our dream of virtual worlds.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=564c7ec07e92" width="1" height="1" alt=""><hr><p><a href="https://medium.com/an-improbable-future/the-future-of-the-game-engine-564c7ec07e92">The future of the game engine</a> was originally published in <a href="https://medium.com/an-improbable-future">An Improbable Future</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How SpatialOS simulations will help us uncover hidden vulnerabilities in the Internet]]></title>
            <link>https://medium.com/an-improbable-future/how-spatialos-simulations-will-help-us-uncover-hidden-vulnerabilities-in-the-internet-c5502cf9d79?source=rss-a1c0470dc7a4------2</link>
            <guid isPermaLink="false">https://medium.com/p/c5502cf9d79</guid>
            <category><![CDATA[cybersecurity]]></category>
            <category><![CDATA[internet]]></category>
            <category><![CDATA[simulation]]></category>
            <dc:creator><![CDATA[Improbable]]></dc:creator>
            <pubDate>Thu, 24 Mar 2016 15:50:16 GMT</pubDate>
            <atom:updated>2016-03-24T16:21:36.279Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Read original post </em><a href="http://bit.ly/1RnzpoM"><em>here</em></a><em>.</em></p><p>This January Telstra, Australia’s largest telecoms provider, suffered a countrywide outage. The outage affected 16.7 million subscribers on its 3G and 4G networks and preventing phone calls; other users reported complete loss of phone and data services for over three hours. The outage not only cost the company millions in compensation (<a href="http://www.abc.net.au/news/2016-03-18/telstra-ceo-apologises-for-four-hour-outages/7256706">just last week, the CEO announced a day of free data</a>) it also placed businesses who rely on mobile phone communications at risk of losing hours of trade.</p><p>These kinds of incidents aren’t<a href="https://bgpstream.com/"> rare</a>; errors happen often, <a href="http://www.bgpmon.net/blog/">frequently at scale</a>. Sometimes they happen by mistake. In 2008, for example, an ISP (Internet Service Provider) in Pakistan famously <a href="http://www.networkworld.com/article/2284273/software/youtube-pakistan-incident--could-something-similar-whack-your-site-.html">cut off access to YouTube</a> for a large portion of the world by accidental misconfiguration of a central server.</p><p>Accidents happen, but more concerning are deliberate attempts to hijack traffic, preventing access to communication channels and content. At the height of the Arab Spring, the Egyptian government closed <a href="http://www.telegraph.co.uk/news/worldnews/africaandindianocean/egypt/8288163/How-Egypt-shut-down-the-internet.html">88% of the Egyptian internet</a>, blocking more than 3,500 routes. This ensured that no website could be accessed and that its citizens could not communicate (through email or social media, among others). Meanwhile, the largest Distributed Denial of Service attack (DDoS) occurred against a non-profit anti-spam organisation Spamhaus, <a href="http://www.nytimes.com/2013/03/27/technology/internet/online-dispute-becomes-internet-snarling-attack.html?&amp;_r=0">affecting millions of ordinary Internet users</a>.</p><p>What does this tell us? That these kinds of events are happening more often, and getting more severe. Businesses, governments, and ordinary people are increasingly reliant on services they get over the Internet. Imagine the implications of these kinds of errors and attacks on the Internet’s infrastructure when our cities, cars and home get connected to the Internet too. Truly understanding what the World Economic Forum once termed “<a href="http://www.wired.co.uk/news/archive/2012-01/12/dark-side-of-connectivity">the dark side of interconnectivity</a>” is critical; but this is well beyond the reach of current analysis tools.</p><h4>Simulating the entire backbone of the Internet on SpatialOS</h4><p>A few weeks ago, a team of two came in from the British government to explore our technology. Their goal was to build a realistic simulation of the internet so that they could take a look at it in a new and unique way, a way that would actually show the internet’s “structure”, or in other words, the vast number of connections between computers and networks that make up the World Wide Web. With the internet under attack from a variety of sources, it’s critical they understand how to make it more resilient.</p><p>Quite an ambitious project, given they had never used our technology, SpatialOS, before and the huge scale and complexity of the system they wanted to simulate.</p><p>In their own words, <em>“Having never developed an application on SpatialOS before, this was a tall order for a three day sprint. However, combining the flexibility of the platform with the experience and enthusiasm of Improbable’s engineers, resulted in a simulation that surpassed all initial expectations.”</em></p><p>The result was <strong>a 1:1 simulation of the backbone of the internet</strong>. We believe it is the largest simulation of its kind ever to have been created.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FNYcFLV1mUx8%3Ffeature%3Doembed&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DNYcFLV1mUx8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FNYcFLV1mUx8%2Fhqdefault.jpg&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/0d3636f3c4abc177e3e74268abc2f84c/href">https://medium.com/media/0d3636f3c4abc177e3e74268abc2f84c/href</a></iframe><p>In precise terms, this is a fully dynamic and interactive simulation of all the <a href="https://en.wikipedia.org/wiki/Autonomous_system_(Internet)">Autonomous Systems</a> (<strong>AS</strong>) on the internet, using a communication system closely modelled on the <a href="https://en.wikipedia.org/wiki/Border_Gateway_Protocol">Border Gateway Protocol</a> (<strong>BGP</strong>). BGP essentially determines how data can flow from one server to another. An AS usually belongs to an ISP such as BT or Comcast, but may also be owned by sites like Netflix or Youtube.</p><p>For example, when you try to load youtube.com, a request is sent to your ISP. Your ISP may not have a direct link to Youtube’s data centre, but thanks to its routing table, it will know how to get one step closer. The request is then passed along to another ISP and might go through several more locations before ending up at Youtube.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-jPZlbo5vVr7NjY-YT_IOw.jpeg" /><figcaption>When you send a request for youtube.com, your ISP will attempt to pass the request on via the best route, according to its internal routing table. The next server will do the same, and so on until the request reaches the destination.</figcaption></figure><p>But what if one of those servers along the way is malfunctioning? Or being rerouted by someone? Perhaps it actually doesn’t know how to reach youtube.com? In that case, nobody using your ISP will be able to reach Youtube.</p><p>By letting you add and delete links in the network, you can ask &quot;what if&quot; questions to determine the impact of interventions. SpatialOS enables you to build large scale simulations which can be used to investigate these kinds of failures and find the routes most likely to be affected. What happens if a rogue AS starts advertising routes to everywhere but then dumps all traffic? Perhaps there are a few key routes we could protect that would mitigate the effects of such failures.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2Fl2QZY3XNeZekl4Dba%2Ftwitter%2Fiframe&amp;src_secure=1&amp;url=http%3A%2F%2Fgiphy.com%2Fgifs%2Fl2QZY3XNeZekl4Dba&amp;image=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2Fl2QZY3XNeZekl4Dba%2Fgiphy.gif&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/15412dec83c64f64ddda5d590e863f4c/href">https://medium.com/media/15412dec83c64f64ddda5d590e863f4c/href</a></iframe><p>With SpatialOS, you could also use such simulations to monitor routes in the real world and detect when they deviate from what the simulation predicts, to spot errors as they happen or to detect more malicious behaviour. Considering how many people rely on the internet, the implications are enormous.</p><h4>Scale: simulating thousands of connections on SpatialOS</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2F3osxY7fKoIYHFUDw2s%2Ftwitter%2Fiframe&amp;src_secure=1&amp;url=http%3A%2F%2Fgiphy.com%2Fgifs%2F3osxY7fKoIYHFUDw2s&amp;image=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2F3osxY7fKoIYHFUDw2s%2Fgiphy.gif&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/e4b706e7f5b85ab4fc06c2dd579117c7/href">https://medium.com/media/e4b706e7f5b85ab4fc06c2dd579117c7/href</a></iframe><p>The whole internet contains about 60,000 AS networks and over half a million routes. Since every AS has to store path information about routes to nearly every other AS, the routing tables can become unmanageably large. This has <a href="http://www.theguardian.com/technology/2014/aug/14/internet-infrastructure-needs-updating-more-blackouts-will-happen">caused problems</a> as the global routing tables of each AS grows.</p><p>In our simulation, <em>every </em>AS is running simultaneously and independently, so storing the full routing table for each AS requires many terabytes of RAM by itself. Doing this sort of simulation on a single server would be almost impossible; we don’t have the full list of networks and routes, but what we do have is enough to require 15 machines and over 1TB of RAM. Building a distributed system of this magnitude without SpatialOS would have been an enormous undertaking and the result wouldn’t have been as flexible or easy to maintain, and building it during a three day sprint would have been unthinkable.</p><p>In SpatialOS terms, every AS and route is represented as an <em>entity</em>. They each act independently with no shared data; information is shared by message passing.</p><p>There are two kinds of <a href="http://improbable.io/learn-more">workers</a> in the simulation. The BGP protocol and the network flows themselves run in the logic workers, as behaviours associated to the AS and route entities. SpatialOS runs as many logic workers as necessary to handle the workload of the simulation.</p><p>The visualisation and interaction layer is built using the <a href="https://unity3d.com/">Unity game engine</a> and the SpatialOS SDK. This is an unmanaged worker that runs on an end-user machine, and serves the dual purpose of visualising the state of the simulation and letting the user influence it. Using a game engine lets us iterate quickly to build very attractive visualisations.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2F3osxY8ktcwIS6yoKSk%2Ftwitter%2Fiframe&amp;src_secure=1&amp;url=http%3A%2F%2Fgiphy.com%2Fgifs%2F3osxY8ktcwIS6yoKSk&amp;image=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2F3osxY8ktcwIS6yoKSk%2Fgiphy-downsized-large.gif&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/dafe3b8a6eb45727ab496d0350b73399/href">https://medium.com/media/dafe3b8a6eb45727ab496d0350b73399/href</a></iframe><p>The full source code of this simulation is available <a href="https://github.com/improbable-public/internet-simulation">here</a>. You need developer access to SpatialOS to build and run it; you can apply <a href="http://improbable.io/get-spatialos">here</a>.</p><h4>Building upon this model</h4><p>But the possibilities don’t end there. This <a href="https://github.com/improbable-public/internet-simulation">model</a> is available for developers can integrate with their existing models today. In fact, in the past week we have begun to integrate this model with an internal project to model cities. It was as simple as importing the code and setting up the locations correctly. Being able to combine simulations in this way will enable companies and organisations to understand how the internet relates to other complex systems: cities, infrastructure, energy, economies.</p><p>With a detailed simulation of the internet we could begin to prepare for cyber attacks before they happen, and understand the cascading effects of various interventions better. This will enable businesses, institutions and even countries to become more resilient in an age of exponential vulnerability online. But more than this, it is also possible to model any kind of network on this scale, be they energy grids or telecommunications. You could, for example, model a national power infrastructure network to comprehend not just its potential vulnerabilities but the impact it could have on a new transport network or the economy.</p><p>If you would like early access to SpatialOS to start creating your own simulations, or models of multiple connected networks, you can <a href="http://improbable.io/get-spatialos">sign up here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c5502cf9d79" width="1" height="1" alt=""><hr><p><a href="https://medium.com/an-improbable-future/how-spatialos-simulations-will-help-us-uncover-hidden-vulnerabilities-in-the-internet-c5502cf9d79">How SpatialOS simulations will help us uncover hidden vulnerabilities in the Internet</a> was originally published in <a href="https://medium.com/an-improbable-future">An Improbable Future</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>