<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Joe Boyle on Medium]]></title>
        <description><![CDATA[Stories by Joe Boyle on Medium]]></description>
        <link>https://medium.com/@3d_jb?source=rss-7ada5d85b73a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*OpDMgN9JyHHPQpH2.jpg</url>
            <title>Stories by Joe Boyle on Medium</title>
            <link>https://medium.com/@3d_jb?source=rss-7ada5d85b73a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 11:42:24 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@3d_jb/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Sharing in AR, Today and Tomorrow]]></title>
            <link>https://medium.com/@3d_jb/sharing-in-ar-today-and-tomorrow-da285881cd65?source=rss-7ada5d85b73a------2</link>
            <guid isPermaLink="false">https://medium.com/p/da285881cd65</guid>
            <category><![CDATA[spatial-computing]]></category>
            <category><![CDATA[ar-cloud]]></category>
            <category><![CDATA[ar]]></category>
            <category><![CDATA[augmented-reality]]></category>
            <category><![CDATA[iot]]></category>
            <dc:creator><![CDATA[Joe Boyle]]></dc:creator>
            <pubDate>Mon, 20 Aug 2018 17:22:04 GMT</pubDate>
            <atom:updated>2018-08-21T17:51:15.911Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://medium.com/@3d_jb/rights-management-in-augmented-reality-983bf195b400">The last post</a> considered <a href="http://www.wassom.com/from-the-archives-augmented-reality-as-free-speech.html">“AR as speech”</a> and the need for sharing controls that will be necessary to allow for a harmonious sharing of Augmented Reality.</p><h4>Platform Selection</h4><p>Discounting AR for a moment, when users decide to share something online, they make a decision about which platform to use: Facebook, Instagram, Medium, Reddit, Pinterest, Yelp, or Twitter — each designed to enable a specific type of authorship and sharing. Each platform has its own set of sharing controls, social norms, and user management challenges. This trend of platform multiplicity will continue, especially if the trend of adding AR as a feature to existing apps also continues.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F77ZJ3jw6EkA%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D77ZJ3jw6EkA&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F77ZJ3jw6EkA%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/175d46368cefeb3ee2d5536579239418/href">https://medium.com/media/175d46368cefeb3ee2d5536579239418/href</a></iframe><p>In the short term, established platforms experimenting with AR features will be a crucible through which we all collectively learn what works and doesn’t work in AR. This will be a messy process with a lot of lasting benefits that will take time, during which sharing controls will vary widely. If history has anything to teach us, soon after this fragmented experimental period we should expect to see mergers, acquisitions, and copycat implementations that take AR into a more uniform, commodity technology. Once the AR industry turns this corner, interoperability, open standards, and winner-take-all incentives will make an integrated, general purpose augmented reality possible — one that allows for discovery of nearby services and simultaneously viewing AR content from disparate sources.</p><p>Let’s look at the types of sharing controls which make sense in a general purpose AR.</p><h3>Saying Things Online</h3><p>While the “grammar” of systems may vary from platform to platform, high level concepts like author, recipient, subject tags and other metadata become the dimensions upon which sharing controls can operate.</p><p>Let’s look at the most relevant ones:</p><h4>Author</h4><p>Content on the internet is generally attributable to an author. The term we use may vary depending on the medium (poster, user, composer, creator, etc), but the concept applies universally to websites, mobile applications, gaming platforms, and so on.</p><p>Like the Internet itself, potential uses of AR are diverse enough to warrant many different user schemes, including strong, provable identities and persistent pseudonyms. However, the integrated nature of any “AR browser” requires that pseudonymous and anonymous content cannot simply be ambiguously intermingled with verified, strongly identified content or bad actors will be able to impersonate authoritative sources and mislead others. (A subsequent post will be dedicated to identity management as it applies to AR.)</p><p>As it is with the rest of the Internet, the creation of untraceable anonymous content is just not compatible with a general purpose AR platform.</p><h4>Recipient</h4><p>As with an author, many online platforms require one to specify a recipient. Any private form of communication, be it a DM, privmsg, whisper, or email works this way. However, on fully public posts, such as tweets, no recipient user specification is necessary (though tagging usernames can call the attention of others to each tweet).</p><p>Between fully public and fully private postings lives group sharing, where users can post to a subset of users, either by enumerating a list of recipients or by addressing users by a shared attributes. Mailing lists, Facebook Groups, private subreddits, and any group chat exemplify this model.</p><p>Private, public, and in-between — all will apply in AR.</p><h4>Subject</h4><p>Categorizing content also varies widely from platform to platform. A few examples of subject specification include:</p><ul><li>free text fields (such as an email subject line)</li><li>a nested hierarchy of subjects (such as Usenet or a bulletin board)</li><li>a flat hierarchy of subjects (such as Reddit)</li><li>hash-tags (such as Twitter or Instagram)</li></ul><p>Note that the line between subject and recipient is blurred in systems where recipients can be defined as those people interested in a specific subject. For example, a reddit post or facebook group post has no explicit recipients — the members of the forum are the presumed recipients who have opted-in based on their interest in that community’s focus.</p><p>Beyond the present day world of fragmented, special purpose AR applications, filtering content by subject is certainly a baseline immediate requirement for integrated multi-purpose AR tools. If we are ever to achieve a broad utility comparable to a web-browser, robust search and live content filtration tools are necessary. There is no scalable version of a persistent AR world in which all nearby virtual content is shown to the user simultaneously. I’ve seen this filtration referred to as “lenses”, however “AR lens” also means different things to different people.</p><h4>Content</h4><p>The content itself can also be the basis for sharing controls. Examples include limiting image/attachment sizes, blocking black-listed words, <a href="https://cloud.google.com/blog/products/gcp/filtering-inappropriate-content-with-the-cloud-vision-api">auto-detecting inappropriate content</a>, and even human moderation of each post.</p><p>User created AR content does not yet undergo any sort of moderation, but other mature, scalable 3D content platforms (such as SecondLife) do impose polygon limits on user creations.</p><h3>Location</h3><p>Many social networks already incorporate location, especially “mobile first” apps like Snapchat and Instagram where location data is readily available.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FSCgfbw8WwQ4%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DSCgfbw8WwQ4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FSCgfbw8WwQ4%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/22a835bdc0e09e78c8b5054672738549/href">https://medium.com/media/22a835bdc0e09e78c8b5054672738549/href</a></iframe><p>Compared to AR, this style of post annotation is more specifically focused on the map view of the world. (It’s worth noting that there is no widespread anxiety about vandal Snapchatters placing photos and videos on someone else’s property — yet we see this <a href="https://www.businessinsider.com/man-sues-niantic-over-pokemon-go-making-his-backyard-a-pokestop-2016-8">problem in AR when the content attracts too many people to the location with insufficient consideration for the impact</a>.)</p><p>As we established in <a href="https://medium.com/@3d_jb/rights-management-in-augmented-reality-983bf195b400">the previous post</a>, it isn’t feasible to prevent other people from creating augmentations on one’s property, but more importantly, while the conflict surrounding AR rights issues is currently modest compared to other Internet speech issues, bear in mind how small the audience of AR users is today. Any platform which permits content to be placed without any moderation or accountability features will not stand the tests of scaling up to millions of users.</p><h4>Public Spaces</h4><p>The first appearances of persistent multi-user AR suggest a completely public augmented world. Any augmentation, created anywhere is visible to whomever else is nearby, often summarized on a map view so others can more easily find them.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FnEu2CLfcbr4%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DnEu2CLfcbr4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FnEu2CLfcbr4%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/dc418f0cf6dea54475600de82b4978b3/href">https://medium.com/media/dc418f0cf6dea54475600de82b4978b3/href</a></iframe><p>This unrestricted model of sharing is a natural first step, but contributes to some of the misunderstandings and anxiety surrounding AR. The most obvious negative consequences of unrestricted AR are virtual graffiti and vandalism, but the lack of filtration, moderation, and management features would result in a slow loss of usefulness due to virtual objects accumulating and cluttering high traffic locations .</p><p>This gets worse as more and more users begin experimenting with persistent AR applications. Yet high traffic public places stand to benefit greatly with augmented public signage, landmarks, and other interactive <a href="https://en.wikipedia.org/wiki/Smart_city">smart city</a> features.</p><h4>Private Spaces</h4><p>As multi-user persistent AR matures into something with contemporary online sharing controls, being able to share private augmentations with specific individuals will become more useful, particularly in our homes and private spaces. Granting friends access to the AR controls for your <a href="https://en.wikipedia.org/wiki/Internet_of_things">smart home</a> appliances or inviting them to join you in a virtual table top game could be accomplished this way.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FT5Mvk6opwzo%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DT5Mvk6opwzo&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FT5Mvk6opwzo%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/96eeb177f56b8411961a00d5da5cb83e/href">https://medium.com/media/96eeb177f56b8411961a00d5da5cb83e/href</a></iframe><p>Completely private single-user AR features are possible without sharing controls, as they could be stored entirely locally, on-device. However any persistence, backup, or sharing will require storage in a cloud-based geospatial index.</p><h4>Group Spaces</h4><p>Between fully public spaces and private spaces are our shared spaces, home of some of the most valuable use cases in AR. Workplaces are a perfect example where non-public augmentations would benefit from being shared between many users. Industrial facilities such as factories, warehouses, and refineries stand to gain tremendously from AR technology; as do the smart-office applications for indoor navigation, room reservation, and facilities management. Use cases with strict security requirements may justify distinct standalone infrastructure, but the eventual advantages of integrated, multi-use AR will place a pressure on those systems to behave similarly to the public AR platforms.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F5E6qambiCo0%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D5E6qambiCo0&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F5E6qambiCo0%2Fhqdefault.jpg&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/625b6033ba46cacf8425486210487e4f/href">https://medium.com/media/625b6033ba46cacf8425486210487e4f/href</a></iframe><h3>Putting It Together</h3><p>By assembling <strong>author</strong>, <strong>recipient</strong>, <strong>subject</strong>, <strong>location</strong>, and <strong>content</strong> together, we can craft AR sharing controls that make these use cases possible:</p><ul><li><strong><em>Alice</em></strong> shares a <strong><em>board game</em></strong> session with <strong><em>Bob</em></strong> in <strong><em>her dining room</em></strong>.</li><li><strong><em>Charlie</em></strong> shares the virtual <strong><em>HVAC sensors</em></strong> with <strong><em>Daniel</em></strong>, a new hire at <strong><em>the University</em></strong>.</li><li><strong><em>Erin</em></strong> builds a <strong><em>3D model of a long demolished mansion</em></strong> on the site of its <strong><em>former location</em></strong> and shares her creation with her <strong><em>local history group</em></strong>.</li></ul><p>As explained previously, these examples are first going to appear in separate applications, but the inherent advantages of AR integration and interoperability will eventually follow, especially when AR moves into the headset in the form of AR glasses.</p><h3>What is Underway Now</h3><h4>Ubiquity 6</h4><p>Since <a href="https://medium.com/@3d_jb/rights-management-in-augmented-reality-983bf195b400">the last post</a>, <a href="https://ubiquity6.com/">Ubiquity 6</a> has surfaced as an AR platform for sharing and editing AR content. Having <a href="https://www.forbes.com/sites/charliefink/2018/08/14/ubiquity6-raises-37-5m-to-paint-the-world-with-ar-data/">just launched</a>, technical details are scant, nevertheless it is exciting to see AR growing in this direction.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fo0T7QHT0p0U%3Ffeature%3Doembed&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Do0T7QHT0p0U&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fo0T7QHT0p0U%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e095a236414b782886739a5fad65c84a/href">https://medium.com/media/e095a236414b782886739a5fad65c84a/href</a></iframe><p>It’s not realistic to expect Ubiquity 6 (or any other nascent AR platform) to come into the marketplace having already solved all these problems. Watch this series for details on their strategy as it emerges.</p><h4>VERSES</h4><p>Just last week <a href="https://www.prnewswire.com/news-releases/verses-pioneers-global-augmented--virtual-reality-economy-with-blockchain-power-for-the-future-web-300697330.html">VERSES left stealth mode and launched</a> a new blockchain token expressly for AR applications: indexing spaces and smart objects. Blockchain (and more specifically smart contracts) represents an entirely new family of geospatial indices — one which offers to trade the advantages of decentralization with the challenges of blockchain adoption. Watch this space for more detailed information on this strategy as it emerges.</p><h4>Unity Project MARS</h4><p>Even better than a global AR sharing platform would be developer tools and services that allow for AR sharing across otherwise fragmented applications.</p><p>Just a few weeks ago Timoni West, the Director of XR Research at Unity, presented one of the first detailed looks at Unity’s Project MARS — Mixed and Augmented Reality Studio. <a href="https://next.reality.news/news/unity-unveils-project-mars-for-simplified-ar-development-facial-ar-for-easy-mocap-0185339/">Launched at Unite Berlin</a> in June of 2018, Project MARS is a broad set of tools for Unity developers targeting some of the larger problems every MR/AR developer will soon face, including early proposals for UX modalities and most relevant to this post: AR permissions.West expresses her permission model as a matrix for describing application states:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BH7YaD3JCcviKaGZTM0orQ.png" /></figure><p>The entire presentation was captured at a Mozilla event, and can be viewed below. The discussion on AR permissions begins at 23:40.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FupM8suny2Po%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DupM8suny2Po&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FupM8suny2Po%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/a964240ef1adc396c3bb0449d593541a/href">https://medium.com/media/a964240ef1adc396c3bb0449d593541a/href</a></iframe><h4>The Mixed Reality Service</h4><p>Looking back to 2016, the first proposal of a general purpose geospatial index for augmented reality would be <a href="https://medium.com/u/6257c530d185">Mark Pesce</a>’s <a href="https://medium.com/ghvr/mixed-reality-service-ce469783fd68">Mixed Reality Service</a> (or MRS). The MRS proposal is many years ahead of its time. Before any commercialization or traction of any kind, on any AR platform, Pesce drafted a protocol to allow for diverse providers to publish AR services in a discoverable manner.</p><p>Better still, unlike Unity’s Project MARS, the MRS specification is an open standard, and a <a href="https://www.w3.org/community/mixedreality/">working group with the W3C</a> has been created to oversee its development, though little progress has been made since <a href="http://www.mixedrealitysystem.org/spec/Mixed_Reality_Service_Specification_THIRD_DRAFT.pdf">the most recent draft</a> was published almost two years ago. One would hope that the imminent success of AR hardware and persistent AR platforms will soon attract more interest in the open standards and web development communities.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FAXlFOPJqnEo%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DAXlFOPJqnEo&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FAXlFOPJqnEo%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/75b50dd1ee8591dbfaa792408b0f4528/href">https://medium.com/media/75b50dd1ee8591dbfaa792408b0f4528/href</a></iframe><p>In its current form, the MRS proposal is not yet complete, but serves as a stake in the ground around which proponents of an open AR web can gather.</p><h3>What’s Missing?</h3><p>Tune in next time for a closer look at the gaps and obstacles in front of these AR Cloud platforms and standards.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=da285881cd65" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Rights Management in Augmented Reality]]></title>
            <link>https://medium.com/@3d_jb/rights-management-in-augmented-reality-983bf195b400?source=rss-7ada5d85b73a------2</link>
            <guid isPermaLink="false">https://medium.com/p/983bf195b400</guid>
            <category><![CDATA[legal-technology]]></category>
            <category><![CDATA[xr]]></category>
            <category><![CDATA[real-estate-technology]]></category>
            <category><![CDATA[augmented-reality]]></category>
            <category><![CDATA[ar]]></category>
            <dc:creator><![CDATA[Joe Boyle]]></dc:creator>
            <pubDate>Mon, 30 Jul 2018 19:34:06 GMT</pubDate>
            <atom:updated>2019-03-23T06:59:55.054Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://medium.com/@3d_jb/first-steps-into-an-ar-cloud-relocalization-58d04ea93aa">Previous</a> <a href="https://medium.com/@3d_jb/global-relocalization-a-better-gps-b6b98d2db20c">posts</a> in this series have been about foundational AR technology already being built and deployed. This post will branch in a new direction, summarizing issues surrounding rights and permissions in AR — proposing some answers to the questions about who controls the creation and display of virtual augmentations in our real world spaces.</p><p>These questions have been posed before, often voiced with a palpable anxiety — “What if someone defaces my property in AR?” Articles from <a href="https://amp.slate.com/technology/2018/06/can-you-prevent-augmented-reality-ads-from-appearing-on-your-house.html">Slate</a>, <a href="https://techcrunch.com/2018/05/06/we-love-augmented-reality-but-lets-fix-things-that-could-become-big-problems/">Techcrunch</a>, <a href="https://www.newscientist.com/article/2142857-augmented-reality-graffiti-will-lead-to-advertising-ambush-wars/">NewScientist</a>, <a href="https://www.wired.com/2009/09/augmented-reality-ar-graffiti/">Wired</a>, <a href="https://www.fastcompany.com/90164445/googles-new-experiment-lets-you-tag-digital-graffiti-in-the-real-world">FastCompany</a>, and <a href="https://www.citylab.com/design/2018/03/whos-in-charge-of-the-augmented-city/554324/">several more</a> <a href="https://qz.com/1072528/the-next-trend-in-street-art-will-be-graffiti-in-augmented-reality/">tech journalism sites</a> exemplify a growing concern without solution — and while some of this coverage extends the long tradition of stoking techno-<a href="https://en.wikipedia.org/wiki/Fear,_uncertainty_and_doubt">FUD</a> for clicks, there are a number of legitimate concerns worthy of discussion. Furthermore, while early AR applications today are largely frivolous, the recent availability of persistence in AR, we should expect to see a wide range of digital content and services distributed via mobile AR interfaces. When these begin to arrive, it will become even easier to imagine AR becoming <a href="http://time.com/4654944/this-technology-could-replace-the-keyboard-and-mouse/">the next major computer interface</a> — and with it a huge amount of highly valuable unclaimed virtual territory.</p><p>Here’s some important distinctions that will make a more productive conversation possible.</p><h4>Material Reality vs Augmented Realities</h4><h3>Nathan Gitter on Twitter</h3><p>Time travel in AR! ⏳✈️ The artwork in this public graffiti park changes daily. With the power of #ARKit, it&#39;s possible to see into the past. Built with #ARKit2 on #iOS12. https://t.co/APa2Nj8jdC</p><p>Our collective understanding of rights management in physical spaces rests heavily upon the singular, mutually exclusive nature of physical objects in physical space. More plainly, two real objects cannot occupy the same place at the same time. This is true for every tangible object in the real world, and our social mores and laws justifiably presume as much. In AR, things do not work this way, so the rules we use to govern AR should reflect this.</p><p>There are two situations when my use of AR in a space need not conflict with anyone else’s real or virtual use of the same space: where we are using separate AR apps, and where sharing controls limit the visibility of our creations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/510/1*lm7FWvVhhJ0WvsWJfi_QSQ.png" /><figcaption>An example of Facebook’s sharing controls, circled in red.</figcaption></figure><p>Let’s look at both.</p><h4>Separate Geospatial Indices</h4><p>In the same way that photos created with Instagram do not appear in Snapchat, augmentations created in one AR application will not appear in another. This is largely because every AR application developer is currently left to their own devices when it comes to building the back-end infrastructure used to manage and store their augmentations. However, much of the debate surrounding AR property rights management is based on the assumption of a <em>singular, </em>integrated<em> </em>Augmented Reality.</p><p>In the future, we can expect to see platform-as-a-service type companies popping up to provide back-end geospatialized augmentation storage infrastructure to AR application developers. <a href="http://www.escherreality.com/">Escher Reality</a> (<a href="https://techcrunch.com/2018/02/01/niantic-buy-escher-reality-ar-startup/">recently acquired and taken off the market by Niantic</a>) is an example of such a service. However, even in instances where two applications use the same back-end server stack, it is more likely to expect those augmentations to be isolated in some way, such that the augmentations never conflict with each other.</p><p>Until we see investment in the development of sharing controls, we should not expect to see integrated geospatial indices, alleviating many of the concerns about AR “graffiti” appearing in the wrong place at the wrong time.</p><h4>Sharing and Filtering Controls</h4><p>Even within a singular AR application, it is possible for conflicted use of the same space to arise. Within each application there are often no controls limiting access to created augmentations. Eventually, the creation of responsible sharing controls will be necessary to manage these rights and augmentations.</p><p>Unfortunately, if history has anything to teach us about this, before any standards will make interoperability between applications possible, we should expect to see a wide variety of incompatible sharing controls popping up. In the same way that sharing on Facebook is completely different from sharing within Google services, we can expect AR to follow suit. Furthermore, though standards-based protocols and controls are a necessary eventuality, standards take time to mature, often informed by the mistakes of a past which hasn’t yet transpired.</p><h3>A Picture Is Worth 1000 Words</h3><p>In the same way that the spoken word, the written word, the printed word, illustrations, musical performance, sculpture, photography, cinematography, videography, and video games have each been through their journey of testing and expanding our collective understanding of “speech”, so will virtual augmentations find their place. While these augmentations are surely different and novel in that they are contextualized in their surrounding environment, so are other forms of expression. Yet merely including augmentations as speech does not solve the issues we can expect to see in the future — AR has at least as much potential for contention and conflict as any other form of speech, especially online.</p><p>In the absence of AR sharing controls, some people feel strongly that property owners should have the right to remove augmentations from the spaces under their real world control. At first this may seem reasonable, but in practice this idea amounts to asserting that no one can make a particular kind of statement about a place under someone else’s control, which is unreasonably restrictive (and extraordinarily difficult in practice).</p><p>For an example of what these statements might look like, imagine a<a href="https://en.wikipedia.org/wiki/Machine-readable_data"> machine-readable</a> version of the following “plain” english sentence:</p><blockquote>“There is an otherwise invisible, intangible avocado represented by the following glTF model at the world coordinates of <a href="https://www.google.com/maps/place/The+White+House/@38.8976763,-77.0365298,15z/data=!4m5!3m4!1s0x0:0x715969d86d0b76bf!8m2!3d38.8976763!4d-77.0365298">38.8976763,-77.0365298</a>. <a href="https://raw.githubusercontent.com/KhronosGroup/glTF-Sample-Models/master/2.0/Avocado/glTF/Avocado.gltf">https://raw.githubusercontent.com/KhronosGroup/glTF-Sample-Models/master/2.0/Avocado/glTF/Avocado.gltf</a>”</blockquote><p>Instead, it will be necessary to take under consideration the venue and audience amongst whom the speech is shared. In the case of augmentations, this venue is the geospatial index where the augmentation is stored. (Until we share a common index for our augmentations, one could make the case that these augmentations are private, only for the enlisted users of a given application, and therefore landlords have no rights to limit this “private” speech.)</p><h4>Some Informed Legal Perspectives</h4><p>At this point, it may be obvious that the ideas put forth in this article are not born of legal expertise. However, these issues <a href="https://docs.justia.com/cases/federal/district-courts/wisconsin/wiedce/2:2017cv00569/76938/27">have already been tested in courts</a>, resulting in a supportive outcome for the creators of AR applications, classifying those augmentations as protected speech.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F_Ywl4672V_A%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D_Ywl4672V_A&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F_Ywl4672V_A%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/eafa6275747ab6607c02383b2096a379/href">https://medium.com/media/eafa6275747ab6607c02383b2096a379/href</a></iframe><p>As far as I know, the very notion of “AR as speech” is attributable to Brian Wassom, and he <a href="http://www.wassom.com/from-the-archives-augmented-reality-as-free-speech.html">explains this position in this post from 2013</a>. More recently, at AWE2018 a panel of attorneys (including Wassom) revisit the question of property rights in AR:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FDln8rEd5ygE%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DDln8rEd5ygE&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FDln8rEd5ygE%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/77d10fbc9e25657a41104a54e9f04ef9/href">https://medium.com/media/77d10fbc9e25657a41104a54e9f04ef9/href</a></iframe><p>Secondly, dozens of AR applications with persistent augmentations are already widely deployed without consideration for this issue. So long as it is possible for the producers of AR apps to ask for forgiveness instead of permission, persistent virtual objects are going to be created in great number around the world without a thought given to the imagined rights of property owners. Attempting to put that toothpaste back in the tube is an unenviable and ultimately unrealistic task.</p><h4>Some Informed Counterpoint</h4><p>Between AR creator rights and landlord rights, there are indeed qualified legal opinions which favor the landlord. In 2017, Declan T. Conroy published <a href="https://repository.law.umich.edu/cgi/viewcontent.cgi?article=1232&amp;context=mttlr">“Property Rights in Augmented Reality” in the Michigan Telecommunications and Technology Law Review</a>, concluding in part:</p><blockquote>“While this technology can undoubtedly enhance our environments for the better and inspire people to connect with the world in entirely new ways, the antipodal potential for such technology to digitally vandalize locations or instigate trespassory conduct poses a legitimate threat to owners of real property. However, due in large part to this technology’s intimate connection to the physical world, crafting a regime of property rights in site-specific AR would ultimately be a simple and straightforward exercise.”</blockquote><p>While recognizing that the perspective of this post is primarily technical (and not legal), crafting a regime of property rights in site-specific AR will be an extremely challenging, contentious process, as evidenced by the multitude of contemporary non-AR issues surrounding any other form of speech on the Internet.</p><h4>Getting Ahead of The Problem</h4><p>For the moment, AR developers are focused on a different set of more immediate problems, such as tracking, asset production, and discovering new spatial user interactions. As persistent AR continues to reach more people, we can expect an increase in both the conflict surrounding AR and the attention it garners.</p><p>Instead, AR developers must begin to work together to create and establish flexible sharing controls that reduce the friction of user on-boarding, allow for diverse use cases, and most importantly: enable a harmonious, responsible sharing of space.</p><p><a href="https://medium.com/@3d_jb/sharing-in-ar-today-and-tomorrow-da285881cd65">The next post</a> will introduce more specific examples of sharing controls, including some early proposals for sharing in AR.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=983bf195b400" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Global Relocalization: A Better GPS]]></title>
            <link>https://medium.com/@3d_jb/global-relocalization-a-better-gps-b6b98d2db20c?source=rss-7ada5d85b73a------2</link>
            <guid isPermaLink="false">https://medium.com/p/b6b98d2db20c</guid>
            <category><![CDATA[augmented-reality]]></category>
            <category><![CDATA[gps-tracking-system]]></category>
            <category><![CDATA[gps]]></category>
            <category><![CDATA[relocation]]></category>
            <category><![CDATA[ar]]></category>
            <dc:creator><![CDATA[Joe Boyle]]></dc:creator>
            <pubDate>Wed, 27 Jun 2018 11:01:01 GMT</pubDate>
            <atom:updated>2019-03-27T16:33:27.680Z</atom:updated>
            <content:encoded><![CDATA[<p>Since first appearing in a cellular phone twenty years ago, GPS has provided <a href="http://gpsworld.com/the-economic-benefits-of-gps/">billions of dollars in value annually</a> to mobile device users. With the maturing world of IoT coming online, the importance of GPS is increasingly evident. However, in the nascent world of Augmented Reality, while still foundational to AR Cloud infrastructure, GPS alone is not sufficient for the purposes of building AR. As discussed in <a href="https://medium.com/@3d_jb/first-steps-into-an-ar-cloud-relocalization-58d04ea93aa">my previous post</a> on relocalization, emergent positional technologies are some of the most exciting AR developments in 2018. By combining legacy GPS with some specific techniques, we will have a worthy successor to GPS: global relocalization. This post will discuss what will be required to reach this point, laying out the relationship between these new highly accurate SLAM techniques and the adjacent global positioning system found in all our phones and tablets.</p><p>First, it’s important to understand the two types of positioning to be considered: local and global. Local positioning is a device’s understanding of its position in relation to the immediate environment (such as the room it’s in), whereas global positioning is a device’s understanding of its location in relation to the entire planet. As mentioned above, GPS alone itself misses the mark. Here are the deal-breakers:</p><ol><li>It doesn’t work well indoors or in built-up urban environments</li><li>Even where it does work indoors, GPS based tech has a very poor understanding of device elevation and most 2D map apps fail to accurately represent multi-story indoor locations.</li><li>At the best of times, the range of <a href="https://en.wikipedia.org/wiki/Error_analysis_for_the_Global_Positioning_System">error in satellite-based GPS is complex and large</a>; registered in meters, not centimeters.</li><li><em>Orientation</em> is not provided, so applications must rely on unreliable electromagnetic compasses for heading.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/296/1*Cd7y1X68rPjM1VatlY8IDw.png" /></figure><p>However, GPS is still useful to us in several ways. First, by extending the decimal precision of the <a href="https://en.wikipedia.org/wiki/Geographic_coordinate_system">highly developed coordinate systems</a> (like <a href="https://en.wikipedia.org/wiki/ECEF">ECEF</a> or <a href="https://en.wikipedia.org/wiki/World_Geodetic_System">WGS84</a>), we can build upon and extend decades of <a href="https://en.wikipedia.org/wiki/Geographic_information_system">GIS</a> software development.</p><p>Let’s break the journey toward global relocalization into two key techniques: AR feature mapping and georeferencing.</p><h3>AR Feature Mapping</h3><p>All contemporary AR apps work by building a 3D model of the surrounding environment. ARKit and ARCore do this using only a single RGB camera, though other systems also include beacons, markers, or IR depth sensing. Irrespective of sensor type, a model is made from a few different types of 3D model data including point clouds and meshes. For the purposes of this post, these varied digital representations of a given environment are treated interchangeably and boiled down to “AR features” or “feature maps”. The sections below outline how these points will be detected and used by AR applications.</p><p><strong>Tracking Feature Points</strong></p><p>Using existing ARKit/ARCore applications involves a fickle initialization process: your device will ask you to point the camera at large horizontal surfaces like a table top or the floor. After the user has waved their device around for long enough, the initialization will complete and the app will start up.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FeJtgfw07jDI%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DeJtgfw07jDI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FeJtgfw07jDI%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/f83e7df2037b667890030a91736a98fd/href">https://medium.com/media/f83e7df2037b667890030a91736a98fd/href</a></iframe><p>The camera is searching for points of reference (called feature points) throughout an environment. High contrast surface transitions work well: a magazine on a coffee table, a knot of wood in the floorboards, where the couch meets the carpet, etc. With enough features, and by incorporating other sensor data, a device can begin to <a href="https://en.wikipedia.org/wiki/Simultaneous_localization_and_mapping">continuously infer its position and orientation with a high degree of accuracy.</a></p><p>In other words,<strong><em> </em></strong>at this stage of localization, <em>the app only knows where it is in relation to where it was when it first started up. </em>Since no two startup sequences are the same, object coordinates will be different at each start up of the app, which is why no persistence or multi-user is possible in the 2017 launch versions of ARKit or ARCore. The environment is unique to each user, device and every start up. Without agreement on a shared environment, these and similar applications will have limited capabilities with tracked features.</p><p><strong>Sharing feature maps = Multi-user AR</strong></p><p>The next step then, is to have devices come to agreement on a shared environment where feature points must be shared between them. Either device sends its feature points to the other, or they mutually share what they know and try to align the two maps into one coherent view of the world. This alignment process is challenging to say the least.</p><p>Additionally, instead of whimsically assigning the origin to wherever the device happened to be during startup, the devices must also come to consensus on a new shared origin and orientation for this new shared world.</p><p>Of the technologies listed in my<a href="https://medium.com/@3d_jb/first-steps-into-an-ar-cloud-relocalization-58d04ea93aa"> previous post</a>, this is what is provided by <a href="https://www.blog.google/products/google-vr/experience-augmented-reality-together-new-updates-arcore/">ARCore Cloud Anchors</a> and <a href="https://www.apple.com/newsroom/2018/06/apple-unveils-arkit-2/">ARKit </a>2.0’s <a href="https://developer.apple.com/documentation/arkit/arworldmap">ARWorldMap</a>.</p><p>How devices share the feature map data also varies by implementation, but the usual options apply: local wireless networking or a cloud service. In the case of Google Cloud Anchors, they provide a cloud storage component for sharing their feature maps, but Google deliberately <a href="https://codelabs.developers.google.com/codelabs/arcore-cloud-anchors/index.html">discards this data after just 24 hours</a>. There is some speculation that this was a strategic decision to avoid the highly topical <a href="https://en.wikipedia.org/wiki/General_Data_Protection_Regulation">General Data Protection Regulation</a> requirements, though this decision may have more to do with the completing the work involved in the steps outlined below.</p><p><strong>Saving feature maps = Persistent AR</strong></p><p>Once the industry can get two devices to agree on a local feature map, the next step will be to establish capabilities to save and reload feature maps, without the requirement to rescan a given space. This is the step with the most promise to make AR apps launch faster, but to also ensure it is possible to create persistent augmentations whereby virtual characters and objects can be reliably saved and reloaded into the same position.</p><p>Saving and loading these feature maps introduces new challenges: upon launching a persistent AR app, how will it know which feature map to use?</p><p>Also unclear, is how sensitive relocalization algorithms are to routine environmental changes such as closed doors, open windows, moving furniture, or even people. Relocalization algorithms would benefit tremendously from being able to ignore commonly “transient” objects using <a href="http://blog.qure.ai/notes/semantic-segmentation-deep-learning-review">semantic segmentation</a>, but in the meantime, we should expect performance to be poor in some environments. <a href="https://medium.com/u/3bff0521bc4a">Ryan Hickman</a>’s <a href="https://medium.com/@ryanhickman/augmented-reality-robotics-bb6db40ab754">badly behaved virtual cat</a> exhibits such challenges really well.</p><p>The most favorable solution to this problem of loading feature maps would be a system that just assesses and determines location all on its own, though it’s more likely we will see applications asking for some human intervention here; perhaps prompting users to select from a list of previously scanned locations.</p><p><strong>Geo-indexing feature maps</strong></p><p>A cloud service that stores feature maps and makes them available for users to reload at a later time was noted earlier, and although current GPS is not accurate enough to use directly for AR, it will be essential as a means to <a href="https://en.wikipedia.org/wiki/Spatial_database">geo-spatially index feature maps in databases</a> for ease of access and management. In other words, when creating and saving feature map data into a cloud service, the device’s current GPS should be saved with location data, allowing AR applications to load only those feature maps created nearby, and not those created on the other side of town .</p><h3>Georeferencing Feature Maps</h3><p><strong>Turning local coordinates into global coordinates</strong></p><p>We can add a great deal of accuracy to our devices by telling them exactly where each of these feature maps are in relation to the world, not by using legacy GPS, but by using a process called <a href="https://en.wikipedia.org/wiki/Georeferencing"><em>georeferencing</em></a><em>. </em>Georeferencing is<em> </em>the practice of aligning local and global maps so coordinates can be easily translated from local to global or vice-versa.</p><p>In this <a href="https://github.com/wrld3d/wrld-indoor-maps-api/blob/master/TUTORIAL.md#georeference-floor-plan">georeferencing tutorial </a>provided by <a href="https://www.wrld3d.com/">WRLD3D</a> (a 3d map provider), we can see how to use <a href="https://qgis.org/en/site/">QGIS</a>, an open source tool that supports georeferencing assets onto known coordinates.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/624/0*FTRq1x8Ta0OVEHdV.png" /><figcaption>WRLD3D headquarters in Scotland</figcaption></figure><p>In their example, the building on the left is recognizable in both the architectural drawings and satellite imagery. By aligning points on both images, <strong><em>a global positional offset and rotation can be determined</em></strong>. This offset can then be used to determine the global coordinates for features only visible on the “local”, architectural drawing.</p><p>If we georeference a machine-readable AR feature map instead of an architectural drawing, one can see how it is now possible to translate the local 3D scene coordinates to global 3D coordinates. (This positional offset can also replace the less accurate GPS based data in the geospatial index.)</p><p>The accuracy of this approach is bound only by the accuracy of the local positioning system and the georeferencing process itself. Once this is complete, it is possible for each device to know its global position with centimeter accuracy.</p><h3>Scaling Up</h3><p>Above we outline some steps that require human intervention (such as selecting stored feature maps and later georeferencing them), though it’s possible we will see automated solutions to both problems in the future.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FIMSozUpFFkU%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DIMSozUpFFkU&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FIMSozUpFFkU%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/91b8ff703518c2993bc32e983a3d29c3/href">https://medium.com/media/91b8ff703518c2993bc32e983a3d29c3/href</a></iframe><p>Once the friction to collect and use these feature maps is reduced or eliminated, <em>georeferenced feature maps can be used to georeference other feature maps</em>, in a continuous generative process that “fills in the blanks” on unmapped areas of the world. It isn’t clear how long this could take, but given the GPS and camera capabilities already in every smartphone, crowd-sourced 3D mapping is a significant opportunity.</p><p>The <a href="https://arcloudfoundation.com/">AR Cloud Foundation</a> has recognized this opportunity, and has started outlining a block-chain-based incentive scheme to reward users for scanning and sharing 3D map data.</p><p>Naturally, as is the case with any form of crowd-sourced data, the accuracy of the resulting maps should be suspect and subject to constant revalidation.</p><h3>Looking forward</h3><p>The <a href="https://www.pcworld.com/article/2000276/a-brief-history-of-gps.html">first GPS enabled smartphone</a> appeared in 1999. Today we can hardly imagine a phone without GPS, or the rich wealth of services made possible by GPS. It’s easy to take these services for granted, but in retrospect, it has taken almost twenty years to get where we are today. Looking forward, we can be sure of a few things:</p><ol><li>An improved accurate 3D version of the global map will surely be an ongoing process many years in the making.</li><li>It will never be “finished” in exactly the same way today’s 2D map is never finished — a current map will require continuous scanning and rescanning.</li><li>Globally localized mobile AR based use cases are possible right now in those spaces where the benefit provided can justify the cost of manual feature map selection and manual georeferencing.</li><li>As this kind of map comes online, 3D map based indoor applications will explode in popularity and number.</li></ol><p>Just like mobile applications, not every AR application will benefit from or require global relocalization — many amazing AR experiences will be had without bringing in the outside world. However, those experiences are distant from the inherent opportunity in AR: to make the computers in our pockets observing participants that understand where they are even more accurately than we do.</p><p>In the coming posts, issues surrounding AR privacy and security (specifically access control and rights management) will be introduced and analyzed.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b6b98d2db20c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[First steps into an AR Cloud: Relocalization]]></title>
            <link>https://medium.com/@3d_jb/first-steps-into-an-ar-cloud-relocalization-58d04ea93aa?source=rss-7ada5d85b73a------2</link>
            <guid isPermaLink="false">https://medium.com/p/58d04ea93aa</guid>
            <category><![CDATA[xr]]></category>
            <category><![CDATA[augmented-reality]]></category>
            <category><![CDATA[spatial-computing]]></category>
            <category><![CDATA[ar]]></category>
            <category><![CDATA[emerging-technology]]></category>
            <dc:creator><![CDATA[Joe Boyle]]></dc:creator>
            <pubDate>Fri, 08 Jun 2018 04:03:43 GMT</pubDate>
            <atom:updated>2018-06-08T04:03:43.814Z</atom:updated>
            <content:encoded><![CDATA[<p>Back in June of 2017, on the tails of the <a href="https://developer.apple.com/videos/play/wwdc2017/602/">WWDC 2017 ARKit announcement</a>, when the world’s imagination began to run wild with the sudden availability of smartphone AR features on tens of millions of iOS devices, Ori Inbar of <a href="http://www.superventures.com/">Super Ventures</a> wrote a <a href="https://medium.com/super-ventures-blog/arkit-and-arcore-will-not-usher-massive-adoption-of-mobile-ar-da3d87f7e5ad">seminal post about the “AR Cloud”</a>. In his post, Ori explains what is needed to transition from the novelty AR apps we see today toward the AR future of which we’ve been dreaming — a revolutionary new ubiquitous computing interface providing natural interaction with virtual objects and interfaces. <a href="http://twitter.com/keiichiban">@keiichiban</a>’s Augmented City paints a vivid picture:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F3TL80ScTLlM%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D3TL80ScTLlM&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F3TL80ScTLlM%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/6433de72f4a4a6dac4442e6c973483da/href">https://medium.com/media/6433de72f4a4a6dac4442e6c973483da/href</a></iframe><p>If you missed Ori’s article, <a href="https://medium.com/super-ventures-blog/arkit-and-arcore-will-not-usher-massive-adoption-of-mobile-ar-da3d87f7e5ad">check it out</a>, or watch his supplementary video here:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FTAlJ5t73Rr8%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DTAlJ5t73Rr8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FTAlJ5t73Rr8%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/4703fac59877b0cd9a2f026fdf81d165/href">https://medium.com/media/4703fac59877b0cd9a2f026fdf81d165/href</a></iframe><p>Since then, <a href="https://medium.com/search?q=%22ar%20cloud%22">many more “AR Cloud” posts have appeared on Medium</a>, largely supporting Ori’s position, with some finer points made about the relationship between AR Cloud and ephemeral shared state solutions like <a href="https://blog.google/products/google-vr/experience-augmented-reality-together-new-updates-arcore/">Google Anchors</a>.</p><p>As of today, just days after the <a href="https://augmentedworldexpo.com/">Augmented World Expo 2018</a>, the most topical and hotly contested AR Cloud technology is relocalization. (Relocalization enables your device to accurately learn its position and orientation in real world space, which in turn enables multi-user and persistent AR experiences.) Until fairly recently, AR positioning has required markers, such as those used by Vuforia:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FJwWVS5rv0d0%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DJwWVS5rv0d0&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FJwWVS5rv0d0%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/0220db16445877fabff57fe9010e04a8/href">https://medium.com/media/0220db16445877fabff57fe9010e04a8/href</a></iframe><p>Markers have been a staple in AR tracking for years, but markerless versions of AR tracking tech are now rapidly emerging. By my count there are now over half a dozen markerless relocalization plays in motion. Here they are, in no particular order:</p><h3>Google’s Visual Positioning Service (VPS)</h3><p>On the tails of Apple’s launch of ARKit, Google responded by launching it’s own functional equivalent: <a href="https://developers.google.com/ar/discover/">ARCore</a>. While it may seem that Google is playing catchup here, ARCore is really a stripped down repackaging of 2014’s <a href="https://en.wikipedia.org/wiki/Tango_(platform)">Google Tango</a> project. Through the use of “area definition files”, Tango offered relocalization three years ago. Unfortunately, Tango also required an infrared depth camera, which severely limited the number of supported devices.</p><p>Once Apple launched ARKit, Google made the decision to drop the depth camera requirement, and pursue RGB-only based <a href="https://en.wikipedia.org/wiki/Simultaneous_localization_and_mapping">SLAM</a> techniques. In doing so, area definition files were dropped from the newly repackaged ARCore, and Android-based relocalization has been off the table ever since. Almost a year later, Google’s “Visual Positioning System” is resurfacing, which promises relocalization <em>without</em> an infrared depth camera.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FL6-KF0HPbS8%3Ffeature%3Doembed&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DL6-KF0HPbS8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FL6-KF0HPbS8%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/73d0fea0e22c52a3f604950b0adaf239/href">https://medium.com/media/73d0fea0e22c52a3f604950b0adaf239/href</a></iframe><p>As of the writing of this post, Google VPS is not yet available to the public.</p><h3>Vertical’s Placenote SDK</h3><p><a href="https://placenote.com/">S</a>hortly after the launch of ARKit, <a href="http://vertical.ai/">Vertical</a> published <a href="https://itunes.apple.com/us/app/placenote-ar/id1294715515">Placenote AR</a>, a free app demonstrating relocalization as a core feature. I spoke to the founders last year, and learned that their relocalization offering was created out of necessity, in response to Google’s announcement that they were sunsetting Tango.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FbWRXwk58_qI%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DbWRXwk58_qI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FbWRXwk58_qI%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/11a290db2ed9f27e068a530ea262b530/href">https://medium.com/media/11a290db2ed9f27e068a530ea262b530/href</a></iframe><p>On it’s own, this free app is not an all in-one AR Cloud solution, but instead offers people the opportunity to experiment with relocalization today. For app developers, Vertical has launched their <a href="https://placenote.com/">Placenote SDK</a> so that more specific use cases can be addressed. Unlike many of the other SDKs outlined in this post, the Placenote SDK is openly available now.</p><h3>6D.AI</h3><p>To get a sense of the excitement and opportunity surrounding relocalization, look no further than <a href="http://www.6d.ai/">6D.AI</a>. Less than a year ago, <a href="https://medium.com/@mattmiesnieks">Matt Miesnieks wrote many of the most widely shared and insightful articles about the coming challenges in AR</a>. Since then he has transitioned his full attention to founding and building 6D.AI, which promises fast, cross-platform relocalization:</p><h3>Matt Miesnieks on Twitter</h3><p>Another @6d_ai update. Our Persistent Anchors in action. The UX &quot;Just Works&quot;. 0:03 - 0:04 anchor auto-loaded 0:05 - 0:07 relocalized 0:10 - manually place some content 0:22 - player 2 starts (from 180 deg and long distance away) 0:28 - anchor auto-loaded for P2 (0 click) cont... https://t.co/HolgVuAtEP</p><p>Additionally, 6D has recently released video demonstrating real-time mesh construction, which has interesting use cases for AR applications that benefit from topological information about the immediate environment.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F3Nv-jHP0j08%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D3Nv-jHP0j08&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F3Nv-jHP0j08%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/692d4f7726a7ad2ba4d55a7ba8182479/href">https://medium.com/media/692d4f7726a7ad2ba4d55a7ba8182479/href</a></iframe><p>As of the writing of this article, the 6D.AI SDK is in beta via invitation only. Applications <a href="https://www.6d.ai/beta/">can be submitted here</a>.</p><h3>Fantasmo’s Camera Positioning Standard (CPS)</h3><p>Among the contenders in the relocalization space, <a href="https://fantasmo.io/">Fantasmo</a> stand outs in a few ways:</p><ol><li>At first glance, the size of the tracked space appears to be larger than some of the other demo videos.</li><li>They call out autonomous robot navigation as a use case for a “machine readable” 3D map.</li><li>Their pending <a href="https://blog.fantasmo.io/proposing-an-open-camera-positioning-standard-64113b6012e">Camera Positioning Standard (CPS)</a> proposal promises future interoperability (though the standard itself appears to be unpublished at the time of the writing of this article). Details here are scant, but these are the first traces of shared open infrastructure in the service of creating AR common spaces. More on this later.</li></ol><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FpYoAVyF19FI%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DpYoAVyF19FI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FpYoAVyF19FI%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/0b94aa8f1d91640914d41735c2df6ca2/href">https://medium.com/media/0b94aa8f1d91640914d41735c2df6ca2/href</a></iframe><p>Fantasmo’s approach is an ambitious one. To learn more, visit <a href="https://www.camerapositioning.io/">camerapositioning.io</a>.</p><h3>Selerio</h3><p><a href="http://www.selerio.io/"><em>Selerio</em></a><em> </em>is unique amongst the SDKs listed here in that the core proposition appears to be recognizing objects in scene, thus enabling virtual interactions with real world objects. Neverthless, relocalization appears to be a part of their proposition:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FrzM6gs9nA0E%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DrzM6gs9nA0E&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FrzM6gs9nA0E%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/36cd6c34c3edc5d5f9d5622b809e157a/href">https://medium.com/media/36cd6c34c3edc5d5f9d5622b809e157a/href</a></iframe><p>Selerio is also <a href="http://www.selerio.io/#application">accepting applications to access their SDK</a>.</p><h3>Scape</h3><p>In the outdoor category, <a href="http://scape.io/">Scape</a> is offering relocalization in city-sized outdoor spaces. This initially struck me as a larger, more difficult version of the indoor positioning problem (given that the outdoor environment is subject to seasonal changes). However, in conversation with <a href="https://medium.com/u/9640f942b365">Edward Miller</a> at AWE2018, he explained that though the tech would work just as well indoors, his team’s background and expertise was strategically positioned to have a greater impact on the city-scale version of this problem. For those of us with concerns about receiving our AR services from ad-motivated data-stockpiling tech giants, this is a promising alternative to Google VPS.</p><p>Unlike the indoor propositions, Scape’s relocalization is <em>global</em>, meaning that once your device is localized, you know your precise position in world coordinates, not just local, relative coordinates. (More on this in an upcoming post.)</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2F26FfjfcRElOoQOfFC%2Ftwitter%2Fiframe&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2Fscape-technologies-point-cloud-fly-though-26FfjfcRElOoQOfFC&amp;image=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2F26FfjfcRElOoQOfFC%2Fgiphy-downsized-large.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/b63a6b7b45d54e22361dff012c965a3a/href">https://medium.com/media/b63a6b7b45d54e22361dff012c965a3a/href</a></iframe><p>Scape is currently accepting <a href="https://scapetechnologies.typeform.com/to/rqWVF1">applications for their private beta</a> which offers relocalization in London, England.</p><h3>Sturfee</h3><p>Like Scape, <a href="https://sturfee.com/">Sturfee</a> is offering city-scale outdoor positioning. They appear to be targeting outdoor location based AR game developers, but I have no doubt that a variety of gaming and non-gaming applications are possible.</p><p>I can’t be sure, but I believe Sturfee may also provide global coordinates.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fy5zXoK4K6g4%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dy5zXoK4K6g4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fy5zXoK4K6g4%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/b88488dd8b2ef63e3415786225b0e69c/href">https://medium.com/media/b88488dd8b2ef63e3415786225b0e69c/href</a></iframe><p>Sturfee is <a href="https://sturfee.com/sdk.html">accepting signups for its SDK now</a>.</p><h3>Vuforia’s Object Tracking</h3><p>Previously, I mentioned Vuforia as an example of a mature marker based AR tracking system. Vuforia continues to develop their marker-based tracking, with support for a variety of different marker types beyond 2D images. Most notably, 3D objects are identifiable as scene markers, meaning relocalization may be achievable by referencing unique 3D objects in your location, such as distinctive architecture or furniture elements.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F6W7_ZssUTDQ%3Ffeature%3Doembed&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D6W7_ZssUTDQ&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F6W7_ZssUTDQ%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/eea35df4101eabddace11d849e614505/href">https://medium.com/media/eea35df4101eabddace11d849e614505/href</a></iframe><p>In previous releases, Vuforia has showcased “Extended Tracking” which like ARKit and ARCore, uses sensor data to track device pose even when markers have left the camera field of view. (It is unclear to me if this work is compatible with ARkit or ARCore.)</p><h3>Jidomaps</h3><p>The persistence contender I confess to knowing the least about is <a href="http://www.jidomaps.com/">Jidomaps</a>. Jidomaps differentiates themselves from other relocalization offerings by clarifying that they are not an AR Cloud service. Instead, they describe themselves as a simpler way to achieve multi-user AR experiences, though I imagine their state could additionally be saved to a cloud based service if they chose to pursue that capability. Having not yet popped the hood, I believe Jidomaps shares state for multi-user AR via local networking, but please do correct me if you know better.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FF1OpctWRsog%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DF1OpctWRsog&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FF1OpctWRsog%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/5a096fa34ba3c9f453eec600e9158628/href">https://medium.com/media/5a096fa34ba3c9f453eec600e9158628/href</a></iframe><h3>Apple’s Arkit 2.0</h3><p>Apple’s annual WWDC was earlier this week, and the AR community is still processing the <a href="https://www.apple.com/newsroom/2018/06/apple-unveils-arkit-2/">announcement of ARKit 2.0</a>. Much like Jidomaps, Apple is <a href="http://not yet tackling the cloud aspects of sharing state between">enabling developers to share state between two AR-enabled devices</a>, but has not yet created a cloud infrastructure to store and retrieve these <a href="https://developer.apple.com/documentation/arkit/arworldmap">3D feature maps</a> on behalf of the developers. Instead it is up to the developer to serialize, save, transmit, or restore the AR feature maps themselves. Nevertheless, this reaffirms Apple’s position as an important leader in the AR space.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FgZhQCVSvq5E%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DgZhQCVSvq5E&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FgZhQCVSvq5E%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/41850774e5b45923d0658c25cc3dd65a/href">https://medium.com/media/41850774e5b45923d0658c25cc3dd65a/href</a></iframe><p>Now, just a few days later, the first multi-user ARKit 2.0 test applications are appearing:</p><h3>Jesse Ringrose on Twitter</h3><p>an early test of a co-op augmented reality mode. pretty neat so far #ARKit2 #madewithunity #spek https://t.co/GaofMxqjRp</p><h3>In Summary</h3><p>It is important to call out two caveats to everything that has been said thus far:</p><p>First, no attempt has been made to test the veracity of any of the claims made by these companies; nor has any testing been done to measure the relative performance of the techniques employed under the myriad of possible environments and circumstances. There is a huge amount of work to be done by the community at large to sift through and sort out how best to use these technologies. In other, more mature industries, I have seen head to head “shoot-out” tests in which third party labs test for accuracy and speed — this style of testing is a much needed next step for those of us collectively trying to build in Augmented Reality.</p><p>Second, and yet more important to my own thesis, relocalization is just the most immediate of many obstacles between the novelty pop-up-book AR and <a href="https://vimeo.com/166807261">the AR future of our dreams (or nightmares)</a>. My hope is that the discussion around “AR Cloud” will come to include some of the other necessary services toward internet-scale AR. Challenges surrounding privacy, security, application fragmentation, open standards development, content management, identity management, community management — each in their own right difficult in a world without augmented reality, yet exacerbated in an augmented world.</p><p>Hopefully this post will help set the stage for that discussion. Stay tuned for follow up posts and please feel free to reach out with comments or questions.</p><p><a href="http://twitter.com/3d_jb">Joe Boyle (@3d_jb) | Twitter</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=58d04ea93aa" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>