<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Scott Satkin on Medium]]></title>
        <description><![CDATA[Stories by Scott Satkin on Medium]]></description>
        <link>https://medium.com/@scott_limbix?source=rss-a3f999012c6e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*yZammXUakJL-eg3o.</url>
            <title>Stories by Scott Satkin on Medium</title>
            <link>https://medium.com/@scott_limbix?source=rss-a3f999012c6e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 17:05:54 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@scott_limbix/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Driving Across the Uncanny Valley]]></title>
            <link>https://medium.com/@scott_limbix/driving-across-the-uncanny-valley-5ab1a8ba651b?source=rss-a3f999012c6e------2</link>
            <guid isPermaLink="false">https://medium.com/p/5ab1a8ba651b</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[therapy]]></category>
            <category><![CDATA[virtual-reality]]></category>
            <dc:creator><![CDATA[Scott Satkin]]></dc:creator>
            <pubDate>Fri, 14 Apr 2017 20:20:36 GMT</pubDate>
            <atom:updated>2017-04-15T17:14:03.926Z</atom:updated>
            <content:encoded><![CDATA[<h4>How we engineered Limbix VR Therapy for car accident PTSD</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4fNdvNoOmMsOUCWN79Pn-A.png" /><figcaption>Screenshot from <a href="https://blog.limbix.com/treat-car-accident-ptsd-with-vr-therapy-264f734d51b4">Limbix VR Therapy for Driving Phobia</a>.</figcaption></figure><p>When we set out to create Virtual Reality exposures for driving phobia, we wanted to ensure our simulations would elicit authentic emotional responses from patients. From an engineering perspective, we had two very different options:</p><ol><li>Follow the typical “video game” approach and create computer animations of driving scenes</li><li>Design a cutting-edge VR camera and renderer to immerse patients in real-world environments</li></ol><h3>Animating is slow. Driving is fast.</h3><p>Animation for VR is a time-consuming art form, with major constraints. Every object in a scene must be individually crafted and animated — down to each hair on a person or each leaf on a tree. Not only is it a tremendous amount of work to create the details of these 3D scenes, it also pushes the limits of what computers and phones are capable of displaying. Every object, animation, and lighting effect must be rendered twice, once for each eye. As the number of objects in a scene grows, it can quickly exceed a device’s computational capabilities, causing the VR scene to grind to a halt.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/430/1*NMuXUYeky9AbccBbST905Q.jpeg" /><figcaption>The Uncanny Valley, a phenomenon causing more realistic imagery to appear less familiar.</figcaption></figure><p>To deal with these computational limits, artists typically simplify models as much as possible, reducing the total number of polygons in a scene to make it render more efficiently. However, these simplified models can have a cartoon-like appearance. Surprisingly, efforts to make 3D models appear more photorealistic often have a negative effect. People are generally forgiving of imperfections on avatars and animated characters; however, as the fidelity of animations approach photographic quality, small anomalies stand out and become increasingly objectionable. This phenomena of more realistic content being perceived negatively is known as the “<a href="https://en.wikipedia.org/wiki/Uncanny_valley">Uncanny Valley</a>.”</p><p>On the other side of the uncanny valley is photorealism — what you experience in a headset running Limbix VR. Our team leveraged the latest advances in computer vision and graphics research to produce an immersive image-based driving experience.</p><p>We could have spent months hand-crafting every mile of road and then rendered 3D models of the entire landscape. However, that approach would not have been visually appealing or displayed smoothly in VR. Instead, we built a world-class VR camera rig to capture 360 degree driving scenes at 60 miles per hour, at a smooth 60fps (frames per second), in vivid 6k resolution.</p><p>Here’s how we did it.</p><h3>360 degrees, 0 seams.</h3><p>Filming VR content poses additional challenges beyond traditional video production. If the resolution is too low, the world appears blurry. If the frame rate is too low, motions appear choppy and can induce nausea. That is why we build a custom camera to capture the world at 6k resolution and 60fps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yIHQZw0hyYYygiUAARnAEg.png" /><figcaption>6k video has more than twice as many pixels as 4k video and 19x as many pixels as most HDTVs.</figcaption></figure><p>6k resolution exceeds the cinematic fidelity of typical Hollywood movie productions, and is the highest resolution video you can display on any VR headset, desktop or mobile. It would require 19 HDTVs arranged in a grid to display every pixel we capture. To handle the massive number of pixels we are rendering, our team engineered a custom video format which more efficiently stores and decodes the pixels using a compact <a href="https://en.wikipedia.org/wiki/Texture_atlas">texture atlas</a> and omits pixels which are not visible through our virtual car’s windows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8Y77cqhtM1pYtAfXet4QwQ.png" /><figcaption>Our custom-built 6k VR rig has five camera lenses for capturing real-world driving scenarios.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9eSIIRHCgJPe9QfZcqIhGw.jpeg" /><figcaption>Example images from our five cameras.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4znxo1o1PdsMN5Vd4gHwXQ.jpeg" /><figcaption>One seamless panorama computed from the five views.</figcaption></figure><p>To capture all these pixels, we used five cameras, pointing in different directions, and stitch the images together to create one panorama. Panoramas often exhibit “stitching artifacts” or anomalies, such as the one shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/675/1*B8iLkXeDEOhxgLIX4hIBXA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/675/1*XCKO7uwjfIkVtGZWxUTBtA.jpeg" /><figcaption>Comparison of stitching artifacts from misaligned cameras (left) and seamless stitching (right).</figcaption></figure><p>Our engineers went through tremendous effort to eliminate these artifacts, which focused on two sources of <a href="https://en.wikipedia.org/wiki/Parallax">parallax</a>:</p><ol><li><strong>Relative camera positions<br></strong>Objects appear different from different viewpoints. That is why we have two eyes — giving us a sense of depth. The further apart our eyes (or cameras) are from each other, the greater the difference between each view. This difference in views causes a seam at image boundaries, common in panoramas.</li><li><strong>Camera motion<br></strong>In 1/60th of a second a lot can happen. Although our video cameras capture the world at an impressive 60fps (compared to your TV which plays at 24fps), at highway speeds, oncoming traffic moves multiple feet in that instance. Due to the tremendous motion in the scene, even the slightest time shift between camera shutters will result in major artifacts, causing objects to stretch, shrink, sheer, and jump as they cross image boundaries.</li></ol><p>To alleviate the first issue, we built a custom rig to mount our five cameras as physically close to each other as possible. To resolve the second issue, we used special camera firmware and built a custom cable to synchronize the shutters of each camera, ensuring the pixels of each frame are captured at the identical time. Special care was also taken to balance individual camera exposures and harmonize colors to produce a beautiful 360° panorama without any seams.</p><h3>“Presence”</h3><p>In VR, you don’t feel like you are watching a movie. The movie surrounds you in every direction you look, and immerses you in the environment. When you put on a headset running Limbix VR you gain a sense of “<a href="https://en.wikipedia.org/wiki/Immersion_(virtual_reality)#Presence">presence</a>,” the layers of technological complexity fade away, and you are left in the moment… in another world. It feels real.</p><p>With Limbix, therapists can offer effective exposures to their patients, and provide them with the sense of presence necessary to realistically face their fears. Our photorealistic environments enable therapists to perform the equivalent of <a href="https://en.wikipedia.org/wiki/Exposure_therapy#Techniques">graduated in vivo exposure therapy</a> from the comfort and convenience of their office.</p><p>To achieve a sense of presence in our driving simulation, and ensure we don’t get caught in the uncanny valley, we leveraged state-of-the-art computer vision and graphics techniques. Our VR car began as a detailed 3D model with over 3 million vertices (far more complex than can be rendered in real-time). We then simulated how light travels around the vehicle, bouncing off surfaces, passing through windows, and casting shadows — a computationally expensive process that takes days to run. After simulating billions of light paths, we created compact image-based representations of the vehicle appearance, geometry, and illumination.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RDzgopR40A1VLE5jOVoKmw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z_gqkCOoE3bLQ39Zc0ZxDQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DVK4jo1quQeM9uZGjrAX0g.png" /><figcaption>Representations of ambient illumination (top), surface normals (left), and depths (right) used to immerse patients in a photorealistic car.</figcaption></figure><p>This data is used to render 3D views of the vehicle for each eye in real-time, at 60fps. We project light from our 6k panoramic video through the vehicle’s windows and off its rear-view mirrors to the viewer. The vehicle is dynamically illuminated with a <a href="https://graphics.stanford.edu/papers/envmap/envmap.pdf">spherical harmonics</a> lighting basis computed from the video, to provide a natural appearance as you drive though diverse conditions (e.g., dimming surfaces when entering a tunnel, and brightening different regions based on the direction and color of each light source). Although lighting effects are subtle, they make a big difference.</p><p>Proper illumination transforms our vehicle from looking like a plastic toy to feeling real. It’s the little things that make a big difference in VR, and create a sense of presence for our users, enabling effective exposures.</p><p>Our engineering team is incredibly proud of the technical accomplishments showcased in our driving experience — but they are even more proud to watch the layers of complex technology melt away as patients immerse themselves in our driving simulation, gain a sense of presence, and face their fears.</p><p><a href="https://www.limbix.com/#early_access"><strong>Sign Up</strong></a> to hear from Limbix when we publicly launch later this year.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5ab1a8ba651b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>