<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Aahana on Medium]]></title>
        <description><![CDATA[Stories by Aahana on Medium]]></description>
        <link>https://medium.com/@asp032909?source=rss-ecaf3e1a112c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*wZ2MyIoLwXWPnpUh</url>
            <title>Stories by Aahana on Medium</title>
            <link>https://medium.com/@asp032909?source=rss-ecaf3e1a112c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 14 Nov 2026 15:51:59 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@asp032909/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Augmented Wishes: Saving Trees One Birthday Card at a Time]]></title>
            <link>https://medium.com/antaeus-ar/augmented-wishes-saving-trees-one-birthday-card-at-a-time-fad8648a3216?source=rss-ecaf3e1a112c------2</link>
            <guid isPermaLink="false">https://medium.com/p/fad8648a3216</guid>
            <category><![CDATA[augmented-reality]]></category>
            <dc:creator><![CDATA[Aahana]]></dc:creator>
            <pubDate>Sat, 30 Nov 2024 00:43:52 GMT</pubDate>
            <atom:updated>2024-11-30T00:43:52.024Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*K0GjzwILXFNuSJ2P" /></figure><p>More than <strong>3.4 billion gift cards</strong> were sold in the United States in 2021.</p><p>Almost all these cards are thrown away after the first few days of receiving them, resulting in increased waste in landfills.</p><p>Recently, I have been looking into this issue, and I’ve created a solution that can help us reduce paper waste from cards. I’ve used recreated a similar project by Yuma from ‘Anyone Can Code,’ using <strong>Apple’s ARKit</strong> and <strong>Xcode. </strong>My project is an <strong>augmented birthday card</strong> that can be reprogrammed and redesigned based on the person you are gifting it to.</p><h4>Project Overview:</h4><p>This project works by installing an app onto my phone. When the code is connected, it opens that app, and goes to a camera-like screen which is constantly searching for the tracking image (in my case, a birthday cake icon from Google). Once this image is detected, an augmented birthday card animation pops up, and essentially “sticks” onto the image in real life. The animation can then play either a video, or display an image.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FaGlWDjUrcvA%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fshorts%2FaGlWDjUrcvA&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FaGlWDjUrcvA%2Fhq2.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/392d16996ece0379e144da7438093390/href">https://medium.com/media/392d16996ece0379e144da7438093390/href</a></iframe><h4>Here’s a detailed breakdown of how I created it:</h4><h4>Step 1: Set Up</h4><p>The first step is to install ARKit and Xcode using these two links:</p><ul><li><a href="https://developer.apple.com/augmented-reality/">https://developer.apple.com/augmented-reality/</a></li><li><a href="https://developer.apple.com/xcode/">https://developer.apple.com/xcode/</a></li></ul><p>Once these finish downloading, you can start running Xcode.</p><p>The first thing that you will see are three options; Create New Project, Clone Git Repository, or Open Existing Project. Since we are creating a new project, we can click on the first option.</p><p>This leads us to a page that looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YmhHPG4PorrH1rcfMhyaHw.png" /><figcaption>ARKit project templates</figcaption></figure><p>In this case, since our app is going to be an augmented birthday card, we can chose <strong>Augmented Reality app</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5U4olTCTZ-rGydR51IiAaA.png" /><figcaption>ARKit project options</figcaption></figure><p>Once this page pops up, you can set your project name, team, and organization to anything you would like, however, ensure all the other settings are the same as these:</p><ul><li><strong>Interface:</strong> Storyboard</li><li><strong>Language:</strong> Swift</li><li><strong>Testing System:</strong> Swift Testing with XCTest UI Tests</li><li><strong>Content Technology:</strong> SceneKit</li></ul><p>After this, is will ask you to save your project in your files. For my project, I saved it in my TKS projects folder, since it is accessible and easy to find.</p><p>Once you have created your project and saved it into your files, we can start building the project.</p><h4>Step 2: Visual Birthday Message</h4><p>The next step is creating the actual project, and making the visual of how you want the birthday card to look.</p><p>On the left side of your window, would will find a few starting files. Click on the dropdown next to <strong>art.scnassets</strong>, and go to the file called ship. You will find a couple pre-installed images, however, we will not be needing them so we can delete those, but keep the actual file itself.</p><p>Once you have deleted everything, you should see an empty plane where we can create our animation.</p><p>Near the top right corner of the screen, you will find a plus sign, which when clicked, will open a dropdown menu. Drag the <strong>empty node</strong> into the <strong>Scene Graph</strong>, and rename it “container.” On the right, you will find a toolbar with options to modify our container. Click on the icon that looks like a cube, and change the <strong>Transforms</strong> options to look something like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/436/1*SxOXHztFI0eMo48h5-Fnow.png" /><figcaption>Container measurements</figcaption></figure><p>This container will serve as our main building space, and will allow us to move around our augmented animation, and make it lay over our tracking image. Within this container, we can drag in two <strong>3D text objects</strong>. These texts will be our birthday message. Be sure to name both these objects something similar to “text,” so you can easily write it in the code.</p><p>For mine, I made the first one say “Happy Birthday,” and the second one say “Aahana!”</p><p>Once everything is put together, the measurements and positions of the texts should be:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/446/1*p5Yot1E7Ivt79SP1Dow9Dw.png" /><figcaption>Happy Birthday text measurements and positioning.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/442/1*pyj095Up-Oq6QrNJ_ZojIw.png" /><figcaption>Name text measurements and positioning.</figcaption></figure><p>Once we have the text set up, we can work on the video. For the video to display, we need to first upload it as an <strong>mp4</strong>. Within the project file, I uploaded a test video to use as a demonstration.</p><p>Now that we have the video file uploaded, we can create a box for the video to actually be shown in. To do this, we need to go back to our Scene Graph, and insert another object called<strong> plane</strong>. This allows us to display our video in a rectangular frame once the tracking image is detected. Make sure to also go into materials component of this display, and change the shading properties to <strong>physically based</strong>. This essentially allows us to display a clear image of our video. The measurements and position should look something like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/436/1*mRbhMbKpbyWf408OchcpcA.png" /><figcaption>Measurements and positioning of the video.</figcaption></figure><p>An optional step of this process would be to add <strong>directional</strong> and <strong>ambient</strong> <strong>lighting</strong> to the texts, however, this does not change the functionality of the app, so it is not required.</p><p>After adding in all these components, we should have a simple visual that looks something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/1*8Z-Vzh0-ClsLPjUknpS1EA.png" /><figcaption>Final birthday message visual.</figcaption></figure><h4>Step 3: AR Resource Group for Image Tracking</h4><p>In order to create the tracking image, we first need to upload a <strong>png</strong> of the image we are going to track in real life. Once we have uploaded the image into the project files, we need to print that exact same image (in my case, a birthday cake icon).</p><p>Now we can work on building the AR Resource Group. This will allow us to identify the image that we will be tracking.</p><p>After creating the project, you should have a file called Assets.xcassets. Inside this file, click on the empty space under Applcon. This should open a dropdown with a few options. Click on <strong>AR</strong> and <strong>Textures</strong>, then <strong>New AR Resource Group</strong>. You can name this new folder AR Resources. You can now drag your tracking image, into the thin rectangle next to the AR Resources file. If a yellow triangle shows up on the bottom right of the image, you will have to click the image and go into attributes inspector to set size values to your image. Whatever image size you set, should correlate to its real world size.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*05-mHlwA_CHxP0COCVrJ6w.png" /><figcaption>Attributes Inspector for the image.</figcaption></figure><p>Once you have this AR Resource Group and the visual birthday image set up, you can set up an info.plist file.</p><h4>Step 4: Info.plist File</h4><p>This file will allow our device to access the camera for image tracking, and it is simple to implement.</p><ul><li>Right-click on the folder where you want to create the new file.</li><li>Select New File From Template and choose Property List.</li><li>Click Next and name the file Info.plist</li></ul><p>Within this file, you will find a Key, Type, and a Value. In the key, enter “NSCameraUsageDescription,” make sure the type is set to string, and set the value to something like “We need access to your camera for augmented reality features.” This lets the program know to enable the camera as soon as the app opens.</p><p>Now we are ready for the programming!</p><h4>Step 5: Programming</h4><p>All of our code will be in the <strong>ViewController.swift file</strong>, and will be done in Swift.</p><ol><li>We can start by importing all our frameworks, or libraries.</li></ol><pre>import UIKit<br>import SceneKit<br>import ARKit<br>import AVFoundation<br>import SpriteKit</pre><p>These libraries will allow us to combine audio/visual elements and custom UI features for our augmented reality app.</p><p>2. When you initially open this file, you will see some pre-existing code. You can delete all the code under:</p><pre> // Pausing the view&#39;s session<br> sceneView.session.pause()</pre><p>At the top of your source code, you should have these lines. If not, make sure you add them in:</p><pre>class ViewController: UIViewController, ARSCNViewDelegate {<br>    <br>    @IBOutlet var sceneView: ARSCNView!<br>    <br>    override func viewDidLoad() {<br>        super.viewDidLoad()</pre><p>The first line is declaring a class named ViewController which will provide the functionality for the screen that will display our video. Next, we are declaring an IBOutlet, which will connect our code with the UI of our app. The ARSCNView is also going to enable AR content, and the viewDidLoad() function will initially set up our app.</p><p>Then, we can work on the audio aspect of our AR video display.</p><pre>// Set up the audio session to play audio<br>            do {<br>                let session = AVAudioSession.sharedInstance()<br>                try session.setCategory(.playback, mode: .default, options: .mixWithOthers)<br>                try session.setActive(true)<br>            } catch {<br>                print(&quot;Audio session setup error: \(error)&quot;)<br>            }</pre><p>This code will enable the audio on the video we uploaded earlier. The line try session.setActive(true), activates the audio system and tells the app that audio is ready to be played. The line, catch { print(“Audio session setup error: \(error)”) }, handles any errors that may occur during this audio setup session.</p><p>The next step would be to set the 3D scene.</p><pre>  UIApplication.shared.isIdleTimerDisabled = true<br>        <br>        // Setting the view&#39;s delegate<br>        sceneView.delegate = self<br>        <br>        // Creating a new scene<br>        let scene = SCNScene(named: &quot;art.scnassets/ship.scn&quot;)!<br>        <br>        // Setting the scene to the view<br>        sceneView.scene = scene<br>}</pre><p>Some of this code may already be there, however this is important for our 3D scene to show up, so ensure this is in your code. These lines allow us to keep the screen active when we are running our program, prepare the AR view, and load the 3D model that will be displayed.</p><p>Now that we have our 3D model loaded, we can work on image tracking.</p><pre>override func viewWillAppear(_ animated: Bool) {<br>        super.viewWillAppear(animated)<br>        <br>        // Creating a session configuration<br>        let configuration = ARImageTrackingConfiguration()<br>        <br>        guard let arImages = ARReferenceImage.referenceImages(inGroupNamed: &quot;AR Resources&quot;, bundle: nil) else { return }<br>        configuration.trackingImages = arImages<br>        <br>        // Running the view&#39;s session<br>        sceneView.session.run(configuration)<br>    }</pre><p>This portion of our code does a few things:</p><ul><li>Configures an ARKit session to detect and track 2D images (in our case, the birthday cake image).</li><li>Uses images from our AR Resources Group to reference for image tracking.</li><li>Enables image tracking by running the AR Session.</li></ul><p>This basically allows our program to only detect the specific image, an display the AR content only after that image is detected.</p><p>Now that we have made the AR content display upon detection of a certain image (birthday cake image), we also have to make it disappear when the image is no longer detected. This is how you would do that:</p><pre>override func viewWillDisappear(_ animated: Bool) {<br>        super.viewWillDisappear(animated)<br>        <br>        // Pausing the view&#39;s session<br>        sceneView.session.pause()<br>    }<br>    <br>    func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {<br>        guard anchor is ARImageAnchor else { return }</pre><p>After the AR content is able to be displayed and concealed, we can move onto container section.</p><pre>// Container<br>        guard let container = sceneView.scene.rootNode.childNode(withName: &quot;container&quot;, recursively: false) else { return }<br>        container.removeFromParentNode()<br>        node.addChildNode(container)<br>        container.isHidden = false</pre><p>This source code allows us to move our 3D object, “container,” to a specific location. In our case, this location would be an AR anchor like the detected image. For this program specifically, we would use this approach to move our video container to the position of the card (once detected).</p><p>Once we have completed this step, we can move onto the second last step in programming this app, which is setting up the video on a 3D object in an ARKit SceneKit scene. This can be done with the following code:</p><pre>// Video<br>        let videoURL = Bundle.main.url(/service/forresource: &quot;video_test1&quot;, withExtension: &quot;mp4&quot;)!<br>        <br>        let videoPlayer = AVPlayer(url: videoURL)<br><br>        <br>        let videoScene = SKScene(size: CGSize(width: 720.0, height: 1280.0))<br>        let videoNode = SKVideoNode(avPlayer: videoPlayer)<br>        videoNode.position = CGPoint(x: videoScene.size.width / 2, y: videoScene.size.height / 2)<br>        videoNode.size = videoScene.size<br>        videoNode.yScale = -1<br>        videoNode.play()<br>        <br>        videoScene.addChild(videoNode)<br>        <br>        <br>        guard let video = container.childNode(withName: &quot;video&quot;, recursively: true) else { return }<br>        video.geometry?.firstMaterial?.diffuse.contents = videoScene</pre><p>This code takes our video file that we uploaded earlier (video_test1) and makes it ready to play in our container.</p><p>This step is also combined with an a few more lines to make the video look clear and actually play. When running my code, I realized there was a white hue on the video container that prevented the video from playing clearly. This first line of code fixes this issue:</p><pre> // Setting transparency to 1.0 to avoid the white hue on the video<br>        video.geometry?.firstMaterial?.transparency = 1.0<br>        <br>        // Start audio from video<br>            videoPlayer.play()</pre><p>Finally, we can work on animations on the 3D nodes of our project.</p><pre> // Animations<br>        guard let videoContainer = container.childNode(withName: &quot;videoContainer&quot;, recursively: false) else { return }<br>        <br>        guard let text = container.childNode(withName: &quot;text&quot;, recursively: false) else { return }<br>        <br>        guard let textaahana = container.childNode(withName: &quot;textaahana&quot;, recursively: false) else { return }<br>        <br>        videoContainer.runAction(SCNAction.sequence([SCNAction.wait(duration: 1.0), SCNAction.scale(to: 1.0, duration: 0.5)]))<br>        text.runAction(SCNAction.sequence([SCNAction.wait(duration: 1.5), SCNAction.scale(to: 0.08, duration: 0.5)]))<br>        textaahana.runAction(SCNAction.sequence([SCNAction.wait(duration: 2.0), SCNAction.scale(to: 0.07, duration: 0.5)]))<br>        <br>    }<br>    <br>}</pre><p>Overall, this code does a few things:</p><ul><li><strong>videoContainer</strong> — Waits 1 second, then scales up to full size.</li><li><strong>text</strong> — Waits 1.5 seconds, then scales down to 8% of its size.</li><li><strong>textaahana</strong> — Waits 2 seconds, then scales down to 7% of its size.</li></ul><p>These lines allow for simple animations in our app, but can be modified to implement more interesting features in the future.</p><p>Once you have added all the code, you can now move onto the last step of developing our app.</p><h4>Step 6: Testing</h4><p>In order to test your app, go to the top on the window and select IOS Devices. Here, you can select the device you want to pair your program with. Once your device is connected, upon running the program, an app should install in our device. This app will open up and start a camera-like display, and once the tracking image is shown on the screen, your AR content should be displayed.</p><p>Here’s a video of me going through the programming and design behind this project:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FEJq1jflHlS8%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DEJq1jflHlS8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FEJq1jflHlS8%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e47f6d36f73b4ef9b0a94b0a066a332e/href">https://medium.com/media/e47f6d36f73b4ef9b0a94b0a066a332e/href</a></iframe><h4>Overall Experience</h4><p>Despite the final product looking simple and easy, this project has taken me a few weeks to build, with every day coming with a new learning opportunity. I got a chance to learn Swift, and how to use applications such as ARKit, which I never would have even known about. Overall, I really enjoyed working on this project, and I would love to expand my knowledge in the future by improving the AR Birthday Card App (maybe even displaying an interactive game tailored to the recipient of the card👀)!</p><h4>About The Author</h4><p>Hey there! My name is Aahana Patel, a 15-year old exploring the intersection of AI in VR/AR. I’m super passionate about creating life-like digital experiences, that contort the boundaries between what’s real and what’s virtual. I hope to one day create a digital utopia for people to connect with friends and family in a way like never before.</p><p>If you enjoyed this article, be sure to stay in touch with my profile for more interesting tech. If you wish to connect, you can find my LinkedIn below!</p><p><strong>LinkedIn</strong>: <a href="https://www.linkedin.com/in/aahanapatel/">https://www.linkedin.com/in/aahanapatel/</a></p><h4>References</h4><p>Wholf, T. J. (2022, 11 25). <em>Paper or plastic? Gift cards don’t need to be trash</em>. abcNEWS. <a href="https://abcnews.go.com/Business/paper-plastic-gift-cards-trash/story?id=93502280">https://abcnews.go.com/Business/paper-plastic-gift-cards-trash/story?id=93502280</a></p><p>Yuma. (2018, 09 22). <em>ARKit Image Tracking Tutorial</em>. YouTube. <a href="https://www.youtube.com/watch?v=3FKzSYAGfJ0&amp;t=167s">https://www.youtube.com/watch?v=3FKzSYAGfJ0&amp;t=167s</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fad8648a3216" width="1" height="1" alt=""><hr><p><a href="https://medium.com/antaeus-ar/augmented-wishes-saving-trees-one-birthday-card-at-a-time-fad8648a3216">Augmented Wishes: Saving Trees One Birthday Card at a Time</a> was originally published in <a href="https://medium.com/antaeus-ar">Antaeus AR</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Battle For Realism in 3D Rendering: A Brief Overview of NeRFs vs. Gaussian Splatting]]></title>
            <link>https://medium.com/antaeus-ar/the-battle-for-realism-in-3d-rendering-a-brief-overview-of-nerfs-vs-gaussian-splatting-580cff4d8801?source=rss-ecaf3e1a112c------2</link>
            <guid isPermaLink="false">https://medium.com/p/580cff4d8801</guid>
            <category><![CDATA[neural-radiance-fields]]></category>
            <category><![CDATA[3d-rendering]]></category>
            <category><![CDATA[gaussian-splatting]]></category>
            <dc:creator><![CDATA[Aahana]]></dc:creator>
            <pubDate>Sun, 13 Oct 2024 15:48:25 GMT</pubDate>
            <atom:updated>2024-10-13T15:48:25.295Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zcypcRrxs07bB8vHEIlFmg.jpeg" /></figure><p>Envision a world where we unlock artificial realism that blends the boundary between virtual and reality. Well this utopia is currently being created in the form of advanced 3D rendering technologies.</p><h4><strong>What is a Neural Network?</strong></h4><p>This is a term I’m sure is familiar when we talk about <a href="https://www.youtube.com/watch?v=aircAruvnKk&amp;t=232s">deep learning</a>, however, the meaning is still unknown to many. The name neural network is derived from the human brain, and the concept of these networks is closely related to the function of our brains as well. From the name alone, we can understand that it is a network of neurons. In this context, a neuron is essentially a unit that processes information.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rh3E22XAJBHR22X9VM4K7w.jpeg" /><figcaption>Fig. 1: Neural network layers, Photo Credit: Ravindra Parmar</figcaption></figure><p><strong>A network contains </strong><a href="https://www.ibm.com/topics/neural-networks"><strong>4 main types</strong></a><strong> of layers:</strong></p><ul><li>The <strong>input layer</strong> is where the network receives data, such as images, sounds, text, etc. From here, this data is converted to a numerical format for the network to be able to understand.</li><li>Next comes one or more <strong>hidden layers</strong>, depending on the complexity of the task. These hidden layers are responsible for analyzing the input with artificial neurons. Each individual neuron detects patterns within the input data, and is able to identify distinct aspects that help us understand the input data; similar to the human brain.</li><li>After analyzing this data, the neurons pass the information through the <strong>activation function</strong>, deciding whether or not to send a signal, based on the received data. This layer ensures that our model can solve real-world problems, by introducing non-linearity. A model without this layer would only be able to solve problems where the relationship between input and output is linear. This allows our model to learn complex patterns and make more accurate predictions.</li><li>Finally, the <strong>output layer</strong> provides us with the final decision or prediction, based on what the network has learned with the data.</li></ul><p>For this model to learn and make more accurate predictions in the future, the network needs to be trained. During this process, the network is fed large amounts of data, through which it has to make predictions, while comparing it to the correct answer. This process of error minimization is called <a href="https://builtin.com/machine-learning/backpropagation-neural-network">backpropagation</a>, and it essentially ensures that our network can provide precise results.</p><h4>What is a Neural Radiance Field (NeRF)?</h4><p><a href="https://arxiv.org/abs/2003.08934">NeRFs</a> are essentially a use of neural networks specially designed for synthesizing high-quality 3D scenes from 2D images. By taking 2D pictures of something from different angles, NeRFs are able to take that data, and generate smooth 3D representations of that scene. NeRFs focus on something called <em>volumetric rendering</em>. This means that it models how light moves through a space, rather than having to rely on polygonal structures like traditional <a href="https://www.matthewtancik.com/nerf">3D models</a>. This technique allows NeRFs to have the upper-hand when creating highly detailed models, with complex lighting and shadows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5ZKue4bil2P9leAsMaSBRQ.jpeg" /><figcaption>Fig. 2: Scene rendering with input images, Photo Credit: Oscar de la Hera Gomez</figcaption></figure><p><a href="https://medium.com/@sarita_68521/basic-understanding-of-neural-network-structure-eecc8f149a23"><strong>Step by step</strong></a><strong> explanation:</strong></p><ol><li><strong>Input</strong>: A NeRF requires multiple 2D images taken from different angles. An input for a NeRF would be something like a 3D coordinate and the direction of the camera (described as a vector).</li><li><strong>Architecture</strong>: NeRFs require multiple hidden layers of artificial neurons within its neural networks. The input layer consists of the viewing directions and 3D positions. The hidden layers process the input and access how the lighting plays with the environment. The network then outputs important information about the input points. The RGB colour, and the density, or how much light is being absorbed at that point.</li><li><strong>Volume Rendering</strong>: Once thee two values are estimated, we need to combine them. This technique is used to take all the colours and densities along a ray of light from the camera, through to the 3D points that are intersected. Meaning that as the light ray moves, NeRFs are able to predict the colour and density at every point along that path.</li><li><strong>Training</strong>: NeRFs are trained using a set of images, like the picture of input images above. During this process, the network uses backpropagation to learn to predict the colours and density at any given point, and be able to reduce differences between the predictions, and the actual 2D input images. After training the NeRF, it is able to produce photorealistic 3D representations with various viewpoints just based off of a couple 2D input images.</li><li><strong>Output</strong>: As mentioned, after the training process, NeRFs are able to accurately synthesize new views of a scene, regardless of whether or not the training data showed that angle of the scene already. In a virtual setting, you may move the camera wherever, and the NeRF is able to predict what that angle would look like, based on its training.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*bO5yZxEMUUWZG5rgTirm2A.png" /><figcaption>Fig. 3: Breakdown of how NeRFs work, Photo Credit: Sergios Karagiannakos</figcaption></figure><p>In figure 3 for example, the system starts with input information, which includes the position and direction. This information is passed through a neural network. The network then predicts colour (RGB) and the density (how transparent or solid the point is). From here, it simulates the rays of light that pass through the scene. By combining all these effects of rays, we can create an image of a scene from a specific viewpoint. The rendering loss layer basically compares the generated image to the given input, and calculates any errors, which are called “loss”. This loss is then used to adjust the network and make it more accurate for future results.</p><h4>Disadvantages of NeRFs</h4><p>Despite being able to achieve photorealism and generate hyper-realistic 3D representations of 2D images, NeRFs have their downsides.</p><ul><li><strong>High Computational Demand</strong>: NeRFs require a lot of power to train and render. A NeRF model can take hours or even days to train, and rendering can be very slow because the NeRF need to trace rays and compute colours and density while doing so.</li><li><strong>Data Requirements</strong>: In order for NeRFs to create accurate 3D representations, it requires a large amount of high-resolution images from multiple angles. Without this data, the scene can become choppy, and not properly constructed.</li><li><strong>Memory Usage</strong>: For complex scenes, NeRF models tend to be memory-intensive. The model can become extremely large once the scene grows in detail, size and complexity, making NeRFs overly large.</li><li><strong>Dynamic Scene Rendering</strong>: NeRFs are usually designed to handle <em>static scenes </em>meaning objects don’t move or change over time. Updating models in real-time can be difficult since they aren’t designed to work with <em>dynamic scenes</em>.</li></ul><h4>Introducing Gaussian Splatting</h4><p><a href="https://arxiv.org/abs/2308.04079">Gaussian Splatting </a>is an advanced technique used for 3D rendering and scene reconstruction. This technique is often used as a faster and more flexible alternative to traditional NeRFs, as they have their advantages.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0mmfJyFv-E5E1EU9IAsfkQ.png" /><figcaption>Fig. 4: Representation of how Gaussian Splats are blended to create a seamless image, Photo Credit: Innoarea Projects</figcaption></figure><p><strong>Breakdown of How it Works:</strong></p><ol><li><strong>Scene Representation</strong>: Instead of using a collection of grid points like NeRFs, Gaussian Splatting uses <em>3D Gaussian Distributions</em>. Each Gaussian is described by the position (the 3D location within the scene), colour density (RGB value, and how the light is absorbed), and size &amp; shape (how “blurry” or “sharp” the scene would be in a particular spot).</li><li><strong>Projection onto a 2D Image Plane</strong>: Gaussian Splatting works by projecting each Gaussian “blob” onto the camera view. This is essentially telling us how each Gaussian would appear from different viewpoints of the camera. Visually, on the image plane, a Gaussian can appear as an elliptical spot, or a “<a href="https://scaniverse.com/news/intro-gaussian-splats">splat</a>” as shown below.</li><li><strong>Blending Gaussians</strong>: After these splats are projected onto a 2D plane, they are blended together based on their positions in 3D space. By blending the colours and densities and creating that gradual transition, we can achieve a smooth, seamless image, like the one above.</li><li><strong>Handling Camera Rays</strong>: Similar to NeRFs, Gaussian Splatting works by simulating rays of light that pass through the scene. However, unlike NeRFs, rather than tracing these rays through grids, Gaussian Splatting calculates how each Gaussian contributes to each individual light ray, by considering the colour and density.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/603/1*T19FL9GYxEeSjebzl5xPag.jpeg" /><figcaption>Fig. 5: How NeRFs trace rays through grids vs. how Gaussian Splatting calculates how each Gaussian contributes to the light ray using elliptical splats, Photo Credit: Kate Yurkova</figcaption></figure><p>Once all the Gaussians are blended and projected on to the 2D plane, the result is a realistic rendering of the 3D scene from the current camera angle.</p><h4>Is Gaussian Splatting Better than NeRFs?</h4><p>There are often times when speed, adaptability, and efficiency are needed, and Gaussian Splatting is sometimes seen as a better alternative to traditional NeRFs.</p><p><strong>Let’s walk through why Gaussian Splatting often outperforms NeRFs:</strong></p><ol><li><strong>Real-Time Rendering</strong>: When it comes to rendering new views or angles of a scene, Gaussian Splatting is far more efficient than NeRFs. The real-time rendering that Gaussian Splatting does is crucial for interactive applications such as live simulations and VR/AR.</li><li><strong>More Simple Computation</strong>: Gaussian Splatting represents a scene using 3D Gaussian Distributions, unlike NeRFs that use grids or volumetric models. Each Gaussian is a simple structure that represents colour, density, and shape efficiently. However, NeRFs require complex algorithms that calculate how light passes through each volumetric pixel (voxel) in a grid. This can be both time consuming and expensive.</li><li><strong>Rendering Complex Scenes</strong>: As I mentioned earlier, NeRFs are trained to handle static scenes, and any changes to the scene would require the model to be re-trained for accurate 3D representation. Gaussian Splatting on the other hand, is better suited for dynamic elements, and can adapt to changing scenes without having to be completely re-trained.</li><li><strong>Data Requirements</strong>: NeRFs require large amount of data in the form of 2D images from various angles to be able to accurately represent a scene. Gaussian Splatting however, uses flexible and light-weight Gaussian representation, which allows it to work with the small data sets more effectively.</li><li><strong>Memory</strong>: Compared to NeRF’s volumetric grids, Gaussian Blobs are much more memory efficient due to their ability to work with smaller data sets. This can allow Gaussian Splatting to render complex scenes using less resources.</li><li><strong>Training Time</strong>: Gaussian Splatting uses simpler mathematical models, and doesn’t require some of the complex techniques that NeRFs use. Once the 3D Gaussians are defined, we are able to project and render them quickly. However, while NeRFs are capable of producing high-quality scenes, they require large amounts of training data time, which limits their application.</li></ol><h4>What Does This Mean for Us?</h4><p>Techniques such as Gaussian Splatting are reforming how we create hyper-realistic, immersive virtual environments. This technique specifically provides a fast, efficient, and flexible method of representing 3D scenes. Instead of having to rely on traditional rendering techniques that involve sharp polygons, or dense grids of voxels, Gaussian Splatting uses smooth and light blobs to describe the interaction of light within a 3D space. In turn, we are left with smooth blending, fast computation, and efficiency in memory. This approach is revolutionizing the <a href="https://www.magnopus.com/blog/the-rise-of-3d-gaussian-splatting">3D rendering industry</a>, and is bringing new heights to quality and speed of visual production.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/860/1*88uBZ2ZwQv7R8HXQfyKGrQ.jpeg" /><figcaption>Fig. 6: Meta Codec Avatars using Gaussian Splatting, Photo Credit: Tomislav Bezmalinovic</figcaption></figure><p><strong>Where Can Gaussian Splatting Be Used?</strong></p><ol><li><strong>Virtual Reality (VR) and Augmented Reality (AR)</strong>: Rendering life-like virtual worlds requires advanced 3D rendering techniques. Gaussian Splatting can improve the realism in <a href="https://www.pcmag.com/news/augmented-reality-ar-vs-virtual-reality-vr-whats-the-difference">VR/AR</a>, and make the overall experience much more seamless.</li><li><strong>Visual Effects (</strong><a href="https://www.youtube.com/watch?v=srNT6DIjcD4"><strong>VFX</strong></a><strong>)</strong>: In film production, Gaussian Splatting can allow us to render special visual effects, where efficient blending is crucial to achieve that “real” look.</li><li><strong>Scientific Visualization</strong>: In fields such as astronomy, medical imaging, and particle physics, Gaussian Splatting can help us visualize data in a continuous way, without making it overly complex with geometry (like NeRFs).</li></ol><h4>TL;DR</h4><p>In a world where immersive technologies are constantly advancing, techniques such as Gaussian Splatting and NeRFs are transforming 3D rendering. NeRFs (Neural Radiance Fields) generate seamless 3D representations with 2D input images by using neural networks, and are able to produce highly-detailed scenes. On the other hand, Gaussian Splatting uses 3D Gaussian Blobs that create smooth scenes. Ultimately, both technologies have their advantages, however, Gaussian Splatting is starting to become the preferred choice due to its speed and efficiency.</p><h4>About The Author</h4><p>Hey there! My name is Aahana Patel, a 15-year old exploring the intersection of AI in VR/AR. I’m super passionate about creating life-like digital experiences, that contort the boundaries between what’s real and what’s virtual. I hope to one day create a digital utopia for people to connect with friends and family in a way like never before.</p><p>If you enjoyed this article, be sure to stay in touch with my profile for more interesting tech. If you wish to connect, you can find my LinkedIn below!</p><p><strong>LinkedIn</strong>: <a href="https://www.linkedin.com/in/aahanapatel/">https://www.linkedin.com/in/aahanapatel/</a></p><h4>References</h4><p><em>[2003.08934] NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis</em>. (2020, March 19). arXiv. Retrieved October 13, 2024, from <a href="https://arxiv.org/abs/2003.08934">https://arxiv.org/abs/2003.08934</a></p><p><em>[2308.04079] 3D Gaussian Splatting for Real-Time Radiance Field Rendering</em>. (2023, August 8). arXiv. Retrieved October 13, 2024, from <a href="https://arxiv.org/abs/2308.04079">https://arxiv.org/abs/2308.04079</a></p><p>Al-Masri, A., &amp; Urwin, M. (n.d.). <em>Backpropagation in a Neural Network: Explained</em>. Built In. Retrieved October 13, 2024, from <a href="https://builtin.com/machine-learning/backpropagation-neural-network">https://builtin.com/machine-learning/backpropagation-neural-network</a></p><p><em>An Intro to Gaussian Splats — Scaniverse</em>. (2024, September 26). Scaniverse. Retrieved October 13, 2024, from <a href="https://scaniverse.com/news/intro-gaussian-splats">https://scaniverse.com/news/intro-gaussian-splats</a></p><p><em>Basic Understanding of Neural Network Structure | by Sarita, PhD</em>. (2023, October 3). Medium. Retrieved October 13, 2024, from <a href="https://medium.com/@sarita_68521/basic-understanding-of-neural-network-structure-eecc8f149a23">https://medium.com/@sarita_68521/basic-understanding-of-neural-network-structure-eecc8f149a23</a></p><p>Brennan, M. (2024, March 14). <em>3D Gaussian Splatting! — Computerphile</em>. YouTube. Retrieved October 13, 2024, from <a href="https://www.youtube.com/watch?v=VkIJbpdTujE">https://www.youtube.com/watch?v=VkIJbpdTujE</a></p><p><em>But what is a neural network? | Chapter 1, Deep learning</em>. (2017, October 5). YouTube. Retrieved October 13, 2024, from <a href="https://www.youtube.com/watch?v=aircAruvnKk">https://www.youtube.com/watch?v=aircAruvnKk</a></p><p><em>Facebook is building the future of connection with lifelike avatars</em>. (n.d.). Tech at Meta. Retrieved October 13, 2024, from <a href="https://tech.facebook.com/reality-labs/2019/3/codec-avatars-facebook-reality-labs/">https://tech.facebook.com/reality-labs/2019/3/codec-avatars-facebook-reality-labs/</a></p><p><em>Gaussian Splatting and the Future of Photogrammetry — Innoarea Projects</em>. (n.d.). Innoarea Projects. Retrieved October 13, 2024, from <a href="https://innoarea.com/en/noticias/gaussian-splatting/">https://innoarea.com/en/noticias/gaussian-splatting/</a></p><p>Greenwald, W. (2023, June 6). <em>Augmented Reality (AR) vs. Virtual Reality (VR): What’s the Difference?</em> PCMag. Retrieved October 13, 2024, from <a href="https://www.pcmag.com/news/augmented-reality-ar-vs-virtual-reality-vr-whats-the-difference">https://www.pcmag.com/news/augmented-reality-ar-vs-virtual-reality-vr-whats-the-difference</a></p><p><em>NERFs (No, not that kind) — Computerphile</em>. (2023, December 21). YouTube. Retrieved October 13, 2024, from <a href="https://www.youtube.com/watch?v=wKsoGiENBHU">https://www.youtube.com/watch?v=wKsoGiENBHU</a></p><p><em>NeRF: Neural Radiance Fields</em>. (n.d.). Matthew Tancik. Retrieved October 13, 2024, from <a href="https://www.matthewtancik.com/nerf">https://www.matthewtancik.com/nerf</a></p><p>Parmar, R. (2018, September 11). <em>Training Deep Neural Networks. Deep Learning Accessories | by Ravindra Parmar</em>. Towards Data Science. Retrieved October 13, 2024, from <a href="https://towardsdatascience.com/training-deep-neural-networks-9fdb1964b964">https://towardsdatascience.com/training-deep-neural-networks-9fdb1964b964</a></p><p>Regalbuto, A. (2023, December 8). <em>The rise of 3D Gaussian Splatting: what is it and how is it changing the immersive media industry? — Magnopus</em>. Magnopus. Retrieved October 13, 2024, from <a href="https://www.magnopus.com/blog/the-rise-of-3d-gaussian-splatting">https://www.magnopus.com/blog/the-rise-of-3d-gaussian-splatting</a></p><p>Takacs, D. (2024, August 27). <em>2 Visual Effects using Gaussian Splatting and AI</em>. YouTube. Retrieved October 13, 2024, from <a href="https://www.youtube.com/watch?v=srNT6DIjcD4">https://www.youtube.com/watch?v=srNT6DIjcD4</a></p><p><em>3D Gaussian Splatting — Explained!</em> (2023, November 5). YouTube. Retrieved October 13, 2024, from <a href="https://www.youtube.com/watch?v=sQcrZHvrEnU">https://www.youtube.com/watch?v=sQcrZHvrEnU</a></p><p><em>3D Gaussian Splatting — Why Graphics Will Never Be The Same</em>. (2023, August 31). YouTube. Retrieved October 13, 2024, from <a href="https://www.youtube.com/watch?v=HVv_IQKlafQ">https://www.youtube.com/watch?v=HVv_IQKlafQ</a></p><p><em>What is a Neural Network?</em> (n.d.). IBM. Retrieved October 13, 2024, from <a href="https://www.ibm.com/topics/neural-networks">https://www.ibm.com/topics/neural-networks</a></p><p><em>What is 3D Gaussian Splatting?</em> (2023, September 28). YouTube. Retrieved October 13, 2024, from <a href="https://www.youtube.com/watch?v=Tnij_xHEnXc">https://www.youtube.com/watch?v=Tnij_xHEnXc</a></p><p>Yurkova, K. (2023, December 23). <em>A Comprehensive Overview of Gaussian Splatting | by Kate Yurkova</em>. Towards Data Science. Retrieved October 13, 2024, from <a href="https://towardsdatascience.com/a-comprehensive-overview-of-gaussian-splatting-e7d570081362#fbb7">https://towardsdatascience.com/a-comprehensive-overview-of-gaussian-splatting-e7d570081362#fbb7</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=580cff4d8801" width="1" height="1" alt=""><hr><p><a href="https://medium.com/antaeus-ar/the-battle-for-realism-in-3d-rendering-a-brief-overview-of-nerfs-vs-gaussian-splatting-580cff4d8801">The Battle For Realism in 3D Rendering: A Brief Overview of NeRFs vs. Gaussian Splatting</a> was originally published in <a href="https://medium.com/antaeus-ar">Antaeus AR</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>