AR with 3D Models
On Advanced Multimedia Technology’s second assignment of the semester we were tasked to create an interactive AR (Augmented Reality) model exploring various technologies, including Polycam, Substance 3D Stager, and SparkAR.
Since the beginning of the course we were given chances to experiment these technologies that enables us to create AR experiences using various types of graphics, including images, 3d models and text.
Process
The first thing we had to do was understanding how we can make a 3D model of a physical object. There are many tools that can help us with this task, but the one that we have used is called “Polycam”. It’s a free mobile app where we take multiple pictures (the more, the better) of the object and Polycam would do all the work. After the render is complete, we have the option to export the model as GLB format.
Now that we have the 3D model in the right format, we can use it in any software that accepts it. We explored Substance 3D Stager, which allows us to create an environment from a picture, set our model inside it and arrange the lightning. Stager also has a set of 3D models ready, where some of them can be customized to our needs. Objects from Stager can be exported as 3D models as well — I’ll be mentioning more of this later on.
Exploring the process of creating a 3D model and also learning how to export it, import it in Stager, and then export it back, made me feel more comfortable and confident to start working on the final project.
Planning
For this assignment, we were tasked to showcase the Advanced Chocolate and Confectionery Artistry Humber program’s work in an AR environment. We can create these experiences either by using a target tracker or a plane tracker. The space where the AR experience would happen has some great spots to be used as a target tracker — however, it wouldn’t be as efficient and practical as these spots could be being used by someone else. For this matter, I decided to use a plane tracker, displaying the cakes on top of a table.
Get Zena A’s stories in your inbox
Join Medium for free to get updates from this writer.
The first step was to have the cake models optimized and ready to use — Fortunately, our instructor greg goralski had many of them ready for us to use (thank you!). I chose 3 (three) cakes to showcase, so my next step was to display them on top of a table. For this matter, I created a table inside Stager and imported the cake models, creating the visuals I needed before jumping to the next stage.
Having the model ready, I exported it in GLB format from Stager. Then in SparkAR, I exported it inside the file. First thing done was to create a Plane Tracker, and inside of it I placed the 3D model imported from Stager. Inside this model I can select the table and each of the cakes to play with it.
For the purpose of this showcase, since the cakes will be displayed on a table, the user should be able to scale them for better visualization and also rotate them to see all of its angles. To do that, I selected the cake model and created a set of patches like this:
To scale up the cake, I didn’t want the anchor point to be in the middle of it: to solve this problem, I created a Null Object, set the anchor point where I want it to be, and placed the cake model inside of it. That way, I can interact with the Null Object itself instead of the cake model. This is how the patch editor looks like:
Final Result
While creating all these interactions in SparkAR, I was constatly testing them on my mobile device. Unfortunately I couldn’t test them at Humber’s Bakery space itself because the file was too big to be shared as an Instagram effect. The only way to test it out was through an USB cable.
The final AR experience shows the table with all the cakes and a text guiding the user how to interact with the models. I like how the final result turned out to be despite all the difficulty I had with sizing issues. Once the user triggers the AR, it stays in the same place no matter how they rotate their phone.
Augmented Reality has been with us for several years now, but its capabilities and resources are increasing rapidly. In our technology era, considering to use AR in our projects can be very helpful in creating an even better experience to the user.
Thanks for reading.
