VR, MR & AR
“If life is not already complicated, God added fiction to human’s life” - Anonymous
When I read this quote, the first thought that came to my mind was ‘What is wrong with fiction and why there has to be so much negativity in that quote?’
As per me, fiction brings joy and color to the otherwise dull, lifeless real world. In general, tech scientists are dull, gloomy, and solemn in nature.
‘The bigger the technology, the more the solitude’
remarked one tech professional, whom I recently met during one of the tech conferences in Silicon Valley, however the other engineering and technological reality is changing the way the tech scientists and engineers are envisioned by normal public, by bringing out the funny part of these ‘STEM ers’ and enhancing their opportunities to interact with the normal world, extremely well, through their technological marvels.
Just like fiction to literary industry, the replacement reality technologies bring life and pleasure to the field of IT and computers.
The advancements made in Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) are a boon for the entertainment industry in broader sense, but have also helped in manufacturing, medicine, engineering, aeronautics, and space research. Although the three technologies are similar in simulating a ‘real life situation’ using computer graphics and code, they are distinctive from each other in some ways. While AR assists the human beings in their miserable world by enhancing physical reality, MR supports them to bring some brightness and ease into their wretched world by combining the physical world and the digital world, VR transports them to a completely new happy imaginary reality which occludes the physical reality.
Simulation of the real world through computer graphics has been happening for more than three decades and it has helped many engineers, pilots, drivers, and medical students in getting the grips with the original environment generated though computer graphics. It has also helped innumerable educators in gaining vital knowledge and many professionals in training the best minds. Augmented Reality has enhanced simulation of various activities by adding another important feature to it, ‘Superimposition’. AR can simulate a specific real-life situation, which can be easily projected on any non-reflective surface to help humans. Say for example, it can simulate an environment of assembling a machinery in graphics and this computer-generated imagery can then be projected on to any shiny surface to help the shop floor person in assembling the machinery through easy step by step instructions.
In a smart connected factory of the future, every machine is paired with the other and they can transmit data to and from each other, thus, helping the factory managers to monitor and control their workload, optimize, and maintain the connected machines and in achieving the shop floor analytics. The data from IIoT ( Industrial Internet of Things) connected machines, combined with the AR experience can help the shop floor engineers and managers to identify the state of the machines, the status of the jobs that is currently being pursued by the machine etc., Also these connected factories do not leave the human out of the loop, even the engineers and managers get included in their processes once 5G is fully deployed and their work schedule can be fully linked to the connected machines, to ensure a fully automated environment.
The same can be said about AR when used in training a surgeon in hands-on complex surgeries. The step-by-step instructions can be projected on a small LCD screen to help him perform the operation and thus improve his accuracy and precision. However as per the experiments conducted with AR, the most important gain is enhanced speed. The speed with which the engineers, surgeons and astronauts have performed these procedures has gone up exponentially compared to those who performed the same using manual instructions either written or oral.
The recent development in AR was in the field of Sports. Audience of 2018 winter Olympics, which was held in the city of PyeongChang, South Korea, experienced the first quality AR/ VR tech, combined with 5G improvements. Communication giants like Cisco and Samsung joined hands and used 5G technology to provide the best- ever known augmented reality experience. This technology did not merely bring the real time experience of the participants to the audience vividly, but it enhanced the experience of the viewers by superimposing the biometric data of the players like heart rate, muscle tension etc., along with other statistics like their speed, angular velocity, lateral positions, their lead time, and distance against their nearest opponents etc.,
A new form of AR developed by PTC called ‘Vuforia Chalk’ enables the sharing of the work environment with the peers or the executive managers, who sit remotely and offers IIoT, and enterprise IoT Solutions. The operative can ask for help and solutions when faced with problems. It works, similar to ‘GoogleDocs’ where the two people involved in solving a problem share the same workspace. In this case, instead of sharing a notebook /spreadsheet/ presentation, ‘Vuphoria Chalk’ allows sharing of real / workspace videos among the two people in real time, while the peer or the manager makes the right decision or gives the right instructions to turn on or turn off a specific equipment or a function in each machine, in order to solve a specific problem. The mentee can follow his instructions to do so accordingly, while the remote peer watches the workspace solution happening through the shared screen on the gadget (smartphone) held in his mentee’s hands.
Augmented Reality is recently employed in Apple’s iOS12 called ARKit2. Apple Inc., has come up with plenty of features such as real object detection and then use this scanned image to find information about the physical object. For example if the real time scanner identifies a store card with the help of its image recognition algorithm, then the software automatically digs into the database of the store and find all the details pertaining to the card like who owns the card, what is the worth of the card in loyalty points or cash value ( if it serves as a credit card like target red card) , recent purchases etc.,
The above image shows a real shadow cast by AR images on real objects, in this case a coffee table. The image was taken in the author’s home at San Jose, California.
ARKit2 is expected to do plenty more like face and eye tracking and acting based on the functions desired by the eyes, like blinking twice can expand a pop-up window, blinking once can close the same etc., It also has brought upon another feature where a real object can cast a shadow on another object created by VR software, like a material knife placed on a existent table casts its shadow on a virtual strainer prepared by a AR/ VR software and placed on the same table. This new feature has taken immersion in Augmented, Virtual and Mixed Reality a level up, beyond the regular standards.
Another important application of AR in business, as envisioned in the future, is the floating user interface, where the AR technology can transfer all the documents, charts, and presentations into the user interface and managers can swipe the data in the air screen and manipulate it or use it, without any gadgets but with the help of an internet connection and the reflective surface.
Mixed Reality is the field of combining the computer world and the real world. When Mixed reality was first invented in 1994, by two academics, Paul Ingram and Fumio Kishino, they called it Augmented Virtuality, an intermediary between AR and VR. The most important aspect to be considered is the sharpness with which a computer-generated imagery can be superimposed on a real-life video or still image. The concept of superimposition is similar to AR, however, there are some subtle differences in how the images are projected. In AR, the computer imagery is fully formed and functional when it is projected on a screen, while in MR, the computer imagery is formed and placed in a suitable position in the real world in the first step. Then the image is captured along with the real environment it is in, in order to be rendered as a functional combined imagery. Just as in Virtual reality, mixed reality users use a MR headset to get the full MR experience.
There are many aspects which need to be considered when mixed reality is being used. They are
- The ability of the camera to capture images
- The computer imagery to be combined and
- The environment in which mixed reality is being used.
For example, the environment can be a house, or an industry or an office or a mall or a city etc., the image rendered can be an object, a human, an animal or even a building. Also, the MR concept of combining real world scenery and computer-generated imagery allows more flexibility to the user, because they can pick and choose and drag and drop images from computer on to their personal world around them to make it increasingly appealing and get the feeling of wholesome, into their life. This concept allows a lot of imagination compared to AR. Hence MR is extremely entertaining compared to AR and thus used in several interactive videos and PC games. The use of headset to improve the gaming experience makes it extensively expensive compared to AR.
Get Uma Chandrasekhar’s stories in your inbox
Join Medium for free to get updates from this writer.
The field of engineering and medicine gets the maximum benefits with the use of MR. Whether it is oil rigs or emergency Operation theatre, MR can come in handy in many applications. While AR software like ‘Vuforia Chalk’ helps to share a screen, MR headsets help the operators and surgeons to completely project their physical environment on to their peers or supervisor’s computer and gets assistance while facing a difficult procedure to perform.
Virtual Reality as the name suggests is a completely imaginary world generated using computer graphics. The three I’s of VR are Immersion, Interaction, and Imagination. The main difference between AR and VR is AR brings augmented graphics technology to the workplace using which one can accomplish some difficult to do jobs, however VR takes the user into the computer world and immerse them into this software generated world and everything which happens inside this code generated world is virtual. In VR, there is no combination of reality and virtuality. To rephrase it, if a player gets assualted physically or sexually, it does not impact the player's real life in any way at all.
With AR technology it is possible to repair an actual smart grid, while using VR technology the intricacies of the smart grid operations such as how the fault can be detected, serviced and maintained, what are procedures to be followed in case of an increased surge or other emergency etc., can be experienced using a number of ‘not- so- real’ computer generated avatars in an artificially generated environment. Thus, it is made feasible to implement the techniques learned using VR experience, once the user is out of the virtual world, making it, an excellent learning tool as well, like AR, but here learning is not through step-by-step instructions, but through animated movie like experiences which are extensively authentic using enhanced graphics technology.
VR experiences are made powerful by providing the five sensory pleasures of sight, sound, haptic, taste and speech, nevertheless the technology has so far made possible only vision and audio capabilities to enhance the VR experience, though there are ample amount of research, which is being performed currently to include haptics, which is the next sensory organ, to be included into the VR experience. Haptic gloves can be used to make the touch, travel through network to reach the other person, who is teaming with the player. The main problem faced with the haptic experience is the latency of the network, however with the low latency 5G deployments, it’s become successively easier to deal with haptic transmissions. So, the poke in the Facebook (Meta Inc.,) can be truly felt once Facebook fully provides the oculus experience to its users.
VR when used in medicine and healthcare, can help train surgeons and physicians about newest technologies in their respective field of expertise, without performing the surgery or meeting up with the patients in a totally virtual world filled with virtual content in all respects.
The recent development in VR is mind controlled VR, where the need of a VR remote to enjoy this imaginary world is eliminated. As per Rames Alcaide, the founder and CEO of Neurables, this technology works merely with a headset. The electronics inside the headset connect to the user brain ‘s neural network and the user can use thought process to move the images on the VR screen. BCI[UC1] is made possible through a similar technology like EEG, where the brain’s potential is converted into digital signals of 0 and 1 and this binary bit of data enters the electronic system of the Mind controlled VR headset and in turn enables the brain to perform tasks through merely thinking about it. This helps in hands free operation, and they are planning to do simple tasks like typing, playing games, moving the mouse, clicking etc.,
https://www.technologyreview.com/video/609888/technology-spotlight-mind-controlled-vr/
To view the above video, you might need a subscription to MIT Tech review. But it shows a live demo of Neurable- a mind- controlled VR game and a discussion with Rames Alcaide, the Neurable co-founder.
Gone are the days when coders and game developers are considered retarded, unshaven, unhygienic nerds, who sit in their basement and do nothing other than building and testing new software. The replacement reality technologies are making them genuine by giving an opportunity for them to contribute to many other fields such as education and training, healthcare, retail, manufacturing, and business applications, thus enabling them to interact with the normal world other than through interactive entertainment. With the advent of interactive games on smart phones, a new piece of life was thrown to the coders where they have learned to strive, thrive, and survive in a bolder, better, and bigger way, by building mobile apps for these games.
Besides AR, VR and MR, the San Diego based communication firm, Qualcomm Technologies’ CEO, Mr. Steven Mollenkopf and his team, has produced a fourth stage of reality called Extended Reality (XR) which Qualcomm introduced at the Augmented World Expo (AWE) in May 2018. Previously, the term XR used to describe the combined form of the all the three-proxy reality- AR, MR and VR and everything in-between, however Qualcomm has patented the term XR with a graphic ‘X’, and thus unveiling the new mobile computing platform of the future. As per Qualcomm, their XR technology makes the augmented experience extensively enhanced using Artificial Intelligence, thus enabling better interaction, increased performance, and improved efficiencies. The Qualcomm’s Snapdragon XR1 platform is a new mobile computing platform for the future. It is a combination of ARM[UC2] based CPU[UC3] , GPU, Vector processor and Qualcomm AI engine. With the improved performing XR1 chip called Snapdragon 845 and the associated enhanced hardware and efficient machine learning based computer vision algorithms, it enables the XR devices to provide a better image, good audio and reduced noise using a better ISP (Image Signal Processor) chip in their headset. The increased performing head tracking and controllers with 3 and 6 Degrees of Freedom respectively, enables free movement in the virtual world and using their integrated sensor equipment. XR devices provide a better immersion and good interaction to the user, making the experience as close to the real world as possible.
The future of these reality technologies is bright as there are plenty of applications which can be envisioned using them.
1) The predictive analytics along with these Proxy Reality technologies might allow our successive generations to work in real and foretell the future without a crystal ball and prepare the business world for the oncoming eventuality.
2) Crowd sourcing and crowd funding platforms like Amazon’s Mechanical Turk and Goldieblock with the help VR, AR and MR might increase social funded resources leading to more collaboration.
3) Virtual interfaces and virtual workplaces have already become the norm and remove the need to travel to work in future, thanks to Covid -19 shut down in 2020 and 2021.
4) VR/ AR combined holographic images are likely to improve the customer’s purchasing decisions and help to build the retail industries, through efficient use of AI based recommender Systems.
5) VR/ MR influenced dating Apps and other social media apps might spring up leading to increased marital matches made through these proxy reality technologies.
6) Smart City management can be influenced by VR /MR technologies where the operator wearing a VR headset can enter the VR representation of the Smart City and monitor all the details of the city, without even visiting the city in real time. All updates to the city’s improvements can be fed into the VR city in regular intervals of 24–36 hours.
While I listed the six positive ways these substitution reality technologies can influence our future generations, I can think of many negative applications too such as bank heists, terrorist plots, mass shootings, battleground scenarios and many more use cases where this technology can be misused. Hence, it’s the responsibility of every one of us, the researchers, the developers, the sellers, the buyers, and the users, to prevent such evil use cases and keep the future generations clean and safe.
I want to conclude by saying
“Just like a tool is as good as the technician, the technology is also as good as its users”.
[UC1] Brain controlled Interface.
[UC2] Advanced Risk Machine, one of the earliest processor cores with RISC (Reduced Instruction Set Computing) architecture
[UC3] RISC architecture CPUs have the same performance as that of CISC (Complex Instruction Set Computing) except with reduced cost and simpler hardware.
Originally published at https://www.linkedin.com.
