AR and VR with
Gamification

October 31, 2021
Maya Tayake
A trend of entities shifting to incorporate augmented reality (AR) and virtual reality (VR) into their business practices is not so uncommon anymore. Technology sectors like NASA use VR to study drone air traffic control (NASA Video, 2020). IKEA, a furniture retail company, uses an AR catalog to help consumers visualize their space with IKEA goods (IKEA, 2013). Gaming companies such as Niantic, with collaboration with Nintendo and The Pokémon Company, launched Pokémon Go, an AR mobile app game, in 2016, which brought a sensational experience for many players worldwide.
Facebook, a social media giant, recently rebranded its company name and logo to Meta, shortened for Metaverse, a virtual world where people come together to work, learn, play, etc. Over the years, the company has been expanding its cross-platform capabilities to enable teleporting a VR world to another as smoothly as possible to heighten immersive experience (Meta, 2021).
When it comes to educational institutions, Knox Grammar School in Australia enriches students' minds by fully incorporating immersive experiences using AR and VR technologies, helping to expand the learners' perspectives (Immersive Learning Research Network, 2021). When applied in an educational setting, one of the advantages AR and VR technologies have is that they can be experienced together—meaning AR technology embedded in the VR technology by applying AR information into a VR world. For example, at Knox, kindergarteners experiment using AR to understand how the forces impact the world around us by first applying AR into the real world using the camera feature of a mobile device, i.e., bouncing ball. Then apply that experience into an underwater simulation setting using VR to see the differences in force, such as gravity. This integration of AR and VR is clever, helping learners connect prior knowledge to working knowledge.
AR and VR technologies have advantages and disadvantages. AR is often used with mobile devices to access the camera feature to augment the real world. No AR headset is required, but the virtual objects can only be viewed using the device's app. On the other hand, VR uses a VR headset, and the users can fully immerse themselves into the entirely fictional worlds. However, since both technologies heavily rely on the visual sense, the technology may not be for everyone.
While all the technologies are not perfect, used appropriately can bring new learning experiences to the learners, i.e., applying gamification into AR or VR for instructional design purposes. Aoyama and Tse (2017) highlighted the AR's flexibility to the learning experience by embedding digital information into the real-world setting, allowing learners to interact with the content and stay engaged. Using GPS location-based AR app, their research explored the effectiveness of applying the technology during the historical tours and emotional engagements the learners had during the gamification in practice. Aoyama and Tse recommended mixing the familiar with a novel in the learning experience by using unique yet straightforward graphics or short videos. In addition, to incorporate problem-based activities.
Knox Grammar School incorporated VR for the learners to curate a virtual tour of Canberra, Australia, during the pandemic since physical field trip was not permitted (Immersive Learning Research Network, 2021). Their approach offered freedom for the learners to explore and experiment using their creativity. Having the learners create their virtual world within the virtual space using avatars, already rendered 3D graphic assets in the program allowed this learning experience success. This affordance of technology where the learners do not need to code to create the VR experience is still relatively new yet powerful.
When thinking about the mobile app that integrates AR technology, Museum Alive AR with David Attenborough by Alchemy Immersive offers excellent information about the prehistoric world (see figure 1).
Figure 1
Museum Alive AR with David Attenborough

Note. [Museum Alive AR with David Attenborough]. (2021). https://alchemyimmersive.com/productions/museumalive/
Apple App Store: https://apps.apple.com/us/app/museum-alive/id1548503337
The target audience of the app is anyone nine years or older due to fantasy violence. The professionally recorded voice of David Attenborough and background sound provides an immersive experience. When first opening the app, it requests to access the camera feature of the mobile device to connect the real world to the virtual world using AR technology. The app can be used to bring the museum to your living space. The camera’s zoom-in and zoom-out features enhance the experience while listening to Attenborough’s narration.
The AR app that can be adapted for educational use is Google Translate (see figure 2). The app has a feature where the users can use a camera feature of a mobile device, and the app will translate the image into the language of your choice by overlaying AR.
Figure 2
Google Translate

Note: [Google Translate]. https://apps.apple.com/us/app/google-translate/id414706506#?platform=iphone
The target audience of the app is four years old and above. For beginning language learners, the app is appropriate to guess the meaning of a diagram or everyday object. In addition, the app can offer the learners a way to practice site translation for intermediate to advanced language learners.
With Meta in full swing to create virtual worlds for everyone globally, the VR-infused learning opportunities will expand significantly soon, which is exciting. However, this requires the instructional designers to always be on top of the current trends in technology. With the AR contact lens (Mojo Vision, 2020) soon available for visually impaired people, the VR contact lens is not too far behind, ideal for people who get motion sickness wearing the VR headset. The AR and VR learning experience will be a science-fiction come true in mixed reality.
References
Aoyama, R., & Tse, H. (2017). Using augmented reality and gamification to make history field trips more engaging for university students. International Conference on Language, Education, Humanities, and Innovation. https://icsai.org/procarch/6iclehi/6iclehi-050.pdf
[Google Translate]. https://apps.apple.com/us/app/google-translate/id414706506#?platform=iphone
IKEA. (2013, July 26). Place IKEA furniture in your home with augmented reality. [Video]. YouTube. https://www.youtube.com/watch?v=vDNzTasuYEw&t=1s
Immersive Learning Research Network. (2021, May 17). iLRN2021: Extended educational realities--virtual and augmented reality showcases engaging remote. [Video]. YouTube. https://www.youtube.com/watch?v=EGbiW3-QCC4
Meta. (2021, October 28). The Metaverse and how we’ll build it together—Connect 2021. [Video]. YouTube. https://www.youtube.com/watch?v=Uvufun6xer8
Mojo Vision. (2020). Augmented reality: A brighter future for the visually impaired. Mojo Vision. https://www.mojo.vision/webinars/augmented-reality-a-brighter-future-for-the-visually-impaired.
[Museum Alive AR with David Attenborough]. (2021). https://alchemyimmersive.com/productions/museumalive/
NASA Video. (2020, November 15). Augmented reality for drone air traffic management. [Video]. YouTube. https://www.youtube.com/watch?v=92YKZVlsrzg

The Prototype Idea for AR App
The target audience of this game is beginning interpreters working in a technology company. During the language interpretation, an interpreter must be able to read the speaker's body language and hand gestures unique to the industry to understand the situation. In this augmented reality game, the learners will select the background and how many people to appear in the scene. Then guess how many body language cues they can spot. Also, the learners can rotate the view 360 degrees for a closer look.
The beauty of this AR app is its flexibility.
Here are possible use cases for this app:
Cyber security training
Workplace harassments training
Cultural training
New hire training
Ideal for used with the Just-in-time training to reinforce the learning objectives.
