How Does Mixed Reality Expand on Augmented Reality. Recently, Mixed Reality (MR) has attracted a lot of attention.
MR will be explained in this article, along with examples of its use in business.
Additionally, it discusses the differences between VR and AR, MR’s merits, and its relationship with the Metaverse, which has been a hot topic in recent years.
You can find examples of MR business implementation here if you wonder, “What is MR?”
What is a Key Feature of Mixed Reality?
Mixed Reality refers to goggles that recognize the shape of actual space (spatial mapping) and project holograms onto a display so they can be viewed in the real world. Technology is referred to as such.
Using a device’s camera or sensor, spatial mapping captures the shape and distance of floors, walls, desks, and other objects in three dimensions. Mixed Reality is a technology that combines Reality and Virtual Reality.
As an example of MR, a service that turns a car into a virtual object and displays it in real space, reproduces the vehicle in the same size as the actual car, allows you to view the exterior from any viewpoint, and allows you to open the door and see inside. It might be mentioned.
By using MR technology, it is now possible to provide experiences previously only possible with natural objects.
The characteristics of MR make it an appealing next-generation technology that combines real space with holograms to visualize virtual information in various business settings.
You will better understand MR by watching the video below that introduces Canon’s MR goggles.
How MR works
A unique head-mounted display (MR goggles) is used to experience MR.
Sensors and cameras in the MR goggles recognize the shape and size of the actual space (spatial mapping) and display a hologram on the see-through display part.
Using this mechanism, the hologram overlaps in real space, making it appear accurate.
It is also possible to check the hologram while walking around since the position of the person wearing the MR goggles is also accurately detected.
In other words, the positional relationship between the user and the hologram affects each other, so one of its significant features is that it can be viewed from any angle and that the user can also perform operations that directly interfere with the hologram, such as enlarging, reducing, and rotating it.
In addition, users in remote locations can share the identical hologram in real time using MR devices. Users can view 3D holograms in the same space in all directions, regardless of location.
As a result, MR requires advanced scanning technology that accurately recognizes real space, precise 3DCG depiction technology, and an interaction system that allows Reality and Virtual Reality to come together.
Experience with MR requires the following devices
MR goggles
For example, let’s look at Microsoft’s Hololens 2 MR goggles.
It is a stand-alone MR goggle that does not require a computer or other peripherals and is a holographic device with Windows OS.
Hololens2 has multiple cameras and sensors that map the space around the user and project holograms.
As part of these features, it has hand-tracking and eye-tracking functions that do not require a controller. The user’s hand-tracking can touch, grab, and move holograms naturally.
In real time, the eye tracking function also recognizes the content the user is viewing and detects eye movements. You can move the cursor and follow the menu window utilizing just your eye developments.
As a result of these features, Hololens 2 gave the impression that it was a “device of the future” operated by the user’s hands and eyes.
For work efficiency, training, simulation, and other purposes, Hololens 2 is primarily used in manufacturing, medical care, and education for work efficiency, training, and simulation.
Apple’s MR headset “Vision Pro”
“Vision Pro” is Apple’s wearable computer, announced in June 2023.
MR became Apple’s entry into the MR business, a new product category, and a hot topic worldwide.
This is the first product you can ‘see’ rather than stare at. You can see and hear digital content as if it were actually in your room. A whole new way to relive your most precious memories.”
The iPhone introduced smartphones to the world in the past, and “Vision Pro” may do the same for spatial computing.
As with “Hololens 2,” the “Vision Pro” also displays a floating screen superimposed on the real world.
Below are some of the main features of “Vision Pro”.
- As a result of the external battery, the headset is lightweight. A dial on the headset allows you to adjust the ratio of Reality to virtuality.
- A 4K display with an unlimited field of view (FOV)
- When someone approaches you, you can see them on your screen using a feature called Eyesight.
According to these features, it is an unprecedented next-generation MR headset incorporating Apple’s unique ideas. Early in 2024, it will be available in the United States.
In the future, “Vision Pro” is expected to be used in various fields and industries, not just as an MR headset.
How Does Mixed Reality Expand on Augmented Reality
The difference between MR and VR/AR, which often need clarification, will be explained here.
VR (Virtual Reality)
Virtual Reality is an abbreviation for “Virtual Reality” and refers to an artificially created virtual space you can see and experience.
VR goggles are usually used to provide visual and auditory information in real-time in a 360-degree virtual space, giving users the feeling of being in the real world.
Reference Article : What is VR (Virtual Reality)? What are the 3 types of VR
AR (Augmented Reality)
AR stands for Augmented Reality and refers to a technology that “virtually augments” the real world by adding digital visual information.
In standard AR technology, CG content is displayed in real-world scenery when a specific location is captured with a smartphone camera, or information and explanations are displayed in response to the captured area.
Reference Article : What is AR (Augmented Reality)? Explanation of Augmented Reality Scope
Differences between VR, MR and AR
Because VR enters a virtual space and is entirely covered by CG, it completely differs from MR, which is based on real space.
This technology is similar to augmented Reality, which is also based on real space. Still, it differs from AR because it can directly interfere with holograms projected in real space.
As a result, MR attracts attention for its ability to display and manipulate real space and virtual information while influencing each other, which is impossible with VR/AR. As the need for MR becomes more active, technological innovations will continue to be promoted to achieve more accurate and detailed spatial recognition and reproducibility.
There has been a blurring of the boundaries between VR and AR, and the number of services that cross borders has increased, leading to the creation of a new term, XR (cross Reality).
There has been a lot of talk about the Metaverse in recent years, and further technological developments and collaborations with the Metaverse are expected in the future.
Reference Article : What is a Defining Feature of the Metaverse? How to Start a Business in the Metaverse
Table of comparisons for each technology
| AR (Augmented Reality) | MR (Mixed Reality) | VR (virtual reality) | |
|---|---|---|---|
| Overview | Integrating virtual and real information | Influencing each other and integrating real and virtual information | Separate from real space is virtual space |
| Immersion level | low | During ~ | high |
| Main compatible devices | Smartphone/tablet AR glasses | MR goggles | VR goggles |
| Introduction cost | cheap | expensive | expensive |
| Use scene | 1. Advertisement 2. Navigation 3. Entertainment 4. Retail/EC 5. Education | 1. Simulation 2. Medical care 3. Virtual exhibition 4. Training/training 5. Business efficiency | 1. Entertainment 2. Education 3. SNS 4. Sightseeing 5. Watching the event |
Advantages of MR
Productivity and skill improvement
Similar to traditional web conferencing, it is easier to identify problems when communicating with a realistic 3D hologram instead of a flat display.
3D holograms allow information to be shared with fewer differences in perception, improving productivity by preventing misunderstandings.
Additionally, MR allows trainees to repeat simulations from an actual perspective for tasks that are difficult to simulate in Reality or have a high degree of technical difficulty.
Enhancing the efficiency of education and training
We contribute to streamlining work education and training by providing 3D educational content that reproduces a variety of scenarios.
By overlaying the operating manual onto the equipment and animating the steps, anyone can intuitively learn how to operate the equipment.
The manual is displayed in your field of vision and can be checked hands-free, improving your work efficiency.
This makes MR a suitable technology for improving the efficiency of education and training environments.
Construction period and cost reduction
MR can significantly reduce construction costs and speed up construction by digitizing prototypes.
For example, the creation of a prototype, the identification of improvements, and the design of a prototype may be repeated.
Digital mockups, on the other hand, do not require an actual mockup, and they take much less time to create. This is especially useful when dealing with large products or buildings that are easier to grasp once you see them in person.
Support for accurate remote work
Improved accuracy and speed result from remote work support between MR users.
A hand-tracking function in MR makes it possible to reproduce and communicate manual tasks accurately.
VR can support similar business support, but even hand movements are expressed using CG, so advanced motion capture technology is required to convey minute finger movements.
In MR, the image can be reproduced with the hands of an actual person, allowing it to be communicated more easily and clearly than in VR.
Uses of MR in business
Education
When on-site support is complex, we provide remote support.
NASA, for example, introduced MR on missions to the International Space Station (ISS).
Astronauts can share real-time images from their HoloLens with the ground team, which can provide instructions and advice.
By following the instructions on the hologram, work can be performed more efficiently, and operational errors can be prevented.
Construction
With MR, completed images of constructions can be shared, and details can be observed from multiple angles.
Koyanagi Construction uses it to improve the efficiency of planning, construction, inspection, safety management and communication.
There are many applications for MR, such as visualizing the internal structure of a building and displaying traceability, which is vital for after-sales maintenance.
Medical care
MR is being used in an online telemedicine system to make telemedicine more accessible to people living on remote islands or in remote areas.
By utilizing MR to implement 3D and real-time observation and evaluation, Nagasaki University Hospital is hoping to provide rheumatoid arthritis patients with medical treatment that is more precise than ever. His opinion was expressed.
Plans include collaboration with 5G high-speed communication technology and education and research focused on practical training for students who have faced difficulties due to the Coronavirus pandemic.
Manufacturing
3D holograms are replacing traditional paper and web work procedure manuals to improve productivity, standardize technical skills, and improve proficiency.
Mechanics will be able to inspect vehicles by wearing HoloLens and projecting 3D work procedures and repair manuals onto the vehicle’s body while wearing HoloLens.
Anyone can understand the work procedure intuitively since it is displayed above the part of the car that needs to be inspected.
Thus, working hours are significantly reduced, which is expected to result in work-style reforms for mechanics.
Aviation
With MR goggles alone, the aviation industry can create virtual simulations that do not require actual aircraft or physical simulators.
With its MR application system, JAL presents a realistic virtual cockpit right in front of you.
Training can be conducted on real-size 3D CAD data in an immersive MR environment.
Additionally, operations can be performed by using a line of sight and gestures, and during training, exercises can be performed, such as observing visually and switching switches.
JAL evaluates the MR application as an educational tool that can improve pilot proficiency quickly.
Future of MR
It is predicted that MR will be used in an even wider variety of industries as technology develops.
As a result of its characteristics, MR is also capable of improving operational efficiency and saving labor, and it is expected to contribute to resolving future labor shortage issues.
Combining MR and AI (artificial intelligence) will enable more advanced and accurate information processing.
Workers are more efficient by displaying information analyzed by AI in real-time using MR.
MR’s future outlook is also attracting attention, and if higher-quality devices and creative usage methods can be developed, the expansion of industrial use will speed up even further.
Collaboration between MR and Metaverse will allow it to be used in business settings and in the daily lives of general consumers.
