A collaborative AR platform for training
maintenance technicians
One of the greatest logistical challenges Rolls Royce faces is training maintenance technicians across the world on their jet engines. Currently, Rolls Royce sends out their experts and practice engines to train technicians or flies technicians in to their offices in Indianapolis, Indiana for a 5-7 day course.
This was the prompt given to us for our industry class project by Rolls Royce and our idea aims to help solve the problem by creating a virtual collaborative training platform using Microsoft's Hololens.
4 weeks
(Jan-Feb 2019)
Aditya Sankhe
Taylor Childers
Ragan Kelley
Steven Peavey
Adobe XD
After Effects
Product Designer
Operations Manager
How can we improve the efficiency of the maintenance training process?
Based on my analysis and the requirement that Rolls Royce put forth, I decided that to accomplish the stakeholders' requirements, the design direction needed to accomplish the following objectives:
• Provide a way for people to remotely access training and stay updated on procedures
• Minimize the risk to both the trainees and the engines
• Improve the rate of assimilation of trainees by using different channels of communication
HoloLens' gesture recognition and object recognition creates an AR engine overlay for trainees to practice on naturally and without risk.
Using Microsoft's Remote Assist and collaboration tools, users can actively interface with trainers remotely while also seeing trainers movements as an overlay on their engines.
An automated tracker that recognizes task completion, automatically filling the list.
To get a better understanding of the current scenario I decided to focus on the following aspects:
We studied the existing training process to identify key pain points to address
To gauge the feasibility of our design we analyzed successful implementations from our competitors in the aviation industry.
.png)
Typical training consists of onsite courses spanning a period of 5-7 days at limited training facilities. In situations where onsite training is not possible, trainers and engines are transported out for remote training, which proves to be more expensive.
Not only does it become risky for trainees to practice and learn on working engines, but it becomes detrimental to the engine itself due to the unnecessary wear and tear involved.
We found that manuals and books were provided for references during maintenance which posed issues of translating for international technicians. There were also inconsistencies present with the way trainers taught and the information present in the manuals.
Through our findings, I was able to create iterations of wireframes. Testing with 5 participants showed promising results in terms of learnability, but the usability aspect of our design needed to be more understandable.
An collaborative virtual training environment that enables hands on training with Augmented Reality
Based on our analysis and the requirements that Rolls Royce put forth, I decided that to accomplish the stakeholders' requirements, the design direction needed to accomplish the following objectives:
I designed concepts integrating the features required by the stakeholders and created sketches illustrating the process and the aspects involved. While conceptualizing I realized the only way to meet the requirements in a way that did not compromise on the quality of training would be to use Augmented Reality.
Given the scope of the project, the features, and the projected timeline, I decided to focus on the HoloLens. After discussing my designs with the rest of the team, we agreed that this was the most viable direction and proceeded with the concept.
By creating a detailed and segmented 3D model of the engine and interacting with it in physical space we can eliminate the problem of physical dependency and the risk associated while maintaining the quality of learning.
We can also create a collaborative environment by syncing two HoloLens devices around the engine, acting as an anchor.
I proceeded to make a user flow to visualize the process and to help me design the screens for each page.
Since there was no easy way to prototype our concept I decided to create hybrid prototypes for my phone that would help me get the sense of usability for the screens. Since there was no external input device, there were just two ways of interaction:
1) Direct Manipulation
2) Finger Ray Casting
I identified that one of the major drawbacks of having such limited hands on training leads to a lack of practice. By providing access to unrestricted, risk free practice modules, as well as live guided instructions with Microsoft's remote assist functionality, we can add to the value proposition.
Syncing the technician's and the instructor's HoloLens, we can create a shared virtual space anchored around the 3D model of the engine itself. Doing so will bring the level of granularity as close to the existing system as possible.
During the pre-recorded practice sessions, the technicians can select the type of scenario they want to practice.
An equipment check is done prior to beginning, so that the technician is prepared with the right tools.
Finally, an automated assistant popups for the technician to know which step needs to be performed along with an illuminated element on the engine to highlight the placement and the motions of involved.
Collaborative Augmented Reality Training for Rolls Royce, is the next step in the evolution of maintenance training. It helps technicians practice where ever and when ever they want, with step by step instructions along with 3D models to enhance the learning experience without the any of the risk involved.
1. We eliminated the physical dependency on engines, the transportation costs as well as the risks involved.
2. We also added improved the obsolete and vague learning structure by a visually enhanced and action-oriented instructional system.
Making our platform more accessible means that it should be available for a wide variety of users and devices. Expanding to mobile options will be the next step, which will bring more challenges and opportunities.
Using audio and speech to communicate more freely and practice pronunciation will be fundamental to speaking any language more fluently. To do so, discovering strategies that allow users a form of digital validation of their speech will be instrumental.
Creating samples of the training process and testing the basic usability of the HoloLens will allow us to pin point the next steps and formulate a comprehensive plan.
Companies like AirBus and Boeing can be core partners for this training process and help scale the business.
There were quite a lot of speculation at the time surrounding the launch of the HoloLens 2 with enhanced functionality in late February of 2019. Looking back at the the feature list, the technical feasbility of our project has improved significantly and the best move forward would be to develop and prototype using the HoloLens 2.
I pushed myself to think beyond the ordinary at the same time think about the business feasibility of the project. I realized that AR is a medium of communication and needs to be used as such.
Head on over to my other projects to check out more of my work!
