The future of pilot training could be found in a pair of virtual reality goggles.
When 15 officers and five enlisted airmen convene next month in Austin, Texas, for the Air Force’s new Pilot Training Next program, they could help create a cutting-edge method of teaching airmen to fly ― one that uses advanced biometrics, artificial intelligence, and virtual reality systems such as the Oculus Rift.
“We want to know, in the pilot training context, what this technology can do for us,” Lt. Col. Robert Vicars, director of the six-month program, said in a Jan. 9 interview. “What you get when you use VR is … a more immersive simulation experience than your legacy simulators. With how fast VR is advancing, it’s cheap and it’s pretty ubiquitous.”
Each student will get his or her own $500 to $600 VR helmet, as well as a stick, throttle and rudder pedals to create the cockpit environment, Vicars said. Each student also might get a second helmet to practice at home so they don’t have to carry their equipment around, he said, but it hasn’t been decided yet. Other VR headsets that will being tested are the HTC Vive and the Varjo headset.
Such an immersive environment has a more physical effect on students than current simulators, where they climb into a mock cockpit and see a screen in front of them, Vicars said. That means the suspension of disbelief is much more effective.
“People will be in a VR environment and they’ll end up walking next to a cliff, and they will get nervous,” Vicars said. “It has a bit more impact on our mind than I think the legacy simulators do. We want to know if we can leverage that for pilot training and build more capable simulators.”
And the AI will track students’ biometrics ― including their stress or emotions ― and tailor the simulation environment to optimize the stress load and more effectively train them, Vicars said. For example, if the simulation is too easy, the system could detect if someone is getting bored and make it tougher in order to hold their interest. But if it’s too hard and students are showing signs of frustration, the simulation could ease up a bit.
While a lot of effort has been put into the legacy simulators to make their faux cockpits look like the real thing, it might not be engaging enough to keep students’ attention and keep them from getting tired.
“It’s very realistic, but not that engaging,” Vicars said.
VR simulators might sacrifice some realism, he said. They’ll use digital renderings, and although they will have sticks and pedals, there won’t actually be switches to flip. But they could prove to be a more engaging and effective trainer ― and less tiring ― simply because they’ll be more fun.
“People sit down in front of [video] games for hours and hours,” Vicars said. “It creates a sense of enjoyment. There’s just something fun about it that you don’t get in the legacy sims. There’s this internal energy you get from experiencing it that way.”
One of the key things Vicars hopes the test will show is whether VR’s immersive experience will lead to students performing better when they actually enter a cockpit. For example, will a student who’s wrestled his way out of a dangerous dive simulation in virtual reality learn more about how his body registers fear and alarm in that situation, and as a result, react more coolly if he has a similar emergency in a real plane?
Vicars acknowledges that this test might not work, but even if it doesn’t, he hopes it will lay the groundwork for eventually making such technology ubiquitous in pilot training.
“Does the immersive experience equate to the performance of an airplane?” Vicars said. “We’re asking something very difficult of the technology, which is, make military pilots in six months. And that’s a lot to ask. For the 1,000 reasons this technology probably won’t work, we’re hoping to find all 1,000, and fix them.”
The program will include a mixture of VR simulations and actual flights in the T-6 trainer, Vicars said, and part of the study will be to determine what the most effective balance is between the two. Vicars said previous studies on legacy pilot simulations showed that students need to spend about 1/3 of their time in the plane, but VR could be different.
If the immersive VR environment works, students might not need to fly as much, he said. But some aspects of pilot training must be done in an actual plane, he said, so the study needs to figure out what those are.
This technology could, for example, replace the so-called “chair flying” training procedures that pilots like Vicars used to do when they were students. When “chair flying,” students sit and look at a piece of paper mocked up with cockpit controls, and repeatedly go through their ground operations procedures, touching the “switches” they need to flip, in the right order, over and over to build muscle memory.
But in a VR environment, a student could use the VR headset to move his head around and look at the switches and run through that checklist. Sensors mounted on the helmet will monitor where students’ hands move and register when they flip switches. Vicars said gloves would likely be too cumbersome when trying to control the stick and throttle.
If this study shows promise, other career fields could adopt VR training, Vicars said. Explosive ordnance disposal technicians could learn how to defuse bombs in a safe, virtual environment, or dentists could learn how to care for patients’ teeth.
This program first entered the public spotlight when a Nov. 30 email emerged from Maj. Gen. Timothy Leahy, commander of the Second Air Force, that said the program will provide data to Air Education and Training Command “on the potential for enlisted members to train to fly modern combat aircraft.”
AETC later said that while the program includes teaching some enlisted airmen to fly, it isn’t meant to create enlisted aviators. Enlisted participants would move on to their predetermined technical training after finishing Pilot Training Next, AETC said.
In the interview, Vicars said his job is to study what the technology can do, and that the Air Force will decide whether to consider enlisted pilots. But, he said, the Air Force needs to look at different, non-traditional demographics to see how they learn.
“This absolutely will broaden the idea of who can become a pilot,” Vicars said. “Does that suggest that there will be enlisted [pilots]? I don’t necessarily think that would be the conclusion to draw now, for sure. But we’re certainly looking at how we broaden the talent pool in our selection and accessions domain.”
Stephen Losey is the air warfare reporter for Defense News. He previously covered leadership and personnel issues at Air Force Times, and the Pentagon, special operations and air warfare at Military.com. He has traveled to the Middle East to cover U.S. Air Force operations.
