AUSTIN, Texas — In a classroom at the Armed Forces Reserve Center in Austin, Texas, a visitor climbed into one of 20 stations lining the walls and strapped on a virtual reality headset.
Instantly, he was transported into the cockpit of a Navy F/A-18 Hornet fighter jet. Over his right shoulder he saw the wing loaded with missiles. The whine of the Hornet’s engines and chatter from instructors piped directly into his ears.
Throttle, stick, heads-up display, instruments, the battlespace — it was all there, in a 360-degree display, everything down to the handle for the ejector seat between where his legs would be.
He grabbed the stick and throttle attached to the station, and the simulation began. An instructor told the virtual pilot to ease the stick to the left and bank the “plane.” All across his field of vision, even in the periphery, he saw the horizon turn sharply. And though he knew his physical body was perfectly still, on the ground at the Austin-Bergstrom International Airport, he felt his stomach sway as if he were actually banking a fighter jet.
A little later, the instructor told him to open up the throttle and pull back on the stick to execute a loop, and then guided him through a rudimentary barrel roll. All the while, his mind tricked him into feeling the sensations and excitement of executing those basic aerial maneuvers — and more importantly, he started to get the hang of the finer points on how to pull them off.
This could be how the Air Force of the future trains its new pilots.
The Air Force launched the first Pilot Training Next class in April to try to find a new, cutting-edge way to teach airmen, using advanced biometrics, artificial intelligence and virtual reality systems.
RELATED
Because the service has such a pressing need for pilots, especially fighter pilots, Air Force leaders are looking for ways to streamline the training pipeline and graduate more aviators each year. They want to find out if using an all-enveloping VR headset will create a faster, more effective way to train — though officials stress it isn’t meant to replace the traditional undergraduate pilot training program.
The Air Education and Training Command program integrates various technologies to produce pilots in an accelerated, cost efficient and learning-focused training environment. Students will still do many of the things they did before. For example, they’ll still go through drag training to learn to disconnect their parachutes as they are being dragged along the ground after landing. They will still climb into a hyperbaric chamber so they can experience hypoxia, or lack of oxygen in the bloodstream, in a controlled environment. And, of course, they will fly T-6 trainers.
The emerging technologies are being used to decrease the time and cost of training, without sacrificing depth of learning.
The Air Force began its VR training experiment with 20 pilot students — 15 officers and five enlisted airmen without college degrees. Thirteen of those students pinned on silver pilot wings and graduated Aug. 3, four months after beginning the class, according to AETC officials. Typically, pilot training takes an entire year.
But the service hopes what it learns from Pilot Training Next will also revolutionize training for airmen in scores of other career fields, such as medical or maintenance.
Pilot Training Next also tracks students’ biometrics and uses artificial intelligence to tailor the level of difficulty they face in different scenarios and teach them more effectively. Students wear a Zephyr “puck” near their heart, wearable technology that measures heart rate, breathing rate, pulse, and stress levels to see how well they’re responding to a task.
For example, it could detect if someone is growing too comfortable and throw a challenge at him. Maybe he loses one of his two engines, or the mission changes, or he runs into bad weather, enemy fighters or surface-to-air missiles. Or, if a student is becoming frustrated and discouraged, the simulator could back off and let him gain confidence.
When you stack this VR simulator up against older, legacy simulators — the kind that use projectors and hydraulics to move mock cockpits around — the difference is like watching a movie on an IMAX screen versus a 32-inch television, officials say.
“There’s a reason that people go to IMAX theaters,” said Maj. Scott Van De Water, deputy director of the Pilot Training Next program, in a recent interview. “When you go to an IMAX theater, and you’re watching the Blue Angels fly in formation, there’s a level of immersiveness. You look around, and you’re with the Blue Angels. That is compelling.”
Similarly, pilot students wearing the HTC VIVE Pro headsets see nothing but the virtual environment enveloping them, and hear the sounds pumped into their ears. This makes it easier for “suspension of disbelief” to take hold, and the students’ brains start to react as if they were actually in a cockpit.
“What we’re finding is that … some of the things that we’re expecting to be hurdles” aren’t a problem, Van De Water said. For example, “I can’t reach out and grab the gear handle [because] there’s nothing physically there. But because of the level of immersiveness in the visual environment, your brain makes the connections.”
Sometimes, he said, students get so lost in the simulator that they try to grab something in the virtual cockpit to help them turn around in the seat, forgetting that it’s not actually there.
On that day, three instructors guided about eight students through lessons on aerial combat. The students softly chattered to one another, occasionally whipping their goggled heads around without warning to look over their shoulders.
The Air Force considered several commercial off-the-shelf VR headsets, like the Oculus Rift, but chose the VIVE Pro because its organic light-emitting diode display had the best resolution. It’s not yet at perfect “eyeball resolution,” Van De Water said — there are still some details of instruments that are a little blurrier than they’d like — but it’s good enough, and the technology is getting better all the time.
Despite its greater authenticity, the new technology is much cheaper than legacy simulators. Each VIVE costs less than $1,000. And when the seat, computer, software, display screens for instructors, stick, throttle, and other equipment is factored in, each VR simulator setup runs about $15,000. So, the entire suite of 20 simulators in the “sim bay” cost about $300,000.
A legacy simulator, on the other hand, would cost about $4.5 million, he said. And students would still have to stand in line to get their shot at the controls, one after another.
In the Pilot Training Next classroom, there’s nothing keeping all 20 students from flying at the same time — even alongside or against one another in the same virtual airspace.
At the end of the day, they can squeeze in a little more flight time back at the dorm rooms. Every two students share their own additional VIVE Pro, stick and throttle setup at home for after-hours practice. The system’s AI will see if a student is developing a bad habit when practicing by himself and correct him before it sinks in.
Van De Water said that students in a traditional pilot training environment typically average 1 ½ sorties per day, both in the simulator and flying trainers such as the T-6 Texan.
Pilot Training Next students, on the other hand, average two sorties per day, he said.
And there are other advantages as well to this kind of a simulator, Van De Water said. If an aspiring pilot needs to practice flying a loop in a T-6, he’s got to wait until a Texan is available, fueled up and ready to fly. He’s got to spend time taking off and landing, and other activities that have to be done when flying a real plane, which limits how much time he can spend flying loops.
But in a VR simulator, he said, there’s no wait. The instructor can start him off in mid-air, and he can get right to practicing his loops. So when it comes time for that Pilot Training Next student to fly in a real trainer, he’s already got the fundamentals of flying a loop down cold.
“They have unfettered access,” Van De Water said. “They can come down [to work on flying] whenever they want, as opposed to having a very limited resource, that they only get a certain number of touch points throughout the course of instruction.”
And because the simulators are side-by-side, barely feet from one another, a single instructor can easily sit between two students, observing them both and offering guidance, Van De Water said.
Instructor pilot Capt. John “Flop” Joern said Pilot Training Next is a much less linear way to learn than the traditional model, which usually has a set, year-long syllabus of one lesson after another that must be followed.
But under Pilot Training Next, instructors can tailor the lessons to what each individual student needs, Joern said. That means, if a student is a natural at one flight technique, but is lacking in another, the instructor has the freedom to concentrate on the student’s deficient skill and not waste time on something he’s already good at.
“We found that just by unchaining the students from the syllabus, they can progress much faster,” Joern said. “I don’t need to go out [with a student] for the next seven rides and practice that one maneuver [he’s already mastered]. Whether it’s a cloverleaf, a Cuban eight, a barrel roll, the list goes on. We spend, in the traditional model, a lot of time trying to get students proficient at these maneuvers, when they really could be focusing that time on other things such as instrument flying, formation flying, or introducing new concepts to them.”
Jeorn also said that tailoring lessons to each individual student — as opposed to following a set syllabus — forces him to be more creative in how he teaches.
If the students can more quickly move on to greater challenges — such as flying with another aviator in the same airspace — they rise to meet those challenges, Joern said.
The expensive hydraulics used in a standard simulator to move a cockpit around don’t add much to the learning process, Van De Water noted. And as the graphics of a simulation get better and better, the value of recreating physical motion decreases.
“You can’t simulate five or six [G-forces], even with a motion simulator,” Van De Water said. “You’d have to put him in a centrifuge, or on a roller coaster. Our eyes do a pretty phenomenal job of tricking our brains into thinking that the motion is there.”
Lt. Gen. Steven Kwast, head of Air Education and Training Command, agreed, and suggested the days of the hydraulic-operated simulator that physically feels like the real thing may be coming to an end.
“The human brain does not care about motion,” he said. “You do not need perfect fidelity in the way it feels to your fingers, or the way it feels to your body. The human body learns those things fast. What the body needs is a lot of cognitive repetitions. So now, instead of spending millions of dollars on a simulator that is anchored in one location and can only have one person at a time in it, I can afford to give everybody their own simulator that they can take home with them in their pocket. And they can go through as many cognitive reps as they want.”
Sometimes, the accuracy of old-style simulators is “more than lacking,” Kwast said. For example, the F-35 simulator used by the Air Force is “nothing like the real aircraft.”
“Why? Because we can’t afford it, because it’s too expensive to upgrade the software in the simulator, and upgrade the software in the jet,” Kwast said. “So we upgrade it in the jet, so the jet is leading edge. But the entire training pipeline is training on something they know does not exist in the jet. … And when they get to their fighter squadron, they kind of have to start all over. What a waste.”
Pilot Training Next uses commercially-available flight simulator software like X-Plane and Lockheed Martin’s Prepar3D program to recreate dozens, or even hundreds of different civilian and military aircraft — anything from a T-6 Texan trainer, to a C-130 Hercules, to an H-60 helicopter, to an F-35.
And with all those options, Van De Water said, instructors can throw curveballs at a student who’s getting a little too comfortable, and put her in an unfamiliar aircraft. That way, she has to figure out quickly how fast to fly, how to land, and other techniques unique to that strange plane. This will make her more of a “generalist” with the aviation skills needed to fly most, or all airplanes — not limiting her to a specialization in one particular plane.
But while the first batch of Pilot Training Next graduates learned to fly much more quickly than students in the traditional year-long curriculum, Van De Water said the Air Force isn’t aiming to completely replace pilot training as it exists with this new program.
“The overall objective [is], how do we expedite learning, how do we increase the durability of the learning?” Van De Water said. “As they’re learning these concepts, how do we arrive at the end point faster? How do we become more efficient with our resources, not for the sake of resources, but for the sake of student time — getting them out to the Air Force … quicker so that we have more time with them as viable combat aviators.”
The second test class of Pilot Training Next students will begin next January, Kwast said. When they’re finished, Air Force Secretary Heather Wilson and Chief of Staff Gen. Dave Goldfein will review the data from the two classes and decide whether and how to use this kind of a model throughout the Air Force.
Stephen Losey is the air warfare reporter for Defense News. He previously covered leadership and personnel issues at Air Force Times, and the Pentagon, special operations and air warfare at Military.com. He has traveled to the Middle East to cover U.S. Air Force operations.
