New rollout for the Army’s $22 billion ‘mixed reality’ combat goggles

FORT BELVOIR, Va. — The Army’s mixed reality goggle is headed to soldiers in a three-step process in which developers expect two early versions released next year will help with redesigns for a third version to roll out across the Army.

The Integrated Visual Augmentation System, or IVAS, is a nearly $22 billion program that the Army is developing to bring night vision, thermal vision, tactical edge computing and the situational awareness of a fighter pilot down to the lowest-level infantry soldier. The device will likely be the most advanced single technology ever fielded exclusively to the close-combat, squad-level soldier in military history.

The Army will deliver 5,000 IVAS 1.0 versions and another 5,000 IVAS 1.1 versions this coming year, said Brig. Gen. Christopher Schneider, the Program Executive Office-Soldier commanding general. Previous budget reports indicate the Army will purchase 40,000 devices or more.

The combined 1.0 and 1.1 fielding is a delay from the previous first unit equipped planned date of September 2022, which was delayed from another previous initial fielding planned for late 2021. But the short-term holdup is critical to ensure soldier acceptance of the device, according to Army officials.

Schneider spoke with Army Times about the IVAS program recently, noting the new rollout plan will field the two existing versions and begin developing the IVAS 1.2 version in parallel with those first fieldlings. IVAS 1.2, which will have a new low-light sensor and hardware redesign, is scheduled to field in the fiscal year 2025.

“We don’t want to rush IVAS to the field until it’s ready,” Schneider said.

The Army will send 5,000 IVAS 1.0 devices to units in Training and Doctrine Command, such as the Maneuver Center of Excellence, for use in training, schoolhouses and experimentation.

The 5,000 IVAS 1.1 devices will go to not-yet-identified operational units and can be used in garrison, field training and on combat deployments. Units such as the 82nd Airborne Division, 75th Ranger Regiment and 101st Airborne Division often receive such technology first, due to their unique missions.

The 1.0 and 1.1 versions look similar from the outside. Both have night vision as good or better than most fielded systems. Both also have thermal sights in the same device, present only in one other goggle that’s had limited fielding to a select group of close combat forces, the Enhanced Night Vision Device-Binocular.

Both versions also allow for “passive targeting.” This lets users identify aiming points and targets without being seen by other night vision users.

“If we have to plan to fight a near peer threat, they’re going to be equipped with night vision, too. And if you use a laser, they see a laser too,” Schneider said.

With passive targeting, that’s not a concern.

“They have no idea you’re targeting them until a bullet hits them,” Schneider said.

The challenge is that the IVAS is expected to be more than a night vision goggle. It’s designed to provide mixed reality, information sharing, situational awareness tools and a host of other capabilities to the individual soldier.

“People think of IVAS as a night vision goggle,” Schneider said. But it’s more about learning “how to use the cloud at the tactical edge. How to train with the device, execute missions and mission planning.”

What the Army wants

IVAS is a standalone system that contains situational awareness tools in one package, beyond any of the Army’s current systems, including Nett Warrior, a long-running soldier-borne computer in a smartphone or tablet.

The device also works as a vehicle, ground or air integration tool for dismounted soldiers.

That’s part of ongoing work with soldiers integrating the IVAS with helicopters, the Bradley Fighting Vehicle and the Stryker, Brig. Gen. Larry Burris, Cross Functional Team-Soldier Lethality director and Infantry School commandant told Army Times recently.

Soldiers using wireless routers in the Bradley were able to see the same view as the vehicle commander, driver or gunner through their optics, Burris said.

But the most success so far, he added, has come with the Stryker.

“You look around and it’s as if there’s no armor on the vehicle,” Burris said, referencing what it’s like to wear the device from the point of view of vehicle occupants. “I turn my head to the right and it automatically detects the camera that’s off the right side of the vehicle.”

And in all platforms, soldiers passed data and used voice communications with the device, he added.

That work has involved PEO-Ground Systems and the Army’s Combat Capabilities Development Command.

With the current fielding plan, Burris expects to see novel applications and get more feedback from TRADOC and operational units as they use the device.

Mixed reality is a kind of augmented reality common now in gaming devices that places simulated objects, terrain or characters overlaid on the real world. It gives soldiers a tool only available now in laboratories or training facilities.

That feature can be used on the fly, in a barracks parking lot, on a training range or hours before an actual mission for rehearsal.

“We can pipe in ISR feeds into the device, we can scan areas with the device and build 3D models,” Schneider said.

The simulation tech is promising but still needs some work. Schneider said indoor facilities are no problem for scanning, mapping and rehearsals. But outdoor locations challenge the current technology.

If a soldier scans an area, then a car moves, or the sun shifts and shadows change, it can alter a soldier’s view. But, Schneider said, once resolved, a soldier will be able to see a computer-generated avatar that can interact with the real world. That provides a realistic training option that previously required human role players.

Technology limits

But the Army still has some technical hurdles to overcome.

Current tech limitations create challenges to having both a digital low-light sensor and thermal sight in the same device with the level of data fusion, integration and information-sharing that the Army wants.

The legacy technology, known as I2, has been developed over nearly 70 years and uses an analog, tube-based design. That is an extremely capable technology for providing night vision, Schneider said.

“The current I2 systems, high resolution, works really well, but it doesn’t allow us to share or leverage what a soldier sees,” Schneider said. “You can’t employ computer vision software, you can’t employ (Artificial Intelligence) with the analog solution today.”

For example, thermal sights cannot see through glass, but low-light sensors can. Meanwhile, low-light sensors can’t see through dust, smoke or other obscurants, but thermals can. By combining the two, users get the best of both.

After 70 years of fine tuning, the I2 technology has nearly reached its optimum performance levels, Schneider said.

The tradeoff is that the existing low-light sensor and camera works on digital technology. That provides a good night vision capability, to a point. But the existing camera will drop off under dark, less than quarter-moon conditions. The legacy tech can work at those levels, but it does degrade. At a certain point, the digital tech basically stops working.

“We’re trying to replace that technology with something that’s new, we think we’ve finally found a digital low-light sensor that will do it,” Schneider said.

Army Chief of Staff Gen. James McConville restated his own commitment to the program and its eventual use, providing some context from his own career as a helicopter pilot, during remarks at the Maneuver Warfighter Conference at Fort Benning, Georgia, on Sept. 13.

“If you flew back when I was flying with night vision goggles in ‘81 they were big, full-face goggles and how we flew with them, I don’t know,” McConville said. “We had weights in the back of our head. It was not very smart, but we weren’t very smart then, we just did it.”

The days of pushing out hard-to-use new tech are no longer the norm, the chief said.

Program delays

The Army planned to have the device to soldiers in 2021, but problems with software reliability, display warping and moisture issues paused the plan. PEO Soldier conducted additional tests through most of the past year.

The Army pushed back release for additional testing, as well as some hardware and software fixes. Those included reducing the originally ambitious field of view to resolve image distortion problems. After further testing and funding approvals, the device release has again been pushed, now to 2023 with versions 1.0 and 1.1.

Most existing night vision goggles have about a 40-degree field of view, meaning you can see about a 40-degree “wedge” in front of the goggle. That does not cover a user’s peripheral vision.

IVAS and Microsoft developers of the HoloLens, the commercial device the IVAS started with, initially sought to double that to an 80-degree field of view.

But image distortion required a scale back to 70 degrees, still nearly twice the existing range, officials previously told Army Times.

A Department of Defense Inspector General’s audit of the IVAS program, released earlier this year, directed developers to seek more soldier input to avoid, “wasting up to $21.88 billion in taxpayer funds to field a system that soldiers may not want to use or use as intended.”

Despite dozens of soldier touch points, the report noted, the Army hadn’t determined how it would measure soldier acceptance of the device.

PEO Soldier spokesman David Patterson told Army Times shortly after the IG report released that officials recognized the importance of soldier acceptance of the goggle, but it was not the only factor.

Soldier feedback was incorporated into numerous design decisions and updates in the prototyping process, Patterson said.

The published IG audit included soldier feedback, but those portions were redacted from the publicly released version.

A test report planned for completion in October could determine the next round of funding. Members of Congress in both chamber’s appropriation committees recommended cuts to the program following previous delays.

The next funding installment in the current plan would be for more than $424 million.

Another report on the most recent reliability fixes and anticipated changes is due in October, officials said.

Changes

The IVAS 1.2 version will see a “form factor” redesign. The current systems use straps to fix it to the user’s face and over the helmet, almost like ski goggles, 1.2 will be helmet mounted, like currently fielded night vision devices.

A cable that in versions 1.0 and 1.1 runs down the front of the user and connects to batteries will be moved to the rear of the helmet off the user’s chest and shoulder. The 1.2 version will also be slimmer and lighter.

The entire IVAS 1.0 system, excluding the radio, is about 3.4 pounds. Of that, 2.4 pounds is on the soldier’s head.

In version 1.2, developers seek to cut that total weight to 2.85 pounds or less, same or better than the ENVG-B, officials said.

Humidity control and moisture problems noted in last year’s review have been fixed for IVAS 1.0 and beyond.

All three versions work with the Family of Weapons Sights-Individual and Rapid Target Acquisition software. FWS-I was included in the ENVG-B fielding and allows users to wirelessly link their goggle to their rifle optic to view targets without shouldering the weapon.

RTA uses software to cue a target within the optic or goggle view, much like what vehicle-mounted soldier systems do. The combination also gives soldiers a “picture-in-picture” view option that lets them see the terrain but also a smaller version of the optic view.

The current ENVG-B lowlight sensor does outperform the IVAS 1.0 lowlight sensor. But an improved sensor is planned for both 1.1 and 1.2 versions, officials said.

A lowlight sensor being examined for 1.1 already exceeds all other individual night vision optics, except for the ENVG-B.

Both Schneider and his predecessor at PEO-Soldier, Maj. Gen. Anthony Potts, repeatedly acknowledged unplanned delays in the program but defended some of those with the ambitious strides the organization is taking on the edges of existing technology to merge all the capabilities into one individual-soldier-borne device.

Despite those challenges, Schneider stressed in his interview with Army Times that the fielded IVAS devices need to work as advertised to achieve soldier acceptance.

“This plan allows that. We are listening, we’re going to fix the camera and we’re going to get the form factor right. We have to get this right for soldier adoption,” Schneider said. “And we’ve got to maintain trust with them that we’re doing the right thing. But we’ve also got to push, push the limits.”