The US Military Is Setting Up Their Proto-Metaverse

Table of Contents

Two fighter pilots conducted on May 10 a Proto-Metaverse experiment at a high altitude. They wore special AR headsets to connect to a system that overlay a ghostly, shimmering image of a refueling aircraft flying alongside them in the sky a few thousand feet above the California desert in a pair of Berkut 540 planes.

The virtual tanker was then refueled by one of the pilots while the other stood by and watched. Welcome to the military metaverse, which is still in its infancy.

The metaverse craze isn’t limited to Silicon Valley these days. Many defense startups, contractors, and financiers are increasingly talking about the metaverse, even though its definition and utility aren’t always apparent, just as tech companies and corporations scramble to devise strategies for virtual worlds.

The fundamental technologies required for the metaverse—augmented and virtual reality, head-mounted displays, 3D simulations, and artificial intelligence-powered virtual environments—are currently in use in the military industry.


The end result is less polished, adorable, and roomy than Mark Zuckerberg’s concept of a virtual world, but that’s part of the point. Even if it stutters in the civilian realm, the underlying technology has a strong chance of taking off.

Fighter pilots have been able to practice dogfighting against simulated opponents, such as Chinese and Russian jets, using a combination of augmented reality, artificial intelligence, and video game visuals.

The company behind the technology, Red 6, claims that it provides a significantly more realistic examination of a pilot’s ability than a traditional flight simulator. “We can fly against any attack,” says Daniel Robinson, the company’s creator and CEO. “And that threat may be managed remotely by a person or by artificial intelligence.”

In comparison to consumer AR or VR headsets, Red6’s AR technology must perform in more harsh environments, with reduced latency and higher dependability. Robinson goes on to say that the business is currently developing a platform that will allow a variety of scenarios to be depicted in augmented or virtual reality. He explains, “What we’re developing is actually a military metaverse.” “It’s like a sky-based multiplayer computer game.”

Some of the most recent military systems already use metaverse-related concepts. For example, the high-tech helmet for the new F-35 fighter plane incorporates an augmented reality display that overlays video feeds from around the aircraft with telemetry data and target information.

The US Army stated in 2018 that it would pay Microsoft up to $22 billion to construct a version of Microsoft’s HoloLens augmented reality system for warfighters called the Integrated Visual Augmentation System (IVAS).

Defense Tech and the Proto-Metaverse

In recent years, virtual reality and augmented reality have become commonplace in military training. Project BlueShark, a technology that allows sailors to control ships and collaborate in a virtual environment, was developed in 2014 by the Office of Naval Research and the Institute for Creative Technologies at the University of Southern California.

Another attempt, known as Project Avenger, is presently being used to assist in the training of US Navy pilots. Virtual reality is being used by the US Air Force to teach pilots how to control aircraft and missions.

Veterans with chronic pain and post-traumatic stress disorder can benefit from virtual reality therapy. Boeing has also designed an augmented reality environment that allows mechanics to practice working on planes before boarding a real one.

The US military has recently begun to investigate more complicated virtual worlds. There’s also a rising interest in connecting and integrating virtual worlds in a metaverse-like manner. The US Air Force convened a high-level conference through the virtual environment in December 2021, comprising approximately 250 persons from places ranging from the United States to Japan.

“The promise is integrating these technologies,” says Caitlin Dohrman, general manager of Improbable’s defense division. Improbable, which develops virtual world technologies and has created sprawling virtual battlefields with over 10,000 individually controlled characters for the UK’s military wargames and also works with the US Department of Defense, has created sprawling virtual battlefields with over 10,000 individually controlled characters for the UK’s military wargames (DOD).

“It’s an extraordinarily sophisticated form of simulation,” Dohrman says, “particularly given the fidelity that the military requires.” “You can have live players participate in the simulation, or [characters] can be AI-enabled, as the military frequently does.”

Oculus VR

According to Palmer Luckey, the founder of Oculus, a virtual reality firm bought by Facebook in 2014, Zuckerberg’s decision to go all-in on VR and the metaverse sparked a lot of excitement in the commercial sector. “A week or two later, investors are asking everyone on their quarterly corporate calls, ‘What’s your metaverse play?'” he says.

Luckey cofounded the defense firm Anduril in 2017. Despite all of the current metaverse excitement, he believes there is significant defense potential, mainly due to the importance and cost of military training. However, he claims that the technology does not have to be hyper-realistic to be beneficial, and that Anduril should only employ it when absolutely necessary.

He claims that “what we’re doing with VR is something that is uniquely better than any other alternative.” This may involve employing virtual reality to train people to operate Anduril’s drones or displaying information about a region based on data from sensors on the ground, according to him.

Red6’s AR technology

Newer military systems, like Zuckerberg’s projected metaverse, rely significantly on AI to be effective. Red6’s AR technology was utilized in October 2020 to pit a real fighter pilot against an AI-controlled aircraft produced as part of a Defense Advanced Research Projects Agency (DARPA) AI dogfighting study.

Through trial and error, the AI top gun, designed by another firm called EpiSci, learned how to outmaneuver and outgun an opponent. The AI pilot finally achieved superhuman abilities and was able to consistently outperform its human opponent.

Another DARPA project, Perceptually-enabled Task Guidance, attempts to develop an AI assistant that monitors what a soldier is doing and provides guidance via speech, sound, or visuals. Unlike Boeing’s augmented reality system, which only functions in a restricted environment, such a system would have to understand the real world.

The actual benefit of the technologies being examined by the military, according to Bruce Draper, the DARPA program manager in charge, lies in the melding of the real and the virtual. “The metaverse is primarily virtual, and virtual worlds are useful for training,” he adds, adding that “we exist in the physical world.” “It’s not about an abstract metaverse in the military arena,” says the author.

Challenges encountered

However, attempts to integrate the virtual and actual worlds have run into difficulties. A leaked Microsoft letter from March 2022 allegedly revealed that staff working on IVAS, the US Army’s equivalent of the HoloLens AR headgear, expected it to be poorly welcomed by consumers. And, according to a DOD audit revealed in April 2022, the US Army may be wasting money as a result.

Several statements from high-ranking army officers praising the IVAS’ promise were shared by Jason Kuruvilla, a senior communications manager at Microsoft. He also cited a DOD report from 2021 that emphasizes the significance of creating IVAS quickly so that issues can be dealt out along the way.

The military metaverse’s proponents have only grown more confident as a result of such high-profile and costly initiatives. “I know this is the future of military training,” says Doug Philippone, global defense director at Palantir, a defense firm that has backed Anduril.

Philippone is also a co-founder of Snowpoint Ventures, a venture capital firm that has invested in Red6. “However, I view it as the way the military fights and makes decisions in the future.” As a result, it’s not only about battling; it’s also about making choices.”

Anduril, according to Luckey, is already developing technologies that might be used in training missions and combat. “The next big step for us, which I’m really thrilled about,” he says, “is taking data from our core product and feeding it to heads-up displays that front-line troops will be able to wear.”

But it’s unclear how much of this cutting-edge technology makes it to the front lines—or even into training exercises. Sorin Adam Matei, a professor at Purdue University in West Lafayette, Indiana, who has designed virtual warfare training platforms for the US military, believes that the technology deployed will be far simpler than proponents of the metaverse believe.

He speculates that a less complex version of the IVAS headset could be fitted with an AR rifle scope in the future. “The last thing you want to worry about when you’re out there shooting and being shot at is another piece of equipment,” he says. To be beneficial, technology does not need to be as expansive as a metaverse. “We need to think about this metaverse idea a little bit more—powerful, it’s but it has its limitations.”

Do you have any thoughts on this article? Let us know what you think in the comments section below.

Here is another article that you may find very interesting.

Leave a Comment