In Depth

Elite Engineering: The Brain of the F-35

April 14, 2015

The human brain relies on five senses: sight, smell, taste, touch and hearing. It takes information from each of these sources and analyzes the data to understand our surrounding environment.

Similarly, the F-35 relies on five sensors: Electronic Warfare (EW), radar, Communication, Navigation and Identification (CNI), Electro-Optical Targeting System (EOTS) and the Distributed Aperture System (DAS). The F-35 “brain”—the process that combines this stunning amount of information into an integrated picture of the environment —is known as sensor fusion.

LM Senior Fellow Tom Frey and Research Scientist Kent Engebretson are part of the world-class Lockheed Martin team of experts who have made sensor fusion a reality on the F-35.

LM Senior Fellow Tom Frey, left, and Research Scientist Kent Engebretson explain one of the thousands of algorithms that fuse the staggering amount of data the F-35 receives in-flight.

Defining “Fusion”

At any given moment, a huge influx of data flows into fusion from sensors around the aircraft—plus additional information from datalinks with other in-air F-35s. Fusion takes all information from those various sources and combines it into a centralized view of activity in the jet’s environment.

Many 4th generation aircraft were designed for a crew of two. The pilot flew, and the “back-seater” analyzed data displayed on various screens. For a single-seat jet like the F-35, the system must gather relevant data automatically and display it in a way that allows the pilot to fully concentrate on flying the mission ahead.

While the pilot flies, fusion actively interprets real-time sensor data to give him or her perhaps the most valuable advantage of all: reliable situational awareness.

Pieces of the Puzzle

F-35 fusion has the ability to take partial data from each sensor and combine it to make an accurate assessment. It not only combines data, but figures out what additional information is needed and automatically tasks sensors to gather it—without the pilot ever having to ask.

Given this unique capability, the way F-35 sensors had to adjust how they “think about” and report incoming data to take full advantage of the fusion system.

“Fusion is the core of our 5th Generation system,” Kent remarks. “We’re asking the sensors to send us not only their answer, but we want to know the reasoning and details behind that answer. That is what we combine during fusion to give us the whole picture.”

The F-35 changes the way data is displayed for pilots. The full Panoramic Cockpit Display (PCD) enables data from all sensors to be shown on one screen in simplified form, instead of multiple. It even allows each pilot to customize the size and layout of displays. This makes it much easier for the pilot to assess the situation and make smarter decisions in the battle space.

It’s All About Math

So, what is this entity that works so tirelessly to “fuse” all the information together?

The answer: math equations.

That’s right, thousands of algorithms encoded onto a standard processing chip simultaneously fuse the staggering amount of data. And what’s more: they are constantly changing.

Kent, Tom and their counterparts can take the software code they write, test it in simulators and make adjustments based on the lessons learned. This enables them to rapidly mature the fusion software along with the capabilities of the F-35.

“Fusion is easy when all the data agrees—but every now and then, there are discrepancies,” Tom reveals. “It makes it harder when sensors give misinformation or are in conflict.”

It’s math that figures out what data to believe, when to believe and how much to believe. No one knows this better than Kent, who has been the Target Identification (ID) expert in F-35 fusion for the last 13 years following his time in the U.S. Air Force.

“For ID fusion, it’s a lot of probability theory,” he shares.

While there are many standard equations at the base of F-35 fusion, the team creates faster or more efficient implementations to handle all of the aircraft’s fusion needs.

The Fusion Evolution

While the concept for fusion was first conceived in the 1970s on the F-15 program, no one ever fully succeeded in standing it up in an aircraft system until the F-22.

With 18 years spent as a representative on the F-22 fusion team, Tom is one of only a handful of people who have intimate knowledge of both the F-35 and F-22 fusion systems.

“Some innovations had to happen mathematically to deal with data the way they were sharing it before the F-22” he says. “By the time the F-22 came along, the computers and technology finally caught up, and we launched the first real 5th Generation fusion on an aircraft.”

That was “Fusion 1.0.” The F-35 takes it one step further.

“The F-35 not only has the ability to proactively collect and analyze data, but it adds the ability to share it amongst the fleet and work as a pack,” he explains. “That’s ‘Fusion 2.0.’”

When asked about what’s ahead for sensor fusion, both Tom and Kent see it continuing to evolve.

“We do things with fusion now that a decade ago, we said were impossible,” Tom elaborates. “When you have capability that no one ever dreamed of when it all seemed to be ‘too hard’—and it all of a sudden becomes available—it changes the way we operate and fight.”

Kent adds that, “I am very excited looking to the future, because there’s this influx of additional technology that will surely enable us to do even more.”