Partner Spotlight: NLR

About NLR and their role in RESPONSE

The Royal Netherlands Aerospace Centre (NLR) is an ambitious applied research organisation, driven to make aerospace more sustainable, safer, more efficient and more effective through innovation. To this end, NLR offers specific, innovative solutions and technical expertise. Within the RESPONSE project, NLR’s Safety & Human Performance and Air Traffic Management departments are involved.

In Work Package 1 (WP1) – Pilot Incapacitation Transition Monitoring, NLR is investigating the extent to which functional near-infrared spectroscopy (fNIRS) in combination with eye tracking can be used as a reliable indicator of fatigue, and to what extent this combination of tools can predict whether a person is becoming fatigued before they are too tired to fly.

In Work Package 2 (WP2) – Safe Return to Land CONOPS Evolution, the NLR team is working on enabling a “safe return to land” from the air traffic control perspective, in situations where a pilot has become incapacitated and is no longer able to control the aircraft.

In RESPONSE, different Concepts of Operation (ConOps) are written about what to do in case of pilot incapacitation in the current dual pilot operations (DPO), but also for more automated (by the help of a digital assistant) concepts.

Current Research: Fatigue Detection and Safe Return Operations

NLR is currently conducting a flight simulator experiment in which subjects fly a scenario while fNIRS and eye tracking data are being recorded. The 1.5-hours scenario is designed to have varying task load, inducing boredom and mental fatigue in subjects whilst also studying their performance when faced with sudden high-task load conditions after a prolonged period of low workload. Throughout the flight, the flow of oxygenated blood to the brain (fNIRS) is measured as a primary indicator of activity, and the percentage of eye closure (measured via eye tracking) is used as a primary indicator of fatigue. Data analysis will commence once all data has been recorded.

The experiment aims to determine whether fNIRS and eye tracking provide similar insights into fatigue levels in operators or whether they can be seen as complementary to each other. One may provide an indication of fatigue at an earlier stage, or one may be a better predictor in terms of the ability to increase performance and alertness levels again after a long and monotonous flight.

Further work is being conducted on five ConOps for different scenarios of operation, e.g. current dual pilot framework, different levels of automation up until single pilot operations, with respect to pilot incapacitation. The work on the first ConOps has been completed, which can be seen as the benchmark framework of the current DPO. Currently, work is being done on the second ConOps which, compared to the first, introduces digital assistants to support pilots and air traffic controllers, by reducing cognitive workload and enhancing situational awareness.

Contributing to safer operations

NLR’s results will contribute to the success of the project by addressing two critical questions related to operating under higher levels of automation and also Reduced Crew Operations concepts: how to effectively detect and predict pilot incapacitation due to fatigue, and how to ensure a safe landing in case of pilot incapacitation. Important are periods during which one pilot is alone in the cockpit, making it essential to monitor the pilot’s state to take immediate action in case of total or partial pilot incapacitation.

The RESPONSE project starts by identifying fatigue as one of the most common forms of pilot incapacitation. NLR’s study will help determine whether detecting pilot incapacitation due to fatigue can become more effective and stable when a combination of fNIRS and eye tracking is used, whether it is possible to identify when someone will become fatigued before they actually fall asleep, and whether someone will be able to act efficiently when needed after flying a monotonous trajectory for an extended period.

The results from the different safe return to land concepts of operations are needed to provide a safe landing in case of pilot incapacitation, by enhancing air-ground integration and reducing the workload for pilots and ATCOs through the introduction of advanced automation in air and ground digital assistants. NLR helps in constructing the concept of operations of the ground side, e.g. a ground side (ATC) digital assistant.

Research Challenges and Solutions

“In our opinion”, commented the leader of the NLR team in RESPONSE, Rolf Zon – “the biggest challenge that NLR is facing in the RESPONSE project is that we are conducting research on pilot state at a scientific fundamental level. The fNIRS and eye tracking data can be measured independently. Now we want to compare them and make an overall statement about the fatigue level of the pilot at the time of measurement, as well as a projection of the fatigue level into the future. Creating an algorithm that can do this in a robust manner will be challenging, but will make the approach much more operationally applicable.

To overcome this challenge, we will work with a team of specialists from different disciplines, including psychologists, human factors researchers, AI specialists, ATC researchers and pilots. Together, they will interpret the data and optimise the way the data is processed.

Ultimately, when the project is completed, the next step would be to apply the findings at an operational level. Therefore, the tools to measure fNIRS and eye tracking should be installed in the cockpit and should be less intrusive than they are now.

Another challenge for NLR lies in the way the ATC digital assistant should and can be implemented. Implementing such a system and getting the users onboard requires time and trust from ATCOs. The ATC environment has historically, and often rightfully, proven to be conservative, especially when it comes to any safety-related changes. The main way to overcome this conservatism is to show to the users what the benefits of such a system can be. Also, it has to be proven that such a system is stable, reliable and fail-safe and actively helps in reducing the workload of an ATCO in case of a pilot incapacitation”.

Impact on the Future of European Aviation

We expect that the RESPONSE project will have a significant impact on European aviation by delivering a robust method for measuring fatigue and reliable means to automatically and safely land the aircraft once pilot incapacitation is established. If successful, this will be a key step towards enabling safe flights with increased levels of automation or even with reduced crew concepts. We do not believe that all the necessary steps for reduced crew operations or single pilot operations can be taken within a short timeframe. We are convinced that other challenges related to reduced crew operations, which are beyond the scope of this project, need to be addressed first. Examples include cross-checking, which a pilot cannot do alone, or dealing with complex technological failures that a single pilot needs to solve alone.

However, we’re confident that the outcomes will contribute to a better assessment of pilot states and that all the findings of the project will be applied, provided that it can be done with an equivalent, or even higher, level of safety.