One of the most compelling questions in human-centric AI is how to detect signs of strain or overload—without adding more complexity to the operator’s task. In Air Traffic Management, where timing and attention are critical, the answer may lie in combining unobtrusive sensors with intelligent data processing.

During the recent face-to-face DIALOG meeting in Oslo, Zia Uddin (SINTEF) shared key developments from Work Package 2, which focuses on the real-time assessment of air traffic controllers’ (ATCOs) attention and workload. This work underpins the Teammate Awareness Service, one of DIALOG’s three core AI components.

Sensing Workload Without Disrupting the Task

A variety of minimally invasive technologies are being explored to track cognitive state in real time:

  • Gaze and behavior monitoring: Eye-tracking and task-related behavior offer real-time insights into where a controller’s attention is focused—and how it shifts across activities.
  • Speech analysis: Subtle changes in voice—such as pace, pitch, or hesitation—can act as passive indicators of cognitive load.
  • Physiological signals: Heart rate variability and electrodermal activity reflect internal states, offering additional clues without requiring active input.
  • Facial expressions: Emotion-related cues from facial analysis help complement the understanding of mental effort and stress.

These data sources are non-intrusive by design—a key requirement in high-stakes environments like ATM.

Why Multimodality Matters

The power of this approach lies in data fusion. Isolated signals are informative, but it’s when they are combined—across visual, auditory, and physiological channels—that a fuller, more accurate picture of mental workload emerges. In fact, low synchrony between modalities often signals high workload. Machine learning models help detect these patterns and transform them into actionable insights.

From Monitoring to Support

The ultimate goal is not just to monitor, but to enable adaptive support: AI agents that can recognize when a controller is becoming cognitively saturated and adjust their assistance accordingly.

This work directly supports DIALOG’s broader ambition: creating AI teammates that are context-aware, non-intrusive, and truly collaborative—ensuring that support never becomes an additional burden.

🧠 DIALOG is a SESAR Exploratory Research project reimagining how humans and AI collaborate in air traffic control.
 💬 Want to learn more about our technical progress? Explore the project updates 
 📡 Follow us to stay updated on the future of ATM.