| Author: | Li, Zhimin |
| Title: | Seeing the unseen through fixation dynamics : large language model-augmented detection failure analytics in air traffic control |
| Advisors: | Li, Fan (AAE) |
| Degree: | Ph.D. |
| Year: | 2026 |
| Department: | Department of Aeronautical and Aviation Engineering |
| Pages: | xx, 181 pages : color illustrations |
| Language: | English |
| Abstract: | Detection failure (DF) poses a critical challenge in air traffic control (ATC), where undetected alerts threaten decision-making and safety. Since DF is an unavoidable byproduct of human behavior in dynamic contexts, its accurate, proactive identification is vital, protecting thousands of lives under controllers' care. DF arises from inattentional blindness due to attentional constraints, yet encompasses broader perceptual and cognitive factors. Conventional methods rely on the presence or absence of fixations on the target to measure DF, but they overlook the cognitive continuum of visual processing, where fixation does not guarantee awareness, and human peripheral vision may detect target without direct fixations. This gap underscores the need for a structured DF classification that accounts for these divergent cognitive pathways and address complex "look-but-fail-to-see" and "seeing-without-looking" scenarios. Further complicating DF analysis is the spatiotemporal uncertainty of alerts in ATC, the dynamic nature of pre-alert attentional states, and class imbalance in DF datasets, which degrade the recall and F1-scores of current methods, necessitating a comprehensive analytical framework. This thesis proposes systematic DF analytics leveraging fixation dynamic-based feature representation and large language model (LLM)-augmented intelligent analysis. First, a gaze-behavioral DF classification grounded in Endsley's situational awareness theory is developed, categorizing DFs into distinct types by awareness levels and identifying their key gaze indicators through a four-phase theoretical framework. Second, an ATC-specific hierarchical DF recognition framework is developed, integrating peripheral vision tracking, spatiotemporal uncertainty of alerts, and multitasking demands. Third, proactive DF prediction is achieved through a fixation-state transition feature set capturing pre-alert attention shifts, coupled with an LLM-driven probabilistic threshold tuning method to adaptively calibrate class-specific predictive probability boundaries. Several ATC-specific case studies were conducted to validate the proposed methods. The results revealed significantly distinct gaze patterns across different DF types. The ATC-specific hierarchical DF recognition framework further identifies DF types through an Adaptive Symbolic Alert Detection method (95% precision in alert-region annotation) combined with LLM-based attention evaluation for reliable labeling, and a novel fixation coordinate-sensitive feature set that effectively captured spatiotemporal-frequency characteristics of diverse DF types (93% accuracy vs. 84% traditional features). Additionally, in proactive DF prediction, our proposed feature set outperforms existing features (78% vs. 71%–73% accuracy), with the LLM-enhanced prediction method further enhancing overall accuracy and significantly boosting DF-type-specific recall and F1-scores, showcasing superior robustness in class imbalance. In conclusion, this research establishes a systematic paradigm for analyzing a broader and structured DF, introducing a streamlined feature set and a novel LLM-driven pathway for safety-critical systems. These developments lay the groundwork for designing intelligent, human-centered ATC support systems to improve alert detection reliability and aviation safety. |
| Rights: | All rights reserved |
| Access: | open access |
Copyright Undertaking
As a bona fide Library user, I declare that:
- I will abide by the rules and legal ordinances governing copyright regarding the use of the Database.
- I will use the Database for the purpose of my research or private study only and not for circulation or further reproduction or any other purpose.
- I agree to indemnify and hold the University harmless from and against any loss, damage, cost, liability or expenses arising from copyright infringement or unauthorized usage.
By downloading any item(s) listed above, you acknowledge that you have read and understood the copyright undertaking as stated above, and agree to be bound by all of its terms.
Please use this identifier to cite or link to this item:
https://theses.lib.polyu.edu.hk/handle/200/14614