| Author: | Lau, Ka Lok |
| Title: | Artificial intelligence (AI) large language models (LLMS) for medical consultation and triage in the accident & emergency department (AED) |
| Advisors: | Hui, Vivian (SN) |
| Degree: | M.Sc. |
| Year: | 2025 |
| Subject: | Artificial intelligence -- Medical applications Hospitals -- Emergency services Emergency medicine Triage (Medicine) Hong Kong Polytechnic University -- Dissertations |
| Department: | Faculty of Health and Social Sciences |
| Pages: | vii, 56 pages : color illustrations |
| Language: | English |
| Abstract: | Purpose In recent years, large language models (LLMs) in artificial intelligence (AI) have emerged as powerful tools for understanding human language. LLMs can classify patient triage results in the Accident & Emergency Department (AED). This study aims to estimate the accuracy of LLMs in AED triage results and assess the clinical relevance of these results. Methods The triage data in seven categories of chief complaints in AED from the open dataset, PhysioNet, were utilized in this study. A multi-query script was employed to query the LLMs using retrieval-augmented generation (RAG) based on the AED Emergency Severity Level (ESI). The models were compared, and qualitative methods were employed to collect feedback from domain experts. Results Approximately 500 validation cases in AED were analyzed, representing the first existing results from LLMs. The total average accuracy rate was 94.8% after adjustments based on domain experts" comments. The BLEU-4 rate, which checks the similarity between LLM results and original AED notes, was 39.15%. The ROUGE-Lsum score, which assesses the similarity of summarization between LLM explanations and the AED ESI triage guidelines, was 40.65%. Conclusions This study demonstrates that current LLMs can generate reliable information for low acuity levels (2-5), as verified by domain experts in the AED, potentially reducing misdiagnosis and improving outcomes. Domain experts found the explanations provided by LLMs regarding triage outcomes to be comprehensible. Importantly, the triage results generated by LLMs, including those with hallucination' content, were reviewed by domain experts, thereby reducing the incidence of incorrect or nonsensical outcomes. |
| Rights: | All rights reserved |
| Access: | restricted access |
Files in This Item:
| File | Description | Size | Format | |
|---|---|---|---|---|
| 8953.pdf | For All Users (off-campus access for PolyU Staff & Students only) | 1.46 MB | Adobe PDF | View/Open |
Copyright Undertaking
As a bona fide Library user, I declare that:
- I will abide by the rules and legal ordinances governing copyright regarding the use of the Database.
- I will use the Database for the purpose of my research or private study only and not for circulation or further reproduction or any other purpose.
- I agree to indemnify and hold the University harmless from and against any loss, damage, cost, liability or expenses arising from copyright infringement or unauthorized usage.
By downloading any item(s) listed above, you acknowledge that you have read and understood the copyright undertaking as stated above, and agree to be bound by all of its terms.
Please use this identifier to cite or link to this item:
https://theses.lib.polyu.edu.hk/handle/200/14208