| Author: | Yin, Yue |
| Title: | Human-guided robot programming and learning for smart manufacturing : a mixed reality-assisted digital twin approach |
| Advisors: | Zheng, Pai (ISE) |
| Degree: | Ph.D. |
| Year: | 2025 |
| Department: | Department of Industrial and Systems Engineering |
| Pages: | xv, 121 pages : color illustrations |
| Language: | English |
| Abstract: | Traditional robot programming methods often lack intuitiveness and adaptability, failing to meet the growing demands for flexibility and efficiency in the mass personalization paradigm. In addition, insufficient attention has been paid to exploring the role of human interaction in robot skill acquisition and the methodologies that support it. To address these challenges, this study presents a systematic approach built on mixed reality-assisted digital twin (MR-assisted DT) technology, enabling non-expert users to guide robot programming and skill learning effectively. Targeting human-in-the-loop (HITL) robotic assembly applications, the research tackles three key objectives: First, it develops a skill-based low-code robot programming system leveraging MR-assisted DT to enhance the intuitiveness and efficiency of human-robot interaction. Second, it proposes a scene-centric object manipulation approach enabling task intent expression through object’s DT manipulation, achieved via zero-shot segmentation in the MR environment. Lastly, it presents a large language model (LLM)–enhanced hybrid robot skill imitation approach combining human demonstrations with autonomous learning to boost the efficiency and generalization. This work provides an innovative solution from fundamental human-robot interaction (HRI) and control, to enhanced robot perception and skill execution, for HITL robotic assembly in complex and dynamic environments. Experimental results show significant improvements in the intuitiveness and efficiency of human-guided robot programming across perception, interaction, and execution dimensions. This thesis hopes to adopt an apprentice-like approach, enabling seamless robot programming through natural interactions and sustained skill imitations. |
| Rights: | All rights reserved |
| Access: | open access |
Copyright Undertaking
As a bona fide Library user, I declare that:
- I will abide by the rules and legal ordinances governing copyright regarding the use of the Database.
- I will use the Database for the purpose of my research or private study only and not for circulation or further reproduction or any other purpose.
- I agree to indemnify and hold the University harmless from and against any loss, damage, cost, liability or expenses arising from copyright infringement or unauthorized usage.
By downloading any item(s) listed above, you acknowledge that you have read and understood the copyright undertaking as stated above, and agree to be bound by all of its terms.
Please use this identifier to cite or link to this item:
https://theses.lib.polyu.edu.hk/handle/200/13831