| Author: | Li, Shufei |
| Title: | A systematic exploration methodology to achieve proactive human-robot collaboration in the human-centric smart manufacturing environment |
| Advisors: | Zheng, Pai (ISE) Lee, K. M. Carman (ISE) |
| Degree: | Ph.D. |
| Year: | 2023 |
| Subject: | Human-robot interaction Robots, Industrial Hong Kong Polytechnic University -- Dissertations |
| Department: | Department of Industrial and Systems Engineering |
| Pages: | xxiii, 192 pages : color illustrations |
| Language: | English |
| Abstract: | The manufacturing industry is striving for flexible automation and collaboration between humans and machines, due to the urgent challenges of an aging workforce, labor shortages, and increasing demand for mass personalised products. Human-Centric Smart Manufacturing (HCSM) is emerging to combine advanced technologies such as artificial intelligence, robotics, and industrial internet of things with human-centered design principles to create a collaborative and adaptive production environment. The concept of HCSM places emphasis on the role of humans in future factories, with the goal of enhancing productivity, safety, and operator wellbeing, while also improving the efficiency, flexibility, and quality of production processes. In this context, Human-Robot Collaboration (HRC) emerges as a promising solution to enhance manufacturing performance by leveraging the strengths of both human cognitive flexibility and adaptability and robots’ exceptional precision, strength, and repeatability. To achieve the objective of HCSM, extensive research has been conducted to devise a variety of HRC systems and technologies, which enable robots and humans to collaborate efficiently. These technologies include sensing and perception systems, decision-making methods, user-friendly interfaces, and planning and control algorithms. An effective HRC system can be applied in a wide variety of manufacturing tasks, like machining, welding, and assembly. It provides humans with opportunities to learn new skills and engage in tasks requiring problem-solving and creative abilities, while robots work alongside humans to perform repetitive, dangerous, or physically taxing tasks, resulting in improved productivity and worker satisfaction. However, current HRC applications have not yet fully achieved the best complementarity between human and robot capabilities, and are facing several challenges in the following aspects. • Limited adaptability: Most HRC systems are currently designed to operate within specific environments, lacking the necessary flexibility to adapt to changing work environments or task requirements. • Weak context awareness: The context-awareness capability of HRC sticks to a non-semantic perception level, which fails to understand the context of the work environment and human-robot-task relations. • Unidirectional communication: Current communication methods between humans and robots are normal in a master-slave mode and fail to exchange precise information in a short time. • Uneffective decision-making: Today’s HRC systems heavily rely on preprogrammed instructions, which limit their ability to make real-time decisions and proactively adapt to changes in the work environment. • Poor trust: The bidirectional understanding between humans and robots is lacking, as humans may not fully trust the capabilities of robots, and robots may not fully comprehend human intentions. Motivated by the above challenges, firstly, this study explores a foreseeable HRC paradigm, i.e., Proactive HRC that coincides with today’s HCSM transformation (Chapter 3). The Proactive HRC aims to cultivate intellectual abilities similar to humans and encourage proactive behaviour in both humans and robots. Four pivotal modules are vital for the implementation of Proactive HRC systems, i.e., 1) mutual-cognition and empathy, 2) predictable spatio-temporal collaboration, 3) self-organising multi-agent teamwork, and 4) intelligent robot control and human assistant system. In this context, multiple humans in different roles and robotic agents with various workloads work on complex manufacturing tasks in close proximity, with the characteristics of bi-directional communication, holistic cognition ability, mutual empathetic teamwork, adaptive decision-making mechanisms, and proactive robot control. This collaboration is achieved by considering each other’s operation needs, desired resources, and qualified complementary capabilities. Secondly, with the purpose to permeate mutual-cognitive intelligence and empathetic skills into Proactive HRC, a Mixed Reality (MR) and visual reasoning-enabled method is explored to infer scene interpretation and to assign task arrangements meeting bi-directional operation goals (Chapter 4). The robot motion follows ergonomic rules for easy interaction with humans, while the human can obtain super information awareness abilities in an MR environment. The proposed approach facilitates HRC systems from a perceptual loop to a knowledge-enriched cognition level, which allows for high-level intuitive behaviours of humans and the proactive role of robots in co-working. Thirdly, a multimodal transfer learning-based method is proposed to consider human operation intentions in advance for proactive robot motion, assuring predictable collaboration in the near future (Chapter 5). The multi-modal fusion network is capable of accurately predicting ongoing human intentions despite pattern confusion in various fine-grained operations, such as wedging pins and screwing bolts. The transfer learning tackles time-consuming problems for massive sensor data annotation and allows for easy adaptation of the method to new manufacturing scenarios. A dynamic decision-tree mechanism allows showing robot future motions in a clear sequential sequence for human reference, achieving predictable and expected collaboration. Lastly, a temporal subgraph reasoning-based method is proposed for selforganising task planning between multiple humans and robots, solving the problem of predefined task arrangements in HRC (Chapter 6). The temporal subgraph reasoning mechanism forecasts each agent’s next operations through knowledge learning of prior experience, task stage progression, and environment changes. Meanwhile, the temporal subgraph provides an interpretable graphical structure for hierarchical and sequential task arrangement between humans and robots in task-agent-operation layers. Humans obtain a global understanding of the HRC task from the graph. Robots parse graph triples of robot-action-component connections to automatic motions and perform teamwork. Most of the research work in this thesis has been reported in four journal papers and three conference papers. It is hoped that the research findings will initiate insightful discourse for researchers worldwide and provide a practical production framework for contemporary enterprises and factories to meet the changing market demands. |
| Rights: | All rights reserved |
| Access: | open access |
Copyright Undertaking
As a bona fide Library user, I declare that:
- I will abide by the rules and legal ordinances governing copyright regarding the use of the Database.
- I will use the Database for the purpose of my research or private study only and not for circulation or further reproduction or any other purpose.
- I agree to indemnify and hold the University harmless from and against any loss, damage, cost, liability or expenses arising from copyright infringement or unauthorized usage.
By downloading any item(s) listed above, you acknowledge that you have read and understood the copyright undertaking as stated above, and agree to be bound by all of its terms.
Please use this identifier to cite or link to this item:
https://theses.lib.polyu.edu.hk/handle/200/12631