| Author: | Zhou, Peng |
| Title: | LaGeo : a latent and geometrical framework for path and manipulation planning |
| Advisors: | Navarro-Alarcon, David (ME) |
| Degree: | Ph.D. |
| Year: | 2022 |
| Subject: | Robots -- Control systems Hong Kong Polytechnic University -- Dissertations |
| Department: | Department of Mechanical Engineering |
| Pages: | xxix, 170 pages : color illustrations |
| Language: | English |
| Abstract: | Path and manipulation planning has been gaining popularity within the robotics community all the time due to its potential applications in many economically important applications. In the past decades, although great progress has been achieved in many different path and manipulation planning tasks, most state-of-the-art methods are case-specific; They can only be used to perform single type of planning task (e.g. 2D pixel based path planning, 3D point cloud path planning), as their feature extraction algorithms for the sensory data of external environments typically rely on "handcrafted" features. Besides, many applications only consider planning tasks in Euclidean action space instead of adaptively planning the manipulation in transformed action space based on the extracted feature (e.g. latent variables after Auto-Encoder (AE) network, landmarks generated from a dense descriptor network). Moreover, traditional path and manipulation planning methods can only perform on rigid objects, which suffer from the manipulation planning problems for deformable objects. In this thesis, we present LaGeo, a general framework for path and manipulation tasks by interleaving the latent space and geometrical models. Our new method introduces internal latent representation layers between low-level geometric feature extraction and high-level semantic observation analysis; This allows the compact and meaningful representation for the extremely complex sensing space and the formation of a valid subgoal status generator from the low-level geometric feature extraction levels. The proposed latent and geometrical framework makes path and manipulation planning tasks more generic (independent from the object's geometry, and its mechanical properties) and scalable (it can work with 1D/2D/3D tasks), thus it can both applied to rigid object manipulation and deformable object manipulation tasks. Its high-level semantic layer enables to perform (quasi) path planning tasks with both rigid and soft objects. Besides, a valuable and underexplored capability in many deformable manipulation tasks, namely, deformation planning can be solved with LaGeo. To validate this new framework, we report detailed experimental studies with real robotic manipulators and simulation for four path and manipulation planning tasks, that are as follows: 1. Path planning with automatic seam extraction over point cloud models for robotic arc welding: In this task, we leverage on geometrical models to extract a point cloud into a latent space (edge feature), then based on that we generate 6-DOF welding path on external Euclidean space. The method is tested on multiple workpieces with different joint types and poses. Experimental results prove the robustness and efficiency of this robotic system on automatic path planning for welding applications. 2. Latent space-based shape deformation planning for 1D and 2D deformable objects: In this task, we present LaSeSOM, a new feedback latent representation framework for semantic soft object manipulation. Our new method introduces internal latent representation layers between low-level geometric feature extraction and high-level semantic shape analysis; This allows the identification of each compressed semantic function and the formation of a valid shape classifier from different feature extraction levels. The proposed latent framework makes soft object representation more generic (independent from the object's geometry and its mechanical properties) and scalable (it can work with 1D/2D/3D tasks). Its high-level semantic layer enables to perform (quasi) shape planning tasks with soft objects, a valuable and underexplored capability in many soft manipulation tasks. To validate this new methodology, we report a detailed experimental study with robotic manipulators. 3. Interactive perception for deformable object manipulation: In this task, we address interactive perception considering deformable objects and active vision. Our approach models the regularity/structure of the integrated perception-action space by using a manifold-with-boundary formulation in a POMDP context. We develop a simulation setup to demonstrate the validity and effectiveness of the approach and perform a comparison with the state-of-the-art method. 4. Imitating garment folding from a single visual observation using hand-object graph dynamics model: In this task, we propose a new learning from demonstrations method that enables robots to autonomously fold garments with an assistive folding board. In contrast with traditional methods (that rely on low-level pixel features), our new solution uses a dense visual descriptor to encode the demonstration into a high-level hand-object graph (HoG) that allows to efficiently represent the interactions between the manipulated object and robots. With that, we leverage on Graph Neural Networks (GNN) to autonomously learn the forward dynamics model from HoGs; Then, given only a single demonstration, the imitation policy is optimized with a Model Predictive Controller (MPC) to accomplish the folding task. To validate the proposed approach, we conduct a detailed experimental study on a robotic platform instrumented with vision sensors and a custom-made end-effector that interacts with the folding board. 5. Reactive path planning with neural RMPS for robotic sealing: In this task, we present a new autonomous sealing system capable of online path planning and control in complex dynamic environments; By using real-time feedback, our method avoids acquiring accurate 3D models of the workpiece. To this end, a fractional order differentiation-based edge detection is implemented to robustly extract the sealing seam from low signal-to-noise ratio (SNR) images (which are typically captured by cheap commodity cameras). Subsequently, we introduce a novel framework for reactive path planning that leverages on a Riemannian motion policy (RMP) [1] and deep learning techniques. RMP models the interaction between the robot and the working environment as a unified representation, which is defined by an acceleration policy and its corresponding Riemannian metric. By utilizing the forward kinematics, optimal commands for the robot control can be generated computationally. Experimental results prove the robustness and efficiency of this general framework on above-mentioned path and manipulation task for rigid and deformable objects. |
| Rights: | All rights reserved |
| Access: | open access |
Copyright Undertaking
As a bona fide Library user, I declare that:
- I will abide by the rules and legal ordinances governing copyright regarding the use of the Database.
- I will use the Database for the purpose of my research or private study only and not for circulation or further reproduction or any other purpose.
- I agree to indemnify and hold the University harmless from and against any loss, damage, cost, liability or expenses arising from copyright infringement or unauthorized usage.
By downloading any item(s) listed above, you acknowledge that you have read and understood the copyright undertaking as stated above, and agree to be bound by all of its terms.
Please use this identifier to cite or link to this item:
https://theses.lib.polyu.edu.hk/handle/200/12133