| Author: | Zheng, Hongyi |
| Title: | Deep unfolding based image restoration |
| Advisors: | Zhang, Lei (COMP) |
| Degree: | Ph.D. |
| Year: | 2023 |
| Subject: | Image reconstruction Deep learning (Machine learning) Hong Kong Polytechnic University -- Dissertations |
| Department: | Department of Computing |
| Pages: | xvi, 126 pages : color illustrations |
| Language: | English |
| Abstract: | Image restoration aims to improve the quality of degraded images, such as those noise corrupted, blurred, or compressed images. Over the years, numerous techniques have been developed for image restoration. Traditional methods are generally model based ones, which incorporate prior knowledge about the image's properties and the degradation process. Deep learning based methods have emerged as a powerful approach to image restoration in the past decade, which leverage a large amount of training data to learn complex mappings from degraded images to their corresponding high-quality versions. Deep unfolding methods leverage the advantages of traditional model based and recent deep learning based methods by unfolding the optimization process of model based methods under the deep learning framework. By doing so, deep unfolding methods can effectively incorporate the prior knowledge as well as utilize the learning capability of deep neural networks. In this thesis, we investigate new deep unfolding methods to improve the efficiency and effectiveness of image restoration. In the first work, we propose a deep unfolding method called deep convolutional dictionary learning (DCDicL), which aims to address the image denoising problem. DCDicL unfolds the representation model of convolutional dictionary learning and integrates it into an iterative deep learning framework. It can adaptively generate a set of dictionaries for each input image based on its content and apply the dictionaries for enhancing the image denoising performance. DCDicL has demonstrated leading denoising performance in terms of both quantitative metrics and visual quality, reproducing subtle image structures and textures that are hard to recover by many existing denoising deep neural networks. In the second work, we propose a method called unfolded deep kernel estimation (UDKE), which is designed for the blind image super-resolution problem. UDKE unfolds the mathematical process of the super-resolution problem and employs an iterative framework to solve it. It can jointly learn image and blur kernel priors in an end-to-end manner, and then effectively exploit the information in both training data and image degradation models. It has achieved significantly better blind image super-resolution performance than state-of-the-art methods on benchmark datasets and real-world data. In the third work, we propose a new transformer model based on a novel self-attention module called Kernel Attention Transformer (KAT). By investigating the process of traditional spatial self-attention, we design a novel method to unfold the calculation of self-attention in feature space into its equivalent kernel space, which can reduce computational and memory consumption. KAT can effectively capture image self-attention across both spatial and channel dimensions, which has achieved state-of-the-art results in typical image restoration tasks, including denoising, super-resolution, and compression artifact reduction while requiring less memory and run-time than other transformer methods. In summary, in this thesis, we propose three new deep unfolding methods for improving the performance of image denoising, super-resolution and compression artifact removal, whose effectiveness and efficiency are validated on benchmark datasets. The developed methods shed new insights on how to integrate image modeling into deep prior learning. |
| Rights: | All rights reserved |
| Access: | open access |
Copyright Undertaking
As a bona fide Library user, I declare that:
- I will abide by the rules and legal ordinances governing copyright regarding the use of the Database.
- I will use the Database for the purpose of my research or private study only and not for circulation or further reproduction or any other purpose.
- I agree to indemnify and hold the University harmless from and against any loss, damage, cost, liability or expenses arising from copyright infringement or unauthorized usage.
By downloading any item(s) listed above, you acknowledge that you have read and understood the copyright undertaking as stated above, and agree to be bound by all of its terms.
Please use this identifier to cite or link to this item:
https://theses.lib.polyu.edu.hk/handle/200/12663