| Author: | Liang, Zhetong |
| Title: | New methods for image enhancement and camera ISP learning |
| Advisors: | Zhang, Lei (COMP) |
| Degree: | Ph.D. |
| Year: | 2021 |
| Subject: | Image processing -- Digital techniques Signal processing -- Digital techniques Neural networks (Computer science) Hong Kong Polytechnic University -- Dissertations |
| Department: | Department of Computing |
| Pages: | xviii, 126 pages : color illustrations |
| Language: | English |
| Abstract: | The captured images by modern camera sensor are color-mosaicked signals which contain incomplete color information, noise, less vivid colors and improper tones. To reconstruct a high-quality displayable image, an image signal processing (ISP) pipeline is employed onboard a camera to enhance the captured raw images by a cascade of image processing components, including demosaicking, white balance, noise removal, color space conversion, tone mapping and detail enhancement. However, there are two challenges in designing an ISP pipeline. First, the individual components in an ISP pipeline may have limited performance due to simple design. Second, there could be limitations on the whole ISP pipeline, which are designed in a divide-and-conquer manner with error accumulation. In this thesis, we leverage new optimization and learning methods to tackle the two challenges. To address the first challenge, we make several improvements on the design of individual image processing components. In the first work, we propose a new method for tone mapping component, which aims to convert a high dynamic range (HDR) image to a standard dynamic range image with improved perceptual quality. We design a hybrid l1-l0 norm optimization approach for tone mapping, and address the halo artifacts and over-enhancement problem in existing methods in the literatures. In the second work, we propose a deep-learning-based approach for single image denoising. Unlike the common end-to-end architecture, we adopt a two-stage convolutional neural network (CNN) architecture with smooth-first and enhance-later strategy. The proposed architecture removes the noise in the first stage and hallucinates high-frequency details back to the image in the second stage by adversarial learning. The proposed method can produce detail-enriched results and outperforms the existing denoising methods in terms of perceptual quality on both synthetic and real-world noisy images. In the third work, we propose a novel learning scheme for real-world burst denoising which leverages multiple images. To apply deep learning to burst denoising, it is difficult to construct a dataset for this purpose because of the object motions in a scene. We bypass this obstacle by designing a decoupled learning method to leverage two complementary datasets. With the designed network and the decoupled learning scheme, we achieve leading performance in real-world burst denoising without the need of a real-world burst dataset for training. To address the second challenge, we propose a data-driven framework for camera ISP learning. Different from the existing camera ISPs that rely on manual design of individual image processing components, we design a deep CNN as an ISP and train it with pairwise datasets to reconstruct high-quality displayable images from raw counterparts. The challenge for this work is to properly characterize the diverse image processing components inside an ISP. We tackle this problem by designing a two-stage CNN architecture, where image restoration related subtasks are addressed in the first stage and image enhancement related subtasks in the second stage. The proposed ISP model achieves high image quality and outperforms the state-of-the-art ISP learning methods on several publicly available benchmark datasets. In summary, in this thesis, we present a novel tone mapping algorithm, and two deep CNN-based methods for image denoising and burst denoising, respectively. In addition, we present a data-driven framework for the ISP pipeline design. |
| Rights: | All rights reserved |
| Access: | open access |
Copyright Undertaking
As a bona fide Library user, I declare that:
- I will abide by the rules and legal ordinances governing copyright regarding the use of the Database.
- I will use the Database for the purpose of my research or private study only and not for circulation or further reproduction or any other purpose.
- I agree to indemnify and hold the University harmless from and against any loss, damage, cost, liability or expenses arising from copyright infringement or unauthorized usage.
By downloading any item(s) listed above, you acknowledge that you have read and understood the copyright undertaking as stated above, and agree to be bound by all of its terms.
Please use this identifier to cite or link to this item:
https://theses.lib.polyu.edu.hk/handle/200/11356