| Author: | Yue, Shuwei |
| Title: | Critical considerations for implementing DNN-based AWB and ISP |
| Advisors: | Wei, Minchen (BEEE) |
| Degree: | Ph.D. |
| Year: | 2024 |
| Subject: | Color Signal processing Image processing -- Digital techniques Neural networks (Computer science) Hong Kong Polytechnic University -- Dissertations |
| Department: | Department of Building Environment and Energy Engineering |
| Pages: | xxi, 143 pages : color illustrations |
| Language: | English |
| Abstract: | Color constancy is the human vision’s ability to perceive relatively consistent colors under various lighting conditions. Automatic White Balance (AWB) is one of the most important modules in Image Signal Processor (ISP), which aims to mimic such an ability. Recent advancements in Deep Neural Networks (DNNs) have significantly improved AWB performance, but several key challenges remain. Conventional AWB algorithms struggle with scenes dominated by a single color, a common scenario in smartphone photography. To address this, we developed the ”PolyU Pure Color” dataset and a lightweight feature-based multilayer perceptron (MLP) neural network model called ”Pure Color Constancy (PCC)”. Using just four color features, PCC significantly outperforms the state-of-the-art methods on pure color images while maintaining comparable performance on typical images, with an excellent cross-sensor capability. Notably, PCC achieves such a performance with only about 400 parameters and a processing time of around0.25 ms per image, making it highly suitable for practical deployment. Moreover, cross-sensor compatibility in illuminant estimation typically requires extensive sensor-specific data collection. Our proposed dual-mapping strategy, the DMCC method, addresses this challenge by only requiring white points under a D65 illumination for both training and testing sensors. This approach reconstructs image and illuminant data, maps them to sparse features, and trains a lightweight MLP model that can be directly applied to new sensors without additional data collection. The DMCC method achieves performance comparable to the state-of-the-art methods across multiple datasets, significantly reducing the need for sensor-specific data and offering faster processing speeds. For multi-illuminant scenes, we developed a robust pixel-wise illumination estimation method. Our analysis revealed that conventional pixel-wise algorithms suffer significant accuracy losses (up to 30%) when applied on lower bit-depth images, which are preferred in ISP pipelines. We identified that such a reduction was due to the loss of details and increased noise. Our proposed method, combining L1 loss optimization with physical-constrained post-processing, achieves around 40% higher estimation accuracy compared to the state-of-the-art DNN-based methods, effectively addressing the challenges of multi-illuminant environments and lower bit-depth images. Finally, we investigated the optimal integration of AWB and denoising in modern ISPs. Through extensive experiments, we demonstrated that processing AWB and denoising individually yields better results than an end-to-end approach. Importantly, we found that performing denoising before AWB leads to significant improvements, with an increase of nearly 6 dB in PSNR and a 30% reduction in mean angular error compared to the reverse sequence. These findings provide crucial insights for the design and optimization of future DNN-based ISPs. In summary, this thesis advances AWB and ISP optimization, offering practical solutions for real-world imaging devices and paving the way for more adaptive imaging systems in next-generation digital devices. |
| Rights: | All rights reserved |
| Access: | open access |
Copyright Undertaking
As a bona fide Library user, I declare that:
- I will abide by the rules and legal ordinances governing copyright regarding the use of the Database.
- I will use the Database for the purpose of my research or private study only and not for circulation or further reproduction or any other purpose.
- I agree to indemnify and hold the University harmless from and against any loss, damage, cost, liability or expenses arising from copyright infringement or unauthorized usage.
By downloading any item(s) listed above, you acknowledge that you have read and understood the copyright undertaking as stated above, and agree to be bound by all of its terms.
Please use this identifier to cite or link to this item:
https://theses.lib.polyu.edu.hk/handle/200/13288