| Author: | Jiang, Biyi |
| Title: | Intelligent vision devices and systems |
| Advisors: | Chai, Yang (AP) |
| Degree: | Ph.D. |
| Year: | 2026 |
| Department: | Department of Applied Physics |
| Pages: | vii, 140 pages : color illustrations |
| Language: | English |
| Abstract: | Intelligent vision systems that can sense, understand, and process the visual scenes in real time hold great promise for versatile and complex vision scenarios. Conventional vision system with Von Neumann architecture cannot process these scenarios efficiently. Emerging intelligent device-based systems hold promise, but they still suffer from limited reconfigurability and advanced processing capabilities at device level, as well as the lack of co-designed circuits and algorithms at the system level. To solve the limited reconfigurability, we employ a multi-paradigm device with four sensory computing modes, including 1) photo-spiking neuron (PN) and 2) photo-synaptic (PS) modes with spiking and non-spiking in-sensor low-/mid-level processing capabilities for neuromorphic imaging (NI) functions, and 3) electrical-synaptic (ES) and 4) electrical-neuron (EN) modes with spiking and non-spiking in-memory high-level processing capabilities for artificial intelligence (AI) functions. With the co-designed mode control circuit and reconfigurable dataflow architecture, we achieve an intelligent vision system with hierarchical reconfigurability from device to system level. It is capable of on-demand allocating resources between NI and AI functionalities towards high-quality smart imaging and high-accuracy recognition tasks, and spiking/non-spiking mode transitions towards frameless dynamic and frame-based static scenarios. Exceptional power efficiencies of up to 52.6 TOPS/W for NI-centric computing and 75.5 TOPS/W for AI-centric computing are achieved, which are up to two orders of magnitude larger than the state-of-the-art intelligent vision system. To address the limited processing capability in complex multi-target detection scenarios, systems capable of efficiently perceiving multidimensional optical information are highly desirable. Inspired by the multilevel biological visual filters, we introduce a multidimensional and multilevel MoS₂/HZO ferroelectric field-effect transistor (FeFET) device. It exhibits enhanced wavelength selectivity for low-level colour clustering and mid-level multidimensional decoupling due to the polarization-enhanced interfacial barrier height, as well as polarization-enhanced multi-photoresponsivity states for high-level processing. With the co-designed hierarchical visual filter bank algorithm (HVBF), a multidimensional hierarchical intelligent vision system is constructed, capable of performing low-level colour clustering, mid-level segmentation, and high-level recognition within a single MoS₂/HZO device array. It achieves 93.8% accuracy in identifying overlapping digits in noisy scenarios, demonstrating high robustness and precision. For scenarios requiring three-dimensional (3D) understanding, we demonstrate a bipolar MoS₂/In₂Se₃ device that generates photocurrent with different polarities for near-infrared (NIR) and red, green, and blue (RGB) light under the same biasing condition based on the visible light-induced MoS₂/Au interfacial barrier-lowering mechanism. Combining with near-infrared marker-based depth perception technology, the MoS₂/In₂Se₃ array-based RGB-D camera can efficiently sense and decouple two-dimensional (2D) RGB and 3D depth images simultaneously. An intelligent visual system with 3D scene understanding capability is realized, including the RGB-D camera for sensing and decoupling, and two neural networks (the convolutional neural network (CNN) and the long short-term memory (LSTM) network) for processing the 2D and 3D images, respectively. In the augmented reality (AR) task involving two hands interacting with one object, the system recognizes static objects and dynamic gestures, and places virtual annotations near their real-world counterparts with 91.8% accuracy. |
| Rights: | All rights reserved |
| Access: | open access |
Copyright Undertaking
As a bona fide Library user, I declare that:
- I will abide by the rules and legal ordinances governing copyright regarding the use of the Database.
- I will use the Database for the purpose of my research or private study only and not for circulation or further reproduction or any other purpose.
- I agree to indemnify and hold the University harmless from and against any loss, damage, cost, liability or expenses arising from copyright infringement or unauthorized usage.
By downloading any item(s) listed above, you acknowledge that you have read and understood the copyright undertaking as stated above, and agree to be bound by all of its terms.
Please use this identifier to cite or link to this item:
https://theses.lib.polyu.edu.hk/handle/200/14422