Full metadata record
| DC Field | Value | Language |
|---|---|---|
| dc.contributor | Department of Data Science and Artificial Intelligence | en_US |
| dc.contributor.advisor | Huang, Jian (DSAI) | en_US |
| dc.creator | Gao, Yuan | - |
| dc.identifier.uri | https://theses.lib.polyu.edu.hk/handle/200/13974 | - |
| dc.language | English | en_US |
| dc.publisher | Hong Kong Polytechnic University | en_US |
| dc.rights | All rights reserved | en_US |
| dc.title | When differential equations meet generative modeling : regularity, approximation, and convergence | en_US |
| dcterms.abstract | In recent years, continuous generative models based on ordinary differential equations (ODEs) and stochastic differential equations (SDEs) have played a central role in the rapidly expanding field of generative AIs. These generative AIs have shown remarkable empirical success across various applications, including large-scale image synthesis, protein structure prediction, and molecule generation. In this thesis, we aim to investigate the theoretical properties of these continuous generative models by considering the regularity of the differential equations, the ability to approximate them with deep neural networks, and the non-asymptotic convergence rate of these continuous generative models. | en_US |
| dcterms.abstract | In the first part, we address the regularity of a class of simulation-free continuous normalizing flows (CNFs) constructed with ODEs. Through a unified framework of the flow models termed Gaussian interpolation flows, we establish the Lipschitz regularity of the flow velocity field, the existence and uniqueness of the flow, and the Lipschitz continuity of the flow map and the time-reversed flow map for several rich classes of target distributions. This analysis also sheds light on the auto-encoding and cycle consistency properties of Gaussian interpolation flows. Our findings offer valuable insights into the learning techniques and accumulations of errors when employing Gaussian interpolation flows for generative modeling. | en_US |
| dcterms.abstract | In the second part, we study the theoretical properties of continuous normalizing flows with linear interpolation in learning probability distributions from a finite random sample, using a flow-matching objective function. We establish non-asymptotic error bounds for the distribution estimator based on CNFs, in terms of the Wasserstein-2 distance. We present a convergence analysis framework that encompasses the error due to velocity estimation, the discretization error, and the early stopping error. A key step in our analysis involves establishing the regularity properties of the velocity field and its estimator for CNFs constructed with linear interpolation. This necessitates the development of uniform error bounds with Lipschitz regularity control of deep ReLU networks that approximate the Lipschitz function class. Our nonparametric convergence analysis offers theoretical guarantees for using CNFs to learn probability distributions from a finite random sample. | en_US |
| dcterms.abstract | The last part of the thesis addresses the convergence properties of a Bayesian fine-tuning approach for large diffusion models. Diffusion models are a class of continuous generative models built with SDEs whose generation ability has been largely reinforced by various fine-tuning procedures. However, the mystery of fine-tuning has seldom been uncovered from a statistical perspective. In this part, we address the gap in the systematic understanding of the advantages of fine-tuning mechanisms from a statistical perspective. We prove that a pre-trained large diffusion model can gain a faster convergence rate from the Bayesian fine-tuning procedure when adapted to perform conditional generation tasks. This improvement in the convergence rate justifies that a pre-trained large diffusion model would perform better on a downstream conditional generation task than a standard conditional diffusion model, whenever an appropriate fine-tuning procedure is implemented. | en_US |
| dcterms.extent | x, 188 pages : color illustrations | en_US |
| dcterms.isPartOf | PolyU Electronic Theses | en_US |
| dcterms.issued | 2025 | en_US |
| dcterms.educationalLevel | Ph.D. | en_US |
| dcterms.educationalLevel | All Doctorate | en_US |
| dcterms.accessRights | open access | en_US |
Copyright Undertaking
As a bona fide Library user, I declare that:
- I will abide by the rules and legal ordinances governing copyright regarding the use of the Database.
- I will use the Database for the purpose of my research or private study only and not for circulation or further reproduction or any other purpose.
- I agree to indemnify and hold the University harmless from and against any loss, damage, cost, liability or expenses arising from copyright infringement or unauthorized usage.
By downloading any item(s) listed above, you acknowledge that you have read and understood the copyright undertaking as stated above, and agree to be bound by all of its terms.
Please use this identifier to cite or link to this item:
https://theses.lib.polyu.edu.hk/handle/200/13974