扩散模型研究一：去噪扩散概率模型DDPM

作者: 引线小白-本文永久链接：httpss://www.limoncc.com/post/1c60669bbe56769f/
知识共享许可协议: 本博客采用署名-非商业-禁止演绎4.0国际许可证

一、基本介绍

扩散模型大放异彩，从原理上搞清楚运作机制非常关键。下面我们定义一些符号：对于观测变量 $\mathit{x}$，与VAE对应一个隐变量 $\mathit{z}$不同，扩散模型对应一组隐变量 ${\displaystyle {\mathcal{D}}_T^{\mathit{z}}=\{{\mathit{z}}_t{\}}_{t=1}^T}$, 而且我们假设${\mathcal{D}}_T^{\mathit{z}}$有马尔可夫性质。这样我们的模型就不是 ${\displaystyle p(\mathit{x})=\int p(\mathit{x}\mid \mathit{z})p(\mathit{z})d\mathit{z}}$而是

$$\begin{array}{}\text{(1)}& p(\mathit{x})& =\int p(\mathit{x}\mid {\mathcal{D}}_T^{\mathit{z}})d{\mathcal{D}}_T^{\mathit{z}}\text{(2)}& & =\int p(\mathit{x}\mid {\mathit{z}}_1)p({\mathit{z}}_1\mid {\mathit{z}}_2)\cdots p({\mathit{z}}_{t-1}\mid {\mathit{z}}_T)p({\mathit{z}}_T)d{\mathcal{D}}_T^{\mathit{z}}\text{(3)}& & =\int p(\mathit{x}\mid {\mathit{z}}_1)p({\mathit{z}}_T)\prod _{t=2}^{T}p({\mathit{z}}_{t-1}\mid {\mathit{z}}_t)d{\mathcal{D}}_T^{\mathit{z}}\end{array}$$

下面我们来解释一下这么做的动机：我们认为上帝是这么来生成一张图片的。先掷骰子(对，没错我们又假设上帝投骰子)，然后慢慢画出图像(先色块，再形状，再细节，就像美术的厚涂法一样)。这对应着解码过程(decode)，反过来就是编码过程(encode)。

$$\begin{array}{}\text{(4)}& \text{encode}:& \mathit{x}\to {\mathit{z}}_1\to {\mathit{z}}_2\to \cdots \to {\mathit{z}}_{T-1}\to {\mathit{z}}_T=\mathit{z}\text{(5)}& \text{decode}:& \mathit{z}={\mathit{z}}_T\to {\mathit{z}}_{T-1}\to {\mathit{z}}_{T-2}\to \cdots \to {\mathit{z}}_1\to \mathit{x}\end{array}$$

也就是说有三个关键分布
1、编码分布 ${\displaystyle p(\mathit{z}\mid \mathit{x})}$
2、生成分布(解码分布) ${\displaystyle p(\mathit{x}\mid \mathit{z})}$
3、先验分布 ${\displaystyle p(\mathit{z})}$

二、损失函数

考虑到编码分布的复杂性，我们一般使用易于计算的分布 ${\displaystyle q(\mathit{z}\mid \mathit{x})}$。我们使假设的模型分布尽可能与真实数据靠拢

$$\begin{array}{}\text{(6)}& \mathbb{KL}[q(\mathit{x},{\mathcal{D}}_T^{\mathit{z}})|p(\mathit{x},{\mathcal{D}}_T^{\mathit{z}})]& =\int q(\mathit{x},{\mathcal{D}}_T^{\mathit{z}})\log \frac{q(\mathit{x},{\mathcal{D}}_T^{\mathit{z}})}{p(\mathit{x},{\mathcal{D}}_T^{\mathit{z}})}d{\mathcal{D}}_T^{\mathit{z}}d\mathit{x}\text{(7)}& & =\int q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})q(\mathit{x})\log \frac{q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})q(\mathit{x})}{p(\mathit{x},{\mathcal{D}}_T^{\mathit{z}})}d{\mathcal{D}}_T^{\mathit{z}}d\mathit{x}\text{(8)}& & ={\mathbb{E}}_{\mathit{x}\sim q(\mathit{x})}[\int q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})\log q(\mathit{x})d{\mathcal{D}}_T^{\mathit{z}}-\int q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})\log \frac{p(\mathit{x},{\mathcal{D}}_T^{\mathit{z}})}{q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})}d{\mathcal{D}}_T^{\mathit{z}}]\text{(9)}& & =-{\mathbb{H}}_{q(\mathit{x})}[\mathit{x}]+{\mathbb{E}}_{\mathit{x}\sim q(\mathit{x})}[-\int q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})\log \frac{p(\mathit{x},{\mathcal{D}}_T^{\mathit{z}})}{q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})}d{\mathcal{D}}_T^{\mathit{z}}]\text{(10)}& & ⩽{\mathbb{E}}_{\mathit{x}\sim q(\mathit{x})}[-\int q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})\log \frac{p(\mathit{x},{\mathcal{D}}_T^{\mathit{z}})}{q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})}d{\mathcal{D}}_T^{\mathit{z}}]\end{array}$$

令 ${\displaystyle f(\mathit{\theta })=-\int q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})\log \frac{p(\mathit{x},{\mathcal{D}}_T^{\mathit{z}})}{q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})}d{\mathcal{D}}_T^{\mathit{z}}}$ 我们有

$$\begin{array}{}\text{(11)}& f(\mathit{\theta })& =-\int q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})\log \frac{p(\mathit{x},{\mathcal{D}}_T^{\mathit{z}})}{q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})}d{\mathcal{D}}_T^{\mathit{z}}\text{(12)}& & =-\int q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})\log \frac{p(\mathit{x}\mid {\mathit{z}}_1)p({\mathit{z}}_T)\prod _{t=2}^{T}p({\mathit{z}}_{t-1}\mid {\mathit{z}}_t)}{q({\mathit{z}}_1\mid \mathit{x})\prod _{t=2}^{T}q({\mathit{z}}_t\mid {\mathit{z}}_{t-1})}d{\mathcal{D}}_T^{\mathit{z}}\text{(13)}& & =-{\mathbb{E}}_{q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})}[\log \frac{p(\mathit{x}\mid {\mathit{z}}_1)p({\mathit{z}}_T)\prod _{t=2}^{T}p({\mathit{z}}_{t-1}\mid {\mathit{z}}_t)}{q({\mathit{z}}_1\mid \mathit{x})\prod _{t=2}^{T}q({\mathit{z}}_t\mid {\mathit{z}}_{t-1})}]\end{array}$$

考虑到
$$\begin{array}{}\text{(14)}& q({\mathit{z}}_t\mid {\mathit{z}}_{t-1})=q({\mathit{z}}_t\mid {\mathit{z}}_{t-1},\mathit{x})=\frac{q({\mathit{z}}_{t-1}\mid {\mathit{z}}_t,\mathit{x})q({\mathit{z}}_t\mid \mathit{x})}{q({\mathit{z}}_{t-1}\mid \mathit{x})}\end{array}$$

于是有
$$\begin{array}{}\text{(15)}& f(\mathit{\theta })& =-{\mathbb{E}}_{q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})}[\log \frac{p(\mathit{x}\mid {\mathit{z}}_1)p({\mathit{z}}_T)\prod _{t=2}^{T}p({\mathit{z}}_{t-1}\mid {\mathit{z}}_t)}{q({\mathit{z}}_1\mid \mathit{x})\prod _{t=2}^{T}q({\mathit{z}}_t\mid {\mathit{z}}_{t-1})}]\text{(16)}& & =-{\mathbb{E}}_{q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})}[\log \frac{p(\mathit{x}\mid {\mathit{z}}_1)p({\mathit{z}}_T)\prod _{t=2}^{T}p({\mathit{z}}_{t-1}\mid {\mathit{z}}_t)q({\mathit{z}}_{t-1}\mid \mathit{x})}{q({\mathit{z}}_1\mid \mathit{x})\prod _{t=2}^{T}q({\mathit{z}}_{t-1}\mid {\mathit{z}}_t,\mathit{x})q({\mathit{z}}_t\mid \mathit{x})}]\text{(17)}& & =-{\mathbb{E}}_{q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})}[\log p(\mathit{x}\mid {\mathit{z}}_1)+\log \frac{p({\mathit{z}}_T)\cancel{\prod _{t=2}^{T}q({\mathit{z}}_{t-1}\mid \mathit{x})}}{q({\mathit{z}}_T\mid \mathit{x})\cancel{\prod _{t=1}^{T-1}q({\mathit{z}}_t\mid \mathit{x})}}+\sum _{t=2}^T\log \frac{p({\mathit{z}}_{t-1}\mid {\mathit{z}}_t)}{q({\mathit{z}}_{t-1}\mid {\mathit{z}}_t,\mathit{x})}]\text{(18)}& & =-{\mathbb{E}}_{q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})}[\log p(\mathit{x}\mid {\mathit{z}}_1)+\log \frac{p({\mathit{z}}_T)}{q({\mathit{z}}_T\mid \mathit{x})}+\sum _{t=2}^T\log \frac{p({\mathit{z}}_{t-1}\mid {\mathit{z}}_t)}{q({\mathit{z}}_{t-1}\mid {\mathit{z}}_t,\mathit{x})}]\text{(19)}& & =\underset{{\mathcal{L}}_T}{\underbrace{\mathbb{KL}[q({\mathit{z}}_T\mid \mathit{x})|p({\mathit{z}}_T))]}}+\underset{{\mathcal{L}}_{1:T-1}}{\underbrace{\sum _{t=2}^T\mathbb{KL}[q({\mathit{z}}_{t-1}\mid {\mathit{z}}_t,\mathit{x})|p({\mathit{z}}_{t-1}\mid {\mathit{z}}_t)]}}+\underset{{\mathcal{L}}_0}{\underbrace{{\mathbb{E}}_{q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})}[\log p(\mathit{x}\mid {\mathit{z}}_1)]}}\end{array}$$

所以有损失函数:
$$\begin{array}{}\text{(20)}& \mathcal{L}={\mathbb{E}}_{x\sim p(\mathit{x})}[\mathbb{KL}[q({\mathit{z}}_T\mid \mathit{x})|p({\mathit{z}}_T))]+\sum _{t=2}^T\mathbb{KL}[q({\mathit{z}}_{t-1}\mid {\mathit{z}}_t,\mathit{x})|p({\mathit{z}}_{t-1}\mid {\mathit{z}}_t)]+{\mathbb{E}}_{q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})}[\log p(\mathit{x}\mid {\mathit{z}}_1)]]\end{array}$$

这里需要注意一点，我们是根据马尔可夫假设，推导出了损失函数。如果我们仔细观察损失函数的构成，其实我们很容易发现，马尔可夫假设不是必须的。这也为之后的模型提供了优化空间。

三、模型构建的细节

3.1、编码模型的实现

我们首先来考察一个损失函数 ${\displaystyle {\mathcal{L}}_T}$。所谓编码其实就是一个破坏的过程。我们希望通过往正常图片中逐步的加入噪声，来得到噪声。
$$\begin{array}{}\text{(21)}& {\mathit{z}}_t={\alpha }_t{\mathit{z}}_{t-1}+{\beta }_t{\mathit{ϵ}}_t\end{array}$$其中 ${\displaystyle \mathit{ϵ}\sim \mathcal{N}(\mathbf{0},\mathit{I})}$, ${\displaystyle {\mathit{z}}_t\in \{\mathit{x}\}\cup \{{\mathit{z}}_t{\}}_{t=1}^T=\{{\mathit{z}}_t{\}}_{t=0}^T}$
最终得到噪声
$$\begin{array}{}\text{(22)}& p({\mathit{z}}_T)\sim \mathcal{N}(\mathbf{0},\mathit{I})\end{array}$$

这样有：

$$\begin{array}{}\text{(23)}& q({\mathit{z}}_t\mid \mathit{x})=\int q({\mathcal{D}}_t^{\mathit{z}}\mid \mathit{x})d{\mathcal{D}}_{-t}=\mathcal{N}({\alpha }_t{\mathit{z}}_{t-1},{\beta }_t^2\mathit{I})\end{array}$$
对于 ${\displaystyle q({\mathit{z}}_T\mid \mathit{x})}$, 我们展开到 ${\displaystyle \mathit{x}}$就有

$$\begin{array}{}\text{(24)}& {\mathit{z}}_T& ={\alpha }_T{\mathit{z}}_{T-1}+{\beta }_T{\mathit{ϵ}}_T={\alpha }_T[{\alpha }_{T-1}{\mathit{z}}_{T-2}+{\beta }_{T-1}{\mathit{ϵ}}_{T-1}]+{\beta }_T{\mathit{ϵ}}_T\text{(25)}& & ={\alpha }_T{\alpha }_{T-1}[{\alpha }_{T-2}{\mathit{z}}_{T-3}+{\beta }_{T-2}{\mathit{ϵ}}_{T-2}]+{\alpha }_T{\beta }_{T-1}{\mathit{ϵ}}_{T-1}+{\beta }_T{\mathit{ϵ}}_T\text{(26)}& & =\prod _{t=1}^T{\alpha }_t\mathit{x}+[\sum _{t=1}^{T-1}{\beta }_t\prod _{\tau =t}^T{\alpha }_{\tau +1}{\mathit{ϵ}}_t+{\beta }_T{\mathit{ϵ}}_T]\end{array}$$
考察均值与方差
$$\begin{array}{}\text{(27)}& \mathbb{E}[{\mathit{z}}_T]& =\prod _{t=1}^T{\alpha }_t\mathit{x}\text{(28)}& \mathbb{Cov}[{\mathit{z}}_T]& =[\sum _{t=1}^{T-1}{\beta }_t^2\prod _{\tau =t}^T{\alpha }_{\tau +1}^2+{\beta }_T^2]\cdot \mathit{I}\end{array}$$

我们希望最终的 ${\displaystyle {\mathit{z}}_T\sim \mathcal{N}(\mathbf{0},\mathit{I})}$, 若 ${\displaystyle {\alpha }_t^2+{\beta }_t^2=1}$ 则有
$$\begin{array}{}\text{(29)}& \sum _{t=1}^{T-1}{\beta }_t^2\prod _{\tau =t}^T{\alpha }_{\tau +1}^2+{\beta }_T^2& =\sum _{t=1}^{T-1}(1-{\alpha }_t^2)\prod _{\tau =t}^T{\alpha }_{\tau +1}^2+{\beta }_T^2\text{(30)}& & =\sum _{t=1}^{T-1}[\prod _{\tau =t}^T{\alpha }_{\tau +1}^2-\prod _{\tau =t}^T{\alpha }_{\tau }^2]+{\beta }_T^2\text{(31)}& & =\sum _{t=1}^{T-1}[\prod _{\tau =t}^T{\alpha }_{\tau +1}^2-\prod _{\tau =t}^T{\alpha }_{\tau }^2]+{\beta }_T^2\text{(32)}& & =-\prod _{t=1}^T{\alpha }_t^2+{\alpha }_T^2+{\beta }_T^2\text{(33)}& & =1-\prod _{t=1}^T{\alpha }_t^2\end{array}$$

令 ${\displaystyle {\overline{\alpha }}_T=\prod _{t=1}^T{\alpha }_t}$, ${\displaystyle {\overline{\beta }}_T=\sqrt{1-\prod _{t=1}^T{\alpha }_t^2}}$, 这样就有：

$$\begin{array}{}\text{(34)}& q({\mathit{z}}_T\mid \mathit{x})\sim \mathcal{N}({\overline{\alpha }}_T\mathit{x},{\overline{\beta }}_T^2)=\mathcal{N}(\prod _{t=1}^T{\alpha }_t\mathit{x},[1-\prod _{t=1}^T{\alpha }_t^2]\mathit{I})\end{array}$$

我们只要设计合理的 ${\displaystyle {\alpha }_t}$使得 ${\displaystyle \underset{T\to +\mathrm{\infty }}{lim}\prod _{t=1}^T{\alpha }_t=\mathbf{0}}$即可。这样无需参数，只通过一些必要的高斯假设就实现了我们的目的 ${\displaystyle \mathbb{KL}[q({\mathit{z}}_T\mid \mathit{x})|p({\mathit{z}}_T))]=0}$。故我们损失的第一项是零。

3.2、生成模型的实现

3.2.1、${\displaystyle {\mathcal{L}}_{1:T-1}}$

考虑逆过程 ${\displaystyle q({\mathit{z}}_{t-1}\mid {\mathit{z}}_t,\mathit{x})}$，根据贝叶斯定理有：
$$\begin{array}{}\text{(35)}& q({\mathit{z}}_{t-1}\mid {\mathit{z}}_t,\mathit{x})& =\frac{q({\mathit{z}}_t\mid {\mathit{z}}_{t-1},\mathit{x})q({\mathit{z}}_{t-1}\mid \mathit{x})}{q({\mathit{z}}_t\mid \mathit{x})}\text{(36)}& & =\frac{\mathcal{N}({\alpha }_t{\mathit{z}}_{t-1},{\beta }_t^2\mathit{I})\mathcal{N}({\overline{\alpha }}_{t-1}\mathit{x},{\overline{\beta }}_{t-1}^2\mathit{I})}{\mathcal{N}({\overline{\alpha }}_t\mathit{x},{\overline{\beta }}_t^2\mathit{I})}\text{(37)}& & \propto \exp \sum _{i=1}^k[-\frac{1}{2}[\frac{(z_t-{\alpha }_t{z}_{t-1}{)}^2}{{\beta }_t^2}+\frac{(z_{t-1}-{\overline{\alpha }}_{t-1}x{)}^2}{{\overline{\beta }}_{t-1}^2}-\frac{(z_t-{\overline{\alpha }}_{t}x{)}^2}{{\overline{\beta }}_t^2}]]\text{(38)}& & \propto \exp \sum _{i=1}^k[-\frac{1}{2}[(\frac{{\alpha }_t^2}{{\beta }_t^2}+\frac{1}{{\overline{\beta }}_{t-1}^2})z_{t-1}^2-2(\frac{{\alpha }_t}{{\beta }_t^2}{z}_t+\frac{{\overline{\alpha }}_{t-1}}{{\overline{\beta }}_{t-1}^2}x)z_{t-1}]]\end{array}$$
根据高斯分布，我们配平方 ${\displaystyle a{x}^2-2bx=a(x-\frac{b}{a}{)}^2+C}$可以得到：

均值有 $$\begin{array}{}\text{(39)}& \mathbb{E}[{\mathit{z}}_{t-1}\mid {\mathit{z}}_t,\mathit{x}]& =(\frac{{\alpha }_t}{{\beta }_t^2}{\mathit{z}}_t+\frac{{\overline{\alpha }}_{t-1}}{{\overline{\beta }}_{t-1}^2}\mathit{x})/(\frac{{\alpha }_t^2}{{\beta }_t^2}+\frac{1}{{\overline{\beta }}_{t-1}^2})=\frac{{\overline{\beta }}_{t-1}^2}{{\overline{\beta }}_t^2}{\alpha }_t{\mathit{z}}_t+\frac{{\beta }_t^2}{{\overline{\beta }}_t^2}{\overline{\alpha }}_{t-1}\mathit{x}\end{array}$$

方差有
$$\begin{array}{}\text{(40)}& \mathbb{Cov}[{\mathit{z}}_{t-1}\mid {\mathit{z}}_t,\mathit{x}]& =1/(\frac{{\alpha }_t^2}{{\beta }_t^2}+\frac{1}{{\overline{\beta }}_{t-1}^2})\mathit{I}=1/(\frac{{\alpha }_t^2(1-{\overline{\alpha }}_{t-1}^2)+{\beta }_t^2}{{\beta }_t^2(1-{\overline{\alpha }}_{t-1}^2)})\mathit{I}=\frac{1-{\overline{\alpha }}_{t-1}^2}{1-{\overline{\alpha }}_t^2}{\beta }_t^2=\frac{{\overline{\beta }}_{t-1}^2}{{\overline{\beta }}_t^2}{\beta }_t^2\mathit{I}\end{array}$$

也就是说：
$$\begin{array}{}\text{(41)}& q({\mathit{z}}_{t-1}\mid {\mathit{z}}_t,\mathit{x})=\mathcal{N}(\frac{{\overline{\beta }}_{t-1}^2}{{\overline{\beta }}_t^2}{\alpha }_t{\mathit{z}}_t+\frac{{\beta }_t^2}{{\overline{\beta }}_t^2}{\overline{\alpha }}_{t-1}\mathit{x},\frac{{\overline{\beta }}_{t-1}^2}{{\overline{\beta }}_t^2}{\beta }_t^2\mathit{I})\end{array}$$

在编码过程(变成噪声)中，我们有 ${\displaystyle q({\mathit{z}}_t\mid \mathit{x})\sim \mathcal{N}({\overline{\alpha }}_t\mathit{x},{\overline{\beta }}_t^2)=\mathcal{N}(\prod _{\tau =1}^t{\alpha }_{\tau }\mathit{x},[1-\prod _{\tau =1}^t{\alpha }_{\tau }^2]\mathit{I})}$ 我们改变一下表现形式

$$\begin{array}{}\text{(42)}& {\mathit{z}}_t={\overline{\alpha }}_t\mathit{x}+{\overline{\beta }}_t\mathit{ϵ}\Rightarrow \mathit{x}=\frac{1}{{\overline{\alpha }}_t}({\mathit{z}}_t-{\overline{\beta }}_t\mathit{ϵ})\end{array}$$我们带入上式，进一步考察均值
$$\begin{array}{}\text{(43)}& \mathbb{E}[{\mathit{z}}_{t-1}\mid {\mathit{z}}_t,\mathit{x}]& =\frac{{\overline{\beta }}_{t-1}^2}{{\overline{\beta }}_t^2}{\alpha }_t{\mathit{z}}_t+\frac{{\beta }_t^2}{{\overline{\beta }}_t^2}{\overline{\alpha }}_{t-1}\mathit{x}=\frac{{\overline{\beta }}_{t-1}^2}{{\overline{\beta }}_t^2}{\alpha }_t{\mathit{z}}_t+\frac{{\beta }_t^2}{{\overline{\beta }}_t^2}{\overline{\alpha }}_{t-1}\frac{1}{{\overline{\alpha }}_t}({\mathit{z}}_t-{\overline{\beta }}_t\mathit{ϵ})\text{(44)}& & =[\frac{{\overline{\beta }}_{t-1}^2}{{\overline{\beta }}_t^2}{\alpha }_t+\frac{{\beta }_t^2}{{\overline{\beta }}_t^2}\frac{{\overline{\alpha }}_{t-1}}{{\overline{\alpha }}_t}]{\mathit{z}}_t-\frac{{\beta }_t^2}{{\overline{\beta }}_t^2}\frac{{\overline{\alpha }}_{t-1}{\overline{\beta }}_t}{{\overline{\alpha }}_t}\mathit{ϵ}\text{(45)}& & =[\frac{1-{\overline{\alpha }}_{t-1}^2}{{\overline{\beta }}_t^2}{\alpha }_t+\frac{{\beta }_t^2}{{\overline{\beta }}_t^2}\frac{1}{{\alpha }_t}]{\mathit{z}}_t-\frac{{\beta }_t^2}{{\overline{\beta }}_t}\frac{1}{{\alpha }_t}\mathit{ϵ}\text{(46)}& & =[\frac{{\alpha }_t^2+{\beta }_t^2-{\overline{\alpha }}_{t-1}^2{\alpha }_t^2}{{\overline{\beta }}_t^2{\alpha }_t}]{\mathit{z}}_t-\frac{{\beta }_t^2}{{\overline{\beta }}_t}\frac{1}{{\alpha }_t}\mathit{ϵ}\text{(47)}& & =[\frac{1-{\overline{\alpha }}_t^2}{{\overline{\beta }}_t^2{\alpha }_t}]{\mathit{z}}_t-\frac{{\beta }_t^2}{{\overline{\beta }}_t}\frac{1}{{\alpha }_t}\mathit{ϵ}\text{(48)}& & =\frac{1}{{\alpha }_t}{\mathit{z}}_t-\frac{{\beta }_t^2}{{\overline{\beta }}_t}\frac{1}{{\alpha }_t}\mathit{ϵ}\end{array}$$

我们知道 ${\displaystyle \mathbb{KL}[\mathcal{N}({\mu }_1,{\sigma }_1^2)|\mathcal{N}({\mu }_2,{\sigma }_2^2)]=\frac{({\mu }_1-{\mu }_2{)}^2+{\sigma }_1^2}{2{\sigma }_2^2}+\log \frac{{\sigma }_2}{{\sigma }_1}-\frac{1}{2}}$，同时 ${\displaystyle p({\mathit{z}}_{t-1}\mid {\mathit{z}}_t)=\mathcal{N}({\mathit{\mu }}_{\mathit{\theta }}[{\mathit{z}}_t,t],{\mathbf{\Sigma }}_{\mathit{\theta }}[{\mathit{z}}_t,t])}$ 是我们真实的生成分布， 其中 ${\displaystyle {\mathit{\mu }}_{\mathit{\theta }}[{\mathit{z}}_t,t]}$ 是我们要学习的神经网络，用神经网络原因很简单图像数据太复杂。那么还剩下 ${\displaystyle {\mathbf{\Sigma }}_{\mathit{\theta }}[{\mathit{z}}_t,t]}$，我们注意到：

$$\begin{array}{}\text{(49)}& q({\mathit{z}}_{t-1}\mid {\mathit{z}}_t,\mathit{x})=\mathcal{N}(\frac{{\overline{\beta }}_{t-1}^2}{{\overline{\beta }}_t^2}{\alpha }_t{\mathit{z}}_t+\frac{{\beta }_t^2}{{\overline{\beta }}_t^2}{\overline{\alpha }}_{t-1}\mathit{x},\frac{{\overline{\beta }}_{t-1}^2}{{\overline{\beta }}_t^2}{\beta }_t^2\mathit{I})\end{array}$$

注意到损失函数 ${\displaystyle \mathbb{KL}[q({\mathit{z}}_{t-1}\mid {\mathit{z}}_t,\mathit{x})|p({\mathit{z}}_{t-1}\mid {\mathit{z}}_t)]}$，使得最小化的最简单的方法就是令 ${\displaystyle {\mathbf{\Sigma }}_{\mathit{\theta }}[{\mathit{z}}_t,t]={\sigma }_t^2\mathit{I}}$, ${\displaystyle {\sigma }_t^2=\frac{{\overline{\beta }}_{t-1}^2}{{\overline{\beta }}_t^2}{\beta }_t^2}$(论文作者是这么干的)。 ${\displaystyle {\sigma }_t^2}$ 其实可以取其他的值，在这之后的模型中提出的各种方案。那么有

$$\begin{array}{}\text{(50)}& \sum _{t=2}^T{\mathbb{E}}_{\mathit{x}}[\mathbb{KL}[q({\mathit{z}}_{t-1}\mid {\mathit{z}}_t,\mathit{x})|p({\mathit{z}}_{t-1}\mid {\mathit{z}}_t)]]& =\sum _{t=2}^T{\mathbb{E}}_{t,\mathit{x},\mathit{ϵ}}[\frac{1}{2{\sigma }_t^2}|\frac{1}{{\alpha }_t}{\mathit{z}}_t-\frac{{\beta }_t^2}{{\overline{\beta }}_t}\frac{1}{{\alpha }_t}\mathit{ϵ}-{\mathit{\mu }}_{\mathit{\theta }}[{\mathit{z}}_t,t]{|}^2]+Const\end{array}$$

变形神经网络形式如下：${\displaystyle {\mathit{\mu }}_{\mathit{\theta }}[{\mathit{z}}_t,t]=[\frac{1}{{\alpha }_t}{\mathit{z}}_t-\frac{{\beta }_t^2}{{\overline{\beta }}_t}\frac{1}{{\alpha }_t}{\mathit{ϵ}}_{\mathit{\theta }}[{\mathit{z}}_t,t]]}$，那么可以简化得到：

$$\begin{array}{}\text{(51)}& {\mathcal{L}}_{1:T-1}=\sum _{t=2}^T{\mathbb{E}}_{t,\mathit{x},\mathit{ϵ}}[\mathbb{KL}[q({\mathit{z}}_{t-1}\mid {\mathit{z}}_t,\mathit{x})|p({\mathit{z}}_{t-1}\mid {\mathit{z}}_t)]]& =\sum _{t=2}^T{\mathbb{E}}_{t,\mathit{x},\mathit{ϵ}}[\frac{{\beta }_t^4}{2{\sigma }_t^2{\alpha }_t^2{\overline{\beta }}_t^2}|\mathit{ϵ}-{\mathit{ϵ}}_{\mathit{\theta }}[{\mathit{z}}_t,t]{|}^2]+Const\end{array}$$

3.2.2、${\displaystyle {\mathcal{L}}_0}$

接下来我们考察最后一项 ${\displaystyle {\mathcal{L}}_0={\mathbb{E}}_{q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})}[\log p(\mathit{x}\mid {\mathit{z}}_1)]}$

$$\begin{array}{}\text{(52)}& {\mathcal{L}}_0& ={\mathbb{E}}_{\mathit{x}}[{\mathbb{E}}_{q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})}[\log p(\mathit{x}\mid {\mathit{z}}_1)]]={\mathbb{E}}_{\mathit{x}}[\int q({\mathcal{D}}_T^{\mathit{z}}\mid \mathit{x})\log p(\mathit{x}\mid {\mathit{z}}_1)d{\mathcal{D}}_T^{\mathit{z}}]\text{(53)}& & ={\mathbb{E}}_{\mathit{x}}[\int q({\mathit{z}}_1\mid \mathit{x})\log p(\mathit{x}\mid {\mathit{z}}_1)d{\mathit{z}}_1]\end{array}$$

由于 ${\displaystyle {\mathit{z}}_1={\alpha }_1\mathit{x}+{\beta }_1{\mathit{ϵ}}_1}$，${\displaystyle p(\mathit{x}\mid {\mathit{z}}_1)=\mathcal{N}({\mathit{\mu }}_{\mathit{\theta }}[{\mathit{z}}_1,1],{\mathbf{\Sigma }}_{\mathit{\theta }}[{\mathit{z}}_1,1]={\sigma }_1^2)}$，有 ${\displaystyle {\mathit{\mu }}_{\mathit{\theta }}[{\mathit{z}}_1,1]=[\frac{1}{{\alpha }_1}{\mathit{z}}_1-\frac{{\beta }_1^2}{{\overline{\beta }}_1}\frac{1}{{\alpha }_1}{\mathit{ϵ}}_{\mathit{\theta }}[{\mathit{z}}_1,1]]}$于是有：

$$\begin{array}{}\text{(54)}& {\mathcal{L}}_0& ={\mathbb{E}}_{t,\mathit{x},{\mathit{ϵ}}_1}[\frac{1}{2{\sigma }_1^2}|\mathit{x}-{\mathit{\mu }}_{\mathit{\theta }}[{\mathit{z}}_1,1]{|}^2]+Const\text{(55)}& & ={\mathbb{E}}_{t,\mathit{x},{\mathit{ϵ}}_1}[\frac{1}{2{\sigma }_1^2}|\mathit{x}-\frac{1}{{\alpha }_1}[{\alpha }_1\mathit{x}+{\beta }_1{\mathit{ϵ}}_1]+\frac{{\beta }_1^2}{{\overline{\beta }}_1}\frac{1}{{\alpha }_1}{\mathit{ϵ}}_{\mathit{\theta }}[{\mathit{z}}_1,1]{|}^2]+Const\text{(56)}& & ={\mathbb{E}}_{t,\mathit{x},{\mathit{ϵ}}_1}[\frac{1}{2{\sigma }_1^2}|-\frac{{\beta }_1}{{\alpha }_1}{\mathit{ϵ}}_1+\frac{{\beta }_1^2}{{\overline{\beta }}_1}\frac{1}{{\alpha }_1}{\mathit{ϵ}}_{\mathit{\theta }}[{\mathit{z}}_1,1]{|}^2]+Const\text{(57)}& & ={\mathbb{E}}_{t,\mathit{x},{\mathit{ϵ}}_1}[\frac{{\beta }_1^4}{2{\sigma }_1^2{\alpha }_1^2{\overline{\beta }}_1^2}|{\mathit{ϵ}}_1-{\mathit{ϵ}}_{\mathit{\theta }}[{\overline{\alpha }}_1\mathit{x}+{\overline{\beta }}_1{\mathit{ϵ}}_1,1]{|}^2]+Const\end{array}$$

3.2.3、${\displaystyle {\mathcal{L}}_{0:T-1}}$

对于 ${\displaystyle {\mathcal{L}}_{0:T-1}}$，令 ${\displaystyle {\mathcal{L}}_{t-1}={\mathbb{E}}_{t,\mathit{x},\mathit{ϵ}}[\frac{{\beta }_t^4}{2{\sigma }_t^2{\alpha }_t^2{\overline{\beta }}_t^2}|\mathit{ϵ}-{\mathit{ϵ}}_{\mathit{\theta }}[{\mathit{z}}_t,t]{|}^2]+Const}$, 我们去掉与参数无关的权重，简化得到${\displaystyle {\mathcal{L}}_{0:T-1}}$的统一表达式：

$$\begin{array}{}\text{(58)}& {\mathcal{L}}_{t-1}={\mathbb{E}}_{t,\mathit{x},\mathit{ϵ}}[|\mathit{ϵ}-{\mathit{ϵ}}_{\mathit{\theta }}[{\overline{\alpha }}_t\mathit{x}+{\overline{\beta }}_t\mathit{ϵ},t]{|}^2]+Const\end{array}$$

于是有损失函数：

$$\begin{array}{}\text{(59)}& \mathcal{L}=\sum _{t=1}^T{\mathbb{E}}_{t,\mathit{x},\mathit{ϵ}}[\frac{{\beta }_t^4}{2{\sigma }_t^2{\alpha }_t^2{\overline{\beta }}_t^2}|\mathit{ϵ}-{\mathit{ϵ}}_{\mathit{\theta }}[{\mathit{z}}_t,t]{|}^2]+Const\end{array}$$

这样，我们有算法：
$$\begin{array}{}\text{(60)}& \underline{\begin{array}{l}\text{Algorithm 1 Training}\\ 1:\text{repeat}\\ 2:{\mathbf{x}}_0\sim q\left({\mathbf{x}}_0\right)\\ 3:t\sim \mathrm{Uniform}(\{1,\dots ,T\})\\ 4:\mathit{ϵ}\sim \mathcal{N}(\mathbf{0},\mathbf{I})\\ 5:\text{Take gradient descent step on}\\ {\mathrm{\nabla }}_{\theta }|\mathit{ϵ}-{\mathit{ϵ}}_{\theta }[{\overline{\alpha }}_t\mathit{x}+{\overline{\beta }}_t\mathit{ϵ},t]{|}^2\\ 6:\text{until converged}\end{array}}\end{array}$$

对于采样生成有：
$$\begin{array}{}\text{(61)}& \begin{array}{l}\text{Algorithm 2 Sampling}\\ 1:{\mathit{z}}_T\sim \mathcal{N}(\mathbf{0},\mathit{I})\\ 2:\text{for}t=T,\cdots ,1\mathbf{do}\\ 3:\mathit{\xi }\sim \mathcal{N}(\mathbf{0},\mathit{I})\text{ if }t>1,\text{ else }\mathit{\xi }=\mathbf{0}\\ 4:{\mathit{z}}_{t-1}=\frac{1}{{\alpha }_t}{\mathit{z}}_t-\frac{{\beta }_t^2}{{\overline{\beta }}_t}\frac{1}{{\alpha }_t}{\mathit{ϵ}}_{\mathit{\theta }}[{\mathit{z}}_t,t]+{\sigma }_t\mathit{\xi }\\ 5:\text{end for}\\ 6:\text{return }\mathit{x}={\mathit{z}}_0\end{array}\end{array}$$

四、评述

1、推理中的一些关键动机，视乎没有加以说明。一些想法视乎是从天而降。 ${\displaystyle {\alpha }_t^2+{\beta }_t^2=1}$似乎还能说的过去。${\displaystyle {\sigma }_t^2=\frac{{\overline{\beta }}_{t-1}^2}{{\overline{\beta }}_t^2}{\beta }_t^2}$似乎就动机不足了，毕竟有那么多可以取的值，不一定非要这个。

2、在逆向生成中${\displaystyle \mathbb{E}[{\mathit{z}}_{t-1}\mid {\mathit{z}}_t,\mathit{x}]}$， 做变形${\displaystyle {\mathit{z}}_t={\overline{\alpha }}_t\mathit{x}+{\overline{\beta }}_t\mathit{ϵ}\Rightarrow \mathit{x}=\frac{1}{{\overline{\alpha }}_t}({\mathit{z}}_t-{\overline{\beta }}_t\mathit{ϵ})}$带入。就是为了消除 ${\displaystyle \mathit{x}}$，从而使得 ${\displaystyle {\mathit{z}}_t\to {\mathit{z}}_{t-1}}$不依赖 ${\displaystyle \mathit{x}}$。而这一步推导DDPM依赖了马尔可夫假设，显然限制了分布空间。如果我们去掉马尔可夫假设，我们可以得到更大自由度。

3、最终的结果是简单，一定还有更加显然的推理方式，论文DDPM的推理总觉得还是太过迂回，与不显然。接下我们将使用宋大神SDE的思想来解读这个模型。

4、DDPM以后有一系列的改进，也将一一解读，敬请关注。

Ho, J., Jain, A., & Abbeel, P. (2020, December 16). Denoising Diffusion Probabilistic Models. arXiv. http://arxiv.org/abs/2006.11239. Accessed 22 March 2023
Kingma, D. P., & Welling, M. (2022, December 10). Auto-Encoding Variational Bayes. arXiv. http://arxiv.org/abs/1312.6114. Accessed 18 April 2023

---

版权声明
	由引线小白创作并维护的柠檬CC博客采用署名-非商业-禁止演绎4.0国际许可证。 本文首发于柠檬CC [ https://www.limoncc.com ] , 版权所有、侵权必究。
本文永久链接	httpss://www.limoncc.com/post/1c60669bbe56769f/

如果您需要引用本文，请参考：

引线小白. (May. 16, 2023). 《扩散模型研究一：去噪扩散概率模型DDPM》[Blog post]. Retrieved from https://www.limoncc.com/post/1c60669bbe56769f

@online{limoncc-1c60669bbe56769f, title={扩散模型研究一：去噪扩散概率模型DDPM}, author={引线小白}, year={2023}, month={May}, date={16}, url={\url{https://www.limoncc.com/post/1c60669bbe56769f}}, }