基于贝叶斯方法学习分类分布参数

最近在看计算机视觉：模型学习和推理，现在在使用c++实现里面的代码。本篇博客使用c++实现基于贝叶斯方法学习分类分布的参数。

$$\begin{array}{rl}\mathrm{Pr}\left({\lambda }_1\dots {\lambda }_k|x_{1\dots I}\right)& =\frac{{\prod }_{i=1}^I\mathrm{Pr}\left(x_i|{\lambda }_{1\dots k}\right)\mathrm{Pr}\left({\lambda }_{1\dots k}\right)}{\mathrm{Pr}\left(x_{1\dots I}\right)}\\ & =\frac{{\prod }_{i=1}^I{\mathrm{Cat}}_{x_i}\left[{\lambda }_{1\dots k}\right]{\mathrm{Dir}}_{{\lambda }_1\dots k}\left[{\alpha }_{1\dots k}\right]}{\mathrm{Pr}\left(x_{1\dots I}\right)}\\ & =\frac{\kappa \left({\alpha }_{1\dots k},x_{1\dots I}\right){\mathrm{Dir}}_{{\lambda }_{1\dots k}}\left[{\tilde{\alpha }}_{1\dots k}\right]}{\mathrm{Pr}\left(x_{1\dots I}\right)}\\ & ={\mathrm{Dir}}_{{\lambda }_{1\dots k}\left[{\tilde{\alpha }}_{1\dots k}\right]}\end{array}$$

使用贝叶斯去做预测:

$$\begin{array}{rl}\mathrm{Pr}\left(x^{\ast }|x_{1\dots I}\right)& =\int \mathrm{Pr}\left(x^{\ast }|{\lambda }_{1\dots k}\right)\mathrm{Pr}\left({\lambda }_{1\dots k}|x_{1\dots I}\right)d{\lambda }_{1\dots k}\\ & =\int {\mathrm{Cat}}_{x^{\ast }}\left[{\lambda }_{1\dots k}\right]{\mathrm{Dir}}_{{\lambda }_{1\dots k}}\left[{\tilde{\alpha }}_{1\dots k}\right]d{\lambda }_{1\dots k}\\ & =\int \kappa \left(x^{\ast },{\tilde{\alpha }}_{1\dots k}\right){\mathrm{Dir}}_{{\lambda }_{1\dots k}}\left[{\breve{\alpha }}_{1\dots k}\right]d{\lambda }_{1\dots k}\\ & =\kappa \left(x^{\ast },{\tilde{\alpha }}_{1\dots k}\right)\end{array}$$

结果表示为： $$\mathrm{Pr}\left(x^{\ast }=k|x_{1\dots I}\right)=\kappa \left(x^{\ast },{\tilde{\alpha }}_{1\dots k}\right)=\frac{N_k+{\tilde{\alpha }}_k}{{\sum }_{j=1}^k\left(N_j+{\tilde{\alpha }}_j\right)}$$

算法流程如下：

$$\begin{array}{l}\text{ Input : Categorical training data }{\left\{x_i\right\}}_{i=1}^I,\text{ Hyperparameters }{\left\{{\alpha }_k\right\}}_{k=1}^K\\ \text{ Output: Posterior parameters }{\left\{{\tilde{\alpha }}_k\right\}}_{k=1}^K,\text{ predictive distribution }\mathrm{Pr}\left(x^{\ast }|{\mathbf{x}}_1\dots I\right)\\ \text{ begin }\\ \text{ l compute categsorical posterior over }\lambda \\ \text{ for }k=l\text{ to }K\text{ do }\\ \text{ Evaluate new datapoint under predictive distribution }\\ \text{ Evaluate new datapoint under predictive distribution }\\ \text{ for }k=1\text{ to }K\text{ do }\\ \mathrm{tr}\left(x^{\ast }=k|{\mathbf{x}}_{1\dots I}\right)=\widetilde{{\alpha }_k}/\left({\sum }_{m=1}^K{\tilde{\alpha }}_m\right)\\ \text{ end }\end{array}$$

代码如下：

void Bayesian_categorical_distribution_parameters() { vector<int> data; data = generate_categorical_distribution_data(100000); std::map<int, double> hist{}; for (int i = 0; i < data.size(); i++) { ++hist[data[i]]; } vector<double> alpha_v; vector<double> alpha_v_post; //set Drichilet distribution superparameters for (int i = 0; i < hist.size(); i++) { alpha_v.push_back(1.0); } double total_p = 0; for (int i = 0; i < hist.size(); i++) { alpha_v_post.push_back(alpha_v[i]+hist.at(i)); } double down = 0; for (int i = 0; i < hist.size(); i++) { down += alpha_v_post[i]; } for (int i = 0; i < hist.size(); i++) { hist.at(i) = alpha_v_post[i] / down; total_p += hist.at(i); std::cout << hist.at(i) << std::endl; } cout << "total_p: " << total_p << endl; }

在书中作者的代码给出了如下两张图，作为对贝叶斯方法的解释