基于组合优化的Kriging参数估计算法

摘要/Abstract

摘要： 在使用常微分方程组描述的数学模型进行参数估计时，本文使用Kriging代理模型完成优化过程。该代理模型通过少量数据点的训练即可部分替代计算费时的原始目标函数优化过程，因此可以节省大量的计算时间。在Kriging代理模型精化过程中，查找新增点的优化算法对参数估计的结果有重要影响。本文针对非线性且具有sloppiness属性的常微分方程组形式的参数估计问题，组合具有二阶动量特征的Adam算法及一阶动量梯度下降算法的各自优势用于搜索模型精化时所需添加的新样例点，从而提高收敛速度及查找质量。通过与其他优化算法相比对，验证了该组合算法的实际有效性。

关键词: 组合优化, 参数估计, Kriging代理模型

Abstract: Kriging surrogate can be used to identify the parameters for the mathematical models described by ordinary differential equations. By training a relatively small set of samples, Kriging surrogate can partially replace the time-consuming original objective function optimization process, so it can save a lot of computation time. And the optimization algorithm of searching new samples has major impacts on result of the parameter estimation during the process of refining Kriging surrogate models. For the problem of parameters estimation described by ordinary differential equations with nonlinearity and sloppiness, this paper combines the advantages of Adam with second order momentum and SGD with momentum to search for new sample points that need to be added during model refinement, so as to improve the convergence speed and search quality. Compared with other optimization algorithms, the effectiveness of the proposed algorithm is verified.

Key words: combinatorial optimization, parameter estimation, Kriging surrogate model

王红. 基于组合优化的Kriging参数估计算法[J]. 计算机与现代化, 2021, 0(09): 51-56.

WANG Hong. Kriging Parameter Estimation Algorithm Based on Combinatorial Optimization[J]. Computer and Modernization, 2021, 0(09): 51-56.

参考文献

［1］ DUFRESNE E, HARRINGTON H A, RAMAN D V. The geometry of sloppiness［J］. Metric Geometry, 2016,arXiv:1608.05679.
［2］ TAFINTSEVA V, TNDEL K, PONOSOV A, et al. Global structure of sloppiness in a nonlinear model［J］. Journal of Chemometrics, 2015,28(8):645-655.
［3］ QUINN K N, WILBER H, TOWNSEND A, et al. Chebyshev approximation and the global geometry of sloppy models［J］. Numerical Analysis, 2018,arXiv:1809.08280.
［4］ TRANSTRUM M K, MACHTA B B, SETHNA J P. The geometry of nonlinear least squares with applications to sloppy models and optimization［J］. Physical Review E Statal Nonlinear & Soft Matter Physics, 2010,83:036701.
［5］ TAN Q, GHOSAL S. Bayesian analysis of mixed-effect regression models driven by ordinary differential equations［J］. Sankhya B, 2019,83:3-29.
［6］ KIMURA S, IDE K, KASHIHARA A, et al. Inference of S-system models of genetic networks using a cooperative coevolutionary algorithm［J］. Bioinformatics, 2005,21(7):1154-1163.
［7］王红,王希诚,李克秋. 混合Kriging代理模型的高维参数估计优化算法［J］. 大连理工大学学报, 2015(2):215-222.〖HJ1.0mm〗
［8］高月华,王希诚. 基于Kriging代理模型的多点加点序列优化方法［J］. 工程力学, 2012,29(4):90-95.
［9］韩忠华. Kriging模型及代理优化算法研究进展［J］. 航空学报, 2016,37(11):3197-3225.
［10］SIMPSON T W, POPLINSKI J D, KOCH P N, et al. Meta models for computer-based engineering design: Survey and recommendations［J］. Engineering with Computers, 2001,17(2):129-150.
［11］SHIMOYAMA K, JEONG S, OBAYASHI S. Kriging-surrogate-based optimization considering expected hyper volume improvement in non-constrained many-objective test problems［C］// 2013 IEEE Congress on Evolutionary Computation. 2013:658-665.
［12］VIANA F A C, HAFTKA R T. Surrogate-based optimization with parallel simulations using the probability of improvement［C］// The 13th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference. 2010:13-15.
［13］SCHONLAU M, WELCH W J, JONES D R. Global versus local search in constrained optimization of computer models［J］. Lecture Notes-monograph Series, 1998,34:11-25.
［14］SASENA M J, PAPALAMBROS P, GOOVAERTS P. Exploration of meta modeling sampling criteria for constrained global optimization［J］. Engineering Optimization, 2002,34(3):263-278.
［15］BOTTOU L. SGD: Stochastic Gradient Descent［EB/OL］. (2018-09-03)［2020-10-27］. http://leon.bottou.org/projects/sgd.
［16］知乎. 一个框架看懂优化算法之异同［EB/OL］. (2018-09-03)［2020-10-27］. https://zhuanlan.zhihu.com/p/32262540.
［17］DU S S， JIN C， LEE J D, et al. Gradient descent can take exponential time to escape saddle points［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017：1067-1077.
［18］NESTEROV Y． A method of solving a convex programming problem with convergence rate o(1／k2)［J］. Soviet Mathematics Doklady, 1983,27(2):372-376.
［19］SUTSKEVER I, MARTENS J, DAHL G, et al. On the importance of initialization and momentum in deep learning［C］// Proceedings of the 30th International Conference on International Conference on Machine Learning. 2013,28:1139-1147.
［20］REDDI S J, KALE S, KUMAR S. On the convergence of Adam and beyond［J］. Machine Learning, 2019,arXiv:1904.09237.
［21］KESKAR N S, SOCHER R. Improving generalization performance by switching from Adam to SGD［J］. Machine Learning, 2017,arXiv:1712.07628.
［22］YANG K, BAI H J, OUYANG Q, et al. Finding multiple target optimal intervention in disease related molecular network［J］. Molecular Systems Biology, 2008,4(228),DOI:10.1038/msb.2008.60.
［23］BROYDEN C G. Quasi-Newton methods and their application to function mini misation［J］. Mathematics of Computation, 1967,21(99):368-368.
［24］EGEAJ A, RODRIGUES-FERNADEZ M, BANGA J R, et al. Scatter search for chemical and bioprocess optimization［J］. Journal of Global Optimization, 2007,37(3):481-503.

[1]	刘雅文, 蒋妍, 潘大志. 改进二进制和声搜索算法求解多维背包问题[J]. 计算机与现代化, 2022, 0(08): 13-19.
[2]	白雪, 程宗毛. 无线传感网络监测质量的统计分析[J]. 计算机与现代化, 2021, 0(08): 1-5.
[3]	吴学龙，徐维超. 基于AUC的非参数快速变点检测算法[J]. 计算机与现代化, 2015, 0(7): 5-.
[4]	龙钧宇;余爱民. 基于小波包变换的非局部均值去噪方法[J]. 计算机与现代化, 2013, 1(11): 13-16.
[5]	谢盛嘉. 一种改进选择算子的遗传算法在泊松曲线沉降预测模型中的应用[J]. 计算机与现代化, 2012, 203(7): 6-8.
[6]	马晓慧;王红;. 改进的PSO在TSP中的应用[J]. 计算机与现代化, 2011, 193(9): 5-7,11.
[7]	谢盛嘉. 基于自适应并行遗传算法的泊松曲线沉降预测模型[J]. 计算机与现代化, 2011, 1(8): 20-22.
[8]	王兴起;薛晓春. 一种求解欧式平面TSP问题的混合算法[J]. 计算机与现代化, 2010, 1(5): 33-35,3.
[9]	王辉. 用遗传算法求解TSP问题[J]. 计算机与现代化, 2009, 1(7): 12-16,2.