在线咨询
中国工业与应用数学学会会刊
主管：中华人民共和国教育部
主办：西安交通大学
ISSN 1005-3085  CN 61-1269/O1
工程数学学报

工程数学学报 ›› 2026, Vol. 43 ›› Issue (3): 564-578.doi: 10.3969/j.issn.1005-3085.2026.03.012cstr: 32411.14.cjem.CN61-1269/O1.2026.03.012

• • 上一篇    下一篇

基于自适应动量的SGDM在机器学习中的应用

王贝宁1,2,   杨建奎1,2   

  1. 1. 北京邮电大学数学科学学院，北京 100876
    2. 北京邮电大学数学与信息网络教育部重点实验室，北京 100876
  • 收稿日期:2024-11-30 接受日期:2025-09-29 出版日期:2026-04-15 发布日期:2026-08-15
  • 通讯作者: 杨建奎 E-mail: yangjk@bupt.edu.cn
  • 基金资助:
    国家自然科学基金 (12571344).

The Application of SGDM with Adaptive Momentum in Machine Learning

WANG Beining1,2,  YANG Jiankui1,2   

  1. 1. School of Mathematical Sciences, Beijing University of Posts and Telecommunications, Beijing 100876

    2. Key Laboratory of Mathematics and Information Network of the Ministry of Education, Beijing University of Posts and Telecommunications, Beijing 100876

  • Received:2024-11-30 Accepted:2025-09-29 Online:2026-04-15 Published:2026-08-15
  • Contact: J. Yang. E-mail address: yangjk@bupt.edu.cn
  • Supported by:
    The National Natural Science Foundation of China (12571344).

摘要：

随机梯度下降动量法(Stochastic Gradient Descent with Momentum, SGDM)是一种广泛应用于求解机器学习问题的优化方法。该算法通过累积历史梯度来加速训练，但由于噪声的累积可能引发超调现象。基于非线性共轭梯度参数提出了一种自适应动量的SGDM算法---PRPSGDM，该算法通过自适应调整动量系数，合理控制高噪声梯度在加速下降过程中的影响。此外，还对该算法进行了偏差分析和非凸随机优化问题下的收敛性分析。数值实验验证了算法在求解非凸支持向量机问题下的有效性。

关键词: 机器学习, SGDM算法, 自适应动量, 共轭梯度算法, 收敛性

Abstract:

Stochastic gradient descent with momentum (SGDM) is a widely used optimization method for solving machine learning problems. This algorithm accelerates training by accumulating historical gradients; however, the accumulation of noise may lead to overshooting. In this paper, we propose an adaptive momentum SGDM algorithm, named PRPSGDM, based on nonlinear conjugate gradient parameters. The proposed algorithm adaptively adjusts the momentum coefficient to effectively control the impact of high-noise gradients during the accelerated descent process. Additionally, we conduct bias analysis and convergence analysis under non-convex stochastic optimization settings. Numerical experiments demonstrate the effectiveness of the algorithm in solving non-convex support vector machine problems.

Key words: machine learning, SGDM algorithm, adaptive momentum, conjugate gradient algorithm, convergence

中图分类号: