python：scipy.optimize.fmin

简介

优化问题是工程实践中经常遇到的一种问题。简单讲，就是搜索优化出一组自变量参数，使得目标函数达到极小值(极大值)。

如何搜索出这组参数呢：这就是优化算法做的事情。不同的优化算法适用于不同的优化问题。

本文简要介绍在python种NM算法来解决局部优化问题。

注意：scipy.optimize中的fmin和minimize都能调用NM算法来优化。两者区别：

• minimize是更通用的优化算法接口，它不仅包含NM算法，也可以调用其他算法；而fmin就只能用NM算法
• minimize中的NM算法多一个adaptive参数，可以提高高维的优化问题的收敛速度
• fmin的一个好处：支持直接返回优化过程中的迭代参数，有助于可视化。

本文用fmin来重新实现一遍minimize的例子。

接口

实例1：Himmelblau函数

优化流程

代码

def cost_function(x):
    return (x[0]**2+x[1]-11)**2+(x[0]+x[1]**2-7)**2

x_center = np.array([0,0])
step = 0.5
x0 = np.vstack((x_center, x_center+np.diag((step,step))))
xtol,ftol = 1e-3,1e-3
xopt,fopt,iter,funcalls,warnflags,allvecs = fmin(cost_function,x_center,initial_simplex=x0,xtol=xtol,ftol=ftol,disp=1,retall=1,full_output=1)
print(xopt,fopt)

n = 50
x = np.linspace(-6,6,n)
y = np.linspace(-6,6,n)
z = np.zeros((n,n))
for i,a in enumerate(x):
    for j,b in enumerate(y):
        z[i,j] = cost_function([a,b])

xx, yy = np.meshgrid(x,y)
fig, ax = plt.subplots()
c = ax.pcolormesh(xx,yy,z.T,cmap='jet')
fig.colorbar(c, ax=ax)

t = np.asarray(allvecs)
x_, y_ = t[:,0], t[:,1]
ax.plot(x_,y_,'r',x_[0],y_[0],'go',x_[-1],y_[-1],'y+',markersize=6)

fig2 = plt.figure()
ax1 = plt.subplot(221)
ax2 = plt.subplot(222)
ax3 = plt.subplot(212)
ax1.plot(x_)
ax1.set_title('x')
ax2.plot(y_)
ax2.set_title('y')
ax3.plot(ys)

结果

Optimization terminated successfully.
         Current function value: 0.000002
         Iterations: 35
         Function evaluations: 68
[3.00021471 1.99974856] 1.7007332966814985e-06

对初值敏感

代码

def cost_function(x):
    return (x[0]**2+x[1]-11)**2+(x[0]+x[1]**2-7)**2

n = 50
x = np.linspace(-6,6,n)
y = np.linspace(-6,6,n)
z = np.zeros((n,n))
for i,a in enumerate(x):
    for j,b in enumerate(y):
        z[i,j] = cost_function([a,b])
xx, yy = np.meshgrid(x,y)
fig, axes = plt.subplots(2, 2, figsize=(12,8))

centers = [[0,0],[-1,0],[0,-1],[-1,-1]]
for i,center in enumerate(centers):
    x_center = np.array(center)
    step = 0.5
    x0 = np.vstack((x_center, x_center+np.diag((step,step))))
    xtol,ftol = 1e-3,1e-3
    xopt,fopt,iter,funcalls,warnflags,allvecs = fmin(cost_function,x_center,initial_simplex=x0,xtol=xtol,ftol=ftol,disp=1,retall=1,full_output=1)
    print(xopt,fopt)

    ii,jj = i//2,i%2
    ax = axes[ii][jj]
    c = ax.pcolormesh(xx,yy,z.T,cmap='jet')
    fig.colorbar(c, ax=ax)

    t = np.asarray(allvecs)
    x_, y_ = t[:,0], t[:,1]
    ax.plot(x_,y_,'r',x_[0],y_[0],'go',x_[-1],y_[-1],'y+',markersize=6)

结果

Optimization terminated successfully.
         Current function value: 0.000002
         Iterations: 35
         Function evaluations: 68
[3.00021471 1.99974856] 1.7007332966814985e-06
Optimization terminated successfully.
         Current function value: 0.000003
         Iterations: 31
         Function evaluations: 60
[-2.80534914  3.13148297] 2.851032148018009e-06
Optimization terminated successfully.
         Current function value: 0.000001
         Iterations: 40
         Function evaluations: 74
[ 3.58440786 -1.8484002 ] 1.1592413943034185e-06
Optimization terminated successfully.
         Current function value: 0.000002
         Iterations: 37
         Function evaluations: 69
[-3.77937271 -3.28299281] 2.2138273459910166e-06