PyTorch实现L1，L2正则化以及Dropout

最新推荐文章于 2024-07-27 20:21:33 发布

原创最新推荐文章于 2024-07-27 20:21:33 发布 · 911 阅读

0 ·

本内容遵循CC 4.0 BY-SA版权协议

PyTorch 专栏收录该内容

9 篇文章

订阅专栏

本文介绍如何使用PyTorch实现L1和L2正则化以及Dropout技术，通过实际代码演示了这些技术在糖尿病数据集上的应用，展示了如何设置网络结构、损失函数、优化器，并通过训练过程调整模型参数以降低过拟合风险。

PyTorch实现L1，L2正则化以及Dropout

"""
Created on 2019/4/14 20:39
@author: Michael
"""

import torch.nn.functional as F
import torch.nn.init as init
import torch
from torch.autograd import Variable
import matplotlib.pyplot as  plt
import math
import pandas as pd
import numpy as np
from sklearn import datasets

diabetes = np.loadtxt("diabetes.csv",delimiter=",",skiprows=1)
print(diabetes[:, 0:-1])
print(diabetes[:, -1])

x_data = Variable(torch.from_numpy(diabetes[:, 0:-1]))
y_data = Variable(torch.from_numpy(diabetes[:, -1]))
print(x_data.data.shape)
print(y_data.data.shape)
x_data = x_data.float()
y_data = y_data.float()

class Model(torch.nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.l1 = torch.nn.Linear(8, 60)
        self.l2 = torch.nn.Linear(60, 4)
        self.l3 = torch.nn.Linear(4, 1)

    def forward(self, x):
        out1 = F.relu(self.l1(x))
        out2 = F.dropout(out1, p= 0.5)
        out3 = F.relu(self.l2(out2))
        out4 = F.dropout(out3, p=0.5)
        y_pred = F.sigmoid(self.l3(out3))
        return y_pred

model = Model()


criterion = torch.nn.BCELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.1, weight_decay=0.1)

Loss=[]
for epoch in range(100):
        y_pred = model(x_data)
        loss = criterion(y_pred, y_data)
        if epoch%5 == 0:
            print("epoch =", epoch, "loss", loss.item())
            Loss.append(loss.item())
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

hour_var = Variable(torch.randn(1, 8))
print("predict", model(hour_var).data[0]>0.5)

plt.plot(loss.detach().numpy())
plt.show()