卷积神经网络

---

在这教程中，主要学习训练CNN，来对CIFAR-10数据集进行图像分类。

该数据集中的图像是彩色小图像，其中被分为了十类。 一些示例图像，如下图所示：

测试GPU是否可以使用

数据集中的图像大小为32x32x3 。在训练的过程中最好使用GPU来加速。

import torch
import numpy as np

# 检查是否可以利用GPU
train_on_gpu = torch.cuda.is_available()

if not train_on_gpu:
    print('CUDA is not available.')
else:
    print('CUDA is available!')

结果：

CUDA is available!

---

加载数据

数据下载可能会比较慢。请耐心等待。 加载训练和测试数据，将训练数据分为训练集和验证集，然后为每个数据集创建DataLoader。

from torchvision import datasets
import torchvision.transforms as transforms
from torch.utils.data.sampler import SubsetRandomSampler

# number of subprocesses to use for data loading
num_workers = 0
# 每批加载16张图片
batch_size = 16
# percentage of training set to use as validation
valid_size = 0.2

# 将数据转换为torch.FloatTensor，并标准化。
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
    ])

# 选择训练集与测试集的数据
train_data = datasets.CIFAR10('data', train=True,
                              download=True, transform=transform)
test_data = datasets.CIFAR10('data', train=False,
                             download=True, transform=transform)

# obtain training indices that will be used for validation
num_train = len(train_data)
indices = list(range(num_train))
np.random.shuffle(indices)
split = int(np.floor(valid_size * num_train))
train_idx, valid_idx = indices[split:], indices[:split]

# define samplers for obtaining training and validation batches
train_sampler = SubsetRandomSampler(train_idx)
valid_sampler = SubsetRandomSampler(valid_idx)

# prepare data loaders (combine dataset and sampler)
train_loader = torch.utils.data.DataLoader(train_data, batch_size=batch_size,
    sampler=train_sampler, num_workers=num_workers)
valid_loader = torch.utils.data.DataLoader(train_data, batch_size=batch_size, 
    sampler=valid_sampler, num_workers=num_workers)
test_loader = torch.utils.data.DataLoader(test_data, batch_size=batch_size, 
    num_workers=num_workers)

# 图像分类中10类别
classes = ['airplane', 'automobile', 'bird', 'cat', 'deer',
           'dog', 'frog', 'horse', 'ship', 'truck']

查看训练集中的一批样本

import matplotlib.pyplot as plt
%matplotlib inline

# helper function to un-normalize and display an image
def imshow(img):
    img = img / 2 + 0.5  # unnormalize
    plt.imshow(np.transpose(img, (1, 2, 0)))  # convert from Tensor image
   
# 获取一批样本
dataiter = iter(train_loader)
images, labels = dataiter.next()
images = images.numpy() # convert images to numpy for display

# 显示图像，标题为类名
fig = plt.figure(figsize=(25, 4))
# 显示16张图片
for idx in np.arange(16):
    ax = fig.add_subplot(2, 16/2, idx+1, xticks=[], yticks=[])
    imshow(images[idx])
    ax.set_title(classes[labels[idx]])

结果：

查看一张图像中的更多细节

在这里，进行了归一化处理。红色、绿色和蓝色（RGB）颜色通道可以被看作三个单独的灰度图像。

rgb_img = np.squeeze(images[3])
channels = ['red channel', 'green channel', 'blue channel']

fig = plt.figure(figsize = (36, 36)) 
for idx in np.arange(rgb_img.shape[0]):
    ax = fig.add_subplot(1, 3, idx + 1)
    img = rgb_img[idx]
    ax.imshow(img, cmap='gray')
    ax.set_title(channels[idx])
    width, height = img.shape
    thresh = img.max()/2.5
    for x in range(width):
        for y in range(height):
            val = round(img[x][y],2) if img[x][y] !=0 else 0
            ax.annotate(str(val), xy=(y,x),
                    horizontalalignment='center',
                    verticalalignment='center', size=8,
                    color='white' if img[x][y]<thresh else 'black')

结果：

---

定义卷积神经网络的结构

这里，将定义一个CNN的结构。 将包括以下内容：

• 卷积层：可以认为是利用图像的多个滤波器（经常被称为卷积操作）进行滤波，得到图像的特征。

  • 通常，我们在 PyTorch 中使用 nn.Conv2d 定义卷积层，并指定以下参数：

    nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0)
    

    • in_channels 是指输入深度。对于灰阶图像来说，深度 = 1
    • out_channels 是指输出深度，或你希望获得的过滤图像数量
    • kernel_size 是卷积核的大小（通常为 3，表示 3x3 核）
    • stride 和 padding 具有默认值，但是应该根据你希望输出在空间维度 x, y 里具有的大小设置它们的值。

  
  用 3x3 窗口和步长 1 进行卷积运算

• 池化层：这里采用的最大池化：对指定大小的窗口里的像素值最大值。

  • 在 2x2 窗口里，取这四个值的最大值。
  • 由于最大池化更适合发现图像边缘等重要特征，适合图像分类任务。
  • 最大池化层通常位于卷积层之后，用于缩小输入的 x-y 维度 。

• 通常的“线性+dropout”层可避免过拟合，并产生输出10类别。

下图中，可以看到这是一个具有2个卷积层的神经网络。

卷积层的输出大小

要计算给定卷积层的输出大小，我们可以执行以下计算：

这里，假设输入大小为(H,W)，滤波器大小为(FH,FW)，输出大小为 (OH,OW)，填充为P，步幅为S。此时，输出大小可通过下面公式进行计算。

例: 输入大小为(H=7,W=7)，滤波器大小为(FH=3,FW=3)，填充为P=0，步幅为S=1, 输出大小为 (OH=5,OW=5)。如果用 S=2，将得输出大小为 (OH=3,OW=3)。

import torch.nn as nn
import torch.nn.functional as F

# 定义卷积神经网络结构
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        # 卷积层 (32x32x3的图像)
        self.conv1 = nn.Conv2d(3, 16, 3, padding=1)
        # 卷积层(16x16x16)
        self.conv2 = nn.Conv2d(16, 32, 3, padding=1)
        # 卷积层(8x8x32)
        self.conv3 = nn.Conv2d(32, 64, 3, padding=1)
        # 最大池化层
        self.pool = nn.MaxPool2d(2, 2)
        # linear layer (64 * 4 * 4 -> 500)
        self.fc1 = nn.Linear(64 * 4 * 4, 500)
        # linear layer (500 -> 10)
        self.fc2 = nn.Linear(500, 10)
        # dropout层 (p=0.3)
        self.dropout = nn.Dropout(0.3)

    def forward(self, x):
        # add sequence of convolutional and max pooling layers
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = self.pool(F.relu(self.conv3(x)))
        # flatten image input
        x = x.view(-1, 64 * 4 * 4)
        # add dropout layer
        x = self.dropout(x)
        # add 1st hidden layer, with relu activation function
        x = F.relu(self.fc1(x))
        # add dropout layer
        x = self.dropout(x)
        # add 2nd hidden layer, with relu activation function
        x = self.fc2(x)
        return x

# create a complete CNN
model = Net()
print(model)

# 使用GPU
if train_on_gpu:
    model.cuda()

结果：

Net(
  (conv1): Conv2d(3, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (conv2): Conv2d(16, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (conv3): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (fc1): Linear(in_features=1024, out_features=500, bias=True)
  (fc2): Linear(in_features=500, out_features=10, bias=True)
  (dropout): Dropout(p=0.3, inplace=False)
)

选择损失函数与优化函数

import torch.optim as optim
# 使用交叉熵损失函数
criterion = nn.CrossEntropyLoss()
# 使用随机梯度下降，学习率lr=0.01
optimizer = optim.SGD(model.parameters(), lr=0.01)

---

训练卷积神经网络模型

注意：训练集和验证集的损失是如何随着时间的推移而减少的；如果验证损失不断增加，则表明可能过拟合现象。（实际上，在下面的例子中，如果n_epochs设置为40，可以发现存在过拟合现象！）

# 训练模型的次数
n_epochs = 30

valid_loss_min = np.Inf # track change in validation loss

for epoch in range(1, n_epochs+1):

    # keep track of training and validation loss
    train_loss = 0.0
    valid_loss = 0.0
    
    ###################
    # 训练集的模型 #
    ###################
    model.train()
    for data, target in train_loader:
        # move tensors to GPU if CUDA is available
        if train_on_gpu:
            data, target = data.cuda(), target.cuda()
        # clear the gradients of all optimized variables
        optimizer.zero_grad()
        # forward pass: compute predicted outputs by passing inputs to the model
        output = model(data)
        # calculate the batch loss
        loss = criterion(output, target)
        # backward pass: compute gradient of the loss with respect to model parameters
        loss.backward()
        # perform a single optimization step (parameter update)
        optimizer.step()
        # update training loss
        train_loss += loss.item()*data.size(0)
        
    ######################    
    # 验证集的模型#
    ######################
    model.eval()
    for data, target in valid_loader:
        # move tensors to GPU if CUDA is available
        if train_on_gpu:
            data, target = data.cuda(), target.cuda()
        # forward pass: compute predicted outputs by passing inputs to the model
        output = model(data)
        # calculate the batch loss
        loss = criterion(output, target)
        # update average validation loss 
        valid_loss += loss.item()*data.size(0)
    
    # 计算平均损失
    train_loss = train_loss/len(train_loader.sampler)
    valid_loss = valid_loss/len(valid_loader.sampler)
        
    # 显示训练集与验证集的损失函数 
    print('Epoch: {} \tTraining Loss: {:.6f} \tValidation Loss: {:.6f}'.format(
        epoch, train_loss, valid_loss))
    
    # 如果验证集损失函数减少，就保存模型。
    if valid_loss <= valid_loss_min:
        print('Validation loss decreased ({:.6f} --> {:.6f}).  Saving model ...'.format(valid_loss_min,valid_loss))
        torch.save(model.state_dict(), 'model_cifar.pt')
        valid_loss_min = valid_loss

结果：

Epoch: 1 	Training Loss: 2.065666 	Validation Loss: 1.706993
Validation loss decreased (inf --> 1.706993).  Saving model ...
Epoch: 2 	Training Loss: 1.609919 	Validation Loss: 1.451288
Validation loss decreased (1.706993 --> 1.451288).  Saving model ...
Epoch: 3 	Training Loss: 1.426175 	Validation Loss: 1.294594
Validation loss decreased (1.451288 --> 1.294594).  Saving model ...
Epoch: 4 	Training Loss: 1.307891 	Validation Loss: 1.182497
Validation loss decreased (1.294594 --> 1.182497).  Saving model ...
Epoch: 5 	Training Loss: 1.200655 	Validation Loss: 1.118825
Validation loss decreased (1.182497 --> 1.118825).  Saving model ...
Epoch: 6 	Training Loss: 1.115498 	Validation Loss: 1.041203
Validation loss decreased (1.118825 --> 1.041203).  Saving model ...
Epoch: 7 	Training Loss: 1.047874 	Validation Loss: 1.020686
Validation loss decreased (1.041203 --> 1.020686).  Saving model ...
Epoch: 8 	Training Loss: 0.991542 	Validation Loss: 0.936289
Validation loss decreased (1.020686 --> 0.936289).  Saving model ...
Epoch: 9 	Training Loss: 0.942437 	Validation Loss: 0.892730
Validation loss decreased (0.936289 --> 0.892730).  Saving model ...
Epoch: 10 	Training Loss: 0.894279 	Validation Loss: 0.875833
Validation loss decreased (0.892730 --> 0.875833).  Saving model ...
Epoch: 11 	Training Loss: 0.859178 	Validation Loss: 0.838847
Validation loss decreased (0.875833 --> 0.838847).  Saving model ...
Epoch: 12 	Training Loss: 0.822664 	Validation Loss: 0.823634
Validation loss decreased (0.838847 --> 0.823634).  Saving model ...
Epoch: 13 	Training Loss: 0.787049 	Validation Loss: 0.802566
Validation loss decreased (0.823634 --> 0.802566).  Saving model ...
Epoch: 14 	Training Loss: 0.749585 	Validation Loss: 0.785852
Validation loss decreased (0.802566 --> 0.785852).  Saving model ...
Epoch: 15 	Training Loss: 0.721540 	Validation Loss: 0.772729
Validation loss decreased (0.785852 --> 0.772729).  Saving model ...
Epoch: 16 	Training Loss: 0.689508 	Validation Loss: 0.768470
Validation loss decreased (0.772729 --> 0.768470).  Saving model ...
Epoch: 17 	Training Loss: 0.662432 	Validation Loss: 0.758518
Validation loss decreased (0.768470 --> 0.758518).  Saving model ...
Epoch: 18 	Training Loss: 0.632324 	Validation Loss: 0.750859
Validation loss decreased (0.758518 --> 0.750859).  Saving model ...
Epoch: 19 	Training Loss: 0.616094 	Validation Loss: 0.729692
Validation loss decreased (0.750859 --> 0.729692).  Saving model ...
Epoch: 20 	Training Loss: 0.588593 	Validation Loss: 0.729085
Validation loss decreased (0.729692 --> 0.729085).  Saving model ...
Epoch: 21 	Training Loss: 0.571516 	Validation Loss: 0.734009
Epoch: 22 	Training Loss: 0.545541 	Validation Loss: 0.721433
Validation loss decreased (0.729085 --> 0.721433).  Saving model ...
Epoch: 23 	Training Loss: 0.523696 	Validation Loss: 0.720512
Validation loss decreased (0.721433 --> 0.720512).  Saving model ...
Epoch: 24 	Training Loss: 0.508577 	Validation Loss: 0.728457
Epoch: 25 	Training Loss: 0.483033 	Validation Loss: 0.722556
Epoch: 26 	Training Loss: 0.469563 	Validation Loss: 0.742352
Epoch: 27 	Training Loss: 0.449316 	Validation Loss: 0.726019
Epoch: 28 	Training Loss: 0.442354 	Validation Loss: 0.713364
Validation loss decreased (0.720512 --> 0.713364).  Saving model ...
Epoch: 29 	Training Loss: 0.421807 	Validation Loss: 0.718615
Epoch: 30 	Training Loss: 0.404595 	Validation Loss: 0.729914

加载模型

model.load_state_dict(torch.load('model_cifar.pt'))

结果：

<All keys matched successfully>

---

测试训练好的网络

在测试数据上测试你的训练模型！一个“好”的结果将是CNN得到大约70%，这些测试图像的准确性。

# track test loss
test_loss = 0.0
class_correct = list(0. for i in range(10))
class_total = list(0. for i in range(10))

model.eval()
# iterate over test data
for data, target in test_loader:
    # move tensors to GPU if CUDA is available
    if train_on_gpu:
        data, target = data.cuda(), target.cuda()
    # forward pass: compute predicted outputs by passing inputs to the model
    output = model(data)
    # calculate the batch loss
    loss = criterion(output, target)
    # update test loss 
    test_loss += loss.item()*data.size(0)
    # convert output probabilities to predicted class
    _, pred = torch.max(output, 1)    
    # compare predictions to true label
    correct_tensor = pred.eq(target.data.view_as(pred))
    correct = np.squeeze(correct_tensor.numpy()) if not train_on_gpu else np.squeeze(correct_tensor.cpu().numpy())
    # calculate test accuracy for each object class
    for i in range(batch_size):
        label = target.data[i]
        class_correct[label] += correct[i].item()
        class_total[label] += 1

# average test loss
test_loss = test_loss/len(test_loader.dataset)
print('Test Loss: {:.6f}\n'.format(test_loss))

for i in range(10):
    if class_total[i] > 0:
        print('Test Accuracy of %5s: %2d%% (%2d/%2d)' % (
            classes[i], 100 * class_correct[i] / class_total[i],
            np.sum(class_correct[i]), np.sum(class_total[i])))
    else:
        print('Test Accuracy of %5s: N/A (no training examples)' % (classes[i]))

print('\nTest Accuracy (Overall): %2d%% (%2d/%2d)' % (
    100. * np.sum(class_correct) / np.sum(class_total),
    np.sum(class_correct), np.sum(class_total)))

结果：

Test Loss: 0.708721

Test Accuracy of airplane: 82% (826/1000)
Test Accuracy of automobile: 81% (818/1000)
Test Accuracy of  bird: 65% (659/1000)
Test Accuracy of   cat: 59% (590/1000)
Test Accuracy of  deer: 75% (757/1000)
Test Accuracy of   dog: 56% (565/1000)
Test Accuracy of  frog: 81% (812/1000)
Test Accuracy of horse: 82% (823/1000)
Test Accuracy of  ship: 86% (866/1000)
Test Accuracy of truck: 84% (848/1000)

Test Accuracy (Overall): 75% (7564/10000)

显示测试样本的结果

# obtain one batch of test images
dataiter = iter(test_loader)
images, labels = dataiter.next()
images.numpy()

# move model inputs to cuda, if GPU available
if train_on_gpu:
    images = images.cuda()

# get sample outputs
output = model(images)
# convert output probabilities to predicted class
_, preds_tensor = torch.max(output, 1)
preds = np.squeeze(preds_tensor.numpy()) if not train_on_gpu else np.squeeze(preds_tensor.cpu().numpy())

# plot the images in the batch, along with predicted and true labels
fig = plt.figure(figsize=(25, 4))
for idx in np.arange(16):
    ax = fig.add_subplot(2, 16/2, idx+1, xticks=[], yticks=[])
    imshow(images.cpu()[idx])
    ax.set_title("{} ({})".format(classes[preds[idx]], classes[labels[idx]]),
                 color=("green" if preds[idx]==labels[idx].item() else "red"))

结果：

参考资料:

《吴恩达深度学习笔记》

《深度学习入门：基于Python的理论与实现》

https://pytorch.org/docs/stable/nn.html#

https://github.com/udacity/deep-learning-v2-pytorch