文章目录

1. 简介

项目 内容
模型 SE-ResNet（Squeeze-and-Excitation 注意力 + ResNet-18）
任务 三分类图像分类（Normal / Mild / Severe）
数据集 1661 张图像，80/20 划分
最优性能 测试准确率 93.1%，测试损失 0.247

2. 环境

  • 语言环境：Python 3.14.6
  • 编译器：Jupyter Notebook
  • 深度学习环境：PyTorch ( torch 2.12.1 + torchvision 0.27.1 )

3. 代码实现

3.1 前期准备

3.1.1 设置GPU & 导入库

导入 PyTorch、torchvision 等深度学习库，配置 matplotlib 中文字体，自动选择 GPU/CPU 设备.

import numpy as np
import torch
from torch import nn
from torch.nn import init
import os
import matplotlib.pyplot as plt
from PIL import Image
from torch.utils.data import DataLoader
from torchvision import transforms, datasets
import copy
from datetime import datetime
import warnings

warnings.filterwarnings("ignore")
plt.rcParams["figure.dpi"] = 100
plt.rcParams['font.sans-serif'] = ['SimHei']
plt.rcParams['axes.unicode_minus'] = False

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"Using device: {device}")

Using device: cpu

3.1.2 数据加载

data_dir = './Data/data/'

# 可视化样本图片
image_folder = './Data/data/2Mild/'
image_files = [f for f in os.listdir(image_folder) if f.endswith((".jpg", ".png", ".jpeg"))]
fig, axes = plt.subplots(3, 8, figsize=(16, 6))

for ax, img_file in zip(axes.flat, image_files[:24]):
    img_path = os.path.join(image_folder, img_file)
    img = Image.open(img_path)
    ax.imshow(img)
    ax.axis('off')

plt.tight_layout()
plt.show()

# 数据预处理
train_transforms = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])

total_data = datasets.ImageFolder(data_dir, transform=train_transforms)
print(f"Total samples: {len(total_data)}")
print(f"Classes: {total_data.class_to_idx}")

train_size = int(0.8 * len(total_data))
test_size = len(total_data) - train_size
train_dataset, test_dataset = torch.utils.data.random_split(total_data, [train_size, test_size])

batch_size = 4
train_dl = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)
test_dl = DataLoader(test_dataset, batch_size=batch_size)

for X, y in test_dl:
    print(f"Shape of X [N, C, H, W]: {X.shape}")
    print(f"Shape of y: {y.shape}, {y.dtype}")
    break

在这里插入图片描述

Total samples: 1661
Classes: {'0Normal': 0, '2Mild': 1, '4Severe': 2}
Shape of X [N, C, H, W]: torch.Size([4, 3, 224, 224])
Shape of y: torch.Size([4]), torch.int64

3.2 模型建立与训练

3.2.1 定义 SE-ResNet 网络模型

实现带有 SE（Squeeze-and-Excitation）注意力机制 的 ResNet 模型，通过显式建模通道间的相互依赖关系，自适应地重新校准各通道的特征响应。

三个核心组件：

组件 作用
SEAttention SE 注意力模块：全局平均池化（Squeeze）→ 两层全连接网络（Excitation）→ Sigmoid 生成通道权重 → 逐通道加权（Scale）
SEBasicBlock 带 SE 注意力的残差块：标准 ResNet 残差块 + SE 模块，在残差相加前对通道特征进行重标定
SEResNet 整体网络：7×7Conv → BN → ReLU → MaxPool → 4个残差层（每层含 SE）→ GlobalAvgPool → FC

本实验中 num_classes=3，对应三分类任务（Normal / Mild / Severe）。

class SEAttention(nn.Module):
    """Squeeze-and-Excitation 注意力机制"""
    def __init__(self, in_channels=512, reduction=16):
        super().__init__()
        self.avg_pool = nn.AdaptiveAvgPool2d(1)
        self.fc = nn.Sequential(
            nn.Linear(in_channels, in_channels // reduction, bias=False),
            nn.ReLU(inplace=True),
            nn.Linear(in_channels // reduction, in_channels, bias=False),
            nn.Sigmoid()
        )

    def forward(self, x):
        b, c, _, _ = x.size()
        y = self.avg_pool(x).view(b, c)
        y = self.fc(y).view(b, c, 1, 1)
        return x * y.expand_as(x)


class SEBasicBlock(nn.Module):
    """带 SE 注意力的残差块"""
    expansion = 1

    def __init__(self, in_channels, out_channels, stride=1, downsample=None, reduction=16):
        super(SEBasicBlock, self).__init__()
        self.conv1 = nn.Conv2d(in_channels, out_channels, kernel_size=3, stride=stride, padding=1, bias=False)
        self.bn1 = nn.BatchNorm2d(out_channels)
        self.relu = nn.ReLU(inplace=True)
        self.conv2 = nn.Conv2d(out_channels, out_channels, kernel_size=3, stride=1, padding=1, bias=False)
        self.bn2 = nn.BatchNorm2d(out_channels)
        self.se = SEAttention(out_channels, reduction)
        self.downsample = downsample
        self.stride = stride

    def forward(self, x):
        identity = x

        out = self.conv1(x)
        out = self.bn1(out)
        out = self.relu(out)

        out = self.conv2(out)
        out = self.bn2(out)
        out = self.se(out)

        if self.downsample is not None:
            identity = self.downsample(x)

        out += identity
        out = self.relu(out)

        return out


class SEResNet(nn.Module):
    """SE-ResNet 模型"""
    def __init__(self, block, layers, num_classes=3, reduction=16):
        self.in_channels = 64
        super(SEResNet, self).__init__()
        self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)
        self.bn1 = nn.BatchNorm2d(64)
        self.relu = nn.ReLU(inplace=True)
        self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)

        self.layer1 = self._make_layer(block, 64, layers[0], reduction=reduction)
        self.layer2 = self._make_layer(block, 128, layers[1], stride=2, reduction=reduction)
        self.layer3 = self._make_layer(block, 256, layers[2], stride=2, reduction=reduction)
        self.layer4 = self._make_layer(block, 512, layers[3], stride=2, reduction=reduction)

        self.avgpool = nn.AdaptiveAvgPool2d((1, 1))
        self.fc = nn.Linear(512 * block.expansion, num_classes)

        for m in self.modules():
            if isinstance(m, nn.Conv2d):
                nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
            elif isinstance(m, nn.BatchNorm2d):
                nn.init.constant_(m.weight, 1)
                nn.init.constant_(m.bias, 0)

    def _make_layer(self, block, out_channels, blocks, stride=1, reduction=16):
        downsample = None
        if stride != 1 or self.in_channels != out_channels * block.expansion:
            downsample = nn.Sequential(
                nn.Conv2d(self.in_channels, out_channels * block.expansion, kernel_size=1, stride=stride, bias=False),
                nn.BatchNorm2d(out_channels * block.expansion)
            )

        layers = []
        layers.append(block(self.in_channels, out_channels, stride, downsample, reduction))
        self.in_channels = out_channels * block.expansion
        for _ in range(1, blocks):
            layers.append(block(self.in_channels, out_channels, reduction=reduction))

        return nn.Sequential(*layers)

    def forward(self, x):
        x = self.conv1(x)
        x = self.bn1(x)
        x = self.relu(x)
        x = self.maxpool(x)

        x = self.layer1(x)
        x = self.layer2(x)
        x = self.layer3(x)
        x = self.layer4(x)

        x = self.avgpool(x)
        x = torch.flatten(x, 1)
        x = self.fc(x)

        return x


model = SEResNet(SEBasicBlock, [2, 2, 2, 2], num_classes=3).to(device)
print(f"Model created and moved to {device}")

Model created and moved to cpu

3.2.2 模型结构概览

通过 SE 注意力模块的独立测试验证其通道加权机制是否正常工作：

  • 输入：(8, 512, 32, 32) — 8张 512通道 32×32 的特征图
  • 输出：(8, 512, 32, 32) — 经通道注意力加权后的特征图，尺寸不变

SE 模块通过全局平均池化将每个通道压缩为标量，再经两层全连接网络学习通道间的依赖关系，最终用 Sigmoid 生成的权重对各通道进行缩放。

if __name__ == "__main__":
    x = torch.randn(8, 512, 32, 32).to(device)
    se = SEAttention(in_channels=512, reduction=16).to(device)
    out = se(x)
    print(f"\nSE Attention Test:")
    print(f"Input shape: {x.shape}")
    print(f"Output shape: {out.shape}")
    print(f"Output mean: {out.mean().item():.4f}")

SE Attention Test:
Input shape: torch.Size([8, 512, 32, 32])
Output shape: torch.Size([8, 512, 32, 32])
Output mean: -0.0004

3.2.3 定义训练和测试函数

  • train() 函数 — 训练阶段：
    对每个 mini-batch 执行标准的前向传播→计算损失→反向传播→参数更新流程：

  • 累加每个 batch 的损失和正确预测数

  • 返回平均损失和整体准确率（总正确数 / 总样本数）

  • test() 函数 — 测试/验证阶段：

  • 使用 torch.no_grad() 禁用梯度计算，减少内存开销并加速推理

  • 不进行参数更新，仅评估模型在测试集上的泛化性能

  • 同样返回平均损失和整体准确率

def train(dataloader, model, loss_fn, optimizer):
    size = len(dataloader.dataset)
    num_batches = len(dataloader)
    model.train()

    train_loss, train_acc = 0, 0

    for X, y in dataloader:
        X, y = X.to(device), y.to(device)
        pred = model(X)
        loss = loss_fn(pred, y)

        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        train_loss += loss.item()
        train_acc += (pred.argmax(1) == y).type(torch.float).sum().item()

    train_loss /= num_batches
    train_acc /= size

    return train_loss, train_acc


def test(dataloader, model, loss_fn):
    size = len(dataloader.dataset)
    num_batches = len(dataloader)
    model.eval()

    test_loss, test_acc = 0, 0

    with torch.no_grad():
        for X, y in dataloader:
            X, y = X.to(device), y.to(device)
            pred = model(X)
            loss = loss_fn(pred, y)

            test_acc += (pred.argmax(1) == y).type(torch.float).sum().item()
            test_loss += loss.item()

    test_loss /= num_batches
    test_acc /= size

    return test_loss, test_acc

3.2.4 训练模型

训练配置
  • 优化器：AdamW（带动量的自适应学习率优化器，含权重衰减正则化），学习率 lr=1e-4，权重衰减 weight_decay=0.01
  • 损失函数：CrossEntropyLoss（交叉熵损失），适用于多分类任务
  • 训练轮次：10 个 Epoch
训练策略
  • 每个 Epoch 结束后在测试集上评估，记录训练/测试的损失和准确率
  • Best Model 保存：当测试准确率超过历史最佳时，深拷贝当前模型权重
  • 训练结束后将最优模型保存至 ./best_se_resnet.pth

最优模型出现在第 8 个 Epoch，测试准确率达 93.1%。

optimizer = torch.optim.AdamW(model.parameters(), lr=1e-4, weight_decay=0.01)
loss_fn = nn.CrossEntropyLoss()

epochs = 10
train_loss_history, train_acc_history = [], []
test_loss_history, test_acc_history = [], []
best_acc = 0

print("\nStarting training...")
current_time = datetime.now()

for epoch in range(epochs):
    train_epoch_loss, train_epoch_acc = train(train_dl, model, loss_fn, optimizer)
    test_epoch_loss, test_epoch_acc = test(test_dl, model, loss_fn)

    if test_epoch_acc > best_acc:
        best_acc = test_epoch_acc
        best_model_wts = copy.deepcopy(model.state_dict())
        PATH = './best_se_resnet.pth'
        torch.save(best_model_wts, PATH)

    train_loss_history.append(train_epoch_loss)
    train_acc_history.append(train_epoch_acc)
    test_loss_history.append(test_epoch_loss)
    test_acc_history.append(test_epoch_acc)

    lr = optimizer.param_groups[0]['lr']
    template = ("Epoch: {:2d}, Train_acc: {:.1f}%, Train_loss: {:.3f}, Test_acc: {:.1f}%, Test_loss: {:.3f}, Lr: {:.2E}")
    print(template.format(epoch+1, train_epoch_acc*100, train_epoch_loss, test_epoch_acc*100, test_epoch_loss, lr))

print(f'Training completed. Best test accuracy: {best_acc:.4f}')
print('Best model saved to best_se_resnet.pth')

Starting training...
Epoch:  1, Train_acc: 70.3%, Train_loss: 0.765, Test_acc: 76.6%, Test_loss: 0.613, Lr: 1.00E-04
Epoch:  2, Train_acc: 76.4%, Train_loss: 0.623, Test_acc: 85.6%, Test_loss: 0.459, Lr: 1.00E-04
Epoch:  3, Train_acc: 81.1%, Train_loss: 0.495, Test_acc: 84.1%, Test_loss: 0.438, Lr: 1.00E-04
Epoch:  4, Train_acc: 83.8%, Train_loss: 0.451, Test_acc: 82.3%, Test_loss: 0.603, Lr: 1.00E-04
Epoch:  5, Train_acc: 84.6%, Train_loss: 0.420, Test_acc: 88.3%, Test_loss: 0.316, Lr: 1.00E-04
Epoch:  6, Train_acc: 87.0%, Train_loss: 0.362, Test_acc: 83.5%, Test_loss: 0.370, Lr: 1.00E-04
Epoch:  7, Train_acc: 89.1%, Train_loss: 0.315, Test_acc: 88.0%, Test_loss: 0.317, Lr: 1.00E-04
Epoch:  8, Train_acc: 92.8%, Train_loss: 0.223, Test_acc: 93.1%, Test_loss: 0.247, Lr: 1.00E-04
Epoch:  9, Train_acc: 91.6%, Train_loss: 0.246, Test_acc: 89.8%, Test_loss: 0.308, Lr: 1.00E-04
Epoch: 10, Train_acc: 93.4%, Train_loss: 0.201, Test_acc: 89.5%, Test_loss: 0.343, Lr: 1.00E-04
Training completed. Best test accuracy: 0.9309
Best model saved to best_se_resnet.pth

4. 模型评估

4.1 可视化训练过程

绘制训练和测试的准确率、损失曲线，并使用 Matplotlib 将训练集与验证集的准确率（Accuracy）和损失值（Loss）随时间变化的趋势绘制成了两幅直观的折线图。

epochs_range = range(epochs)

plt.figure(figsize=(12, 3))
plt.subplot(1, 2, 1)
plt.plot(epochs_range, train_acc_history, label='Training Accuracy')
plt.plot(epochs_range, test_acc_history, label='Test Accuracy')
plt.legend(loc='lower right')
plt.title('Training and Validation Accuracy')
plt.xlabel(current_time)

plt.subplot(1, 2, 2)
plt.plot(epochs_range, train_loss_history, label='Training Loss')
plt.plot(epochs_range, test_loss_history, label='Test Loss')
plt.legend(loc='upper right')
plt.title('Training and Validation Loss')
plt.tight_layout()
plt.show()

在这里插入图片描述

4.2 加载最优模型并评估

加载训练过程中保存的最优模型权重，在测试集上进行最终评估，输出测试准确率和损失值。

model.load_state_dict(torch.load(PATH, map_location=device))
model.eval()

final_test_loss, final_test_acc = test(test_dl, model, loss_fn)
print(f"\nFinal Test Results:")
print(f"Test Accuracy: {final_test_acc:.4f}")
print(f"Test Loss: {final_test_loss:.4f}")

Final Test Results:
Test Accuracy: 0.9309
Test Loss: 0.2467

5.总结

  1. SE 注意力机制有效提升特征表达：在仅 10 个 Epoch 的训练后，测试准确率达到 93.1%，说明 SE 模块通过自适应地重新校准通道权重，能够让模型更关注有判别力的特征通道，提升分类性能。

  2. 参数效率较高：SE-ResNet 基于 ResNet-18 骨干网络，参数量相对较小，额外的 SE 模块仅引入少量全连接层参数（约增加 10% 参数量），以较小的计算开销换取了明显的性能提升。

  3. 收敛速度较快：模型从第 1 个 Epoch 的 76.6% 快速提升，第 8 个 Epoch 即达到最优的 93.1%，说明 AdamW 优化器配合权重衰减（weight_decay=0.01）能有效加速收敛并抑制过拟合。

  4. 存在轻微过拟合迹象：训练准确率持续上升至 93.4%，而测试准确率在第 8 个 Epoch 达到峰值后略有回落（第 9、10 Epoch 降至 89.5%），建议后续可通过以下方式改善：

    • 增加数据增强（随机翻转、旋转、色彩抖动等）
    • 引入学习率调度策略（如 Cosine Annealing 或 StepLR）
    • 增大 SE 模块的 reduction 参数（如从 16 改为 32）以减少参数量
    • 使用交叉验证替代单次随机划分

  5. 数据规模的影响：1661 张图像的数据集相对较小，测试集仅约 333 张，测试准确率的波动可能与样本量不足有关。增加数据量或使用预训练权重微调有望进一步提升性能。

# 深度学习 # 人工智能 # 分类
Logo
https://edu.csdn.net/learn/39067/627173?utm_source=2019755004

汇聚全球AI编程工具，助力开发者即刻编程。

更多推荐

cover

AI审计手记 #06：沙箱逃逸漏洞与“不修复“的防御策略转移

avatar AI编程社区
cover

Codex 和 Claude Code 怎么订阅？套餐、API计费与付款避坑指南

avatar AI编程社区
cover

手绘草图秒变网页：GPT-Image + Claude 前端开发实战与选型指南

avatar AI编程社区