Python+PyTorch深度学习实战:从零入门到计算机视觉项目落地
最近在辅导一些刚接触AI的同学时,发现很多人对Python、PyTorch、神经网络这些概念感到困惑,网上资料虽然多但不够系统。本文将整合一套完整的AI入门实战教程,从Python基础到PyTorch深度学习框架,再到神经网络和机器学习算法的实际应用,最后通过计算机视觉项目落地,帮助零基础的同学系统掌握AI开发全流程。
1. 人工智能与深度学习基础概念
1.1 什么是人工智能与深度学习
人工智能(AI)是计算机科学的一个分支,旨在创建能够执行通常需要人类智能的任务的机器系统。深度学习是AI的一个子领域,它使用包含多个处理层的神经网络来学习数据中的复杂模式。
深度学习与传统机器学习的最大区别在于特征提取的自动化。传统机器学习需要人工设计特征,而深度学习能够自动从原始数据中学习特征表示,这在图像识别、自然语言处理等领域表现出色。
1.2 为什么选择Python+PyTorch组合
Python因其简洁易学的语法和丰富的科学生态系统,成为AI开发的首选语言。PyTorch作为深度学习框架,以其动态计算图和直观的API设计受到学术界和工业界的广泛欢迎。
PyTorch的核心优势包括:
- 动态计算图:允许在运行时修改网络结构,便于调试和实验
- Pythonic设计:与Python语言无缝集成,学习曲线平缓
- 强大的社区支持:拥有丰富的预训练模型和教程资源
- 生产就绪:通过TorchScript支持模型部署到生产环境
2. 环境搭建与工具配置
2.1 Python环境安装
对于AI开发,推荐使用Anaconda来管理Python环境,它可以轻松处理包依赖和环境隔离。
Windows系统安装步骤:
- 访问Anaconda官网下载最新版本
- 运行安装程序,建议勾选"Add Anaconda to PATH"选项
- 打开命令提示符,验证安装:
conda --version
创建专用环境:
conda create -n ai-tutorial python=3.9
conda activate ai-tutorial
2.2 PyTorch安装配置
根据是否有GPU选择不同的安装命令。可以通过以下命令检查CUDA版本:
nvidia-smi # 查看GPU信息
CPU版本安装:
pip install torch torchvision torchaudio
GPU版本安装(CUDA 11.7):
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117
2.3 开发环境配置
推荐使用VS Code作为代码编辑器,安装Python和Jupyter扩展:
# 安装必要的科学计算库
pip install jupyter matplotlib numpy pandas scikit-learn
创建项目目录结构:
ai-tutorial/
├── data/ # 数据集存放
├── models/ # 模型文件
├── notebooks/ # Jupyter笔记本
├── src/ # 源代码
└── utils/ # 工具函数
3. Python编程基础快速入门
3.1 基本语法与数据结构
Python的易读性是其主要优势之一。以下是必须掌握的基础概念:
# 变量和数据类型
name = "AI学习者" # 字符串
age = 25 # 整数
height = 175.5 # 浮点数
is_student = True # 布尔值
# 列表操作
fruits = ["apple", "banana", "orange"]
fruits.append("grape") # 添加元素
print(fruits[0]) # 访问第一个元素
# 字典使用
person = {"name": "张三", "age": 25, "city": "北京"}
print(person["name"]) # 访问值
3.2 函数与面向对象编程
# 函数定义
def calculate_bmi(weight, height):
"""计算BMI指数"""
bmi = weight / (height ** 2)
return bmi
# 类定义
class Student:
def __init__(self, name, age):
self.name = name
self.age = age
def introduce(self):
return f"我叫{self.name},今年{self.age}岁"
# 使用类
student = Student("李四", 20)
print(student.introduce())
3.3 科学计算库NumPy基础
NumPy是Python科学计算的基础库,PyTorch的张量操作与NumPy数组类似:
import numpy as np
# 创建数组
arr = np.array([1, 2, 3, 4, 5])
print("数组形状:", arr.shape)
# 矩阵运算
matrix_a = np.array([[1, 2], [3, 4]])
matrix_b = np.array([[5, 6], [7, 8]])
result = np.dot(matrix_a, matrix_b) # 矩阵乘法
print("矩阵乘法结果:\n", result)
# 随机数生成
random_data = np.random.randn(3, 3) # 3x3正态分布随机数
print("随机矩阵:\n", random_data)
4. PyTorch核心概念与张量操作
4.1 张量(Tensor)基础
张量是PyTorch的基本数据结构,可以看作是多维数组:
import torch
# 创建张量
scalar = torch.tensor(5.0) # 0维张量(标量)
vector = torch.tensor([1, 2, 3]) # 1维张量(向量)
matrix = torch.tensor([[1, 2], [3, 4]]) # 2维张量(矩阵)
print("标量形状:", scalar.shape)
print("向量形状:", vector.shape)
print("矩阵形状:", matrix.shape)
# 特殊张量创建
zeros_tensor = torch.zeros(2, 3) # 全零张量
ones_tensor = torch.ones(2, 3) # 全一张量
random_tensor = torch.randn(2, 3) # 正态分布随机张量
4.2 张量运算与自动微分
PyTorch的自动微分功能是深度学习训练的核心:
# 张量运算
x = torch.tensor([2.0], requires_grad=True) # 需要梯度计算
y = x ** 2 + 3 * x + 1 # 定义函数 y = x² + 3x + 1
y.backward() # 反向传播计算梯度
print("x的梯度:", x.grad) # 在x=2处,dy/dx = 2x + 3 = 7
# 矩阵运算示例
A = torch.randn(3, 4)
B = torch.randn(4, 5)
C = torch.matmul(A, B) # 矩阵乘法
print("矩阵乘法结果形状:", C.shape)
4.3 GPU加速计算
如果系统有NVIDIA GPU,可以利用CUDA加速计算:
# 检查GPU可用性
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print("使用设备:", device)
# 将张量移动到GPU
x = torch.randn(1000, 1000)
if torch.cuda.is_available():
x_gpu = x.to(device)
# 在GPU上执行运算
result_gpu = torch.matmul(x_gpu, x_gpu.T)
# 移回CPU
result_cpu = result_gpu.cpu()
5. 神经网络基本原理与PyTorch实现
5.1 神经元与感知机
神经网络的基本单位是神经元,模仿生物神经元的工作方式:
import torch.nn as nn
# 实现一个简单的感知机
class Perceptron(nn.Module):
def __init__(self, input_dim):
super(Perceptron, self).__init__()
self.linear = nn.Linear(input_dim, 1)
self.activation = nn.Sigmoid()
def forward(self, x):
x = self.linear(x)
return self.activation(x)
# 测试感知机
perceptron = Perceptron(3)
input_data = torch.tensor([0.5, -0.3, 0.8]).float()
output = perceptron(input_data)
print("感知机输出:", output.item())
5.2 多层神经网络构建
现实问题通常需要更复杂的网络结构:
class SimpleNN(nn.Module):
def __init__(self, input_size, hidden_size, output_size):
super(SimpleNN, self).__init__()
self.fc1 = nn.Linear(input_size, hidden_size)
self.relu = nn.ReLU()
self.fc2 = nn.Linear(hidden_size, output_size)
self.sigmoid = nn.Sigmoid()
def forward(self, x):
x = self.fc1(x)
x = self.relu(x)
x = self.fc2(x)
return self.sigmoid(x)
# 创建网络实例
model = SimpleNN(10, 50, 1)
print("网络结构:")
print(model)
5.3 损失函数与优化器
训练神经网络需要定义损失函数和优化算法:
# 损失函数示例
criterion = nn.MSELoss() # 均方误差损失,适用于回归问题
# 优化器示例
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
# 模拟训练步骤
def train_step(model, x, y, criterion, optimizer):
# 前向传播
y_pred = model(x)
loss = criterion(y_pred, y)
# 反向传播
optimizer.zero_grad() # 清零梯度
loss.backward() # 计算梯度
optimizer.step() # 更新参数
return loss.item()
# 示例训练循环
for epoch in range(100):
# 假设x_train, y_train是训练数据
loss = train_step(model, x_train, y_train, criterion, optimizer)
if epoch % 10 == 0:
print(f"Epoch {epoch}, Loss: {loss:.4f}")
6. 机器学习算法实战
6.1 线性回归实现
线性回归是最基础的机器学习算法:
import torch
import torch.nn as nn
import matplotlib.pyplot as plt
# 生成模拟数据
torch.manual_seed(42)
x = torch.linspace(0, 10, 100).reshape(-1, 1)
y = 2 * x + 1 + torch.randn(100, 1) * 2 # y = 2x + 1 + 噪声
# 定义线性回归模型
class LinearRegression(nn.Module):
def __init__(self):
super(LinearRegression, self).__init__()
self.linear = nn.Linear(1, 1)
def forward(self, x):
return self.linear(x)
model = LinearRegression()
criterion = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
# 训练模型
losses = []
for epoch in range(1000):
y_pred = model(x)
loss = criterion(y_pred, y)
optimizer.zero_grad()
loss.backward()
optimizer.step()
losses.append(loss.item())
if epoch % 100 == 0:
print(f'Epoch {epoch}, Loss: {loss.item():.4f}')
# 可视化结果
plt.figure(figsize=(12, 4))
plt.subplot(1, 2, 1)
plt.scatter(x, y, alpha=0.7, label='真实数据')
plt.plot(x, model(x).detach().numpy(), 'r-', label='预测结果')
plt.legend()
plt.title('线性回归拟合结果')
plt.subplot(1, 2, 2)
plt.plot(losses)
plt.title('训练损失下降曲线')
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.show()
6.2 逻辑回归与分类问题
# 生成二分类数据
from sklearn.datasets import make_classification
x, y = make_classification(n_samples=1000, n_features=2, n_redundant=0,
n_informative=2, n_clusters_per_class=1)
x = torch.tensor(x, dtype=torch.float32)
y = torch.tensor(y, dtype=torch.float32).reshape(-1, 1)
class LogisticRegression(nn.Module):
def __init__(self, input_dim):
super(LogisticRegression, self).__init__()
self.linear = nn.Linear(input_dim, 1)
self.sigmoid = nn.Sigmoid()
def forward(self, x):
return self.sigmoid(self.linear(x))
model = LogisticRegression(2)
criterion = nn.BCELoss() # 二分类交叉熵损失
optimizer = torch.optim.Adam(model.parameters(), lr=0.1)
# 训练逻辑回归模型
for epoch in range(500):
y_pred = model(x)
loss = criterion(y_pred, y)
optimizer.zero_grad()
loss.backward()
optimizer.step()
if epoch % 50 == 0:
# 计算准确率
predictions = (y_pred > 0.5).float()
accuracy = (predictions == y).float().mean()
print(f'Epoch {epoch}, Loss: {loss.item():.4f}, Accuracy: {accuracy.item():.4f}')
6.3 决策树与集成学习
虽然PyTorch主要用于神经网络,但了解传统算法也很重要:
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
# 使用scikit-learn实现决策树
dt_model = DecisionTreeClassifier(max_depth=5)
rf_model = RandomForestClassifier(n_estimators=100)
# 训练和评估
x_train, x_test, y_train, y_test = train_test_split(x.numpy(), y.numpy().ravel())
dt_model.fit(x_train, y_train)
rf_model.fit(x_train, y_train)
dt_score = accuracy_score(y_test, dt_model.predict(x_test))
rf_score = accuracy_score(y_test, rf_model.predict(x_test))
print(f'决策树准确率: {dt_score:.4f}')
print(f'随机森林准确率: {rf_score:.4f}')
7. 卷积神经网络与计算机视觉应用
7.1 CNN基本原理
卷积神经网络专门处理网格状数据,如图像:
import torch.nn as nn
import torch.nn.functional as F
class SimpleCNN(nn.Module):
def __init__(self, num_classes=10):
super(SimpleCNN, self).__init__()
self.conv1 = nn.Conv2d(1, 32, kernel_size=3, padding=1)
self.conv2 = nn.Conv2d(32, 64, kernel_size=3, padding=1)
self.pool = nn.MaxPool2d(2, 2)
self.fc1 = nn.Linear(64 * 7 * 7, 128)
self.fc2 = nn.Linear(128, num_classes)
self.dropout = nn.Dropout(0.5)
def forward(self, x):
x = self.pool(F.relu(self.conv1(x))) # 28x28 -> 14x14
x = self.pool(F.relu(self.conv2(x))) # 14x14 -> 7x7
x = x.view(-1, 64 * 7 * 7) # 展平
x = F.relu(self.fc1(x))
x = self.dropout(x)
x = self.fc2(x)
return x
# 实例化模型
cnn_model = SimpleCNN()
print("CNN参数量:", sum(p.numel() for p in cnn_model.parameters()))
7.2 图像分类实战:手写数字识别
使用MNIST数据集进行实战:
from torchvision import datasets, transforms
from torch.utils.data import DataLoader
# 数据预处理
transform = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize((0.1307,), (0.3081,))
])
# 加载数据集
train_dataset = datasets.MNIST('./data', train=True, download=True, transform=transform)
test_dataset = datasets.MNIST('./data', train=False, transform=transform)
train_loader = DataLoader(train_dataset, batch_size=64, shuffle=True)
test_loader = DataLoader(test_dataset, batch_size=1000, shuffle=False)
# 训练函数
def train_model(model, train_loader, criterion, optimizer, epochs=10):
model.train()
for epoch in range(epochs):
running_loss = 0.0
for batch_idx, (data, target) in enumerate(train_loader):
optimizer.zero_grad()
output = model(data)
loss = criterion(output, target)
loss.backward()
optimizer.step()
running_loss += loss.item()
print(f'Epoch {epoch+1}/{epochs}, Loss: {running_loss/len(train_loader):.4f}')
# 评估函数
def evaluate_model(model, test_loader):
model.eval()
correct = 0
total = 0
with torch.no_grad():
for data, target in test_loader:
output = model(data)
_, predicted = torch.max(output.data, 1)
total += target.size(0)
correct += (predicted == target).sum().item()
accuracy = 100 * correct / total
print(f'测试准确率: {accuracy:.2f}%')
return accuracy
# 训练CNN模型
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(cnn_model.parameters(), lr=0.001)
train_model(cnn_model, train_loader, criterion, optimizer, epochs=5)
accuracy = evaluate_model(cnn_model, test_loader)
7.3 迁移学习与预训练模型
对于复杂任务,可以使用预训练模型:
import torchvision.models as models
# 加载预训练的ResNet模型
resnet = models.resnet18(pretrained=True)
# 修改最后一层用于10分类任务
num_features = resnet.fc.in_features
resnet.fc = nn.Linear(num_features, 10)
# 冻结前面的层,只训练最后一层
for param in resnet.parameters():
param.requires_grad = False
for param in resnet.fc.parameters():
param.requires_grad = True
print("迁移学习模型准备完成")
8. 循环神经网络与序列数据处理
8.1 RNN基础原理
循环神经网络适合处理序列数据:
class SimpleRNN(nn.Module):
def __init__(self, input_size, hidden_size, output_size):
super(SimpleRNN, self).__init__()
self.hidden_size = hidden_size
self.rnn = nn.RNN(input_size, hidden_size, batch_first=True)
self.fc = nn.Linear(hidden_size, output_size)
def forward(self, x):
# x的形状: (batch_size, seq_len, input_size)
out, hidden = self.rnn(x)
# 只使用最后一个时间步的输出
out = self.fc(out[:, -1, :])
return out
# 示例:序列分类任务
rnn_model = SimpleRNN(input_size=10, hidden_size=50, output_size=2)
8.2 LSTM实战应用
长短期记忆网络解决了RNN的梯度消失问题:
class LSTMModel(nn.Module):
def __init__(self, vocab_size, embed_size, hidden_size, num_layers, num_classes):
super(LSTMModel, self).__init__()
self.embedding = nn.Embedding(vocab_size, embed_size)
self.lstm = nn.LSTM(embed_size, hidden_size, num_layers, batch_first=True)
self.fc = nn.Linear(hidden_size, num_classes)
def forward(self, x):
x = self.embedding(x)
out, (hidden, cell) = self.lstm(x)
out = self.fc(out[:, -1, :])
return out
# 文本分类示例
lstm_model = LSTMModel(vocab_size=10000, embed_size=100,
hidden_size=128, num_layers=2, num_classes=3)
9. 模型训练技巧与最佳实践
9.1 数据预处理与增强
高质量的数据处理是模型成功的关键:
from torchvision import transforms
# 图像数据增强
train_transform = transforms.Compose([
transforms.RandomHorizontalFlip(),
transforms.RandomRotation(10),
transforms.ColorJitter(brightness=0.2, contrast=0.2),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
])
# 文本数据预处理
def text_preprocessing(text):
import re
# 清洗文本
text = re.sub(r'[^a-zA-Z\s]', '', text)
text = text.lower().strip()
return text
# 数据加载优化
class CustomDataset(torch.utils.data.Dataset):
def __init__(self, data, labels, transform=None):
self.data = data
self.labels = labels
self.transform = transform
def __len__(self):
return len(self.data)
def __getitem__(self, idx):
sample = self.data[idx]
label = self.labels[idx]
if self.transform:
sample = self.transform(sample)
return sample, label
9.2 超参数调优策略
from torch.optim.lr_scheduler import StepLR, ReduceLROnPlateau
# 学习率调度器
optimizer = torch.optim.Adam(model.parameters(), lr=0.01)
scheduler = StepLR(optimizer, step_size=30, gamma=0.1) # 每30轮学习率乘以0.1
# 或者使用自适应学习率调整
scheduler_plateau = ReduceLROnPlateau(optimizer, mode='min',
patience=5, factor=0.5)
# 早停法实现
class EarlyStopping:
def __init__(self, patience=7, min_delta=0):
self.patience = patience
self.min_delta = min_delta
self.counter = 0
self.best_loss = None
self.early_stop = False
def __call__(self, val_loss):
if self.best_loss is None:
self.best_loss = val_loss
elif val_loss > self.best_loss - self.min_delta:
self.counter += 1
if self.counter >= self.patience:
self.early_stop = True
else:
self.best_loss = val_loss
self.counter = 0
# 使用示例
early_stopping = EarlyStopping(patience=10)
9.3 模型评估与可视化
import matplotlib.pyplot as plt
from sklearn.metrics import classification_report, confusion_matrix
import seaborn as sns
def plot_training_history(train_losses, val_losses, train_accs, val_accs):
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15, 5))
# 损失曲线
ax1.plot(train_losses, label='训练损失')
ax1.plot(val_losses, label='验证损失')
ax1.set_title('训练和验证损失')
ax1.set_xlabel('Epoch')
ax1.set_ylabel('Loss')
ax1.legend()
# 准确率曲线
ax2.plot(train_accs, label='训练准确率')
ax2.plot(val_accs, label='验证准确率')
ax2.set_title('训练和验证准确率')
ax2.set_xlabel('Epoch')
ax2.set_ylabel('Accuracy')
ax2.legend()
plt.tight_layout()
plt.show()
def evaluate_classification(model, test_loader, class_names):
model.eval()
all_preds = []
all_targets = []
with torch.no_grad():
for data, target in test_loader:
output = model(data)
_, preds = torch.max(output, 1)
all_preds.extend(preds.cpu().numpy())
all_targets.extend(target.cpu().numpy())
# 分类报告
print(classification_report(all_targets, all_preds, target_names=class_names))
# 混淆矩阵
cm = confusion_matrix(all_targets, all_preds)
plt.figure(figsize=(8, 6))
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues',
xticklabels=class_names, yticklabels=class_names)
plt.title('混淆矩阵')
plt.ylabel('真实标签')
plt.xlabel('预测标签')
plt.show()
10. 项目实战:完整AI应用开发
10.1 项目需求分析
开发一个猫狗图像分类器,要求:
- 准确区分猫和狗的图像
- 支持实时摄像头识别
- 提供Web界面方便使用
- 准确率达到90%以上
10.2 数据准备与预处理
import os
from PIL import Image
class CatDogDataset(torch.utils.data.Dataset):
def __init__(self, data_dir, transform=None):
self.data_dir = data_dir
self.transform = transform
self.images = []
self.labels = []
# 加载猫的图像
cat_dir = os.path.join(data_dir, 'cats')
for img_name in os.listdir(cat_dir):
if img_name.endswith(('.jpg', '.png')):
self.images.append(os.path.join(cat_dir, img_name))
self.labels.append(0) # 猫标签为0
# 加载狗的图像
dog_dir = os.path.join(data_dir, 'dogs')
for img_name in os.listdir(dog_dir):
if img_name.endswith(('.jpg', '.png')):
self.images.append(os.path.join(dog_dir, img_name))
self.labels.append(1) # 狗标签为1
def __len__(self):
return len(self.images)
def __getitem__(self, idx):
img_path = self.images[idx]
image = Image.open(img_path).convert('RGB')
label = self.labels[idx]
if self.transform:
image = self.transform(image)
return image, label
# 数据增强变换
train_transform = transforms.Compose([
transforms.Resize((224, 224)),
transforms.RandomHorizontalFlip(),
transforms.RandomRotation(20),
transforms.ColorJitter(brightness=0.2, contrast=0.2),
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])
test_transform = transforms.Compose([
transforms.Resize((224, 224)),
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])
10.3 模型架构设计
import torch.nn as nn
import torchvision.models as models
class CatDogClassifier(nn.Module):
def __init__(self, num_classes=2):
super(CatDogClassifier, self).__init__()
# 使用预训练的ResNet作为特征提取器
self.backbone = models.resnet18(pretrained=True)
# 替换最后的全连接层
num_features = self.backbone.fc.in_features
self.backbone.fc = nn.Sequential(
nn.Dropout(0.5),
nn.Linear(num_features, 512),
nn.ReLU(),
nn.Dropout(0.3),
nn.Linear(512, num_classes)
)
def forward(self, x):
return self.backbone(x)
def create_model(device):
model = CatDogClassifier()
model = model.to(device)
return model
10.4 训练流程实现
def train_catdog_model():
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"使用设备: {device}")
# 创建数据集
train_dataset = CatDogDataset('data/train', transform=train_transform)
test_dataset = CatDogDataset('data/test', transform=test_transform)
train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True)
test_loader = DataLoader(test_dataset, batch_size=32, shuffle=False)
# 创建模型
model = create_model(device)
# 定义损失函数和优化器
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
scheduler = ReduceLROnPlateau(optimizer, mode='min', patience=3, factor=0.5)
# 训练循环
best_accuracy = 0
train_losses = []
val_accuracies = []
for epoch in range(50):
# 训练阶段
model.train()
running_loss = 0.0
for images, labels in train_loader:
images, labels = images.to(device), labels.to(device)
optimizer.zero_grad()
outputs = model(images)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
running_loss += loss.item()
avg_loss = running_loss / len(train_loader)
train_losses.append(avg_loss)
# 验证阶段
model.eval()
correct = 0
total = 0
with torch.no_grad():
for images, labels in test_loader:
images, labels = images.to(device), labels.to(device)
outputs = model(images)
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
accuracy = 100 * correct / total
val_accuracies.append(accuracy)
print(f'Epoch {epoch+1}/50, Loss: {avg_loss:.4f}, Accuracy: {accuracy:.2f}%')
# 学习率调整
scheduler.step(avg_loss)
# 保存最佳模型
if accuracy > best_accuracy:
best_accuracy = accuracy
torch.save(model.state_dict(), 'best_catdog_model.pth')
print(f'新的最佳模型已保存,准确率: {accuracy:.2f}%')
print(f'训练完成,最佳准确率: {best_accuracy:.2f}%')
return model, train_losses, val_accuracies
# 执行训练
model, train_losses, val_accuracies = train_catdog_model()
10.5 模型部署与推理
class CatDogPredictor:
def __init__(self, model_path, device='cpu'):
self.device = device
self.model = CatDogClassifier()
self.model.load_state_dict(torch.load(model_path, map_location=device))
self.model.to(device)
self.model.eval()
self.transform = test_transform
def predict_image(self, image_path):
"""预测单张图像"""
image = Image.open(image_path).convert('RGB')
image_tensor = self.transform(image).unsqueeze(0).to(self.device)
with torch.no_grad():
output = self.model(image_tensor)
probabilities = torch.softmax(output, dim=1)
confidence, prediction = torch.max(probabilities, 1)
class_names = ['猫', '狗']
result = {
'prediction': class_names[prediction.item()],
'confidence': confidence.item(),
'cat_prob': probabilities[0][0].item(),
'dog_prob': probabilities[0][1].item()
}
return result
def predict_batch(self, image_paths):
"""批量预测"""
results = []
for image_path in image_paths:
try:
result = self.predict_image(image_path)
results.append(result)
except Exception as e:
print(f"处理图像 {image_path} 时出错: {e}")
results.append(None)
return results
# 使用示例
predictor = CatDogPredictor('best_catdog_model.pth')
result = predictor.predict_image('test_image.jpg')
print(f"预测结果: {result['prediction']}, 置信度: {result['confidence']:.4f}")
11. 常见问题与解决方案
11.1 环境配置问题
问题1:PyTorch安装失败 解决方案:
# 使用国内镜像源
pip install torch torchvision torchaudio -i https://pypi.tuna.tsinghua.edu.cn/simple
# 或者使用conda
conda install pytorch torchvision torchaudio -c pytorch
问题2:CUDA版本不匹配 解决方案:访问PyTorch官网获取正确的安装命令,确保CUDA版本匹配。
11.2 模型训练问题
问题:过拟合 解决方案:
# 1. 增加数据增强
transform = transforms.Compose([
transforms.RandomHorizontalFlip(),
transforms.RandomRotation(10),
transforms.ColorJitter(0.2, 0.2, 0.2),
transforms.ToTensor(),
])
# 2. 添加正则化
optimizer = torch.optim.Adam(model.parameters(), lr=0.001, weight_decay=1e-4)
# 3. 使用Dropout
self.dropout = nn.Dropout(0.5)
问题:梯度消失/爆炸 解决方案:
# 梯度裁剪
torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
# 使用Batch Normalization
self.bn = nn.BatchNorm2d(64)
11.3 性能优化技巧
# 使用混合精度训练(FP16)
from torch.cuda.amp import autocast, GradScaler
scaler = GradScaler()
for input, target in dataloader:
optimizer.zero_grad()
with autocast():
output = model(input)
loss = criterion(output, target)
scaler.scale(loss).backward()
scaler.step(optimizer)
scaler.update()
# 数据加载优化
train_loader = DataLoader(dataset, batch_size=32, shuffle=True,
num_workers=4, pin_memory=True)
12. 学习路线与进阶方向
12.1 阶段性学习计划
第一阶段(1-2个月):基础掌握
- Python编程基础
- PyTorch张量操作
- 简单神经网络实现
- 线性回归、逻辑回归实战
第二阶段(2-3个月):中级进阶
- CNN图像分类项目
- R
更多推荐




所有评论(0)