目录

  1. 概述
  2. 发展历史
  3. 基础概念
  4. 传统方法
  5. 两阶段检测器
  6. 单阶段检测器
  7. 无锚框检测器
  8. Transformer检测器
  9. 关键组件
  10. 评估指标
  11. 数据集与增强
  12. 代表模型详解
  13. 完整代码实现
  14. 应用场景
  15. 参考资料

1. 概述

1.1 什么是目标检测

目标检测是计算机视觉的核心任务，旨在从图像中识别并定位感兴趣的目标，输出目标的类别和边界框位置。

1.2 任务定义

输入: RGB图像 I ∈ R^{H×W×3}
输出: 检测结果集合 {(class_i, bbox_i, score_i)}

其中:
- class_i: 目标类别
- bbox_i: 边界框 [x1, y1, x2, y2]
- score_i: 置信度分数

1.3 应用场景

领域 应用
自动驾驶 车辆、行人、交通标志检测
安防监控 异常行为检测、人脸识别
医学影像 病灶检测、器官定位
工业检测 缺陷检测、质量控制
零售分析 商品检测、客流统计
机器人 物体抓取、场景理解

2. 发展历史

2.1 传统方法时代（2000-2012）

2001: Viola-Jones (Haar特征 + AdaBoost)
2005: HOG特征
2008: DPM (Deformable Parts Model)

2.2 深度学习时代（2012-至今）

2012: AlexNet (深度学习革命)
2014: R-CNN (首个深度学习检测器)
2015: Fast R-CNN, Faster R-CNN, YOLO v1, SSD
2017: FPN, RetinaNet
2018: Mask R-CNN, Cascade R-CNN
2019: EfficientDet
2020: DETR (Transformer检测器)
2021: Swin Transformer, YOLOX
2022: DINO, Co-DETR
2023: YOLOv8, Grounding DINO
2024: YOLOv10, RT-DETR

2.3 技术演进

传统方法: 手工特征 + 分类器
    ↓
两阶段: 候选区域 + 分类回归
    ↓
单阶段: 密集预测，端到端
    ↓
无锚框: 去除预定义锚框
    ↓
Transformer: 集合预测，端到端

3. 基础概念

3.1 边界框表示

# 两种常见格式
# 1. (x1, y1, x2, y2) - 左上角和右下角
bbox_corner = [100, 100, 200, 300]

# 2. (cx, cy, w, h) - 中心点和宽高
bbox_center = [150, 200, 100, 200]

def corner_to_center(bbox):
    x1, y1, x2, y2 = bbox
    cx = (x1 + x2) / 2
    cy = (y1 + y2) / 2
    w = x2 - x1
    h = y2 - y1
    return [cx, cy, w, h]

def center_to_corner(bbox):
    cx, cy, w, h = bbox
    x1 = cx - w / 2
    y1 = cy - h / 2
    x2 = cx + w / 2
    y2 = cy + h / 2
    return [x1, y1, x2, y2]

3.2 交并比 (IoU)

def compute_iou(box1, box2):
    """
    计算两个边界框的IoU

    Args:
        box1, box2: [x1, y1, x2, y2]

    Returns:
        iou: 交并比
    """
    x1 = max(box1[0], box2[0])
    y1 = max(box1[1], box2[1])
    x2 = min(box1[2], box2[2])
    y2 = min(box1[3], box2[3])

    intersection = max(0, x2 - x1) * max(0, y2 - y1)

    area1 = (box1[2] - box1[0]) * (box1[3] - box1[1])
    area2 = (box2[2] - box2[0]) * (box2[3] - box2[1])

    union = area1 + area2 - intersection

    return intersection / (union + 1e-6)

3.3 锚框 (Anchor Box)

def generate_anchors(feature_size, scales, ratios):
    """
    生成锚框

    Args:
        feature_size: 特征图大小 (H, W)
        scales: 尺度列表 [32, 64, 128]
        ratios: 宽高比 [0.5, 1.0, 2.0]

    Returns:
        anchors: 锚框列表 [N, 4]
    """
    anchors = []
    for i in range(feature_size[0]):
        for j in range(feature_size[1]):
            cx = j * stride + stride / 2
            cy = i * stride + stride / 2

            for scale in scales:
                for ratio in ratios:
                    w = scale * np.sqrt(ratio)
                    h = scale / np.sqrt(ratio)
                    anchors.append([cx - w/2, cy - h/2, cx + w/2, cy + h/2])

    return np.array(anchors)

4. 传统方法

4.1 Viola-Jones

class ViolaJones:
    """Viola-Jones检测器"""
    def __init__(self):
        self.classifier = cv2.CascadeClassifier(
            cv2.data.haarcascades + 'haarcascade_frontalface_default.xml'
        )

    def detect(self, image, scale_factor=1.1, min_neighbors=5):
        gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
        faces = self.classifier.detectMultiScale(
            gray, scale_factor, min_neighbors
        )
        return faces

4.2 HOG + SVM

class HOGDetector:
    """HOG + SVM检测器"""
    def __init__(self):
        self.hog = cv2.HOGDescriptor()
        self.hog.setSVMDetector(cv2.HOGDescriptor_getDefaultPeopleDetector())

    def detect(self, image):
        boxes, weights = self.hog.detectMultiScale(
            image, winStride=(8, 8), padding=(4, 4), scale=1.05
        )
        return boxes, weights

4.3 DPM

class DPM:
    """可变形部件模型"""
    def __init__(self):
        # 根滤波器
        self.root_filter = None
        # 部件滤波器
        self.part_filters = []
        # 部件偏移
        self.deformation_costs = []

    def detect(self, pyramid):
        # 计算特征金字塔
        # 应用根滤波器
        # 应用部件滤波器
        # 组合分数
        pass

5. 两阶段检测器

5.1 R-CNN

class RCNN:
    """R-CNN检测器"""
    def __init__(self, backbone, roi_pooling, classifier, regressor):
        self.backbone = backbone
        self.roi_pooling = roi_pooling
        self.classifier = classifier
        self.regressor = regressor

    def detect(self, image):
        # 1. 选择性搜索生成候选区域
        proposals = selective_search(image)

        # 2. 提取每个候选区域的特征
        features = []
        for proposal in proposals:
            roi = crop_roi(image, proposal)
            feat = self.backbone(roi)
            features.append(feat)

        # 3. 分类和回归
        classes = self.classifier(features)
        bbox_deltas = self.regressor(features)

        # 4. 后处理
        results = self.postprocess(proposals, classes, bbox_deltas)
        return results

5.2 Fast R-CNN

class FastRCNN(nn.Module):
    """Fast R-CNN"""
    def __init__(self, backbone, roi_pool_size=7):
        super().__init__()
        self.backbone = backbone
        self.roi_pool = RoIPool(roi_pool_size, roi_pool_size)

        # 分类头
        self.cls_head = nn.Sequential(
            nn.Linear(256 * roi_pool_size * roi_pool_size, 1024),
            nn.ReLU(),
            nn.Linear(1024, num_classes + 1)  # +1 背景
        )

        # 回归头
        self.reg_head = nn.Sequential(
            nn.Linear(256 * roi_pool_size * roi_pool_size, 1024),
            nn.ReLU(),
            nn.Linear(1024, num_classes * 4)
        )

    def forward(self, image, rois):
        # 提取特征
        features = self.backbone(image)

        # ROI池化
        roi_features = self.roi_pool(features, rois)

        # 分类和回归
        cls_scores = self.cls_head(roi_features)
        bbox_preds = self.reg_head(roi_features)

        return cls_scores, bbox_preds


class RoIPool(nn.Module):
    """ROI池化层"""
    def __init__(self, output_size, spatial_scale):
        super().__init__()
        self.output_size = output_size
        self.spatial_scale = spatial_scale

    def forward(self, features, rois):
        # 将ROI坐标映射到特征图
        rois_scaled = rois * self.spatial_scale

        # 对每个ROI进行最大池化
        pooled = []
        for roi in rois_scaled:
            x1, y1, x2, y2 = roi.int()
            roi_feat = features[:, :, y1:y2, x1:x2]
            pooled_feat = F.adaptive_max_pool2d(roi_feat, self.output_size)
            pooled.append(pooled_feat)

        return torch.cat(pooled, dim=0)

5.3 Faster R-CNN

class FasterRCNN(nn.Module):
    """Faster R-CNN"""
    def __init__(self, backbone, rpn, roi_head):
        super().__init__()
        self.backbone = backbone
        self.rpn = rpn  # 区域提议网络
        self.roi_head = roi_head

    def forward(self, images, targets=None):
        # 提取特征
        features = self.backbone(images)

        # RPN生成候选区域
        proposals, rpn_losses = self.rpn(features, targets)

        # ROI Head分类和回归
        detections, roi_losses = self.roi_head(features, proposals, targets)

        return detections, rpn_losses, roi_losses


class RPN(nn.Module):
    """区域提议网络"""
    def __init__(self, in_channels, num_anchors=9):
        super().__init__()
        self.conv = nn.Conv2d(in_channels, in_channels, 3, padding=1)
        self.cls_logits = nn.Conv2d(in_channels, num_anchors, 1)
        self.bbox_pred = nn.Conv2d(in_channels, num_anchors * 4, 1)

        self.num_anchors = num_anchors
        self.anchors = None

    def forward(self, features, targets=None):
        # 卷积特征
        x = F.relu(self.conv(features))

        # 分类分数（前景/背景）
        objectness = self.cls_logits(x)

        # 边界框回归
        bbox_deltas = self.bbox_pred(x)

        # 生成锚框
        if self.anchors is None:
            self.anchors = self.generate_anchors(features)

        # 训练时计算损失
        if self.training:
            proposals, rpn_losses = self.compute_loss(
                objectness, bbox_deltas, self.anchors, targets
            )
        else:
            proposals = self.decode_proposals(objectness, bbox_deltas, self.anchors)
            rpn_losses = {}

        return proposals, rpn_losses

    def decode_proposals(self, objectness, bbox_deltas, anchors):
        """解码提议"""
        # 应用边界框回归
        proposals = self.apply_deltas(bbox_deltas, anchors)

        # NMS
        proposals = self.nms(proposals, objectness)

        return proposals

5.4 FPN (特征金字塔网络)

class FPN(nn.Module):
    """特征金字塔网络"""
    def __init__(self, in_channels_list, out_channels):
        super().__init__()
        self.lateral_convs = nn.ModuleList()
        self.output_convs = nn.ModuleList()

        for in_channels in in_channels_list:
            lateral = nn.Conv2d(in_channels, out_channels, 1)
            output = nn.Conv2d(out_channels, out_channels, 3, padding=1)
            self.lateral_convs.append(lateral)
            self.output_convs.append(output)

    def forward(self, features):
        # features: [C2, C3, C4, C5]
        laterals = [conv(f) for conv, f in zip(self.lateral_convs, features)]

        # 自顶向下融合
        for i in range(len(laterals) - 1, 0, -1):
            upsampled = F.interpolate(laterals[i], size=laterals[i-1].shape[2:])
            laterals[i-1] = laterals[i-1] + upsampled

        # 输出卷积
        outputs = [conv(l) for conv, l in zip(self.output_convs, laterals)]

        return outputs

6. 单阶段检测器

6.1 YOLO v1

class YOLOv1(nn.Module):
    """YOLO v1"""
    def __init__(self, S=7, B=2, C=20):
        super().__init__()
        self.S = S  # 网格大小
        self.B = B  # 每个网格的边界框数
        self.C = C  # 类别数

        # 骨干网络
        self.backbone = nn.Sequential(
            nn.Conv2d(3, 64, 7, stride=2, padding=3),
            nn.MaxPool2d(2, 2),
            nn.Conv2d(64, 192, 3, padding=1),
            nn.MaxPool2d(2, 2),
            # ... 更多卷积层
        )

        # 检测头
        self.head = nn.Sequential(
            nn.Linear(1024 * 7 * 7, 4096),
            nn.ReLU(),
            nn.Dropout(),
            nn.Linear(4096, S * S * (B * 5 + C))
        )

    def forward(self, x):
        features = self.backbone(x)
        features = features.view(features.size(0), -1)
        output = self.head(features)
        output = output.view(-1, self.S, self.S, self.B * 5 + self.C)
        return output

6.2 SSD

class SSD(nn.Module):
    """SSD检测器"""
    def __init__(self, num_classes, backbone):
        super().__init__()
        self.backbone = backbone

        # 多尺度检测头
        self.loc_heads = nn.ModuleList()
        self.cls_heads = nn.ModuleList()

        # 不同尺度的锚框数
        num_anchors = [4, 6, 6, 6, 4, 4]

        for i, num_anchor in enumerate(num_anchors):
            loc_head = nn.Conv2d(
                backbone.out_channels[i], num_anchor * 4, 3, padding=1
            )
            cls_head = nn.Conv2d(
                backbone.out_channels[i], num_anchor * num_classes, 3, padding=1
            )
            self.loc_heads.append(loc_head)
            self.cls_heads.append(cls_head)

    def forward(self, x):
        # 提取多尺度特征
        features = self.backbone(x)

        locs = []
        confs = []

        for i, feat in enumerate(features):
            loc = self.loc_heads[i](feat)
            conf = self.cls_heads[i](feat)

            loc = loc.permute(0, 2, 3, 1).contiguous()
            conf = conf.permute(0, 2, 3, 1).contiguous()

            locs.append(loc.view(loc.size(0), -1, 4))
            confs.append(conf.view(conf.size(0), -1, self.num_classes))

        locs = torch.cat(locs, dim=1)
        confs = torch.cat(confs, dim=1)

        return locs, confs

6.3 RetinaNet

class RetinaNet(nn.Module):
    """RetinaNet检测器"""
    def __init__(self, backbone, fpn, num_classes, num_anchors=9):
        super().__init__()
        self.backbone = backbone
        self.fpn = fpn

        # 分类子网络
        self.cls_subnet = nn.Sequential(
            nn.Conv2d(256, 256, 3, padding=1),
            nn.ReLU(),
            nn.Conv2d(256, 256, 3, padding=1),
            nn.ReLU(),
            nn.Conv2d(256, 256, 3, padding=1),
            nn.ReLU(),
            nn.Conv2d(256, num_anchors * num_classes, 3, padding=1)
        )

        # 回归子网络
        self.reg_subnet = nn.Sequential(
            nn.Conv2d(256, 256, 3, padding=1),
            nn.ReLU(),
            nn.Conv2d(256, 256, 3, padding=1),
            nn.ReLU(),
            nn.Conv2d(256, 256, 3, padding=1),
            nn.ReLU(),
            nn.Conv2d(256, num_anchors * 4, 3, padding=1)
        )

    def forward(self, x):
        # 骨干网络
        features = self.backbone(x)

        # FPN
        fpn_features = self.fpn(features)

        # 多尺度检测
        cls_outputs = []
        reg_outputs = []

        for feat in fpn_features:
            cls_outputs.append(self.cls_subnet(feat))
            reg_outputs.append(self.reg_subnet(feat))

        return cls_outputs, reg_outputs


class FocalLoss(nn.Module):
    """Focal Loss"""
    def __init__(self, alpha=0.25, gamma=2.0):
        super().__init__()
        self.alpha = alpha
        self.gamma = gamma

    def forward(self, pred, target):
        # 计算交叉熵
        ce_loss = F.binary_cross_entropy_with_logits(pred, target, reduction='none')

        # 计算pt
        pt = torch.exp(-ce_loss)

        # Focal权重
        focal_weight = (1 - pt) ** self.gamma

        # alpha权重
        alpha_weight = self.alpha * target + (1 - self.alpha) * (1 - target)

        # 最终损失
        loss = alpha_weight * focal_weight * ce_loss

        return loss.mean()

6.4 YOLO系列

class YOLOv5(nn.Module):
    """YOLOv5简化版"""
    def __init__(self, num_classes=80):
        super().__init__()
        self.num_classes = num_classes

        # 骨干网络 (CSPDarknet)
        self.backbone = CSPDarknet()

        # 颈部网络 (PANet)
        self.neck = PANet()

        # 检测头
        self.head = YOLOHead(num_classes)

    def forward(self, x):
        # 骨干网络
        features = self.backbone(x)

        # 颈部网络
        neck_features = self.neck(features)

        # 检测头
        outputs = self.head(neck_features)

        return outputs


class YOLOHead(nn.Module):
    """YOLO检测头"""
    def __init__(self, num_classes, anchors=None):
        super().__init__()
        self.num_classes = num_classes
        self.num_anchors = 3
        self.stride = [8, 16, 32]

        # 检测层
        self.detect_layers = nn.ModuleList()
        for stride in self.stride:
            detect = nn.Conv2d(256, self.num_anchors * (5 + num_classes), 1)
            self.detect_layers.append(detect)

    def forward(self, features):
        outputs = []

        for i, feat in enumerate(features):
            out = self.detect_layers[i](feat)
            out = out.view(
                out.size(0), self.num_anchors, 5 + self.num_classes,
                out.size(2), out.size(3)
            )
            out = out.permute(0, 1, 3, 4, 2).contiguous()
            outputs.append(out)

        return outputs

7. 无锚框检测器

7.1 CenterNet

class CenterNet(nn.Module):
    """CenterNet检测器"""
    def __init__(self, backbone, num_classes=80):
        super().__init__()
        self.backbone = backbone

        # 热图头（关键点检测）
        self.heatmap_head = nn.Sequential(
            nn.Conv2d(256, 256, 3, padding=1),
            nn.ReLU(),
            nn.Conv2d(256, num_classes, 1)
        )

        # 偏移头
        self.offset_head = nn.Sequential(
            nn.Conv2d(256, 256, 3, padding=1),
            nn.ReLU(),
            nn.Conv2d(256, 2, 1)
        )

        # 尺寸头
        self.size_head = nn.Sequential(
            nn.Conv2d(256, 256, 3, padding=1),
            nn.ReLU(),
            nn.Conv2d(256, 2, 1)
        )

    def forward(self, x):
        features = self.backbone(x)

        heatmap = torch.sigmoid(self.heatmap_head(features))
        offset = self.offset_head(features)
        size = self.size_head(features)

        return heatmap, offset, size

    def decode(self, heatmap, offset, size, threshold=0.3):
        """解码检测结果"""
        # 找到峰值点
        peaks = self.find_peaks(heatmap, threshold)

        detections = []
        for peak in peaks:
            cls, y, x = peak

            # 中心点
            cx = (x + offset[0, :, y, x]) * 4
            cy = (y + offset[1, :, y, x]) * 4

            # 尺寸
            w = size[0, :, y, x]
            h = size[1, :, y, x]

            # 置信度
            score = heatmap[0, cls, y, x]

            detections.append({
                'class': cls,
                'bbox': [cx - w/2, cy - h/2, cx + w/2, cy + h/2],
                'score': score
            })

        return detections

7.2 FCOS

class FCOS(nn.Module):
    """FCOS检测器"""
    def __init__(self, backbone, fpn, num_classes):
        super().__init__()
        self.backbone = backbone
        self.fpn = fpn
        self.num_classes = num_classes

        # 共享卷积
        self.share_convs = nn.Sequential(
            nn.Conv2d(256, 256, 3, padding=1),
            nn.GroupNorm(32, 256),
            nn.ReLU()
        )

        # 分类头
        self.cls_head = nn.Conv2d(256, num_classes, 3, padding=1)

        # 回归头
        self.reg_head = nn.Conv2d(256, 4, 3, padding=1)

        # 中心度头
        self.centerness_head = nn.Conv2d(256, 1, 3, padding=1)

    def forward(self, x):
        features = self.backbone(x)
        fpn_features = self.fpn(features)

        cls_scores = []
        bbox_preds = []
        centerness = []

        for feat in fpn_features:
            shared = self.share_convs(feat)
            cls_scores.append(self.cls_head(shared))
            bbox_preds.append(self.reg_head(shared))
            centerness.append(self.centerness_head(shared))

        return cls_scores, bbox_preds, centerness


def compute_centerness_targets(reg_targets):
    """计算中心度目标"""
    left = reg_targets[:, 0]
    top = reg_targets[:, 1]
    right = reg_targets[:, 2]
    bottom = reg_targets[:, 3]

    centerness = (torch.min(left, right) / torch.max(left, right)) * \
                 (torch.min(top, bottom) / torch.max(top, bottom))

    return torch.sqrt(centerness)

7.3 ATSS

class ATSS:
    """自适应训练样本选择"""
    def __init__(self, top_k=9):
        self.top_k = top_k

    def select_samples(self, anchors, gt_boxes):
        """
        ATSS样本选择

        Args:
            anchors: 锚框
            gt_boxes: 真值框

        Returns:
            pos_indices: 正样本索引
            neg_indices: 负样本索引
        """
        # 计算IoU
        ious = compute_iou_matrix(anchors, gt_boxes)

        # 对每个GT选择top_k个最近的锚框
        selected = []
        for gt_idx in range(len(gt_boxes)):
            # 按中心距离选择
            distances = compute_center_distance(anchors, gt_boxes[gt_idx])
            _, top_k_indices = torch.topk(distances, self.top_k, largest=False)
            selected.append(top_k_indices)

        # 计算阈值
        selected_ious = []
        for gt_idx, indices in enumerate(selected):
            selected_ious.append(ious[indices, gt_idx])

        threshold = torch.stack([ious.mean() + ious.std() for ious in selected_ious])

        # 选择正样本
        pos_indices = []
        neg_indices = []

        for gt_idx, indices in enumerate(selected):
            iou_mask = ious[indices, gt_idx] > threshold[gt_idx]
            pos_indices.extend(indices[iou_mask].tolist())

        # 剩余为负样本
        all_indices = set(range(len(anchors)))
        neg_indices = list(all_indices - set(pos_indices))

        return pos_indices, neg_indices

8. Transformer检测器

8.1 DETR

class DETR(nn.Module):
    """DEtection TRansformer"""
    def __init__(self, backbone, transformer, num_classes, num_queries=100):
        super().__init__()
        self.backbone = backbone
        self.transformer = transformer
        self.num_queries = num_queries

        # 查询嵌入
        self.query_embed = nn.Embedding(num_queries, 256)

        # 位置编码
        self.pos_encoder = PositionalEncoding(256)

        # 预测头
        self.class_head = nn.Linear(256, num_classes + 1)
        self.bbox_head = nn.Sequential(
            nn.Linear(256, 256),
            nn.ReLU(),
            nn.Linear(256, 256),
            nn.ReLU(),
            nn.Linear(256, 4)
        )

    def forward(self, images):
        # 提取特征
        features = self.backbone(images)
        src = features.flatten(2).permute(2, 0, 1)  # [HW, B, C]

        # 位置编码
        pos = self.pos_encoder(features).flatten(2).permute(2, 0, 1)

        # 查询
        query_embed = self.query_embed.weight.unsqueeze(1).repeat(1, images.size(0), 1)
        tgt = torch.zeros_like(query_embed)

        # Transformer
        hs = self.transformer(src, tgt, pos, query_embed)

        # 预测
        outputs_class = self.class_head(hs)
        outputs_coord = self.bbox_head(hs).sigmoid()

        return {
            'pred_logits': outputs_class[-1],
            'pred_boxes': outputs_coord[-1]
        }


class DETRTransformer(nn.Module):
    """DETR Transformer"""
    def __init__(self, d_model=256, nhead=8, num_encoder_layers=6, num_decoder_layers=6):
        super().__init__()
        self.encoder = nn.TransformerEncoder(
            nn.TransformerEncoderLayer(d_model, nhead),
            num_encoder_layers
        )
        self.decoder = nn.TransformerDecoder(
            nn.TransformerDecoderLayer(d_model, nhead),
            num_decoder_layers
        )

    def forward(self, src, tgt, pos, query_embed):
        # 编码器
        memory = self.encoder(src + pos)

        # 解码器
        hs = self.decoder(tgt + query_embed, memory)

        return hs

8.2 Deformable DETR

class DeformableDETR(nn.Module):
    """可变形DETR"""
    def __init__(self, backbone, num_classes, num_queries=300, num_points=4):
        super().__init__()
        self.backbone = backbone
        self.num_queries = num_queries
        self.num_points = num_points

        # 可变形注意力
        self.deformable_attn = DeformableAttention(
            d_model=256, n_heads=8, n_points=num_points
        )

        # 预测头
        self.class_head = nn.Linear(256, num_classes + 1)
        self.bbox_head = nn.Linear(256, 4)

    def forward(self, images):
        # 多尺度特征
        features = self.backbone(images)

        # 可变形注意力
        # 参考点预测
        reference_points = self.predict_reference_points()

        # 应用可变形注意力
        hs = self.deformable_attn(features, reference_points)

        # 预测
        outputs_class = self.class_head(hs)
        outputs_coord = self.bbox_head(hs).sigmoid()

        return outputs_class, outputs_coord


class DeformableAttention(nn.Module):
    """可变形注意力"""
    def __init__(self, d_model, n_heads, n_points):
        super().__init__()
        self.n_heads = n_heads
        self.n_points = n_points

        # 偏移预测
        self.offset_proj = nn.Linear(d_model, n_heads * n_points * 2)

        # 注意力权重
        self.attn_proj = nn.Linear(d_model, n_heads * n_points)

        # 值投影
        self.value_proj = nn.Linear(d_model, d_model)
        self.output_proj = nn.Linear(d_model, d_model)

    def forward(self, query, value, reference_points):
        """
        Args:
            query: 查询特征
            value: 值特征
            reference_points: 参考点坐标
        """
        # 预测偏移
        offsets = self.offset_proj(query)
        offsets = offsets.view(-1, self.n_heads, self.n_points, 2)

        # 采样位置
        sampling_locations = reference_points + offsets

        # 采样特征
        sampled_features = self.bilinear_sample(value, sampling_locations)

        # 注意力权重
        attn_weights = self.attn_proj(query)
        attn_weights = F.softmax(attn_weights.view(-1, self.n_heads, self.n_points), dim=-1)

        # 加权聚合
        output = (sampled_features * attn_weights.unsqueeze(-1)).sum(dim=2)
        output = self.output_proj(output)

        return output

8.3 DINO

class DINO(nn.Module):
    """DINO检测器"""
    def __init__(self, backbone, transformer, num_classes, num_queries=900):
        super().__init__()
        self.backbone = backbone
        self.transformer = transformer
        self.num_queries = num_queries

        # 两组查询：正查询和负查询
        self.pos_query_embed = nn.Embedding(num_queries, 256)
        self.neg_query_embed = nn.Embedding(num_queries, 256)

        # 对比去噪训练
        self.denoising_groups = 100

    def forward(self, images, targets=None):
        # 提取特征
        features = self.backbone(images)

        if self.training and targets is not None:
            # 生成去噪查询
            noised_queries = self.generate_noised_queries(targets)

            # 前向传播
            outputs = self.transformer(features, noised_queries)

            # 计算损失
            losses = self.compute_losses(outputs, targets)
            return losses
        else:
            # 推理
            outputs = self.transformer(features)
            return self.postprocess(outputs)

9. 关键组件

9.1 骨干网络

class ResNetBackbone(nn.Module):
    """ResNet骨干网络"""
    def __init__(self, depth=50):
        super().__init__()
        if depth == 50:
            self.resnet = torchvision.models.resnet50(pretrained=True)
        elif depth == 101:
            self.resnet = torchvision.models.resnet101(pretrained=True)

        # 提取中间特征
        self.feature_layers = nn.ModuleList([
            nn.Sequential(self.resnet.conv1, self.resnet.bn1, self.resnet.relu, self.resnet.maxpool),
            self.resnet.layer1,
            self.resnet.layer2,
            self.resnet.layer3,
            self.resnet.layer4
        ])

    def forward(self, x):
        features = []
        for layer in self.feature_layers:
            x = layer(x)
            features.append(x)
        return features  # [C2, C3, C4, C5]

9.2 非极大值抑制 (NMS)

def nms(boxes, scores, iou_threshold=0.5):
    """
    非极大值抑制

    Args:
        boxes: 边界框 [N, 4]
        scores: 置信度分数 [N]
        iou_threshold: IoU阈值

    Returns:
        keep: 保留的索引
    """
    order = scores.argsort(descending=True)
    keep = []

    while len(order) > 0:
        i = order[0]
        keep.append(i)

        # 计算IoU
        ious = compute_iou(boxes[i:i+1], boxes[order[1:]])

        # 保留IoU小于阈值的
        mask = ious < iou_threshold
        order = order[1:][mask]

    return torch.tensor(keep)


def soft_nms(boxes, scores, sigma=0.5, score_threshold=0.001):
    """软NMS"""
    indices = torch.arange(len(scores))

    for i in range(len(boxes)):
        # 找到最大分数
        max_idx = scores[i:].argmax() + i

        # 交换
        boxes[[i, max_idx]] = boxes[[max_idx, i]]
        scores[[i, max_idx]] = scores[[max_idx, i]]
        indices[[i, max_idx]] = indices[[max_idx, i]]

        # 计算IoU并衰减分数
        ious = compute_iou(boxes[i:i+1], boxes[i+1:])
        weights = torch.exp(-(ious ** 2) / sigma)
        scores[i+1:] *= weights

    # 过滤低分
    keep = scores > score_threshold
    return indices[keep], scores[keep]

9.3 ROI Align

class RoIAlign(nn.Module):
    """ROI Align层"""
    def __init__(self, output_size, spatial_scale, sampling_ratio=2):
        super().__init__()
        self.output_size = output_size
        self.spatial_scale = spatial_scale
        self.sampling_ratio = sampling_ratio

    def forward(self, features, rois):
        """
        Args:
            features: 特征图 [B, C, H, W]
            rois: ROI [N, 5] (batch_idx, x1, y1, x2, y2)
        """
        return roi_align(
            features, rois,
            self.output_size,
            self.spatial_scale,
            self.sampling_ratio
        )

9.4 数据增强

class DetectionAugmentation:
    """检测数据增强"""
    def __init__(self):
        self.transforms = [
            RandomHorizontalFlip(),
            RandomScale(),
            RandomCrop(),
            ColorJitter(),
            Normalize()
        ]

    def __call__(self, image, boxes, labels):
        for transform in self.transforms:
            image, boxes, labels = transform(image, boxes, labels)
        return image, boxes, labels


class MosaicAugmentation:
    """Mosaic数据增强"""
    def __init__(self, size=640):
        self.size = size

    def __call__(self, images, targets):
        # 随机选择4张图
        indices = np.random.choice(len(images), 4, replace=False)

        # 创建马赛克
        mosaic = np.zeros((self.size * 2, self.size * 2, 3))
        mosaic_targets = []

        # 放置4张图
        positions = [
            (0, 0),
            (self.size, 0),
            (0, self.size),
            (self.size, self.size)
        ]

        for idx, (x_offset, y_offset) in zip(indices, positions):
            img = images[idx]
            h, w = img.shape[:2]

            # 缩放
            scale = min(self.size / h, self.size / w)
            img_resized = cv2.resize(img, (int(w * scale), int(h * scale)))

            # 放置
            h_new, w_new = img_resized.shape[:2]
            mosaic[y_offset:y_offset+h_new, x_offset:x_offset+w_new] = img_resized

            # 调整边界框
            for target in targets[idx]:
                bbox = target['bbox'] * scale
                bbox[0::2] += x_offset
                bbox[1::2] += y_offset
                mosaic_targets.append({**target, 'bbox': bbox})

        return mosaic, mosaic_targets

10. 评估指标

10.1 IoU计算

def compute_iou_matrix(boxes1, boxes2):
    """
    计算IoU矩阵

    Args:
        boxes1: [N, 4]
        boxes2: [M, 4]

    Returns:
        iou_matrix: [N, M]
    """
    area1 = (boxes1[:, 2] - boxes1[:, 0]) * (boxes1[:, 3] - boxes1[:, 1])
    area2 = (boxes2[:, 2] - boxes2[:, 0]) * (boxes2[:, 3] - boxes2[:, 1])

    # 计算交集
    lt = torch.max(boxes1[:, None, :2], boxes2[None, :, :2])
    rb = torch.min(boxes1[:, None, 2:], boxes2[None, :, 2:])

    wh = (rb - lt).clamp(min=0)
    intersection = wh[:, :, 0] * wh[:, :, 1]

    # 计算并集
    union = area1[:, None] + area2[None, :] - intersection

    return intersection / (union + 1e-6)

10.2 AP计算

def compute_ap(recall, precision):
    """计算AP"""
    # 插值
    mrec = np.concatenate(([0.], recall, [1.]))
    mpre = np.concatenate(([0.], precision, [0.]))

    # 计算包络线
    for i in range(mpre.size - 1, 0, -1):
        mpre[i - 1] = np.maximum(mpre[i - 1], mpre[i])

    # 计算面积
    i = np.where(mrec[1:] != mrec[:-1])[0]
    ap = np.sum((mrec[i + 1] - mrec[i]) * mpre[i + 1])

    return ap


def evaluate_detections(predictions, ground_truths, iou_threshold=0.5):
    """
    评估检测结果

    Args:
        predictions: 预测结果 [(image_id, class, score, bbox), ...]
        ground_truths: 真值 [(image_id, class, bbox), ...]
        iou_threshold: IoU阈值

    Returns:
        results: 各类别的AP
    """
    results = {}

    # 按类别评估
    classes = set([p[1] for p in predictions])

    for cls in classes:
        # 获取该类别的预测和真值
        cls_preds = [p for p in predictions if p[1] == cls]
        cls_gts = [g for g in ground_truths if g[1] == cls]

        # 按置信度排序
        cls_preds.sort(key=lambda x: x[2], reverse=True)

        # 计算TP和FP
        tp = np.zeros(len(cls_preds))
        fp = np.zeros(len(cls_preds))
        matched = set()

        for i, pred in enumerate(cls_preds):
            best_iou = 0
            best_gt_idx = -1

            for j, gt in enumerate(cls_gts):
                if gt[0] == pred[0] and j not in matched:
                    iou = compute_iou(pred[3], gt[2])
                    if iou > best_iou:
                        best_iou = iou
                        best_gt_idx = j

            if best_iou >= iou_threshold:
                tp[i] = 1
                matched.add(best_gt_idx)
            else:
                fp[i] = 1

        # 计算PR曲线
        tp_cumsum = np.cumsum(tp)
        fp_cumsum = np.cumsum(fp)
        recall = tp_cumsum / len(cls_gts)
        precision = tp_cumsum / (tp_cumsum + fp_cumsum)

        # 计算AP
        ap = compute_ap(recall, precision)
        results[cls] = ap

    # 计算mAP
    results['mAP'] = np.mean(list(results.values()))

    return results

10.3 COCO指标

def compute_coco_metrics(predictions, ground_truths):
    """计算COCO指标"""
    # IoU阈值: 0.5:0.05:0.95
    iou_thresholds = np.arange(0.5, 1.0, 0.05)

    aps = []
    for iou_thresh in iou_thresholds:
        results = evaluate_detections(predictions, ground_truths, iou_thresh)
        aps.append(results['mAP'])

    # AP@[0.5:0.95]
    coco_ap = np.mean(aps)

    # AP@0.5
    ap_50 = aps[0]

    # AP@0.75
    ap_75 = aps[5]

    return {
        'AP': coco_ap,
        'AP50': ap_50,
        'AP75': ap_75
    }

11. 数据集与增强

11.1 常用数据集

class COCODataset(torch.utils.data.Dataset):
    """COCO数据集"""
    def __init__(self, root, annotation, transforms=None):
        self.root = root
        self.coco = COCO(annotation)
        self.transforms = transforms

        # 获取所有图像ID
        self.ids = list(sorted(self.coco.imgs.keys()))

    def __getitem__(self, idx):
        img_id = self.ids[idx]

        # 加载图像
        img_info = self.coco.loadImgs(img_id)[0]
        image = cv2.imread(os.path.join(self.root, img_info['file_name']))

        # 加载标注
        ann_ids = self.coco.getAnnIds(imgIds=img_id)
        anns = self.coco.loadAnns(ann_ids)

        # 提取边界框和标签
        boxes = []
        labels = []
        for ann in anns:
            x, y, w, h = ann['bbox']
            boxes.append([x, y, x + w, y + h])
            labels.append(ann['category_id'])

        boxes = torch.FloatTensor(boxes)
        labels = torch.LongTensor(labels)

        # 数据增强
        if self.transforms:
            image, boxes, labels = self.transforms(image, boxes, labels)

        target = {
            'boxes': boxes,
            'labels': labels,
            'image_id': torch.tensor([img_id])
        }

        return image, target

    def __len__(self):
        return len(self.ids)

11.2 数据增强方法

class RandomHorizontalFlip:
    """随机水平翻转"""
    def __init__(self, p=0.5):
        self.p = p

    def __call__(self, image, boxes, labels):
        if np.random.random() < self.p:
            h, w = image.shape[:2]
            image = np.fliplr(image).copy()

            # 调整边界框
            boxes[:, [0, 2]] = w - boxes[:, [2, 0]]

        return image, boxes, labels


class RandomScale:
    """随机缩放"""
    def __init__(self, scale_range=(0.8, 1.2)):
        self.scale_range = scale_range

    def __call__(self, image, boxes, labels):
        scale = np.random.uniform(*self.scale_range)
        h, w = image.shape[:2]

        # 缩放图像
        new_h, new_w = int(h * scale), int(w * scale)
        image = cv2.resize(image, (new_w, new_h))

        # 缩放边界框
        boxes = boxes * scale

        return image, boxes, labels


class ColorJitter:
    """颜色抖动"""
    def __init__(self, brightness=0.2, contrast=0.2, saturation=0.2):
        self.brightness = brightness
        self.contrast = contrast
        self.saturation = saturation

    def __call__(self, image, boxes, labels):
        # 亮度
        if np.random.random() < 0.5:
            factor = 1 + np.random.uniform(-self.brightness, self.brightness)
            image = np.clip(image * factor, 0, 255)

        # 对比度
        if np.random.random() < 0.5:
            factor = 1 + np.random.uniform(-self.contrast, self.contrast)
            mean = image.mean()
            image = np.clip((image - mean) * factor + mean, 0, 255)

        return image, boxes, labels

12. 代表模型详解

12.1 YOLOv8

class YOLOv8(nn.Module):
    """YOLOv8"""
    def __init__(self, num_classes=80):
        super().__init__()
        self.num_classes = num_classes

        # 骨干网络 (C2f模块)
        self.backbone = CSPDarknetv8()

        # 颈部网络 (PAN-FPN)
        self.neck = PANFPN()

        # 解耦检测头
        self.cls_head = nn.ModuleList()
        self.reg_head = nn.ModuleList()

        for _ in range(3):  # 3个尺度
            cls_head = nn.Sequential(
                nn.Conv2d(256, 256, 3, padding=1),
                nn.BatchNorm2d(256),
                nn.SiLU(),
                nn.Conv2d(256, num_classes, 1)
            )
            reg_head = nn.Sequential(
                nn.Conv2d(256, 256, 3, padding=1),
                nn.BatchNorm2d(256),
                nn.SiLU(),
                nn.Conv2d(256, 64, 1)  # 4 * 16 (DFL)
            )
            self.cls_head.append(cls_head)
            self.reg_head.append(reg_head)

    def forward(self, x):
        # 骨干网络
        features = self.backbone(x)

        # 颈部网络
        neck_features = self.neck(features)

        # 解耦头
        cls_outputs = []
        reg_outputs = []

        for i, feat in enumerate(neck_features):
            cls_outputs.append(self.cls_head[i](feat))
            reg_outputs.append(self.reg_head[i](feat))

        return cls_outputs, reg_outputs

12.2 RT-DETR

class RTDETR(nn.Module):
    """实时DETR"""
    def __init__(self, backbone, decoder, num_classes, num_queries=300):
        super().__init__()
        self.backbone = backbone
        self.decoder = decoder
        self.num_queries = num_queries

        # 查询选择
        self.query_select = nn.Linear(256, num_queries)

        # 预测头
        self.class_head = nn.Linear(256, num_classes + 1)
        self.bbox_head = nn.Linear(256, 4)

    def forward(self, images):
        # 提取特征
        features = self.backbone(images)

        # 查询选择（从特征中选择）
        queries = self.select_queries(features)

        # 解码器
        hs = self.decoder(features, queries)

        # 预测
        outputs_class = self.class_head(hs)
        outputs_coord = self.bbox_head(hs).sigmoid()

        return outputs_class, outputs_coord

13. 完整代码实现

13.1 使用PyTorch实现简单检测器

import torch
import torch.nn as nn
import torchvision

class SimpleDetector(nn.Module):
    """简单目标检测器"""
    def __init__(self, num_classes, num_anchors=9):
        super().__init__()
        self.num_classes = num_classes
        self.num_anchors = num_anchors

        # 骨干网络
        backbone = torchvision.models.resnet50(pretrained=True)
        self.features = nn.Sequential(*list(backbone.children())[:-2])

        # FPN
        self.fpn = FPN([1024, 2048], 256)

        # RPN
        self.rpn = RPN(256, num_anchors)

        # ROI Head
        self.roi_head = RoIHead(256, num_classes)

    def forward(self, images, targets=None):
        # 提取特征
        features = self.features(images)
        fpn_features = self.fpn([features[-2], features[-1]])

        # RPN
        proposals, rpn_losses = self.rpn(fpn_features, targets)

        # ROI Head
        detections, roi_losses = self.roi_head(fpn_features, proposals, targets)

        return detections, rpn_losses, roi_losses


class RoIHead(nn.Module):
    """ROI Head"""
    def __init__(self, in_channels, num_classes):
        super().__init__()
        self.roi_pool = RoIAlign(output_size=7, spatial_scale=1/16)

        self.fc = nn.Sequential(
            nn.Linear(in_channels * 7 * 7, 1024),
            nn.ReLU(),
            nn.Linear(1024, 1024),
            nn.ReLU()
        )

        self.cls_score = nn.Linear(1024, num_classes + 1)
        self.bbox_pred = nn.Linear(1024, (num_classes + 1) * 4)

    def forward(self, features, proposals, targets=None):
        # ROI池化
        roi_features = self.roi_pool(features, proposals)

        # 全连接
        roi_features = roi_features.view(roi_features.size(0), -1)
        roi_features = self.fc(roi_features)

        # 分类和回归
        cls_scores = self.cls_score(roi_features)
        bbox_preds = self.bbox_pred(roi_features)

        if self.training:
            losses = self.compute_loss(cls_scores, bbox_preds, targets)
            return None, losses
        else:
            detections = self.decode_detections(cls_scores, bbox_preds, proposals)
            return detections, {}

13.2 使用Detectron2

from detectron2 import model_zoo
from detectron2.engine import DefaultTrainer
from detectron2.config import get_cfg
from detectron2.data import DatasetCatalog, MetadataCatalog

def setup_detectron2():
    """配置Detectron2"""
    cfg = get_cfg()
    cfg.merge_from_file(model_zoo.get_config_file(
        "COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml"
    ))
    cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(
        "COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml"
    )
    cfg.DATASETS.TRAIN = ("my_dataset_train",)
    cfg.DATASETS.TEST = ("my_dataset_val",)
    cfg.DATALOADER.NUM_WORKERS = 4
    cfg.SOLVER.IMS_PER_BATCH = 4
    cfg.SOLVER.BASE_LR = 0.001
    cfg.SOLVER.MAX_ITER = 10000
    cfg.MODEL.ROI_HEADS.NUM_CLASSES = 80

    return cfg


# 训练
class MyTrainer(DefaultTrainer):
    @classmethod
    def build_evaluator(cls, cfg, dataset_name, output_folder=None):
        return COCOEvaluator(dataset_name, cfg, False, output_folder)


# 推理
from detectron2.engine import DefaultPredictor

def inference(image_path, cfg):
    """推理"""
    predictor = DefaultPredictor(cfg)
    image = cv2.imread(image_path)
    outputs = predictor(image)

    # 获取检测结果
    instances = outputs["instances"]
    boxes = instances.pred_boxes.tensor.cpu().numpy()
    scores = instances.scores.cpu().numpy()
    classes = instances.pred_classes.cpu().numpy()

    return boxes, scores, classes

13.3 使用Ultralytics YOLO

from ultralytics import YOLO

def train_yolo():
    """训练YOLOv8"""
    # 加载模型
    model = YOLO('yolov8n.pt')

    # 训练
    results = model.train(
        data='coco.yaml',
        epochs=100,
        imgsz=640,
        batch=16,
        name='yolov8_custom'
    )

    return results


def inference_yolo(image_path):
    """YOLO推理"""
    model = YOLO('yolov8n.pt')

    # 推理
    results = model(image_path)

    # 解析结果
    for result in results:
        boxes = result.boxes.xyxy.cpu().numpy()
        scores = result.boxes.conf.cpu().numpy()
        classes = result.boxes.cls.cpu().numpy()

        return boxes, scores, classes

14. 应用场景

14.1 自动驾驶

class AutonomousDrivingDetector:
    """自动驾驶检测器"""
    def __init__(self):
        self.model = YOLOv8(num_classes=80)
        self.class_names = ['car', 'truck', 'bus', 'person', 'bicycle', ...]

    def detect(self, image, lidar_points=None):
        """检测"""
        # 图像检测
        detections = self.model(image)

        # 融合激光雷达（可选）
        if lidar_points is not None:
            detections = self.fuse_lidar(detections, lidar_points)

        # 后处理
        detections = self.postprocess(detections)

        return detections

14.2 医学影像

class MedicalDetector:
    """医学影像检测器"""
    def __init__(self, organ_type='lung'):
        self.organ_type = organ_type
        self.model = self.load_model()

    def detect_nodules(self, ct_scan):
        """肺结节检测"""
        # 预处理
        scan = self.preprocess(ct_scan)

        # 3D检测
        detections = self.model(scan)

        # 后处理
        nodules = self.postprocess(detections)

        return nodules

14.3 工业检测

class DefectDetector:
    """缺陷检测器"""
    def __init__(self, product_type='pcb'):
        self.product_type = product_type
        self.model = self.load_model()

    def detect_defects(self, image):
        """缺陷检测"""
        # 预处理
        image = self.preprocess(image)

        # 检测
        defects = self.model(image)

        # 分类缺陷类型
        for defect in defects:
            defect['type'] = self.classify_defect(defect)

        return defects

15. 参考资料

核心论文

  1. R-CNN: “Rich feature hierarchies for accurate object detection” (Girshick et al., 2014)
  2. Fast R-CNN: “Fast R-CNN” (Girshick, 2015)
  3. Faster R-CNN: “Faster R-CNN: Towards Real-Time Object Detection” (Ren et al., 2015)
  4. YOLO: “You Only Look Once: Unified, Real-Time Object Detection” (Redmon et al., 2016)
  5. SSD: “SSD: Single Shot MultiBox Detector” (Liu et al., 2016)
  6. FPN: “Feature Pyramid Networks for Object Detection” (Lin et al., 2017)
  7. RetinaNet: “Focal Loss for Dense Object Detection” (Lin et al., 2017)
  8. DETR: “End-to-End Object Detection with Transformers” (Carion et al., 2020)
  9. DINO: “DINO: DETR with Improved DeNoising Anchor Boxes” (Zhang et al., 2022)

开源库

  • Detectron2: https://github.com/facebookresearch/detectron2
  • MMDetection: https://github.com/open-mmlab/mmdetection
  • Ultralytics: https://github.com/ultralytics/ultralytics
  • YOLOv5: https://github.com/ultralytics/yolov5

数据集

  • COCO: https://cocodataset.org/
  • Pascal VOC: http://host.robots.ox.ac.uk/pascal/VOC/
  • ImageNet: https://www.image-net.org/
  • OpenImages: https://storage.googleapis.com/openimages/web/index.html
# 目标检测 # 人工智能 # 计算机视觉
Logo
https://edu.csdn.net/learn/39067/627173?utm_source=2019755004

汇聚全球AI编程工具，助力开发者即刻编程。

更多推荐

cover

当 AI 修图师装上“万能插头“：MCP 如何改变修图工作流

avatar AI编程社区
cover

Claude API + Python实战：结构化输出后，如何拦住“看起来正确”的JSON

avatar AI编程社区
cover

投资一台B300，部署Claude Opus 5多久才能回本？

avatar AI编程社区