def configure_optimizers(self):
    params = list(model.named_parameters())

    def is_backbone(n): return 'bert' in n

    grouped_parameters = [
        {"params": [p for n, p in params if is_backbone(n)], 'lr': args.lr},
        {"params": [p for n, p in params if not is_backbone(n)], 'lr': args.lr * 100},
    ]

    optimizer = torch.optim.AdamW(
        grouped_parameters, lr=args.lr, weight_decay=0
    )

    return optimizer

我用其他方法时,出现了如下报错,一直没有解决,但换了上面的方法就解决了
optimizer can only optimize Tensors, but one of the params is

Logo

汇聚全球AI编程工具,助力开发者即刻编程。

更多推荐