LLM大模型量化系统学习全指南

适配训练部署从业者，含学习路径、核心论文、书籍、开源工程、前沿方向、代码仓库

一、整体进阶学习路线

阶段1 基础筑基（7天）

掌握量化核心概念
对称/非对称量化、缩放因子、零点、INT4/8、FP16/FP8/NF4
区分PTQ训练后量化、QAT量化感知训练、混合精度量化

阶段2 主流算法攻坚（7天）

吃透工业界三大核心方案
GPTQ → AWQ → SmoothQuant
同步理解QLoRA低比特微调逻辑

阶段3 工程源码实操（14天）

拆解量化内核、格式、推理框架
GGUF格式、CUDA反量化内核、KV缓存量化
落地vLLM、TensorRT-LLM、llama.cpp量化部署

阶段4 前沿深耕（长期）

KV Cache压缩、FP8训练、MoE量化、2bit极低比特、软硬件协同

二、必读核心论文清单

1. 全景综述（优先阅读）

  1. A Survey of Quantization in LLM: Unlocking Potential Hardware Efficiency | JCST 2026
    贴合硬件部署，覆盖FP8、KV量化、混合精度
  2. A Survey of Low-bit Large Language Models | arXiv 2024
    算法+系统+硬件全维度低比特量化汇总
  3. A Survey of Quantization Methods for Neural Network Inference | 2021
    神经网络量化基础理论奠基文献

2. PTQ训练后量化（部署核心）

  1. GPTQ: Accurate Quantization for Generative Pre-trained Transformers | ICLR2023
    二阶海森矩阵逐层量化，经典4bit部署基准
  2. AWQ: Activation-aware Weight Quantization | MLSys2024
    激活感知权重保护，消费卡最优量化方案
  3. SmoothQuant: Accurate Post-Training Quantization | ICML2023
    解决激活离群值，W8A8数据中心推理标配

3. 低比特微调&QAT

  1. QLoRA: Efficient Finetuning of Quantized LLMs | NeurIPS2023
    NF4双重量化，冻结模型4bit高效微调开山之作
  2. LLM-QAT: Data-Free Quantization Aware Training
    无数据量化感知训练，低精度精度修复

4. 极低比特前沿

BitNet: Scaling 1-bit Transformers
1.58bit极简网络，替代矩阵乘，下一代轻量化方向

5. KV缓存量化（长文本核心）

  1. KVQuant: 千万级上下文KV量化
  2. KIVI: 免微调2bit非对称KV压缩
  3. KV Cache Management综述 arXiv:2412.19442

6. 细分拓展

扩散模型量化、二值神经网络、量化软硬件协同设计

三、优质书籍推荐

量化领域论文时效性远高于书籍，以下为必看参考

  1. Neural Networks with Model Compression
    系统讲解量化、剪枝、蒸馏三大压缩技术，学术基础必备
  2. Ultimate ONNX for Deep Learning Optimization
    工程向，量化+ONNX推理+端侧部署实操
  3. 《动手学深度学习》
    补齐张量计算、硬件浮点基础原理
  4. Deep Learning for Computer Architecture
    GPU张量核心、低精度指令硬件适配原理

四、核心工程框架&学习重点

  1. llama.cpp
    吃透GGUF格式、Q4_K_M/IQ量化、CPU/GPU混合离线推理
  2. bitsandbytes
    NF4/FP4实现、8bit优化器、QLoRA底层核心
  3. AutoGPTQ
    GPTQ校准逻辑、量化配置、模型导出
  4. AutoAWQ
    40系显卡最优量化推理适配
  5. vLLM
    量化KV缓存、连续批处理、高吞吐量化服务
  6. TensorRT-LLM
    FP8、算子融合、企业级量产量化部署

五、当下热门研究方向

  1. KV Cache量化
    长上下文显存瓶颈最优解，主流3/4bit压缩路线
  2. FP8混合精度训练推理
    新一代算力芯片通用标准
  3. MoE模型量化
    DeepSeek、Mixtral专家模型量化难点优化
  4. 量化感知微调融合
    GPTQ/AWQ+LoRA联合微调，兼顾压缩与效果
  5. 2bit及以下极限低比特
    BitNet、二值大模型轻量化探索
  6. 国产DCU硬件适配量化
    异构芯片量化算子适配、并行量化推理

六、开源代码仓库汇总

通用量化框架

  • pytorch-quantization：英伟达官方PTQ/QAT工具
  • torchao：PyTorch原生轻量化量化优化
  • NNI Quantizer：自动比特分配量化调优

LLM专项量化

  • gptq：GPTQ算法原生实现
  • llm-awq：AWQ激活感知量化源码
  • smoothquant：激活平滑量化工程代码

一站式推理部署

  • vllm：集成全品类量化高吞吐服务
  • llama.cpp：跨平台低比特本地推理
  • text-generation-inference：HuggingFace官方量化部署

论文&教程合集 看后面的详细介绍

  • Awesome-Quantization-Papers：前沿论文持续更新
  • ModelCompressionTutorial：量化实操入门案例

七、实操学习顺序

  1. 基础概念 → 精读3篇核心综述建立认知
  2. 逐篇吃透GPTQ/AWQ/QLoRA三篇里程碑论文
  3. 运行开源库，完成单模型4bit量化推理实操
  4. 研读CUDA内核、GGUF格式底层源码
  5. 切入KV量化、FP8、MoE量化前沿研究
  6. 适配硬件，落地多卡量化并行部署

下面把**LLM量化核心论文 + 直接可访问链接（arXiv PDF/abs）**补齐，按之前结构整理好，可直接复制打开。

二、必读核心论文清单（带链接）

1. 全景综述（优先读）

  1. A Survey of Quantization in LLM: Unlocking Potential Hardware Efficiency (JCST 2026)
    （偏硬件/系统，国内期刊）
    → 暂无公开arXiv，可检索期刊官网

  2. A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithms (arXiv 2024)
    https://arxiv.org/pdf/2409.16694

  3. A Survey of Quantization Methods for Efficient Neural Network Inference (2021)
    （经典CNN/NN量化基础）
    https://arxiv.org/abs/2103.13630

2. PTQ 训练后量化（部署核心）

  1. GPTQ: Accurate Quantization for Generative Pre-trained Transformers (ICLR 2023)
    https://arxiv.org/pdf/2210.17323

  2. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration (MLSys 2024)
    https://arxiv.org/pdf/2306.00978

  3. SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models (ICML 2023)
    https://arxiv.org/pdf/2211.10438

3. 低比特微调 & QAT

  1. QLoRA: Efficient Finetuning of Quantized LLMs (NeurIPS 2023)
    https://arxiv.org/abs/2305.14314

  2. LLM-QAT: Data-Free Quantization Aware Training for Large Language Models (ACL 2023)
    https://arxiv.org/abs/2310.05074

4. 极低比特前沿

  1. BitNet: Scaling 1-bit Transformers for Large Language Models (2023)
    https://arxiv.org/abs/2310.11453

  2. BitNet b1.58 (2024)
    https://arxiv.org/abs/2402.17764

5. KV Cache 量化（长文本核心）

  1. KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization (NeurIPS 2024)
    https://arxiv.org/abs/2406.09278

  2. KIVI: A Tuning-Free Asymmetric 2-bit Quantization for KV Cache (ICML 2024)
    https://arxiv.org/abs/2405.14526

  3. A Survey on Large Language Model Acceleration based on KV Cache Management (arXiv 2024)
    https://arxiv.org/pdf/2412.19442

6. 其他重要经典（补充）

  • LLM.int8() (2022)
    https://arxiv.org/pdf/2208.07339
  • SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Compression (2023)
    https://arxiv.org/abs/2303.15698
  • OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models (2023)
    https://arxiv.org/abs/2308.13137

全部整理成一份可直接导入Zotero的BibTeX+URL列表

LLM量化论文 BibTeX 合集（带URL，直接导入Zotero）

复制整块文本，Zotero → 文件 → 导入 → 从剪贴板导入即可

@article{lowbitllmsurvey2024,
  title={A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithms},
  author={Zhang, X and others},
  journal={arXiv preprint arXiv:2409.16694},
  year={2024},
  url={https://arxiv.org/pdf/2409.16694}
}

@article{quantSurvey2021,
  title={A survey of quantization methods for efficient neural network inference},
  author={Gholami, Amir and Kim, Sehoon and Dong, Zhen and Yao, Zhewei and Mahoney, Michael W and Keutzer, Kurt},
  journal={arXiv preprint arXiv:2103.13630},
  year={2021},
  url={https://arxiv.org/abs/2103.13630}
}

@article{gptq2023,
  title={GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers},
  author={Frantar, Elias and Alistarh, Dan},
  journal={ICLR},
  year={2023},
  url={https://arxiv.org/pdf/2210.17323}
}

@article{awq2024,
  title={AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration},
  author={Lin, Ji and Tang, Haotian and Qin, Yuxuan and Liu, Zhehan and Han, Song},
  journal={MLSys},
  year={2024},
  url={https://arxiv.org/pdf/2306.00978}
}

@article{smoothquant2023,
  title={SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models},
  author={Xiao, Guangxuan and Lin, Ji and Seznec, Mickael and Wu, Hao and Demouth, Julien and Han, Song},
  journal={ICML},
  year={2023},
  url={https://arxiv.org/pdf/2211.10438}
}

@article{qlora2023,
  title={QLoRA: Efficient Finetuning of Quantized LLMs},
  author={Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke},
  journal={NeurIPS},
  year={2023},
  url={https://arxiv.org/abs/2305.14314}
}

@article{llmqat2023,
  title={LLM-QAT: Data-Free Quantization Aware Training for Large Language Models},
  author={Xu, Peijie and others},
  journal={ACL},
  year={2023},
  url={https://arxiv.org/abs/2310.05074}
}

@article{bitnet2023,
  title={BitNet: Scaling 1-bit Transformers for Large Language Models},
  author={Wang, Hongyu and others},
  journal={arXiv preprint arXiv:2310.11453},
  year={2023},
  url={https://arxiv.org/abs/2310.11453}
}

@article{bitnet1582024,
  title={BitNet b1.58},
  author={Ma, Shuming and others},
  journal={arXiv preprint arXiv:2402.17764},
  year={2024},
  url={https://arxiv.org/abs/2402.17764}
}

@article{kvquant2024,
  title={KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization},
  author={Li, Yuhang and others},
  journal={NeurIPS},
  year={2024},
  url={https://arxiv.org/abs/2406.09278}
}

@article{kivi2024,
  title={KIVI: A Tuning-Free Asymmetric 2-bit Quantization for KV Cache},
  author={Jin, Haoyu and others},
  journal={ICML},
  year={2024},
  url={https://arxiv.org/abs/2405.14526}
}

@article{kvcacheSurvey2024,
  title={A Survey on Large Language Model Acceleration based on KV Cache Management},
  author={Li, Haoyang and others},
  journal={arXiv preprint arXiv:2412.19442},
  year={2024},
  url={https://arxiv.org/pdf/2412.19442}
}

@article{llmint82022,
  title={LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale},
  author={Dettmers, Tim and Lewis, Mike and Shuster, Ari and Pettus, Robert and Zettlemoyer, Luke},
  journal={NeurIPS},
  year={2022},
  url={https://arxiv.org/pdf/2208.07339}
}

@article{spqr2023,
  title={SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Compression},
  author={Damm, Christopher and others},
  journal={arXiv preprint arXiv:2303.15698},
  year={2023},
  url={https://arxiv.org/abs/2303.15698}
}

@article{omniquant2023,
  title={OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models},
  author={Shao, Wenqi and others},
  journal={arXiv preprint arXiv:2308.13137},
  year={2023},
  url={https://arxiv.org/abs/2308.13137}
}
# 人工智能
Logo
https://edu.csdn.net/learn/39067/627173?utm_source=2019755004

汇聚全球AI编程工具，助力开发者即刻编程。

更多推荐

cover

一个 API 入口调用多个大模型：AiiOnly客户端 + CC Switch 打通 Codex AI编程流程

avatar AI编程社区
cover

VSCode Claude Code + MiMo 中转免登录完整配置教程（解决原生登录弹窗 + 模型报错）

avatar AI编程社区
cover

从零开始做一个属于自己的 Skill（保姆级教程）

avatar AI编程社区