Phi-3-mini-4k-instruct-gguf部署教程：Docker Compose编排vLLM服务+Chainlit+Redis会话存储

一朵小小玫

946人浏览 · 2026-04-29 03:36:15

一朵小小玫 · 2026-04-29 03:36:15 发布

Phi-3-mini-4k-instruct-gguf部署教程：Docker Compose编排vLLM服务+Chainlit+Redis会话存储

1. 环境准备与快速部署

在开始部署Phi-3-mini-4k-instruct-gguf模型之前，我们需要确保系统满足以下基本要求：

操作系统：推荐使用Ubuntu 20.04/22.04或兼容的Linux发行版
Docker版本：20.10.0或更高
Docker Compose：v2.0.0或更高
硬件要求：
- 至少16GB内存（推荐32GB）
- 支持CUDA的NVIDIA GPU（推荐RTX 3090或更高）
- 至少20GB可用磁盘空间

1.1 安装Docker和NVIDIA容器工具包

首先安装必要的依赖项：

# 安装Docker
sudo apt-get update
sudo apt-get install -y docker.io docker-compose

# 添加当前用户到docker组
sudo usermod -aG docker $USER
newgrp docker

# 安装NVIDIA容器工具包
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
sudo systemctl restart docker

1.2 准备部署文件

创建项目目录并准备docker-compose.yml文件：

mkdir phi3-deployment && cd phi3-deployment

创建docker-compose.yml文件：

version: '3.8'

services:
  vllm:
    image: vllm/vllm-openai:latest
    runtime: nvidia
    environment:
      - MODEL=Phi-3-mini-4k-instruct-gguf
      - HOST=0.0.0.0
      - PORT=8000
    volumes:
      - ./models:/models
    ports:
      - "8000:8000"
    command: --model /models/Phi-3-mini-4k-instruct-gguf --host 0.0.0.0 --port 8000 --tensor-parallel-size 1
    
  chainlit:
    image: chainlit/chainlit:latest
    depends_on:
      - vllm
      - redis
    environment:
      - OPENAI_API_BASE=http://vllm:8000/v1
      - OPENAI_API_KEY=EMPTY
      - REDIS_URL=redis://redis:6379
    volumes:
      - ./app:/app
    ports:
      - "8001:8000"
    command: run /app/app.py --port 8000 --host 0.0.0.0
    
  redis:
    image: redis:alpine
    ports:
      - "6379:6379"
    volumes:
      - redis_data:/data

volumes:
  redis_data:

创建Chainlit应用文件app/app.py：

import chainlit as cl
from openai import AsyncOpenAI

client = AsyncOpenAI(
    base_url="http://vllm:8000/v1",
    api_key="EMPTY"
)

@cl.on_chat_start
async def on_chat_start():
    await cl.Message(content="Phi-3-mini-4k-instruct模型已准备好，请问您有什么问题？").send()

@cl.on_message
async def on_message(message: cl.Message):
    response = await client.chat.completions.create(
        model="Phi-3-mini-4k-instruct-gguf",
        messages=[
            {"role": "system", "content": "你是一个乐于助人的AI助手"},
            {"role": "user", "content": message.content}
        ],
        temperature=0.7,
    )
    
    await cl.Message(content=response.choices[0].message.content).send()

2. 模型部署与验证

2.1 下载模型并启动服务

下载Phi-3-mini-4k-instruct-gguf模型：

mkdir models
wget -O models/Phi-3-mini-4k-instruct-gguf https://huggingface.co/TheBloke/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-gguf-q4_0.gguf

启动所有服务：

docker-compose up -d

2.2 验证服务状态

检查vLLM服务日志：

docker-compose logs vllm

如果看到类似以下输出，表示模型已成功加载：

INFO 05-20 12:34:56 llm_engine.py:72] Initializing an LLM engine with config: model='/models/Phi-3-mini-4k-instruct-gguf', tokenizer='/models/Phi-3-mini-4k-instruct-gguf', tokenizer_mode=auto, trust_remote_code=False, dtype=auto, tensor_parallel_size=1)
INFO 05-20 12:35:12 llm_engine.py:143] # GPU blocks: 918, # CPU blocks: 512
INFO 05-20 12:35:12 model_runner.py:115] Loading model weights took: 16.23s
INFO 05-20 12:35:12 model_runner.py:119] Model loaded in 16.25s