[CI/CD] - 容器编排平台 - K8s
Kubernetes 是什么?
Kubernetes(常简称为 K8s)是一个开源的容器编排平台,用于自动化部署、扩展和管理容器化应用程序。它源自 Google 的 Borg 系统,是目前容器编排领域的事实标准。
核心架构
1. 控制平面(Control Plane)
text
控制平面组件: ├── API Server:集群前端,处理所有 REST 请求 ├── etcd:分布式键值存储,保存集群状态 ├── Scheduler:调度 Pod 到合适节点 ├── Controller Manager:运行控制器进程 └── Cloud Controller Manager(可选):与云服务商交互
2. 数据平面(Data Plane)
text
工作节点组件: ├── kubelet:节点代理,管理 Pod 生命周期 ├── kube-proxy:网络代理,实现 Service 概念 ├── Container Runtime:容器运行时(Docker、containerd等) └── Pod:最小的部署单元
核心功能
1. 服务发现和负载均衡
-
自动为 Pod 分配 DNS 名称或 IP 地址
-
使用 Service 和 Ingress 暴露应用
2. 存储编排
-
自动挂载存储系统
-
支持本地存储、云存储、网络存储
3. 自动部署和回滚
-
声明式配置管理
-
滚动更新和回滚
4. 自动扩缩容
-
基于 CPU/内存使用率自动扩缩
-
基于自定义指标扩缩
5. 自我修复
-
重启失败容器
-
替换不可用节点
-
杀死不健康容器
6. 密钥和配置管理
-
ConfigMap 和 Secret 管理配置
-
敏感信息加密存储
核心概念
1. Pod
最小的部署单元,包含一个或多个容器
yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx-pod
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.21
ports:
- containerPort: 80
- name: log-sidecar
image: busybox
command: ['sh', '-c', 'tail -f /var/log/nginx/access.log']
2. Deployment
管理 Pod 的副本和更新策略
yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.21
ports:
- containerPort: 80
resources:
requests:
memory: "64Mi"
cpu: "250m"
limits:
memory: "128Mi"
cpu: "500m"
livenessProbe:
httpGet:
path: /
port: 80
initialDelaySeconds: 3
periodSeconds: 3
3. Service
暴露应用服务,提供负载均衡
yaml
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
selector:
app: nginx
ports:
- port: 80
targetPort: 80
nodePort: 30080 # NodePort 类型
type: NodePort # 或 ClusterIP、LoadBalancer
4. ConfigMap 和 Secret
配置和敏感信息管理
yaml
# ConfigMap
apiVersion: v1
kind: ConfigMap
metadata:
name: app-config
data:
app.properties: |
database.host=mysql
database.port=3306
cache.enabled=true
# Secret
apiVersion: v1
kind: Secret
metadata:
name: db-secret
type: Opaque
data:
username: YWRtaW4= # admin
password: cGFzc3dvcmQ= # password
5. Volume 和 PersistentVolume
存储管理
yaml
# PersistentVolumeClaim
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mysql-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
storageClassName: fast
6. Namespace
资源隔离
bash
kubectl create namespace production kubectl create namespace development
7. 其他重要资源
-
StatefulSet:有状态应用
-
DaemonSet:每个节点运行一个 Pod
-
Job/CronJob:批处理任务
-
HorizontalPodAutoscaler:自动扩缩容
-
NetworkPolicy:网络策略
安装 Kubernetes
1. 本地开发环境
bash
# 使用 Minikube(单节点集群) brew install minikube # macOS minikube start --driver=docker --memory=4096 --cpus=2 minikube dashboard # 使用 kind(Kubernetes in Docker) brew install kind kind create cluster --name dev-cluster
2. 生产环境安装
bash
# 使用 kubeadm(官方工具) # 在所有节点上安装 Docker 和 kubeadm sudo apt update sudo apt install -y docker.io sudo systemctl enable docker sudo systemctl start docker # 安装 kubeadm、kubelet、kubectl sudo apt update && sudo apt install -y apt-transport-https curl curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list sudo apt update sudo apt install -y kubelet kubeadm kubectl sudo apt-mark hold kubelet kubeadm kubectl # 初始化控制平面节点 sudo kubeadm init --pod-network-cidr=10.244.0.0/16 # 配置 kubectl mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config # 安装网络插件(Calico) kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
3. 云服务商托管
bash
# AWS EKS eksctl create cluster --name my-cluster --region us-west-2 --nodegroup-name standard-workers --node-type t3.medium --nodes 3 # Google GKE gcloud container clusters create my-cluster --num-nodes=3 --zone us-central1-a # Azure AKS az aks create --resource-group myResourceGroup --name myAKSCluster --node-count 3 --enable-addons monitoring --generate-ssh-keys
基本命令操作
1. 集群管理
bash
# 查看集群信息 kubectl cluster-info kubectl config view kubectl get nodes kubectl describe node <node-name> # 查看资源 kubectl api-resources kubectl explain pod
2. Pod 操作
bash
# 创建 Pod kubectl run nginx --image=nginx:1.21 kubectl apply -f pod.yaml # 查看 Pod kubectl get pods kubectl get pods -o wide kubectl get pods -w # 监视模式 kubectl describe pod <pod-name> # Pod 交互 kubectl logs <pod-name> kubectl logs -f <pod-name> # 实时日志 kubectl exec -it <pod-name> -- /bin/bash kubectl cp <pod-name>:/path/to/file ./local/path # 删除 Pod kubectl delete pod <pod-name> kubectl delete pods --all
3. Deployment 操作
bash
# 创建 Deployment kubectl create deployment nginx --image=nginx:1.21 --replicas=3 kubectl apply -f deployment.yaml # 查看 Deployment kubectl get deployments kubectl describe deployment <deployment-name> kubectl get replicasets # 扩缩容 kubectl scale deployment nginx --replicas=5 kubectl autoscale deployment nginx --min=2 --max=10 --cpu-percent=80 # 更新 kubectl set image deployment/nginx nginx=nginx:1.22 kubectl rollout status deployment nginx kubectl rollout history deployment nginx # 回滚 kubectl rollout undo deployment nginx kubectl rollout undo deployment nginx --to-revision=2 # 删除 kubectl delete deployment nginx
4. Service 操作
bash
# 创建 Service kubectl expose deployment nginx --port=80 --target-port=80 --type=NodePort kubectl apply -f service.yaml # 查看 Service kubectl get services kubectl describe service <service-name> # 端口转发(本地访问) kubectl port-forward service/nginx 8080:80 kubectl port-forward pod/<pod-name> 8080:80 # 删除 Service kubectl delete service nginx
5. 配置管理
bash
# ConfigMap 和 Secret
kubectl create configmap app-config --from-file=config.properties
kubectl create secret generic db-secret --from-literal=username=admin --from-literal=password=secret
# 查看
kubectl get configmaps
kubectl get secrets
kubectl describe configmap app-config
kubectl get secret db-secret -o jsonpath='{.data.password}' | base64 --decode
# 使用配置
kubectl create deployment app --image=myapp --dry-run=client -o yaml > deployment.yaml
# 然后在 YAML 中添加 envFrom 或 volumeMounts
6. 命名空间操作
bash
# 创建命名空间 kubectl create namespace production kubectl create namespace development # 在指定命名空间操作 kubectl get pods -n production kubectl apply -f deployment.yaml -n production # 切换当前命名空间 kubectl config set-context --current --namespace=production
7. 调试和故障排查
bash
# 查看事件 kubectl get events --sort-by='.lastTimestamp' kubectl get events -w # 查看资源使用情况 kubectl top nodes kubectl top pods # 诊断工具 kubectl describe <resource> <name> kubectl logs --previous <pod-name> # 查看前一个容器的日志 # 网络调试 kubectl run busybox --image=busybox --restart=Never -- sleep 3600 kubectl exec busybox -- nslookup kubernetes.default kubectl exec busybox -- wget -qO- http://nginx-service # 资源清理 kubectl delete all --all # 删除所有资源(当前命名空间) kubectl delete all --all --all-namespaces
YAML 配置文件详解
1. 完整的 Deployment 示例
yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-app
namespace: production
labels:
app: web
version: v1.0
annotations:
deployment.kubernetes.io/revision: "1"
spec:
replicas: 3
revisionHistoryLimit: 10
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
selector:
matchLabels:
app: web
tier: frontend
template:
metadata:
labels:
app: web
tier: frontend
version: v1.0
spec:
serviceAccountName: web-sa
terminationGracePeriodSeconds: 30
containers:
- name: web
image: myregistry/web-app:v1.0
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 8080
protocol: TCP
env:
- name: DATABASE_URL
valueFrom:
configMapKeyRef:
name: app-config
key: database.url
- name: API_KEY
valueFrom:
secretKeyRef:
name: app-secrets
key: api-key
envFrom:
- configMapRef:
name: app-config
resources:
requests:
memory: "128Mi"
cpu: "250m"
limits:
memory: "256Mi"
cpu: "500m"
livenessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
httpGet:
path: /ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
timeoutSeconds: 2
volumeMounts:
- name: config-volume
mountPath: /etc/config
readOnly: true
- name: data-volume
mountPath: /var/data
volumes:
- name: config-volume
configMap:
name: app-config
- name: data-volume
persistentVolumeClaim:
claimName: web-pvc
nodeSelector:
disktype: ssd
tolerations:
- key: "special"
operator: "Equal"
value: "true"
effect: "NoSchedule"
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/os
operator: In
values:
- linux
2. 完整的 Service 和 Ingress 示例
yaml
# Service
apiVersion: v1
kind: Service
metadata:
name: web-service
namespace: production
spec:
selector:
app: web
tier: frontend
ports:
- name: http
port: 80
targetPort: 8080
protocol: TCP
- name: https
port: 443
targetPort: 8443
protocol: TCP
type: ClusterIP # 也可以是 NodePort、LoadBalancer
# Ingress (需要 Ingress Controller)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: web-ingress
namespace: production
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/ssl-redirect: "true"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
tls:
- hosts:
- app.example.com
secretName: app-tls
rules:
- host: app.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: web-service
port:
number: 80
3. StatefulSet 示例(有状态应用)
yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mysql
spec:
serviceName: mysql
replicas: 3
selector:
matchLabels:
app: mysql
template:
metadata:
labels:
app: mysql
spec:
containers:
- name: mysql
image: mysql:8.0
env:
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: mysql-secret
key: root-password
ports:
- containerPort: 3306
name: mysql
volumeMounts:
- name: mysql-data
mountPath: /var/lib/mysql
volumeClaimTemplates:
- metadata:
name: mysql-data
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 10Gi
实际应用场景
1. 微服务部署
bash
# 目录结构 microservices/ ├── frontend/ │ ├── deployment.yaml │ └── service.yaml ├── api-service/ │ ├── deployment.yaml │ └── service.yaml ├── user-service/ │ ├── deployment.yaml │ └── service.yaml ├── product-service/ │ ├── deployment.yaml │ └── service.yaml └── kustomization.yaml # 使用 Kustomize 管理 # 使用 Kustomize 部署 kubectl apply -k .
2. CI/CD 集成
yaml
# .gitlab-ci.yml 示例
stages:
- build
- test
- deploy
variables:
K8S_NAMESPACE: production
DOCKER_TAG: $CI_COMMIT_SHORT_SHA
build:
stage: build
script:
- docker build -t $CI_REGISTRY_IMAGE:$DOCKER_TAG .
- docker push $CI_REGISTRY_IMAGE:$DOCKER_TAG
deploy:
stage: deploy
script:
- kubectl config set-context --current --namespace=$K8S_NAMESPACE
- kubectl set image deployment/web-app web-app=$CI_REGISTRY_IMAGE:$DOCKER_TAG
- kubectl rollout status deployment/web-app
3. 蓝绿部署
bash
# 创建 v1 版本
kubectl apply -f deployment-v1.yaml
# 创建 v2 版本(不同标签)
kubectl apply -f deployment-v2.yaml
# 通过 Service 切换流量
kubectl patch service web-service -p '{"spec":{"selector":{"version":"v2"}}}'
# 回滚到 v1
kubectl patch service web-service -p '{"spec":{"selector":{"version":"v1"}}}'
生态系统和工具
1. 监控和日志
bash
# Prometheus + Grafana helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm install prometheus prometheus-community/prometheus helm install grafana grafana/grafana # ELK/EFK 日志栈 kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/1.7/config/samples/elasticsearch.yaml kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/1.7/config/samples/kibana.yaml kubectl apply -f https://raw.githubusercontent.com/fluent/fluentd-kubernetes-daemonset/master/fluentd-daemonset-elasticsearch.yaml
2. 服务网格
bash
# Istio 安装 curl -L https://istio.io/downloadIstio | sh - cd istio-* ./bin/istioctl install --set profile=demo kubectl label namespace default istio-injection=enabled
3. 包管理工具
bash
# Helm 包管理器 helm repo add bitnami https://charts.bitnami.com/bitnami helm install my-release bitnami/nginx # 创建自定义 Chart helm create mychart helm install my-app ./mychart helm upgrade my-app ./mychart
最佳实践
1. 资源管理
yaml
# 总是设置资源限制
resources:
requests:
memory: "256Mi"
cpu: "250m"
limits:
memory: "512Mi"
cpu: "500m"
# 使用 LimitRange 和 ResourceQuota
apiVersion: v1
kind: LimitRange
metadata:
name: mem-limit-range
spec:
limits:
- default:
memory: 512Mi
defaultRequest:
memory: 256Mi
type: Container
2. 安全性
bash
# 使用非 root 用户运行容器
securityContext:
runAsNonRoot: true
runAsUser: 1000
# 限制权限
securityContext:
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
# 使用 NetworkPolicy
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
3. 配置管理
bash
# 使用 ConfigMap 和 Secret,而不是硬编码
# 使用环境变量或配置文件注入配置
# 敏感信息使用 Secret,并启用加密
# 启用 Secret 加密
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
- secrets
providers:
- aescbc:
keys:
- name: key1
secret: <base64-encoded-secret>
4. 监控和告警
bash
# 使用 Prometheus Operator
kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/main/bundle.yaml
# 配置 ServiceMonitor
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: web-app-monitor
spec:
selector:
matchLabels:
app: web-app
endpoints:
- port: web
interval: 30s
path: /metrics
5. 备份和恢复
bash
# 使用 Velero 备份 velero install \ --provider aws \ --bucket my-backup-bucket \ --secret-file ./credentials \ --use-volume-snapshots=false # 创建备份 velero backup create my-backup --include-namespaces production # 恢复 velero restore create --from-backup my-backup
常见问题解决
1. Pod 无法启动
bash
# 查看详细状态 kubectl describe pod <pod-name> # 查看事件 kubectl get events --field-selector involvedObject.name=<pod-name> # 查看容器日志 kubectl logs <pod-name> --previous # 常见原因: # - 镜像拉取失败 # - 资源不足 # - 配置错误 # - 健康检查失败
2. 节点问题
bash
# 查看节点状态 kubectl get nodes kubectl describe node <node-name> # 节点排空(维护前) kubectl drain <node-name> --ignore-daemonsets --delete-emptydir-data # 节点恢复 kubectl uncordon <node-name> # 常见问题: # - 磁盘空间不足 # - 内存压力 # - 网络问题
3. 网络问题
bash
# 测试服务连通性 kubectl run test --image=busybox --rm -it --restart=Never -- wget -qO- http://service-name.namespace.svc.cluster.local # 查看网络策略 kubectl get networkpolicies kubectl describe networkpolicy <policy-name> # 检查 DNS kubectl run dns-test --image=busybox --rm -it --restart=Never -- nslookup kubernetes.default
4. 资源清理
bash
# 清理已停止的容器 kubectl delete pods --field-selector=status.phase=Succeeded # 清理未使用的资源 kubectl delete deployments,services,configmaps,secrets --all # 清理命名空间中的所有资源 kubectl delete all --all -n <namespace>
总结
Kubernetes 是一个强大但复杂的系统,学习曲线相对陡峭。建议的学习路径:
-
基础入门:理解核心概念(Pod、Deployment、Service)
-
实际操作:在本地环境(Minikube/kind)练习
-
深入理解:学习存储、网络、安全等高级主题
-
生产实践:学习监控、日志、CI/CD 集成
-
生态系统:探索 Helm、Istio、Operator 等工具
Kubernetes 已经成为云原生应用的事实标准,掌握它将极大提升你在现代软件开发领域的竞争力。
帮助文档和学习资源
官方文档
-
Kubernetes 官方文档:https://kubernetes.io/docs/home/
-
Kubernetes API 参考:https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/
-
Kubernetes 官方博客:https://kubernetes.io/blog/
中文资源
-
Kubernetes 中文社区:https://www.kubernetes.org.cn/
-
Kubernetes 中文文档:https://kubernetes.io/zh/docs/home/
-
Kubernetes 教程 - 易文档:https://k8s.easydoc.net/
学习平台
-
Kubernetes 官方教程:https://kubernetes.io/docs/tutorials/
-
Katacoda Kubernetes 课程:https://www.katacoda.com/courses/kubernetes
-
Kubernetes 练习场:https://killercoda.com/killer-shell-cka
书籍推荐
-
《Kubernetes in Action》 - Marko Luksa
-
《Kubernetes 权威指南》 - 龚正等
-
《Kubernetes 进阶实战》 - 马永亮
-
《云原生模式》 - Cornelia Davis
认证
-
CKA(Certified Kubernetes Administrator):https://www.cncf.io/certification/cka/
-
CKAD(Certified Kubernetes Application Developer):https://www.cncf.io/certification/ckad/
-
CKS(Certified Kubernetes Security Specialist):https://www.cncf.io/certification/cks/
社区和支持
-
GitHub 仓库:https://github.com/kubernetes/kubernetes
-
Stack Overflow:https://stackoverflow.com/questions/tagged/kubernetes
-
Kubernetes Slack:https://slack.k8s.io/
-
中文社区论坛:https://bbs.k8s.cn/
视频教程
-
Kubernetes 官方 YouTube:https://www.youtube.com/c/KubernetesCommunity
-
Kubernetes 完整课程 - freeCodeCamp:https://www.youtube.com/watch?v=X48VuDVv0do
-
Kubernetes 教程 - 尚硅谷:https://www.bilibili.com/video/BV1w4411y7Go
更多推荐

所有评论(0)