使用 operator 部署 VictoriaMetrics

mnasd

257人浏览 · 2026-06-03 21:16:07

mnasd · 2026-06-03 21:16:07 发布

VictoriaMetrics 架构概览

vmstorage 负责存储数据，是有状态组件。
vmselect 负责查询数据，Grafana 添加 Prometheus 数据源时使用 vmselect 地址，查询数据时，vmselect 会调用各个 vmstorage 的接口完成数据的查询。
vminsert 负责写入数据，采集器将采集到的数据 "吐到" vminsert，然后 vminsert 会调用各个 vmstorage 的接口完成数据的写入。
各个组件都可以水平伸缩，但不支持自动伸缩，因为伸缩需要修改启动参数。
安装 operator

使用 helm 安装:

helm repo add vm https://victoriametrics.github.io/helm-charts
helm repo update
helm install victoria-operator vm/victoria-metrics-operator



image:
  # -- Image registry
  registry: ""
  # -- Image repository
  repository: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/victoriametrics/operator  #修改镜像仓库
  # -- Image tag
  # override Chart.AppVersion
  tag: "v0.53.0"  #修改镜像tag

检查 operator 是否成功启动:

$ kubectl -n monitoring get pod
NAME                                                           READY   STATUS    RESTARTS   AGE
victoria-operator-victoria-metrics-operator-7b886f85bb-jf6ng   1/1     Running   0          20s

安装 VMSorage, VMSelect 与 VMInsert

准备 vmcluster.yaml:

apiVersion: operator.victoriametrics.com/v1beta1
kind: VMCluster
metadata:
  name: demo
  namespace: vmtrics-monitor
spec:
  retentionPeriod: "1"
  replicationFactor: 2
  vmstorage:
    image:
      repository: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/victoriametrics/vmstorage
      tag: v1.110.0-cluster
      pullPolicy: Always
    replicaCount: 1
    storageDataPath: "/vm-data"
    storage:
      volumeClaimTemplate:
        spec:
          accessModes:
            - ReadWriteOnce
          storageClassName: nfs-sc
          resources:
            requests:
              storage: "1Gi"
    resources:
      limits:
        cpu: "1"
        memory: "1Gi"
  vmselect:
    image:
      repository: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/victoriametrics/vmselect
      tag: v1.110.0-cluster
      pullPolicy: Always
    replicaCount: 1
    cacheMountPath: "/select-cache"
    storage:
      volumeClaimTemplate:
        spec:
          accessModes:
            - ReadWriteOnce
          storageClassName: nfs-sc
          resources:
            requests:
              storage: "1Gi"
    resources:
      limits:
        cpu: "1"
        memory: "1Gi"
      requests:
        cpu: "0.5"
        memory: "500Mi"
  vminsert:
    image:
      repository: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/victoriametrics/vminsert
      tag: v1.110.0-cluster
      pullPolicy: Always
    replicaCount: 1
    resources:
      limits:
        cpu: "1"
        memory: "1Gi"
      requests:
        cpu: "0.5"
        memory: "500Mi"

安装:

$ kubectl apply -f vmcluster.yaml
vmcluster.operator.victoriametrics.com/vmcluster created

安装 VMAlertmanager 与 VMAlert

准备 vmalertmanager.yaml:

apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAlertmanager
metadata:
  name: vmalertmanager
  namespace: monitoring
spec:
  replicaCount: 1
  selectAllByDefault: true

安装 VMAlertmanager:

$ kubectl apply -f vmalertmanager.yaml
vmalertmanager.operator.victoriametrics.com/vmalertmanager created

准备 vmalert.yaml:

apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAlert
metadata:
  name: vmalert
  namespace: monitoring
spec:
  replicaCount: 1
  selectAllByDefault: true
  notifier:
    url: http://vmalertmanager-vmalertmanager:9093
  resources:
    requests:
      cpu: 10m
      memory: 10Mi
  remoteWrite:
    url: http://vminsert-vmcluster:8480/insert/0/prometheus/
  remoteRead:
    url: http://vmselect-vmcluster:8481/select/0/prometheus/
  datasource:
    url: http://vmselect-vmcluster:8481/select/0/prometheus/

安装 VMAlert:

$ kubectl apply -f vmalert.yaml
vmalert.operator.victoriametrics.com/vmalert created

安装 VMAgent

vmagent 用于采集监控数据并发送给 VictoriaMetrics 进行存储，对于腾讯云容器服务上的容器监控数据采集，需要用自定义的 additionalScrapeConfigs 配置，准备自定义采集规则配置文件 scrape-config.yaml:

apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: additional-scrape-configs
  namespace: monitoring
stringData:
  additional-scrape-configs.yaml: |-
    - job_name: "tke-cadvisor"
      scheme: https
      metrics_path: /metrics/cadvisor
      tls_config:
        insecure_skip_verify: true
      authorization:
        credentials_file: /var/run/secrets/kubernetes.io/serviceaccount/token
      kubernetes_sd_configs:
      - role: node
      relabel_configs:
      - source_labels: [__meta_kubernetes_node_label_node_kubernetes_io_instance_type]
        regex: eklet
        action: drop
      - action: labelmap
        regex: __meta_kubernetes_node_label_(.+)
    - job_name: "tke-kubelet"
      scheme: https
      metrics_path: /metrics
      tls_config:
        insecure_skip_verify: true
      authorization:
        credentials_file: /var/run/secrets/kubernetes.io/serviceaccount/token
      kubernetes_sd_configs:
      - role: node
      relabel_configs:
      - source_labels: [__meta_kubernetes_node_label_node_kubernetes_io_instance_type]
        regex: eklet
        action: drop
      - action: labelmap
        regex: __meta_kubernetes_node_label_(.+)
    - job_name: "tke-probes"
      scheme: https
      metrics_path: /metrics/probes
      tls_config:
        insecure_skip_verify: true
      authorization:
        credentials_file: /var/run/secrets/kubernetes.io/serviceaccount/token
      kubernetes_sd_configs:
      - role: node
      relabel_configs:
      - source_labels: [__meta_kubernetes_node_label_node_kubernetes_io_instance_type]
        regex: eklet
        action: drop
      - action: labelmap
        regex: __meta_kubernetes_node_label_(.+)
    - job_name: eks
      honor_timestamps: true
      metrics_path: '/metrics'
      params:
        collect[]: ['ipvs']
        # - 'cpu'
        # - 'meminfo'
        # - 'diskstats'
        # - 'filesystem'
        # - 'load0vg'
        # - 'netdev'
        # - 'filefd'
        # - 'pressure'
        # - 'vmstat'
      scheme: http
      kubernetes_sd_configs:
      - role: pod
      relabel_configs:
      - source_labels: [__meta_kubernetes_pod_annotation_tke_cloud_tencent_com_pod_type]
        regex: eklet
        action: keep
      - source_labels: [__meta_kubernetes_pod_phase]
        regex: Running
        action: keep
      - source_labels: [__meta_kubernetes_pod_ip]
        separator: ;
        regex: (.*)
        target_label: __address__
        replacement: ${1}:9100
        action: replace
      - source_labels: [__meta_kubernetes_pod_name]
        separator: ;
        regex: (.*)
        target_label: pod
        replacement: ${1}
        action: replace
      - source_labels: [__meta_kubernetes_namespace]
        separator: ;
        regex: (.*)
        target_label: namespace
        replacement: ${1}
        action: replace
      metric_relabel_configs:
      - source_labels: [__name__]
        separator: ;
        regex: (container_.*|pod_.*|kubelet_.*)
        replacement: $1
        action: keep

再准备 vmagent.yaml:

apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAgent
metadata:
  name: vmagent
  namespace: monitoring
spec:
  image:
    repository: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/victoriametrics/vmagent
    tag: v1.110.0
  selectAllByDefault: true
  additionalScrapeConfigs:
    key: additional-scrape-configs.yaml
    name: additional-scrape-configs
  resources:
    requests:
      cpu: 10m
      memory: 10Mi
  replicaCount: 1
  remoteWrite:
  - url: "http://vminsert-vmcluster:8480/insert/0/prometheus/api/v1/write"

安装:

$ kubectl apply -f scrape-config.yaml
secret/additional-scrape-configs created
$ kubectl apply -f vmagent.yaml
vmagent.operator.victoriametrics.com/vmagent created

VMAuth

创建vmauth.yaml

apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAuth
metadata:
  name: demo
  namespace: vmtrics-monitor
  labels:
    app: vmauth-demo
spec:
  image:
    repository: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/victoriametrics/vmauth
    tag: v1.108.1
    pullPolicy: Always
  selectAllByDefault: true
  userNamespaceSelector: {}
  userSelector: {}

创建vmuser.yaml

apiVersion: operator.victoriametrics.com/v1beta1
kind: VMUser
metadata:
  name: demo
  namespace: vmtrics-monitor
spec:
  name: demo
  username: demo
  generatePassword: true
  targetRefs:
    # vmui + vmselect
    - crd:
        kind: VMCluster/vmselect
        name: demo
        namespace: vmtrics-monitor
      target_path_suffix: "/select/0"
      paths:
        - "/vmui"
        - "/vmui/.*"
        - "/prometheus/api/v1/query"
        - "/prometheus/api/v1/query_range"
        - "/prometheus/api/v1/series"
        - "/prometheus/api/v1/status/.*"
        - "/prometheus/api/v1/label/"
        - "/prometheus/api/v1/label/[^/]+/values"

kubectl get secret -n vm vmuser-demo -o jsonpath="{.data.password}" | base64 --decode

参考

https://imroc.cc/kubernetes/monitoring/victoriametrics/install-with-operator

https://docs.victoriametrics.com/operator/resources/vmalert/
https://docs.victoriametrics.com/operator/quick-start/
https://docs.victoriametrics.com/operator/resources/vmcluster/index.html