环境
  • Rancher  牧场主
  • RKE2/K3S provisioned by Rancher in an infrastructure provider (node driver)
    RKE2/K3S 由 Rancher 在基础设施提供商(节点驱动)中配置
访问Rancher-K8S解决方案博主 :


https://blog.csdn.net/lidw2009
情况

A downstream Kubernetes cluster (RKE2 or K3S) was provisioned through Rancher using any infrastructure provider / node driver. The environment requires the cluster nodes to be migrated to a new network or subnet(s).
通过 Rancher,使用任意基础设施提供商/节点驱动,配置了一个下游的 Kubernetes 集群(RKE2 或 K3S)。环境要求集群节点迁移到新的网络或子网。

Some questions may exist, such as:
可能存在一些问题,例如:

  • Can the existing cluster be edited to swap to the new network/subnet?
    现有集群可以被编辑到新的网络/子网吗?
  • Can nodes be moved to the new network?
    节点可以迁移到新网络吗?
  • Should the cluster be deleted and recreated?
    集群应该被删除并重新创建吗?
解决方案

The recommended and safest approach for RKE2/K3s clusters is a node-by-node migration using new machine pools. It is assumed that nodes can communicate between the old and new subnets, at least during the migration period.
RKE2/K3s 集群推荐且最安全的方法是使用新的机器池逐节点迁移 。假设节点可以在迁移期间,至少在新旧子网之间通信。

1. Prepare the New Subnet
1. 准备新的子网

Before making any changes:
在做任何更改之前:

  • Ensure all firewall rules between the new subnet and all cluster components are allowed (etcd ports, kube-apiserver, kubelet, VXLAN/Canal/Cilium ports, etc.)
    确保新子网与所有集群组件之间的防火墙规则 (etcd 端口、kube-apiserver、kubelet、VXLAN/Canal/Cilium 端口等)都被允许。
  • Confirm port/security groups and DHCP/static IP assignments are correctly configured
    确认端口/安全组和 DHCP/静态 IP 分配配置正确

Take full backups:
全面备份 

  • RKE2/K3s etcd snapshot  RKE2/K3s etcd 快照
  • VM snapshots, if allowed in your environment
    虚拟机快照 ,如果你的环境允许的话
  • Rancher’s cluster config (for reference)
    Rancher 的集群配置(供参考)

2. Create a New Machine Pool using the new Network
2. 使用新网络创建新的机器池

In Rancher:  在牧场主中:

  • Edit the downstream cluster
    编辑下游集群
  • Add a new machine pool
    新增一个机池
  • Assign control-plane/etcd/worker roles as required to mirror an existing machine pool
    根据需要分配控制平面/etcd/工作者角色,以镜像现有的机器池
  • Configure the new subnet, new port group, VLAN, or network settings
    配置新的子网、新的端口组、VLAN 或网络设置
  • Save and allow Rancher to provision new nodes in the new network
    保存并允许 Rancher 在新网络中配置新节点
  • It is strongly recommended to bring up nodes in small batches (even one at a time) and verify:
    强烈建议以小批量 (即使一次只一个)调出节点并验证:
    • Node registration  节点注册
    • Kubelet health  库贝莱特健康
    • Pod networking / overlay network connectivity
      Pod 网络/叠加网络连接
    • Ability to join etcd quorum (for CP/etcd nodes)
      加入 etcd 法定人数(针对 CP/etcd 节点)

Note, this is also a good time to make other node-specific changes in the new machine pool(s) if desired
注意 ,这也是对新机器池进行其他节点特定修改的好时机

3. Validate the New Nodes
3. 验证新节点

Once the new nodes come up
一旦新节点出现

  • Ensure pods are scheduled successfully on worker nodes
    确保 pod 在工作节点上成功调度
  • Verify control-plane logs
    验证控制平面日志
  • Check etcd health  检查 etcd 健康

4. Drain and Remove Old Nodes
4. 排空并移除旧节点

After confirming stability:
确认稳定后:

  • Drain the old node:
    排空旧节点:
  • Remove the node from Cluster Management in the Rancher dashboard
    在 Rancher 仪表盘中移除该节点
  • Rancher deletes the node from the provider
    Rancher 从提供者中删除了该节点
  • Repeat this process node-by-node, always maintaining:
    逐节点重复此过程,始终保持:
    • Etcd quorum  ETCHD 法定人数
    • Control-plane availability
      控制平面可用性

5. Decommission the Old Machine Pool
5. 退役旧机池

  • After all old nodes are safely removed:
    所有旧节点安全移除后:
    • Delete the old machine pool from Rancher
      删除 Rancher 里的旧机器池
    • Clean up any unused objects, IP reservations, or port/security group references
      清理所有未使用的对象、IP 保留或端口/安全组引用

6. Validate Cluster Stability
6. 验证簇稳定性

When the migration is complete:
迁移完成后:

  • Confirm all nodes are in the new subnet
    确认所有节点都在新子网中
  • Validate workloads, ingress, CNI, and storage
    验证工作负载、入口、CNI 和存储
  • Check etcd health and control-plane responsiveness
    检查 etcd 的健康状态和控制平面响应性

This completes the migration without needing to recreate the cluster.
这样迁移就完成了,无需重新创建集群。

原因

Clusters provisioned through the node driver embed network configuration (port/security groups, IP ranges, cloud-init settings, etc.) inside the machine and pool definitions. Existing nodes cannot be reassigned to a new subnet by editing the cluster config alone. Additionally, Kubernetes nodes rely on stable IPs and overlay networking behaviour, which can break if the underlying network is changed in-place.
通过节点驱动配置的集群会将网络配置(端口/安全组、IP 范围、云初始化设置等)嵌入机器和池定义中。仅通过编辑集群配置,无法将现有节点重新分配到新的子网。此外,Kubernetes 节点依赖稳定的 IP 和覆盖网络行为,如果底层网络被原地改变,这些行为可能会崩溃。

To safely migrate to a new subnet without downtime or loss of etcd quorum, new nodes must be introduced into the cluster using the new network configuration.
为了安全迁移到新的子网,且不会停机或 etcd 法定人数损失,必须使用新的网络配置引入新节点。

附加信息
Logo

汇聚全球AI编程工具,助力开发者即刻编程。

更多推荐