2. 首页 > Linux教程 > 正文

Linux教程FG565-大规模K8s集群自动化扩缩容

教程整理：风哥教程  |  更新时间：2026-01-25  |  教程分类：Linux教程  |  文档学习：28

Part01-基础概念与理论知识

1.1 自动扩缩容基本概念

自动扩缩容是Kubernetes的核心功能之一，它可以根据集群的负学习交流加群风哥微信: itpux-com载情况自动调整资源，确保应用的可用性和性能，同时优化资源利用率。在大规模集群中，自动扩缩容尤为重要，可以有效应对流量波动和资源需求变化。

1.2 自动扩缩容类型

Kubernetes提供了三种主要的自动扩缩容机制：

  • HPA（Horizontal Pod Autoscaler）：水平Pod自动扩缩容，根据CPU、内存使用率或自定义指标自动调整Pod数量
  • VPA（Vertical Pod Autoscaler）：垂直Pod自动扩缩容，根据Pod的资源使用情况自动调整CPU和内存的请求和限制
  • CA（Cluster Autoscaler）：集群自动扩缩容，根据集群的资源需求自动调整节点数量

1.3 自动扩缩容原理

自动扩缩容的工作原理如下：

  • HPA：通过监控Pod的资源使用情况，与目标利用率进行比较，计算出需要的Pod数量，然后调整Deployment或ReplicaSet的副本数
  • VPA：通过分析Pod的历史资源使用情况，推荐合适的资源请求和限制，然后更新Pod的配置
  • CA：通过监控集群中未调度的Pod和节点的资源使用情况，决定是否需要添加或删除节点

from PG视频:www.itpux.com

Part02-生产环境规划与建议

2.1 资源需求评估

在实施自动扩缩容前，需要评估应用的资源需求：

  • 峰值流量：分析应用的流量模式，确定峰值时期的资源需求
  • 资源使用模式：监控应用的CPU、内存使用情况，了解资源使用的波动规律
  • 响应时间要求：根据业务需求，确定应用的响应时间目标
  • 容错能力：评估应用的容错能力，确定最小副本数

风哥提示：资源需求评估是自动扩缩容的基础，需要结合历史数据和业务预测进行分析。

2.2 扩缩容策略制定

制定合理的扩缩容策略：

  • 扩缩容触发条件：设置合适的CPU、内存利用率阈值，避免频繁扩缩容
  • 扩缩容速率：控制扩缩容的速率，避免资源抖动
  • 最小/最大副本数：根据应用需求设置合理的副本数范围
  • 冷却时间：设置扩缩容的冷却时间，避免短时间内频繁调整

2.3 监控与告警配置

配置完善的监控与告警系统：

  • 资源监控：监控Pod和节点的CPU、内存使用情况
  • 扩缩容事件监控：监控HPA、VPA、CA的扩缩容事件
  • 告警配置：设置扩缩容相关的告警，及时发现异常情况
  • 可视化 dashboard：使用Grafana等工具创建扩缩容监控dashboard

from Linux:www.itpux.com

Part03-生产环境项目实施方案

3.1 HPA（水平Pod自动扩缩容）配置

配置HPA：

# 创建一个Deployment
$ cat > nginx-deployment.yaml << EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        resources:
          requests:
            cpu: "100m"
            memory: "256Mi"
          limits:
            cpu: "500m"
            memory: "512Mi"
        ports:
        - containerPort: 80
EOF

$ kubectl apply -f nginx-deployment.yaml

# 创建HPA
$ kubectl autoscale deployment nginx --cpu-percent=50 --min=3 --max=10

# 查看HPA状态
$ kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   0%/50%    3         10        3          1m

# 使用自定义指标的HPA
$ cat > custom-metrics-hpa.yaml << EOF
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: nginx-custom
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: nginx
  minReplicas: 3
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 70
EOF

$ kubectl apply -f custom-metrics-hpa.yaml

执行结果：

# 查看HPA状态
$ kubectl get hpa
NAME           REFERENCE          TARGETS                        MINPODS   MAXPODS   REPLICAS   AGE
nginx          Deployment/nginx   0%/50%                        3         10        3          5m
nginx-custom   Deployment/nginx   0%/50%, 0%/70%               3         10        3          1m

# 模拟负载测试
$ kubectl run -i --tty load-generator --image=busybox /bin/sh
$ while true; do wget -q -O- http://nginx; done

# 查看HPA自动扩缩容
$ kubectl get hpa nginx -w
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   0%/50%    3         10        3          10m
nginx   Deployment/nginx   80%/50%   3         10        3          11m
nginx   Deployment/nginx   85%/50%   3         10        6          12m
nginx   Deployment/nginx   45%/50%   3         10        6          13m

3.2 VPA（垂直Pod自动扩缩容）配置

配置VPA：

# 安装VPA
$ kubectl apply -f https://github.com/kubernetes/autoscaler/raw/master/vertical-pod-autoscaler/deploy/vpa更多学习教程公众号风哥教程itpux_com-release.yaml

# 创建VPA配置
$ cat > nginx-vpa.yaml << EOF
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: nginx-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: nginx
  updatePolicy:
    updateMode: "Auto"
  resourcePolicy:
    containerPolicies:
    - containerName: "nginx"
      minAllowed:
        cpu: 100m
        memory: 256Mi
      maxAllowed:
        cpu: 1
        memory: 1Gi
      controlledResources: ["cpu", "memory"]
EOF

$ kubectl apply -f nginx-vpa.yaml

# 查看VPA状态
$ kubectl get vpa
NAME        MODE   CPU   MEM         PROVIDED   AGE
nginx-vpa   Auto   100m  256Mi       True       5m

# 查看VPA推荐
$ kubectl describe vpa nginx-vpa

执行结果：

# 查看VPA状态
$ kubectl get vpa
NAME        MODE   CPU   MEM         PROVIDED   AGE
nginx-vpa   Auto   100m  256Mi       True       10m

# 查看VPA推荐
$ kubectl describe vpa nginx-vpa
Name:         nginx-vpa
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  autoscaling.k8s.io/v1
Kind:         VerticalPodAutoscaler
Metadata:
  Creation Timestamp:  2026-04-03T03:00:00Z
  Generation:          1
  Resource Version:    123456
  UID:                 abcdef12-3456-7890-abcd-ef1234567890
Spec:
  Resource Policy:
    Container Policies:
      Container Name:  nginx
      Controlled Resources:
        cpu
        memory
      Max Allowed:
        Cpu:     1
        Memory:  1Gi
      Min Allowed:
        Cpu:     100m
        Memory:  256Mi
  Target Ref:
    API Version:  apps/v1
    Kind:         Deployment
    Name:         nginx
  Update Policy:
    Update Mode:  Auto
Status:
  Conditions:
    Last Transition Time:  2026-04-03T03:05:00Z
    Status:                True
    Type:                  RecommendationProvided
  Recommendation:
    Container Recommendations:
      Container Name:  nginx
      Lower Bound:
        Cpu:     100m
        Memory:  256Mi
      Target:
        Cpu:     300m
        Memory:  512Mi
      Upper Bound:
        Cpu:     1
        Memory:  1Gi

3.3 CA（集群自动扩缩容）配置

配置CA：

# 安装Cluster Autoscaler
$ helm repo add autoscaler https://kubernetes.github.io/autoscaler
$ helm install cluster-autoscaler autoscaler/cluster-autoscaler --namespace kube-system --set autoDiscovery.clusterName=fgedu-cluster --set replicaCount=2

# 查看Cluster Autoscaler状态
$ kubectl get pods -n kube-system | grep cluster-autoscaler
cluster-autoscaler-56789   1/1     Running   0          5m

# 配置节点池标签
$ kubectl label nodes node1 node2 node3 node-pool=default

# 查看Cluster Autoscaler日志
$ kubectl logs cluster-autoscaler-56789 -n kube-system

执行结果：

# 查看Cluster Autoscaler状态
$ kubectl get pods -n kube-system | grep cluster-autoscaler
cluster-autoscaler-56789   1/1     Running   0          10m

# 查看Cluster Autoscaler日志
$ kubectl logs cluster-autoscaler-56789 -n kube-system
I0403 03:10:00.000000       1 autoscaler.go:123] Starting cluster autoscaler
I0403 03:10:00.000000       1 autoscaler.go:124] Cluster name: fgedu-cluster
I0403 03:10:00.000000       1 autoscaler.go:125] Namespace: kube-system
I0403 03:10:00.000000       1 autoscaler.go:126] Provider: aws
I0403 03:10:00.000000       1 autoscaler.go:127] Cluster autoscaler version: v1.28.0
I0403 03:10:00.000000       1 autoscaler.go:128] Node groups: default
I0403 03:10:00.000000       1 autoscaler.go:129] Auto-discovery: enabled
I0403 03:10:00.000000       1 autoscaler.go:130] Expander: random
I0403 03:10:00.000000       1 autoscaler.go:131] Scale down delay: 10m0s
I0403 03:10:00.000000       1 autoscaler.go:132] Scale up delay: 0s
I0403 03:10:00.000000       1 autoscaler.go:133] Max nodes total: 10
I0403 03:10:00.000000       1 autoscaler.go:134] Min nodes total: 3
I0403 03:10:00.000000       1 autoscaler.go:135] Max node group size: 5
I0403 03:10:00.000000       1 autoscaler.go:136] Min node group size: 1
I0403 03:10:00.000000       1 autoscaler.go:137] Node group auto-discovery: enabled

Part04-生产案例与实战讲解

4.1 Web应用自动扩缩容案例

Web应用的自动扩缩容配置：

# 创建Web应用Deployment
$ cat > webapp-deployment.yaml << EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: webapp
spec:
  replicas: 5
  selector:
    matchLabels:
      app: webapp
  template:
    metadata:
      labels:
        app: webapp
    spec:
      containers:
      - name: webapp
        image: harbor.fgedu.net.cn/library/webapp:v1.0.0
        resources:
          requests:
            cpu: "200m"
            memory: "512Mi"
          limits:
            cpu: "1"
            memory: "1Gi"
        ports:
        - containerPort: 8080
EOF

$ kubectl apply -f webapp-deployment.yaml

# 创建Web应用Service
$ cat > webapp-service.yaml << EOF
apiVersion: v1
kind: Service
metadata:
  name: webapp
spec:
  selector:
    app: webapp
  ports:
  - port: 80
    targetPort: 8080
  type: LoadBalancer
EOF

$ kubectl apply -f webapp-service.yaml

# 创建Web应用HPA
$ cat > webapp-hpa.yaml << EOF
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: webapp
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: webapp
  minReplicas: 5
  maxReplicas: 20
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 60
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 70
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 30
      policies:
      - type: Pods
        value: 5
        periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 600
      policies:
      - type: Pods
        value: 2
        periodSeconds: 60
EOF

$ kubectl apply -f webapp-hpa.yaml

执行结果：

# 查看Web应用HPA状态
$ kubectl get hpa webapp
NAME     REFERENCE           TARGETS                        MINPODS   MAXPODS   REPLICAS   AGE
webapp   Deployment/webapp   10%/60%, 15%/70%              5         20        5          5m

# 模拟高流量测试
$ kubectl run -i --tty load-generator --image=busybox /bin/sh
$ while true; do wget -q -O- http://webapp; done

# 查看HPA自动扩缩容
$ kubectl get hpa webapp -w
NAME     REFERENCE           TARGETS                        MINPODS   MAXPODS   REPLICAS   AGE
webapp   Deployment/webapp   10%/60%, 15%/70%              5         20        5          10m
webapp   Deployment/webapp   75%/60%, 80%/70%              5         20        5          11m
webapp   Deployment/webapp   80%/60%, 85%/70%              5         20        10         12m
webapp   Deployment/webapp   45%/60%, 50%/70%              5         20        10         15m

4.2 数据库自动扩缩容案例

数据库的自动扩缩容配置：

# 创建数据库StatefulSet
$ cat > mysql-statefulset.yaml << EOF
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mysql
spec:
  serviceName: mysql
  replicas: 3
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - name: mysql
        image: mysql:8.0
        resources:
          requests:
            cpu: "1"
            memory: "2Gi"
          limits:
            cpu: "2"
            memory: "4Gi"
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: fgedu123
        - name: MYSQL_DATABASE
          value: fgedudb
        - name: MYSQL_USER
          value: fgedu
        - name: MYSQL_PASSWORD
          value: fgedu123
        ports:
        - containerPort: 3306
        volumeMounts:
        - name: mysql-data
          mountPath: /var/lib/mysql
  volumeClaimTemplates:
  - metadata:
      name: mysql-data
    spec:
      accessModes: ["ReadWriteOnce"]
      storageClassName: "standard"
      resources:
        requests:
          storage: 50Gi
EOF

$ kubectl apply -f mysql-statefulset.yaml

# 创建数据库VPA
$ cat > mysql-vpa.yaml << EOF
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: mysql-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: StatefulSet
    name: mysql
  updatePolicy:
    updateMode: "Off"
  resourcePolicy:
    containerPolicies:
    - containerName: "mysql"
      minAllowed:
        cpu: 1
        memory: 2Gi
      maxAllowed:
        cpu: 4
        memory: 8Gi
      controlledResources: ["cpu", "memory"]
EOF

$ kubectl apply -f mysql-vpa.yaml

# 查看数据库VPA推荐
$ kubectl describe vpa mysql-vpa

执行结果：

# 查看数据库VPA状态
$ kubectl get vpa mysql-vpa
NAME        MODE   CPU   MEM         PROVIDED   AGE
mysql-vpa   Off    1     2Gi         True       10m

# 查看数据库VPA推荐
$ kubectl describe vpa mysql-vpa
Name:         mysql-vpa
Names学习交流加群风哥QQ113257174pace:    default
Labels:       <none>
Annotations:  <none>
API Version:  autoscaling.k8s.io/v1
Kind:         VerticalPodAutoscaler
Metadata:
  Creation Timestamp:  2026-04-03T03:20:00Z
  Generation:          1
  Resource Version:    234567
  UID:                 fedcba98-7654-3210-zyxw-vutsrqponmlk
Spec:
  Resource Policy:
    Container Policies:
      Container Name:  mysql
      Controlled Resources:
        cpu
        memory
      Max Allowed:
        Cpu:     4
        Memory:  8Gi
      Min Allowed:
        Cpu:     1
        Memory:  2Gi
  Target Ref:
    API Version:  apps/v1
    Kind:         StatefulSet
    Name:         mysql
  Update Policy:
    Update Mode:  Off
Status:
  Conditions:
    Last Transition Time:  2026-04-03T03:25:00Z
    Status:                True
    Type:                  RecommendationProvided
  Recommendation:
    Container Recommendations:
      Container Name:  mysql
      Lower Bound:
        Cpu:     1
        Memory:  2Gi
      Target:
        Cpu:     1500m
        Memory:  3Gi
      Upper Bound:
        Cpu:     4
        Memory:  8Gi

4.3 大规模集群自动扩缩容实践

大规模Kubernetes集群的自动扩缩容实践：

# 配置多节点池的Cluster Autoscaler
$ cat > cluster-autoscaler-values.yaml << EOF
autoDiscovery:
  clusterName: fgedu-cluster
replicaCount: 3
cloudProvider:
  name: aws
aws:
  region: us-east-1
nodeGroups:
  - name: default
    minSize: 3
    maxSize: 10
    instanceType: t3.large
  - name: high-performance
    minSize: 1
    maxSize: 5
    instanceType: c5.xlarge
    labels:
      node-type: high-performance
    taints:
      - key: node-type
        value: high-performance
        effect: NoSchedule
  - name: memory-intensive
    minSize: 1
    maxSize: 5
    instanceType: r5.xlarge
    labels:
      node-type: memory-intensive
    taints:
      - key: node-type
        value: memory-intensive
        effect: NoSchedule
EOF

$ helm upgrade --install cluster-autoscaler autoscaler/cluster-autoscaler --namespace kube-system --values cluster-autoscaler-values.yaml

# 配置应用的节点亲和性和容忍度
$ cat > app-with-node-affinity.yaml << EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: high-performance-app
spec:
  replicas: 5
  selector:
    matchLabels:
      app: high-performance-app
  template:
    metadata:
      labels:
        app: high-performance-app
    spec:
      containers:
      - name: app
        image: harbor.fgedu.net.cn/library/high-performance-app:v1.0.0
        resources:
          requests:
            cpu: "2"
            memory: "4Gi"
          limits:
            cpu: "4"
            memory: "8Gi"
        ports:
        - containerPort: 8080
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: node-type
                operator: In
                values:
                - high-performance
      tolerations:
      - key: "node-type"
        operator: "Equal"
        value: "high-performance"
        effect: "NoSchedule"
EOF

$ kubectl apply -f app-with-node-affinity.yaml

# 创建应用的HPA
$ kubectl autoscale deployment high-performance-app --cpu-percent=60 --min=5 --max=20

执行结果：

# 查看Cluster Autoscaler状态
$ kubectl get pods -n kube-system | grep cluster-autoscaler
cluster-autoscaler-12345   1/1     Running   0          5m
cluster-autoscaler-67890   1/1     Running   0          5m
cluster-autoscaler-abcde   1/1     Running   0          5m

# 查看节点状态
$ kubectl get nodes
NAME              STATUS   ROLES    AGE   VERSION
ip-10-0-0-101     Ready    worker   10m   v1.28.0
ip-10-0-0-102     Ready    worker   10m   v1.28.0
ip-10-0-0-103     Ready    worker   10m   v1.28.0
ip-10-0-0-104     Ready    worker   5m    v1.28.0  # 自动扩容的节点

# 查看应用HPA状态
$ kubectl get hpa high-performance-app
NAME                  REFERENCE                        TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
high-performance-app   Deployment/high-performance-app   70%/60%   5         20        10         10m

Part05-风哥经验总结与分享

在大规模Kubernetes集群的自动化扩缩容实践中，我总结了以下经验：

5.1 自动扩缩容最佳实践

  • 合理设置资源请求和限制：为Pod设置准确的资源请求和限制，是自动扩缩容的基础
  • 选择合适的扩缩容策略：根据应用特点选择HPA、VPA或CA，或组合使用
  • 设置合理的扩缩容参数：包括目标利用率、最小/最大副本数、冷却时间等
  • 监控扩缩容效果：定期分析扩缩容事件，优化配置参数

5.2 常见问题与解决方案

  • 频繁扩缩容（抖动）：解决方案：增加冷却时间，调整扩缩容阈值，使用更稳定的指标
  • 扩缩容反应缓慢：解决方案：调整监控指标的采集频率，优化扩缩容策略
  • 资源浪费：解决方案：使用VPA优化资源配置，设置合理的最小副本数
  • 节点扩容失败：解决方案：检查节点池配置，确保有足够的资源配额

5.3 性能优化建议

  • 使用自定义指标：对于特殊应用，使用自定义指标进行扩缩容，如QPS、连接数等
  • 多维度指标结合：同时使用CPU、内存和自定义指标进行扩缩容决策
  • 分层扩缩容：对不同层级的应用采用不同的扩缩容策略
  • 预测性扩缩容：基于历史数据和业务预测，提前进行扩缩容

5.4 未来发展趋势

  • 智能化扩缩容：使用机器学习算法预测流量和资源需求，实现更智能的扩缩容
  • 多集群协同扩缩容：跨多个集群的统一扩缩容管理
  • 边缘集群扩缩容：针对边缘计算场景的特殊扩缩容策略
  • 服务网格集成：与服务网格结合，实现基于服务级别的扩缩容

风哥提示：自动扩缩容是一个持续优化的过程，需要根据应用特点和业务需求不断调整和完善。

from Linux:www.itpux.com

本文由风哥教程整理发布,仅用于学习测试使用,转载注明出处:http://www.fgedu.net.cn/10327.html

相关推荐

联系我们

在线咨询：点击这里给我发消息

微信号：itpux-com

工作日：9:30-18:30，节假日休息