实践领域：
1. 集群架构
– 高可用控制平面
– 多节点池设计
– 网络架构规划
– 存储架构设计

2. 资源管理
– 资源请求与限制
– 命名空间隔离
– 配额管理
– 优先级调度

3. 安全管理
– RBAC权限控制
– 网络策略
– Pod安全策略
– 密钥管理

4. 应用部署
– 健康检查配置
– 滚动更新策略
– 配置管理
– 日志收集

5. 可观测性
– 监控指标
– 日志聚合
– 链路追踪
– 告警管理

# FGedu Kubernetes集群架构
集群规模：
– 生产集群：3个Master + 10个Worker
– 测试集群：3个Master + 5个Worker
– 开发集群：1个Master + 3个Worker

节点池划分：
– 系统节点池：运行系统组件
– 应用节点池：运行业务应用
– 数据库节点池：运行数据库
– GPU节点池：运行AI任务

资源配额：
命名空间 CPU请求 CPU限制 内存请求 内存限制
——– ——- ——- ——– ——–
default 50 100 100Gi 200Gi
fgedu-app 100 200 200Gi 400Gi
fgedu-db 50 100 200Gi 400Gi
monitoring 20 40 50Gi 100Gi

# 1. Pod资源配置
apiVersion: v1
kind: Pod
metadata:
name: fgedu-app
namespace: fgedu-app
spec:
containers:
– name: app
image: fgedu/app:v1.0.0
resources:
requests:
cpu: “500m”
memory: “512Mi”
limits:
cpu: “2000m”
memory: “2Gi”
env:
– name: JAVA_OPTS
value: “-Xms512m -Xmx1536m”

# 2. 资源配置最佳实践
# 生产环境资源配置模板
resources:
requests:
cpu: “1000m” # 保证获得的CPU资源
memory: “1Gi” # 保证获得的内存资源
limits:
cpu: “4000m” # CPU使用上限（可超卖）
memory: “4Gi” # 内存使用上限（不可超卖）

# 资源配置原则：
# 1. requests <= limits # 2. 内存limits必须设置，避免OOM影响其他Pod # 3. CPU limits可选，允许突发使用 # 4. requests决定调度，limits决定运行上限 # 3. LimitRange配置 apiVersion: v1 kind: LimitRange metadata: name: fgedu-limits namespace: fgedu-app spec: limits: - type: Container default: # 默认limits cpu: "1000m" memory: "1Gi" defaultRequest: # 默认requests cpu: "100m" memory: "128Mi" max: # 最大限制 cpu: "8000m" memory: "8Gi" min: # 最小限制 cpu: "50m" memory: "64Mi" # 4. ResourceQuota配置 apiVersion: v1 kind: ResourceQuota metadata: name: fgedu-quota namespace: fgedu-app spec: hard: requests.cpu: "100" # 总CPU请求 requests.memory: "200Gi" # 总内存请求 limits.cpu: "200" # 总CPU限制 limits.memory: "400Gi" # 总内存限制 pods: "50" # Pod数量限制 services: "20" # Service数量限制 secrets: "50" # Secret数量限制 configmaps: "50" # ConfigMap数量限制 # 查看资源配额使用情况 $ kubectl describe resourcequota fgedu-quota -n fgedu-app Name: fgedu-quota Namespace: fgedu-app Resource Used Hard -------- ---- ---- limits.cpu 80 200 limits.memory 150Gi 400Gi pods 25 50 requests.cpu 50 100 requests.memory 100Gi 200Gi # 5. 资源监控 $ kubectl top pods -n fgedu-app NAME CPU(cores) MEMORY(bytes) fgedu-app-7b8f9c-d4e5f 500m 512Mi fgedu-app-7b8f9c-g6h7i 450m 480Mi fgedu-app-7b8f9c-j8k9l 480m 500Mi $ kubectl top nodes NAME CPU(cores) CPU% MEMORY(bytes) MEMORY% k8s-master01 500m 12% 4Gi 12% k8s-worker01 2000m 25% 8Gi 25% k8s-worker02 1800m 22% 7Gi 21% # 6. 资源优化脚本 $ cat /opt/scripts/resource_optimize.sh #!/bin/bash echo "=== Kubernetes资源优化分析 ===" # 检查未设置资源的Pod echo -e "\n1. 未设置资源的Pod:" kubectl get pods -A -o json | jq -r '.items[] | select(.spec.containers[].resources.requests == null) | "\(.metadata.namespace)/\(.metadata.name)"' | head -10 # 检查资源使用率低的Pod echo -e "\n2. 资源使用率低于20%的Pod:" for pod in $(kubectl get pods -n fgedu-app -o name); do usage=$(kubectl top $pod -n fgedu-app 2>/dev/null | tail -1 | awk ‘{print $2}’ | sed ‘s/m//’)
request=$(kubectl get $pod -n fgedu-app -o json | jq -r ‘.spec.containers[0].resources.requests.cpu’ | sed ‘s/m//’)
if [ ! -z “$usage” ] && [ ! -z “$request” ]; then
ratio=$((usage * 100 / request))
if [ $ratio -lt 20 ]; then
echo “$pod: 使用${usage}m, 请求${request}m, 比率${ratio}%”
fi
fi
done

# 检查OOMKilled的Pod
echo -e “\n3. OOMKilled的Pod:”
kubectl get pods -A -o json | jq -r ‘.items[] | select(.status.containerStatuses[].lastState.terminated.reason==”OOMKilled”) | “\(.metadata.namespace)/\(.metadata.name)”‘

$ chmod +x /opt/scripts/resource_optimize.sh
$ ./resource_optimize.sh
=== Kubernetes资源优化分析 ===

1. 未设置资源的Pod:
default/test-pod

2. 资源使用率低于20%的Pod:
pod/fgedu-app-7b8f9c-d4e5f: 使用100m, 请求500m, 比率20%

3. OOMKilled的Pod:
fgedu-app/fgedu-cache-abc123

# 1. 创建ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
name: fgedu-app-sa
namespace: fgedu-app
automountServiceAccountToken: false

# 2. 创建Role（命名空间级别）
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: fgedu-app-role
namespace: fgedu-app
rules:
– apiGroups: [“”]
resources: [“pods”, “services”, “configmaps”, “secrets”]
verbs: [“get”, “list”, “watch”]
– apiGroups: [“”]
resources: [“pods/log”]
verbs: [“get”]
– apiGroups: [“apps”]
resources: [“deployments”, “replicasets”]
verbs: [“get”, “list”, “watch”, “update”, “patch”]

# 3. 创建RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: fgedu-app-rolebinding
namespace: fgedu-app
subjects:
– kind: ServiceAccount
name: fgedu-app-sa
namespace: fgedu-app
roleRef:
kind: Role
name: fgedu-app-role
apiGroup: rbac.authorization.k8s.io

# 4. 创建ClusterRole（集群级别）
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: fgedu-viewer
rules:
– apiGroups: [“”]
resources: [“nodes”, “namespaces”, “persistentvolumes”]
verbs: [“get”, “list”, “watch”]
– apiGroups: [“storage.k8s.io”]
resources: [“storageclasses”]
verbs: [“get”, “list”, “watch”]

# 5. 网络策略配置
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: fgedu-app-network-policy
namespace: fgedu-app
spec:
podSelector:
matchLabels:
app: fgedu-app
policyTypes:
– Ingress
– Egress
ingress:
– from:
– namespaceSelector:
matchLabels:
name: fgedu-app
– podSelector:
matchLabels:
app: fgedu-frontend
ports:
– protocol: TCP
port: 8080
egress:
– to:
– namespaceSelector:
matchLabels:
name: fgedu-db
ports:
– protocol: TCP
port: 3306
– to:
– namespaceSelector: {}
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
– protocol: UDP
port: 53

# 6. Pod安全策略
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: fgedu-restricted
spec:
privileged: false
allowPrivilegeEscalation: false
requiredDropCapabilities:
– ALL
volumes:
– ‘configMap’
– ’emptyDir’
– ‘projected’
– ‘secret’
– ‘downwardAPI’
– ‘persistentVolumeClaim’
hostNetwork: false
hostIPC: false
hostPID: false
runAsUser:
rule: ‘MustRunAsNonRoot’
seLinux:
rule: ‘RunAsAny’
fsGroup:
rule: ‘RunAsAny’
readOnlyRootFilesystem: true

# 7. 安全配置验证
$ kubectl auth can-i list pods -n fgedu-app –as=system:serviceaccount:fgedu-app:fgedu-app-sa
yes

$ kubectl auth can-i delete pods -n fgedu-app –as=system:serviceaccount:fgedu-app:fgedu-app-sa
no

# 8. 安全审计脚本
$ cat /opt/scripts/security_audit.sh
#!/bin/bash

echo “=== Kubernetes安全审计 ===”

# 检查特权Pod
echo -e “\n1. 特权Pod:”
kubectl get pods -A -o json | jq -r ‘.items[] | select(.spec.containers[].securityContext.privileged==true) | “\(.metadata.namespace)/\(.metadata.name)”‘

# 检查以root运行的Pod
echo -e “\n2. 以root运行的Pod:”
kubectl get pods -A -o json | jq -r ‘.items[] | select(.spec.containers[].securityContext.runAsUser==0) | “\(.metadata.namespace)/\(.metadata.name)”‘

# 检查没有网络策略的命名空间
echo -e “\n3. 没有网络策略的命名空间:”
for ns in $(kubectl get ns -o name | cut -d/ -f2); do
count=$(kubectl get networkpolicy -n $ns 2>/dev/null | wc -l)
if [ $count -le 1 ]; then
echo “$ns”
fi
done

# 检查ServiceAccount token自动挂载
echo -e “\n4. 自动挂载ServiceAccount token的Pod:”
kubectl get pods -A -o json | jq -r ‘.items[] | select(.spec.automountServiceAccountToken != false) | “\(.metadata.namespace)/\(.metadata.name)”‘ | head -10

$ chmod +x /opt/scripts/security_audit.sh
$ ./security_audit.sh
=== Kubernetes安全审计 ===

1. 特权Pod:
kube-system/kube-proxy-abc123

2. 以root运行的Pod:
default/test-pod

3. 没有网络策略的命名空间:
default
kube-public

4. 自动挂载ServiceAccount token的Pod:
fgedu-app/fgedu-app-7b8f9c-d4e5f

# 1. Deployment配置
apiVersion: apps/v1
kind: Deployment
metadata:
name: fgedu-app
namespace: fgedu-app
spec:
replicas: 3
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
selector:
matchLabels:
app: fgedu-app
template:
metadata:
labels:
app: fgedu-app
version: v1.0.0
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
– weight: 100
podAffinityTerm:
labelSelector:
matchLabels:
app: fgedu-app
topologyKey: kubernetes.io/hostname
containers:
– name: app
image: fgedu/app:v1.0.0
ports:
– containerPort: 8080
resources:
requests:
cpu: “500m”
memory: “512Mi”
limits:
cpu: “2000m”
memory: “2Gi”
livenessProbe:
httpGet:
path: /health/live
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
httpGet:
path: /health/ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 3
lifecycle:
preStop:
exec:
command: [“/bin/sh”, “-c”, “sleep 15”]

# 2. PodDisruptionBudget配置
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: fgedu-app-pdb
namespace: fgedu-app
spec:
minAvailable: 2
selector:
matchLabels:
app: fgedu-app

# 3. Service配置
apiVersion: v1
kind: Service
metadata:
name: fgedu-app
namespace: fgedu-app
spec:
type: ClusterIP
selector:
app: fgedu-app
ports:
– port: 80
targetPort: 8080
sessionAffinity: None

# 4. HPA配置
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: fgedu-app-hpa
namespace: fgedu-app
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: fgedu-app
minReplicas: 3
maxReplicas: 10
metrics:
– type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
– type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 80

# 5. 查看HPA状态
$ kubectl get hpa -n fgedu-app
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
fgedu-app-hpa Deployment/fgedu-app 50%/70%, 60%/80% 3 10 3 10d

# 6. 查看PDB状态
$ kubectl get pdb -n fgedu-app
NAME MIN AVAILABLE MAX UNAVAILABLE ALLOWED DISRUPTIONS AGE
fgedu-app-pdb 2 N/A 1 10d

# 7. 高可用验证脚本
$ cat /opt/scripts/ha_test.sh
#!/bin/bash

echo “=== 高可用测试 ===”

# 测试Pod重启
echo -e “\n1. 测试Pod自动恢复:”
pod=$(kubectl get pods -n fgedu-app -l app=fgedu-app -o name | head -1)
echo “删除Pod: $pod”
kubectl delete $pod -n fgedu-app

echo “等待Pod恢复…”
sleep 30

ready_pods=$(kubectl get pods -n fgedu-app -l app=fgedu-app –no-headers | grep “Running” | grep “1/1” | wc -l)
echo “就绪Pod数量: $ready_pods”

# 测试节点故障
echo -e “\n2. 测试节点故障恢复:”
node=”k8s-worker01″
echo “模拟节点故障: $node”
kubectl cordon $node

echo “等待Pod迁移…”
sleep 60

kubectl uncordon $node
echo “节点已恢复”

# 验证服务可用性
echo -e “\n3. 验证服务可用性:”
for i in {1..10}; do
status=$(kubectl exec -n fgedu-app deploy/fgedu-app — curl -s -o /dev/null -w “%{http_code}” http://fgedudb:8080/health/live)
echo “请求 $i: HTTP $status”
done

$ chmod +x /opt/scripts/ha_test.sh
$ ./ha_test.sh
=== 高可用测试 ===

1. 测试Pod自动恢复:
删除Pod: pod/fgedu-app-7b8f9c-d4e5f
等待Pod恢复…
就绪Pod数量: 3

2. 测试节点故障恢复:
模拟节点故障: k8s-worker01
等待Pod迁移…
节点已恢复

3. 验证服务可用性:
请求 1: HTTP 200
请求 2: HTTP 200
…
请求 10: HTTP 200

# 1. kubelet参数优化
# /var/lib/kubelet/config.yaml
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
maxPods: 110
imageGCHighThresholdPercent: 85
imageGCLowThresholdPercent: 80
evictionHard:
memory.available: “500Mi”
nodefs.available: “10%”
nodefs.inodesFree: “5%”
imagefs.available: “15%”
evictionSoft:
memory.available: “1Gi”
nodefs.available: “15%”
evictionSoftGracePeriod:
memory.available: “1m30s”
nodefs.available: “2m”
evictionMaxPodGracePeriod: 60
evictionMinimumReclaim:
memory.available: “0Mi”
nodefs.available: “500Mi”

# 2. kube-apiserver参数优化
# /etc/kubernetes/manifests/kube-apiserver.yaml
spec:
containers:
– command:
– kube-apiserver
– –max-requests-inflight=800
– –max-mutating-requests-inflight=400
– –request-timeout=60s
– –enable-watch-cache=true
– –watch-cache-sizes=deployments.apps#100,replicasets.apps#100

# 3. kube-scheduler参数优化
# /etc/kubernetes/manifests/kube-scheduler.yaml
spec:
containers:
– command:
– kube-scheduler
– –kube-api-qps=100
– –kube-api-burst=200

# 4. kube-controller-manager参数优化
# /etc/kubernetes/manifests/kube-controller-manager.yaml
spec:
containers:
– command:
– kube-controller-manager
– –kube-api-qps=100
– –kube-api-burst=200
– –concurrent-deployment-syncs=10
– –concurrent-replicaset-syncs=10

# 5. etcd性能优化
# /etc/etcd/etcd.conf
ETCD_HEARTBEAT_INTERVAL=500
ETCD_ELECTION_TIMEOUT=5000
ETCD_SNAPSHOT_COUNT=10000
ETCD_MAX_REQUEST_BYTES=15728640
ETCD_QUOTA_BACKEND_BYTES=8589934592

# 6. 集群性能监控
$ kubectl get –raw /metrics | grep -E “apiserver_request_duration_seconds|etcd_request_duration_seconds”

# 查看API Server延迟
$ kubectl get –raw /metrics | grep apiserver_request_duration_seconds | grep quantile
apiserver_request_duration_seconds{quantile=”0.5″} 0.001
apiserver_request_duration_seconds{quantile=”0.9″} 0.005
apiserver_request_duration_seconds{quantile=”0.99″} 0.02

# 7. 性能基准测试
$ cat /opt/scripts/performance_test.sh
#!/bin/bash

echo “=== Kubernetes性能测试 ===”

# 测试API Server性能
echo -e “\n1. API Server性能测试:”
for i in {1..100}; do
kubectl get pods -n fgedu-app > /dev/null 2>&1
done

# 测试调度性能
echo -e “\n2. 调度性能测试:”
start_time=$(date +%s)
kubectl run test-pod –image=busybox –replicas=10 — sleep 3600
end_time=$(date +%s)
echo “调度10个Pod耗时: $((end_time – start_time))秒”

# 清理测试Pod
kubectl delete deployment test-pod

# 测试网络性能
echo -e “\n3. 网络性能测试:”
kubectl run iperf-server –image=networkstatic/iperf3 — -s
kubectl run iperf-client –image=networkstatic/iperf3 — -c iperf-server

# 等待测试完成
sleep 30

# 获取测试结果
kubectl logs iperf-client

# 清理
kubectl delete pod iperf-server iperf-client

$ chmod +x /opt/scripts/performance_test.sh
$ ./performance_test.sh
=== Kubernetes性能测试 ===

1. API Server性能测试:
完成100次请求

2. 调度性能测试:
调度10个Pod耗时: 15秒

3. 网络性能测试:
[ 5] 0.00-10.00 sec 1.25 GBytes 1.07 Gbits/sec sender
[ 5] 0.00-10.00 sec 1.25 GBytes 1.07 Gbits/sec receiver

# 1. Pod状态排查
# 查看Pod状态
$ kubectl get pods -n fgedu-app -o wide
NAME READY STATUS RESTARTS AGE IP NODE
fgedu-app-7b8f9c-d4e5f 1/1 Running 0 10d 10.244.1.10 k8s-worker01
fgedu-app-7b8f9c-g6h7i 0/1 CrashLoopBackOff 5 10m 10.244.2.10 k8s-worker02

# 查看Pod详情
$ kubectl describe pod fgedu-app-7b8f9c-g6h7i -n fgedu-app
Events:
Type Reason Age From Message
—- —— —- —- ——-
Normal Scheduled 10m default-scheduler Successfully assigned fgedu-app/fgedu-app-7b8f9c-g6h7i to k8s-worker02
Normal Pulled 9m (x4 over 10m) kubelet Container image “fgedu/app:v1.0.0” already present on machine
Normal Created 9m (x4 over 10m) kubelet Created container app
Normal Started 9m (x4 over 10m) kubelet Started container app
Warning BackOff 1m (x20 over 9m) kubelet Back-off restarting failed container

# 查看Pod日志
$ kubectl logs fgedu-app-7b8f9c-g6h7i -n fgedu-app –previous
Error: Unable to connect to database at fgedu-db:3306
Caused by: java.net.ConnectException: Connection refused

# 2. Service排查
# 查看Service
$ kubectl get svc -n fgedu-app
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
fgedu-app ClusterIP 10.96.100.100 80/TCP 10d

# 查看Endpoints
$ kubectl get endpoints fgedu-app -n fgedu-app
NAME ENDPOINTS AGE
fgedu-app 10.244.1.10:8080,10.244.2.10:8080 10d

# 测试Service连通性
$ kubectl run test-dns –image=busybox –rm -it –restart=Never — nslookup fgedu-app.fgedu-app.svc.cluster.local
Server: 10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name: fgedu-app.fgedu-app.svc.cluster.local
Address 1: 10.96.100.100 fgedu-app.fgedu-app.svc.cluster.local

# 3. 节点排查
# 查看节点状态
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master01 Ready control-plane 30d v1.28.0
k8s-worker01 Ready 30d v1.28.0
k8s-worker02 NotReady 30d v1.28.0

# 查看节点详情
$ kubectl describe node k8s-worker02
Conditions:
Type Status LastHeartbeatTime Reason Message
—- —— —————– —— ——-
MemoryPressure False Fri, 03 Apr 2026 10:00:00 +0800 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Fri, 03 Apr 2026 10:00:00 +0800 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Fri, 03 Apr 2026 10:00:00 +0800 KubeletHasSufficientPID kubelet has sufficient PID available
Ready False Fri, 03 Apr 2026 10:00:00 +0800 KubeletNotReady container runtime not ready

# 4. 故障排查脚本
$ cat /opt/scripts/troubleshoot.sh
#!/bin/bash

echo “=== Kubernetes故障排查 ===”

# 检查异常Pod
echo -e “\n1. 异常Pod:”
kubectl get pods -A –field-selector=status.phase!=Running,status.phase!=Succeeded -o wide

# 检查重启次数多的Pod
echo -e “\n2. 重启次数多的Pod:”
kubectl get pods -A -o json | jq -r ‘.items[] | select(.status.containerStatuses[].restartCount > 5) | “\(.metadata.namespace)/\(.metadata.name): \(.status.containerStatuses[].restartCount)次”‘

# 检查Pending的Pod
echo -e “\n3. Pending状态的Pod:”
kubectl get pods -A –field-selector=status.phase=Pending

# 检查资源不足
echo -e “\n4. 资源不足的节点:”
kubectl describe nodes | grep -A 5 “Allocated resources”

# 检查事件
echo -e “\n5. 最近的事件:”
kubectl get events -A –sort-by=’.lastTimestamp’ | tail -20

# 检查组件状态
echo -e “\n6. 组件状态:”
kubectl get componentstatuses

$ chmod +x /opt/scripts/troubleshoot.sh
$ ./troubleshoot.sh
=== Kubernetes故障排查 ===

1. 异常Pod:
NAMESPACE NAME READY STATUS RESTARTS AGE
fgedu-app fgedu-app-7b8f9c-g6h7i 0/1 CrashLoopBackOff 5 10m

2. 重启次数多的Pod:
fgedu-app/fgedu-app-7b8f9c-g6h7i: 5次

3. Pending状态的Pod:
No resources found.

4. 资源不足的节点:
Allocated resources:
CPU Requests CPU Limits Memory Requests Memory Limits
———— ———- ————— ————-
800m (20%) 2000m (50%) 2Gi (12%) 4Gi (25%)

5. 最近的事件:
NAMESPACE LAST SEEN TYPE REASON OBJECT MESSAGE
fgedu-app 2m Warning BackOff pod/fgedu-app-7b8f9c-g6h7i Back-off restarting failed container

6. 组件状态:
NAME STATUS MESSAGE ERROR
controller-manager Healthy ok
scheduler Healthy ok
etcd-0 Healthy {“health”:”true”,”reason”:””}