本文档风哥主要介绍Linux系统大规模Kubernetes服务暴露与负载均衡,包括K8s服务基础概念、服务类型介绍、负载均衡原理、服务暴露配置、负载均衡器配置、Ingress控制器配置以及实战案例等内容,风哥教程参考Kubernetes官方文档、Linux官方文档Networking等内容,适合系统管理员在生产环境中使用。
Part01-基础概念与理论知识
1.1 K8s服务基础概念
Kubernetes服务(Service)是一种抽象,定义了一组Pod的访问方式。服务为Pod提供了一个稳定的IP地址和DNS名称学习交流加群风哥微信: itpux-com,使Pod可以被其他Pod或外部客户端访问。服务还提供了负载均衡功能,将请求分发到多个Pod实例。
- ClusterIP:集群内部IP,只能在集群内部访问
- NodePort:在每个节点上暴露一个端口,外部可以通过节点IP和端口访问
- LoadBalancer:使用云服务商的负载均衡器,外部可以通过负载均衡器IP访问
- ExternalName:将服务映射到外部DNS名称
- Session Affinity:会话亲和性,确保同一客户端的请求被发送到同一Pod
1.2 服务类型介绍
Kubernetes支持多种服务类型,根据不同的场景选择合适的服务类型:
– ClusterIP:默认类型,为服务分配一个集群内部IP,只能在集群内部访问
– NodePort:在每个节点上暴露一个端口,外部可以通过节点IP和端口访问
– LoadBalancer:使用云服务商的负载均衡器,外部可以通过负载均衡器IP访问
– ExternalName:将服务映射到外部DNS名称,通过CNAME记录实现
# 服务类型选择
– 内部服务:使用ClusterIP
– 外部访问:使用NodePort或LoadBalancer
– 云环境:使用LoadBalancer
– 复杂路由:使用Ingress
1.3 负载均衡原理
Kubernetes的负载均衡原理:
- ClusterIP负载均衡:通过kube-proxy实现,使用iptables或IPVS规则将请求分发到Pod
- NodePort负载均衡:在每个节点上开放一个端口,通过kube-proxy将请求转发到Pod
- LoadBalancer负载均衡:使用云服务商的负载均衡器,将请求分发到节点
- Ingress负载均衡:通过Ingress控制器实现,支持更复杂的路由规则
Part02-生产环境规划与建议
2.1 服务设计规划
生产环境服务设计规划要点:
– 服务命名:使用清晰的命名规范,便于管理和识别
– 服务版本:使用版本号管理服务,便于滚动更新
– 服务依赖:明确服务之间的依赖关系,避免循环依赖
– 服务粒度:合理划分服务粒度,避免服务过大或过小
# 负载均衡策略
– 轮询:默认策略,按顺序分发请求
– 会话亲和性:确保同一客户端的请求被发送到同一Pod
– 最小连接:将请求分发到连接数最少的Pod
– 权重:根据Pod的权重分发请求
2.2 负载均衡器选择
生产环境负载均衡器选择要点:
– 软件负载均衡器:Nginx、HAProxy、Traefik
– 硬件负载均衡器:F5、Citrix、A10
– 云负载均衡器:AWS ELB、Azure Load Balancer、GCP Load Balancer
# 负载均衡器选择因素
– 性能需求:处理并发请求的能力
– 功能需求:支持的协议和特性
– 成本:硬件和维护成本
– 集成性:与Kubernetes的集成程度
– 可扩展性:支持的节点数量和服务数量
2.3 网络考虑因素
生产环境网络考虑因素:
– 服务网络:用于Pod间通信
– Pod网络:用于Pod内部通信
– 外部网络:用于服务暴露
# 网络配置
– 网络插件:选择适合的网络插件,如Calico、Flannel、Cilium
– 网络策略:配置网络策略,限制Pod间通信
– 网络安全:使用TLS加密通信,配置防火墙规则
# 网络性能
– 带宽:确保足够的网络带宽
– 延迟:最小化网络延迟
– 可靠性:确保网络可靠性,避免网络故障
Part03-生产环境项目实施方案
3.1 服务暴露配置
3.1.1 ClusterIP服务配置
$ kubectl create deployment nginx –image=nginx –replicas=3
$ kubectl expose deployment nginx –port=80 –type=ClusterIP
# 查看服务状态
$ kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1
nginx ClusterIP 10.100.123.45
# 测试服务访问
$ kubectl run busybox –image=busybox –restart=Never –rm -it — wget -O – http://nginx
Connecting to nginx (10.100.123.45:80)
Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
For online documentation and support please refer to
nginx.org.
Commercial support is available at
nginx.com.
Thank you for using nginx.
– 100% |*******************************| 612 0:00:00 ETA
3.1.2 NodePort服务配置
$ kubectl create deployment nginx –image=nginx –replicas=3
$ kubectl expose deployment nginx –port=80 –type=NodePort –name=nginx-nodeport
# 查看服务状态
$ kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1
nginx-nodeport NodePort 10.100.234.56
# 测试服务访问
$ curl http://192.168.1.101:30080
Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
For online documentation and support please refer to
nginx.org.
Commercial support is available at
nginx.com.
Thank you for using nginx.
3.1.3 LoadBalancer服务配置
$ kubectl create deployment nginx –image=nginx –replicas=3
$ kubectl expose deployment nginx –port=80 –type=LoadBalancer –name=nginx-loadbalancer
# 查看服务状态
$ kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1
nginx-loadbalancer LoadBalancer 10.100.345.67 203.0.113.123 80:30180/TCP 5m
# 测试服务访问
$ curl http://203.0.113.123
Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
For online documentation and support please refer to
nginx.org.
Commercial support is available at
nginx.com.
Thank you for using nginx.
3.2 负载均衡器配置
3.2.1 配置Nginx负载均衡器
$ sudo dnf install -y nginx
# 配置Nginx负载均衡
$ sudo vim /etc/nginx/nginx.conf
http {
upstream kubernetes {
server 192.168.1.101:30080;
server 192.168.1.102:30080;
server 192.168.1.103:30080;
}
server {
listen 80;
server_name fgedu.net.cn;
location / {
proxy_pass http://kubernetes;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
}
# 启动Nginx服务
$ sudo systemctl start nginx
$ sudo systemctl 学习交流加群风哥QQ113257174enable nginx
# 测试负载均衡
$ curl http://fgedu.net.cn
Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
For online documentation and support please refer to
nginx.org.
Commercial support is available at
nginx.com.
Thank you for using nginx.
3.3 Ingress控制器配置
3.3.1 安装Nginx Ingress控制器
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.8.2/deploy/static/provider/cloud/deploy.yaml
# 查看Ingress控制器状态
$ kubectl get pods -n ingress-nginx
NAME READY STATUS RESTARTS AGE
ingress-nginx-controller-5c6697867c-2q4x2 1/1 Running 0 5m
# 创建Ingress资源
$ kubectl create deployment nginx –image=nginx –replicas=3
$ kubectl expose deployment nginx –port=80 –type=ClusterIP
$ kubectl apply -f – << EOF apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: nginx-ingress annotations: nginx.ingress.kubernetes.io/rewrite-target: / spec: rules: - host: fgedu.net.cn http: paths: - path: / pathType: Prefix backend: service: name: nginx port: number: 80 EOF # 查看Ingress状态 $ kubectl get ingress NAME CLASS HOSTS ADDRESS PORTS AGE nginx-ingress nginx fgedu.net.cn 192.168.1.100 80 5m # 测试Ingress访问 $ curl http://fgedu.net.cn
Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
For online documentation and support please refer to
nginx.org.
Commercial support is available at
nginx.com.
Thank you for using nginx.
Part04-生产案例与实战讲解
4.1 NodePort服务案例
4.1.1 企业级NodePort服务部署
# 环境:3节点K8s集群
# 目标:部署NodePort服务,实现外部访问
# 1. 部署应用
$ kubectl create deployment web –image=nginx –replicas=5
# 2. 创建NodePort服务
$ kubectl expose deployment web –port=80 –type=NodePort –name=web-nodeport
# 3. 查看服务状态
$ kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1
web-nodeport NodePort 10.100.123.45
# 4. 测试服务访问
$ for i in {1..10}; do curl http://192.168.1.101:30080; echo; done
Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
For online documentation and support please refer to
nginx.org.
Commercial support is available at
nginx.com.
Thank you for using nginx.
…
4.2 LoadBalancer服务案例
4.2.1 企业级LoadBalancer服务部署
# 环境:云环境K8s集群
# 目标:部署LoadBalancer服务,实现外部访问
# 1. 部署应用
$ kubectl create deployment api –image=nginx –replicas=3
# 2. 创建LoadBalancer服务
$ kubectl expose deployment api –port=80 –type=LoadBalancer –name=api-loadbalancer
# 3. 查看服务状态
$ kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1
api-loadbalancer LoadBalancer 10.100.234.56 203.0.113.123 80:30180/TCP 5m
# 4. 测试服务访问
$ curl http://203.0.113.123
Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
from PG视频:www.itpux.com
For online documentation and support please refer to
nginx.org.
Commercial support is available at
nginx.com.
Thank you for using nginx.
4.3 Ingress服务案例
4.3.1 企业级Ingress服务部署
# 环境:3节点K8s集群
# 目标:部署Ingress服务,实现复杂路由
# 1. 安装Nginx Ingress控制器
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.8.2/deploy/static/provider/cloud/deploy.yaml
# 2. 部署多个应用
$ kubectl create deployment web –image=nginx –replicas=3
$ kubectl expose deployment web –port=80 –type=ClusterIP
$ kubectl create deployment api –image=nginx –replicas=3
$ kubectl expose deployment api –port=80 –type=ClusterIP
# 3. 创建Ingress资源
$ kubectl apply -f – << EOF
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: fgedu.net.cn
http:
paths:
- path: /web
pathType: Prefix
backend:
service:
name: web
port:
number: 80
- path: /api
pathType: Prefix
backend:
service:
name: api
port:
number: 80
EOF
# 4. 查看Ingress状态
$ kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
app-ingress nginx fgedu.net.cn 192.168.1.100 80 5m
# 5. 测试Ingress访问
$ curl http://fgedu.net.cn/web
Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
For online documentation and support please refer to
nginx.org.
Commercial support is available at
nginx.com.
Thank you for using nginx.
$ curl http://fgedu.net.cn/api
Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
For online documentation and support please refer to
nginx.org.
Commercial support is available at
nginx.com.
Thank you for using nginx.
Part05-风哥经验总结与分享
5.1 K8s服务暴露最佳实践
Kubernetes服务暴露最佳实践:
- 选择合适的服务类型:根据访问需求选择ClusterIP、NodePort、LoadBalancer或Ingress
- 使用Ingress进行复杂路由:对于多服务、多路径的场景,使用Ingress控制器
- 配置健康检查:为服务配置健康检查,确保负载均衡器只将流量发送到健康的Pod
- 使用会话亲和性:对于需要会话保持的应用,配置会话亲和性
- 合理配置负载均衡策略:根据应用特点选择合适的负载均衡策略
- 监控服务状态:实时监控服务的健康状态和流量情况
- 使用TLS加密:为外部访问的服务配置TLS加密
- 定期测试服务可用性:定期测试服务的可用性,确保服务正常运行
5.2 常见问题与解决方案
## 1. 服务无法访问
– 原因:网络配置错误、服务未运行、防火墙阻止
– 解决方案:检查网络配置、查看服务状态、关闭防火墙或开放端口
## 2. 负载均衡器未分配IP
– 原因:云服务商限制、负载均衡器配置错误
– 解决方案:检查云服务商配额、检查负载均衡器配置
## 3. Ingress控制器未启动
– 原因:资源不足、配置错误、网络问题
– 解决方案:增加资源、检查配置、检查网络连接
## 4. 会话亲和性不生效
– 原因:配置错误、负载均衡器不支持
– 解决方案:检查会话亲和性配置、选择支持会话亲和性的负载均衡器
## 5. 服务响应缓慢
– 原因:Pod资源不足、网络延迟、负载均衡器性能不足
– 解决方案:增加Pod资源、优化网络、升级负载均衡器
5.3 性能调优建议
风哥针对
Kubernetes服务暴露与负载均衡性能调优建议:
- 优化Pod配置:为Pod设置合理的资源限制和请求,确保Pod有足够的资源处理请求
- 优化服务配置:根据应用特点配置合适的服务类型和负载均衡策略
- 优化网络:使用高性能网络插件,配置适当的MTU,使用万兆网络
- 优化负载均衡器:选择高性能的负载均衡器,配置适当的参数
- 使用缓存:对于静态内容,使用缓存减少请求处理时间
- 监控与分析:使用Prometheus和Grafana监控服务性能,分析瓶颈
- 水平扩展:根据负载情况自动扩展Pod数量,提高服务处理能力
- 定期维护:清理无用的服务和资源,更新Kubernetes版本
风哥提示:
风哥提示:
本文由风哥教程整理发布,仅用于学习测试使用,转载注明出处:http://www.fgedu.net.cn/10327.html
