此文档在Centos7.X的系统、Kubernetes 1.14.X版本上实际操作

默认端口

Kubernetes默认端口为30000-32767，如果超出会报错

例如：

1
2
3
4
5
6
7
8

[root@k8s-77-189 ~]# kubectl apply -f kubernetes-dashboard.yaml
secret/kubernetes-dashboard-certs unchanged
secret/kubernetes-dashboard-csrf unchanged
serviceaccount/kubernetes-dashboard unchanged
role.rbac.authorization.k8s.io/kubernetes-dashboard-minimal unchanged
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-minimal unchanged
deployment.apps/kubernetes-dashboard unchanged
The Service "kubernetes-dashboard" is invalid: spec.ports[0].nodePort: Invalid value: 39002: provided port is not in the valid range. The range of valid ports is 30000-32767

这就需要我们修改nodePort的端口范围。

修改端口范围

一般的Kubernetes的安装方式分kubeadm、二进制。下面分别介绍两种方式的修改方法

kubeadm方式安装

kubeadm搭建的集群除了kubelet全部容器化了，也就是说kube-apiserver也是容器化的

  1. 修改apiserver的yaml文件
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

[root@k8s-77-189 ~]# cd /etc/kubernetes/manifests
[root@k8s-77-189 manifests]# vim kube-apiserver.yaml
...
spec:
  containers:
  - command:
    - kube-apiserver
    - --advertise-address=172.16.77.189
    - --allow-privileged=true
    - --authorization-mode=Node,RBAC
    - --client-ca-file=/etc/kubernetes/pki/ca.crt
    - --enable-admission-plugins=NodeRestriction
    - --enable-bootstrap-token-auth=true
    - --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt
    - --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
    - --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
    - --etcd-servers=https://127.0.0.1:2379
    - --service-node-port-range=1024-65535
    - --insecure-port=0
    - --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt
    - --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key
    ...

增加 - –service-node-port-range=1024-65535这一行 然后稍等一会

  1. 查看一下kube-apiserver
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

[root@k8s-77-189 manifests]# kubectl get pods -n kube-system
NAME                                    READY   STATUS    RESTARTS   AGE
coredns-d5947d4b-jmvbf                  1/1     Running   9          33d
coredns-d5947d4b-tplqp                  1/1     Running   8          33d
etcd-k8s-77-189                         1/1     Running   0          33d
kube-apiserver-k8s-77-189               1/1     Running   0          24s
kube-controller-manager-k8s-77-189      1/1     Running   65         33d
kube-flannel-ds-amd64-7wdl8             1/1     Running   0          33d
kube-flannel-ds-amd64-xkxm8             1/1     Running   21         33d
kube-proxy-9jsrh                        1/1     Running   0          33d
kube-proxy-dbslp                        1/1     Running   30         33d
kube-scheduler-k8s-77-189               1/1     Running   57         33d
kubernetes-dashboard-5f7b999d65-7sp52   1/1     Running   306        33d

可以发现kube-apiserver的运行时间变成很短的时间

  1. 然后使用kubectl edit看一下
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

[root@k8s-77-189 manifests]# kubectl edit -n kube-system pods kube-apiserver-k8s-77-189
...
spec:
  containers:
  - command:
    - kube-apiserver
    - --advertise-address=172.16.77.189
    - --allow-privileged=true
    - --authorization-mode=Node,RBAC
    - --client-ca-file=/etc/kubernetes/pki/ca.crt
    - --enable-admission-plugins=NodeRestriction
    - --enable-bootstrap-token-auth=true
    - --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt
    - --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
    - --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
    - --etcd-servers=https://127.0.0.1:2379
    - --service-node-port-range=1024-65535
    - --insecure-port=0
    - --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt

可以看到- –service-node-port-range=1024-65535已经应用上了

验证

现在再重新执行最开始的yaml

1
2
3
4
5
6
7
8
9

[root@k8s-77-189 ~]# kubectl apply -f kubernetes-dashboard.yaml
secret/kubernetes-dashboard-certs unchanged
secret/kubernetes-dashboard-csrf unchanged
serviceaccount/kubernetes-dashboard unchanged
role.rbac.authorization.k8s.io/kubernetes-dashboard-minimal unchanged
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-minimal unchanged
deployment.apps/kubernetes-dashboard unchanged
service/kubernetes-dashboard configured
[root@k8s-77-189 ~]#

已可以正常运行

二进制方式安装

二进制方式安装的Kubernetes。kube-apiserver是以系统服务启动的

  1. 修改apiserver的配置文件
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

[root@k8s-77-36 ~]# vim /etc/systemd/system/kube-apiserver.service
...
 --bind-address=172.16.77.36 \
  --insecure-bind-address=127.0.0.1 \
  --authorization-mode=Node,RBAC \
  --kubelet-https=true \
  --kubelet-client-certificate=/etc/kubernetes/ssl/admin.pem \
  --kubelet-client-key=/etc/kubernetes/ssl/admin-key.pem \
  --anonymous-auth=false \
  --service-cluster-ip-range=10.68.0.0/16 \
  --service-node-port-range=1024-65535 \
  --tls-cert-file=/etc/kubernetes/ssl/kubernetes.pem \
  --tls-private-key-file=/etc/kubernetes/ssl/kubernetes-key.pem \
  --client-ca-file=/etc/kubernetes/ssl/ca.pem \
  ...

增加或者修改 - –service-node-port-range=1024-65535这一行

  1. 重启apiserver服务
1
2

[root@k8s-77-36 ~]# systemctl daemon-reload
[root@k8s-77-36 ~]# systemctl restart kube-apiserver.service
  1. 查看apiserver状态
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

[root@k8s-77-36 ~]# systemctl status kube-apiserver.service
● kube-apiserver.service - Kubernetes API Server
   Loaded: loaded (/etc/systemd/system/kube-apiserver.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2019-07-04 16:22:42 CST; 2min 34s ago
     Docs: https://github.com/GoogleCloudPlatform/kubernetes
 Main PID: 24960 (kube-apiserver)
    Tasks: 21
   Memory: 230.8M
   CGroup: /system.slice/kube-apiserver.service
           └─24960 /opt/kube/bin/kube-apiserver --admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota,NodeRestriction,MutatingAdm...

Jul 04 16:22:42 k8s-77-36 kube-apiserver[24960]: I0704 16:22:42.374825   24960 controller_utils.go:1034] Caches are synced for crd-autoregister controller
Jul 04 16:22:42 k8s-77-36 kube-apiserver[24960]: I0704 16:22:42.374851   24960 cache.go:39] Caches are synced for APIServiceRegistrationController controller
...

验证

1
2
3
4
5
6
7
8
9

[root@k8s-77-36 ~]# kubectl apply -f kubernetes-dashboard.yaml
secret/kubernetes-dashboard-certs unchanged
secret/kubernetes-dashboard-csrf unchanged
serviceaccount/kubernetes-dashboard unchanged
role.rbac.authorization.k8s.io/kubernetes-dashboard-minimal unchanged
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-minimal unchanged
deployment.apps/kubernetes-dashboard unchanged
service/kubernetes-dashboard configured
[root@k8s-77-36 ~]#