使用阿里云镜像源快速搭建kubernetes(k8s) on debian10

2022

在集群初始化遇到问题，可以使用下面的命令进行清理后重新再初始化：

kubeadm reset
ifconfig cni0 down
ip link delete cni0
ifconfig flannel.1 down
ip link delete flannel.1
rm -rf /var/lib/cni/


root@debian11:~# apt-cache madison kubelet 
   kubelet |  1.24.2-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.24.1-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.24.0-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.23.8-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.23.7-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.23.6-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.23.5-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.23.4-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.23.3-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.23.2-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.23.1-00 | https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 Packages
   kubelet |  1.23.0-00 | h

思路2021

2020年的 Use aliyun mirrors Install and Configure Kubernetes (k8s) on debian10

现在是直接用aliyun提供的就行了 不用出外网 先在围墙外的机器 pull下来 然后 push到自己的hub.docker 最后在内网的机器再pull 下来 再tag一下

参考一下 然后写成脚本吧 ubuntu 使用阿里云镜像源快速搭建kubernetes 1.15.2集群

初始化时 指定aliyun mirrors 本来是指定 1。17。1版本的 我改了新的

  kubeadm init --apiserver-advertise-address=192.168.88.70 --image-repository registry.aliyuncs.com/google_containers --ignore-preflight-errors=all  --kubernetes-version v1.17.3 --service-cidr=10.96.0.0/16 --pod-network-cidr=10.244.0.0/16

国内及一些快速安装的办法

https://gitee.com/imlzw/Kubernetes-1.12.3-all-auto-install

k3OS 由容器软件提供商 Rancher Labs发布，这是业界首个专为 Kubernetes 而生的极轻量操作系统

k8e 是轻量级、可扩展的企业级 Kubernetes 发行版，允许用户统一管理、保护和获得适用于企业环境的开箱即用 Kubernetes 集群

nginx-ingress-controller_install.sh

 安装nginx-ingress-controller
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/cloud-generic.yaml

info

k8s-master 192.168.88.70
k8-snode1 192.168.88.71
k8-snode2 192.168.88.72

cat >>/etc/hosts <<EOF
192.168.88.70  k8s-master
192.168.88.71  k8s-node1
192.168.88.72  k8s-node2
EOF


每台机器最少2GB内存，2CPUs。
集群中所有机器之间网络连接正常。
打开相应的端口，详见： [ Check required ports https://kubernetes.io/docs/setup/independent/install-kubeadm/#check-required-ports]


Kubernetes要求集群中所有机器具有不同的Mac地址、产品uuid、Hostname。可以使用如下命令查看：

# UUID
 cat /sys/class/dmi/id/product_uuid

# Mac地址
 ip link

Set Hostname and update hosts file

sudo hostnamectl set-hostname "k8s-master"
sudo hostnamectl set-hostname k8s-node1
sudo hostnamectl set-hostname k8s-node2

#Add the following lines in /etc/hosts file on all three systems,

时间

Linux时间同步的那些事儿#debian

安装Docker

https://blog.csdn.net/shykevin/article/details/98811021

安装kubelet，kubeadm，kubectl

基础信息

https://mirrors.aliyun.com/kubernetes/apt/dists/kubernetes-jessie/

 cat /etc/os-release 
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"


下一代 Debian 正式发行版的代号为 "buster" — 发布时间尚未确定
Debian 9（"stretch"） — 当前的稳定版
Debian 8（"jessie"） — 被淘汰的稳定版

kubectl version
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.5",

添加apt key以及源（所有主机）

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

sudo sysctl -p /etc/sysctl.d/k8s.conf



apt update &&  apt install -y apt-transport-https curl

#听说 #kubernetes-xenial 不要改动
 

curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -
#curl -s https://mirrors.ustc.edu.cn/kubernetes/apt/dists/kubernetes-jessie/
#注意  不是追加
#echo "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-stretch main" >/etc/apt/sources.list.d/kubernetes.list

#echo "deb https://mirrors.ustc.edu.cn/kubernetes/apt/dists/kubernetes-xenial/ main" >/etc/apt/sources.list.d/kubernetes.list
echo "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main" >/etc/apt/sources.list.d/kubernetes.list


官方居然是这个 
echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

安装（所有主机）

apt-cache madison kubelet #查看版本

sudo apt update
apt install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl

#Setting up kubeadm (1.22.1-00)

#kubeadm is already the newest version (1.20.5-00).

 apt search  kubelet 
Sorting... Done
Full Text Search... Done
kubelet/kubernetes-xenial 1.24.2-00 amd64
  Kubernetes Node Agent



sudo apt install -y kubelet=1.15.2-00 kubeadm=1.15.2-00 kubectl=1.15.2-00
sudo apt-mark hold kubelet=1.15.2-00 kubeadm=1.15.2-00 kubectl=1.15.2-00

安装kubernetes集群（仅master）

#2022 不指定版本了 
  kubeadm init --apiserver-advertise-address=192.168.88.70 --image-repository registry.aliyuncs.com/google_containers --ignore-preflight-errors=all  --service-cidr=10.96.0.0/16 --pod-network-cidr=192.168.0.0/16 #calico

  kubeadm init --apiserver-advertise-address=192.168.88.70 --image-repository registry.aliyuncs.com/google_containers --ignore-preflight-errors=all  --kubernetes-version v1.20.2 --service-cidr=10.96.0.0/16 --pod-network-cidr=10.244.0.0/16 #flannel


—image-repository 指定镜像源，指定为阿里云的源，这样就会避免在拉取镜像超时，如果没问题，过几分钟就能看到成功的日志输入

sudo kubeadm init --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.15.2 --pod-network-cidr=192.169.0.0/16

[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

init 过程

s-master:~#  kubeadm init --apiserver-advertise-address=192.168.88.70 --image-repository registry.aliyuncs.com/google_containers --ignore-preflight-errors=all  --kubernetes-version v1.20.2 --service-cidr=10.96.0.0/16 --pod-network-cidr=10.244.0.0/16
[init] Using Kubernetes version: v1.20.2
[preflight] Running pre-flight checks
	[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
	[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.5. Latest validated version: 19.03
	[WARNING SystemVerification]: missing optional cgroups: hugetlb
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'



[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.88.70:6443 --token 8bemec.r64nlpkdmyijwrls \
    --discovery-token-ca-cert-hash sha256:a21b898539b970e9efe7490037c0c55710d5541ce1c967ff5b9060be845b2d33 

#2022  一定要指定版本 安装的是 v1.20.6-00 而init 时为 v1.20.6   

kubeadm init --apiserver-advertise-address="${ip}" --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.20.6 --service-cidr=10.96.0.0/12 --pod-network-cidr=10.244.0.0/16


[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.10.148:6443 --token fvixji.55pd7qni1rmp1cjk \
    --discovery-token-ca-cert-hash sha256:d65ef89fab0fdb6bcb32cd3bd4cafe6e0955454cf961bf22cce28ddb94ae44de 
root@debian11:~# 

pod网络安装 flannel

可以下载或者直接复制出来

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

https://docs.projectcalico.org/getting-started/kubernetes/flannel/flannel

https://docs.projectcalico.org/getting-started/kubernetes/quickstart

node join

#在所有node上执行
kubeadm join 192.168.88.70:6443 --token 8bemec.r64nlpkdmyijwrls     --discovery-token-ca-cert-hash sha256:a21b898539b970e9efe7490037c0c55710d5541ce1c967ff5b9060be845b2d33 

常用命令

#等一下就全是Ready
 kubectl get nodes
NAME         STATUS   ROLES                  AGE    VERSION
k8s-master   Ready    control-plane,master   110m   v1.20.5
k8s-node1    Ready    <none>                 93m    v1.20.5
k8s-node2    Ready    <none>                 93m    v1.20.5



root@k8s-master:~# kubectl get pod --all-namespaces
NAMESPACE     NAME                                 READY   STATUS    RESTARTS   AGE
kube-system   coredns-7f89b7bc75-6fcdq             1/1     Running   0          4h23m
kube-system   coredns-7f89b7bc75-9n56h             1/1     Running   0          4h23m
kube-system   etcd-k8s-master                      1/1     Running   0          4h23m
kube-system   kube-apiserver-k8s-master            1/1     Running   0          4h23m
kube-system   kube-controller-manager-k8s-master   1/1     Running   0          4h23m
kube-system   kube-flannel-ds-9kcwd                1/1     Running   0          4h7m
kube-system   kube-flannel-ds-ql2hn                1/1     Running   0          4h9m
kube-system   kube-flannel-ds-vk2qb                1/1     Running   0          4h7m
kube-system   kube-proxy-cgkhf                     1/1     Running   0          4h23m
kube-system   kube-proxy-gcbsk                     1/1     Running   0          4h7m
kube-system   kube-proxy-qnc27                     1/1     Running   0          4h7m
kube-system   kube-scheduler-k8s-master            1/1     Running   0          4h23m
root@k8s-master:~# 

集群测试 万年hello

root@k8s-master:~# cat hello.yml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-world
spec:
  selector:
    matchLabels:
      run: load-balancer-example
  replicas: 2
  template:
    metadata:
      labels:
        run: load-balancer-example
    spec:
      containers:
        - name: hello-world
          image: registry.cn-hangzhou.aliyuncs.com/aliyun_google/google-sample-node-hello:1.0
          ports:
            - containerPort: 8080
              protocol: TCP


root@k8s-master:~# kubectl  apply  -f hello.yml 
deployment.apps/hello-world created
root@k8s-master:~# kubectl  get deployment 
NAME          READY   UP-TO-DATE   AVAILABLE   AGE
hello-world   0/2     2            0           13s
my-httpd      2/2     2            2           39h
my-nginx      2/2     2            2           45h
nginx         1/1     1            1           3d13h
root@k8s-master:~# kubectl expose deployment hello-world --type=NodePort --name=node-service
service/node-service exposed
root@k8s-master:~# kubectl  get service node-service 
NAME           TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
node-service   NodePort   10.96.231.107   <none>        8080:32355/TCP   48s
root@k8s-master:~# 



root@k8s-master:~# kubectl  describe service node-service 
Name:                     node-service
Namespace:                default
Labels:                   <none>
Annotations:              <none>
Selector:                 run=load-balancer-example
Type:                     NodePort
IP Families:              <none>
IP:                       10.96.231.107
IPs:                      10.96.231.107
Port:                     <unset>  8080/TCP
TargetPort:               8080/TCP
NodePort:                 <unset>  32355/TCP
Endpoints:                10.244.2.18:8080,10.244.2.19:8080
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>


可以看到，服务通过30585 端口映射到容器的8080端口，这样我们就可以通过 32355端口访问容器内的web应用了，如下响应正常

http://192.168.88.70:32355/ Hello Kubernetes!

部署应用

K8s使用deploy部署nginx

站内资源

Install and Configure Kubernetes (k8s) on ubuntu

使用kubeadm离线部署kubernetesv1.9.0 on centos7

Proxmox 笔记#proxmox and k8s

用 k3s 部署 k8s

用 k3s 部署 k8s 不过还是原生的感觉好 如果不只是学习
k3s 的好处就是配置十分简单：https://rancher.com/docs/k3s/latest/en/quick-start/。不需要装 docker，也不需要装 kubeadm。

    在第一个 node 上跑：curl -sfL https://get.k3s.io | sh -
    在第一个 node 上获取 token：cat /var/lib/rancher/k3s/server/node-token
    在其他 node 上跑：curl -sfL https://get.k3s.io | K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken sh -

然后就搞定了。从第一个 node 的 /etc/rancher/k3s/k3s.yaml获取 kubectl 配置。

troubleshooting

不然init 有错 

kubeadm init shows kubelet isn't running or healthy

#2022  一定要指定版本 安装的是 v1.20.6-00 而init 时为 v1.20.6   

see also

ubuntu 使用阿里云镜像源快速搭建kubernetes 1.15.2集群

kubernetes安装（国内网络+阿里云ecs）

https://blog.csdn.net/u014636124/article/details/105145674

比Minikube更快，使用Kind快速创建K8S学习环境

kubernetes最新部署文档

构建kubernetes基础系统镜像-基于kubeadm安装kubernetes环境

使用阿里云ecs快速搭建Kubernetes(K8S)1.18

15分钟在笔记本上搭建 Kubernetes + Istio开发环境