Ini adalah catatan bagaimana aku setup kubernetes cluster pakai kubeadm. Aku setup pakai VM (KVM) 1 master dan 1 node worker. Aku pakai vm ubuntu 18.04. Selama ini aku setup kubernetes cluster di debian 10. Sekarang belajar setup di ubuntu 18.04. Meskipun ubuntu berbasis debian, tapi ada beberapa yang berbeda ketika setup kubernetes cluster, yg paling kentara di Debian 10 adalah iptables-legacy.
Rangkuman singkatnya begini:
Ikuti panduan https://docs.docker.com/engine/install/ubuntu/
apt-get update
apt-get install \
apt-transport-https \
ca-certificates \
curl \
gnupg-agent \
software-properties-commoncurl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable" apt-get update
apt-get install docker-ce docker-ce-cli containerd.iobash
cat > /etc/docker/daemon.json <<EOF { “exec-opts”: [“native.cgroupdriver=systemd”], “log-driver”: “json-file”, “log-opts”: { “max-size”: “100m” }, “storage-driver”: “overlay2” } EOF
bash
mkdir -p /etc/systemd/system/docker.service.dbash
systemctl daemon-reload systemctl restart docker systemctl enable docker
## A.3 Install kubeadm
Ikuti panduan https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/
```bash
cat <<EOF | tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --systemapt-get update && apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF | tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectlMatikan swap
bash swapoff -a
Matikan swap pas reboot pakai crontab
@reboot swapoff -a
Disini mau pakai flannel untuk networknya, oleh karena itu pakai parameter `--pod-network-cidr=10.244.0.0/16`
bash
kubeadm init --pod-network-cidr=10.244.0.0/16bash
root@master:/home/sumar# kubectl get po -o wide -A
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kube-system coredns-66bff467f8-jghgf 0/1 Pending 0 4m22s
pod coredns status pending krn belum install flannel,
bash
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.ymlSetelah install flannel, pod coredns jadi jalan
bash
root@master:/home/sumar# kubectl get po -o wide -A
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kube-system coredns-66bff467f8-jghgf 1/1 Running 0 8m19s 10.244.0.3 master
# B. Setup Nodesatu
- Node IP addr: 192.168.100.199
- Node Hostname: nodesatu
- OS: Ubuntu 18.04
- RAM: 2GB
- CPU: 2 core
## B.1 Install docker
Ikuti panduan https://docs.docker.com/engine/install/ubuntu/
apt-get update
apt-get install
apt-transport-https
ca-certificates
curl
gnupg-agent
software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
add-apt-repository
“deb [arch=amd64] https://download.docker.com/linux/ubuntu
$(lsb_release -cs)
stable”
apt-get update apt-get install docker-ce docker-ce-cli containerd.io
## B.2 Ganti driver docker ke systemd, awalnya cgroupfs
bash
# Set up the Docker daemon
cat > /etc/docker/daemon.json <<EOF
{
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
},
"storage-driver": "overlay2"
}
EOFbash mkdir -p /etc/systemd/system/docker.service.d
bash
# Restart Docker
systemctl daemon-reload
systemctl restart docker
systemctl enable dockerIkuti panduan https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/
cat <<EOF | tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --systemapt-get update && apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF | tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectlbash
kubeadm join 192.168.100.198:6443 —token 98vdpp.r7tziuipa5ebwpg8
—discovery-token-ca-cert-hash sha256:d6d44f1588470756cd52f70b5576aa7ddd6b5b9a57559b025dbfa21ecee2c0eb
Langkah A dan B adalah langkah untuk setup cluster. Setelah setup master selesai dan setup nodesatu selesai, kita verifikasi:
bash
root@master:~# kubectl cluster-info
Kubernetes master is running at https://192.168.100.198:6443
KubeDNS is running at https://192.168.100.198:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
root@master:~# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
master Ready master 74m v1.18.5 192.168.100.198 <none> Ubuntu 18.04.4 LTS 4.15.0-108-generic docker://19.3.12
nodesatu Ready <none> 41m v1.18.5 192.168.100.199 <none> Ubuntu 18.04.4 LTS 4.15.0-108-generic docker://19.3.12
root@master:~# kubectl -n kube-system get po -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-66bff467f8-jghgf 1/1 Running 0 77m 10.244.0.3 master <none> <none>
coredns-66bff467f8-zkcdd 1/1 Running 0 77m 10.244.0.2 master <none> <none>
etcd-master 1/1 Running 0 77m 192.168.100.198 master <none> <none>
kube-apiserver-master 1/1 Running 0 77m 192.168.100.198 master <none> <none>
kube-controller-manager-master 1/1 Running 0 77m 192.168.100.198 master <none> <none>
kube-flannel-ds-amd64-rb4wc 1/1 Running 0 69m 192.168.100.198 master <none> <none>
kube-flannel-ds-amd64-rmdw7 1/1 Running 1 44m 192.168.100.199 nodesatu <none> <none>
kube-proxy-hgkn9 1/1 Running 0 44m 192.168.100.199 nodesatu <none> <none>
kube-proxy-lc5ps 1/1 Running 0 77m 192.168.100.198 master <none> <none>
kube-scheduler-master 1/1 Running 0 77m 192.168.100.198 master <none> <none>Dari data diatas, semua pod running. Aku asumsikan semua beres.
Setelah cluster berjalan, aku lanjutkan install aplikasi di atas cluster kubernetes. Kita kembali ke master, dan install-install dari master.
Helm v3 ini akan membatu banget untuk install-install aplikasi diatas cluster kubernetes.
bash curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh
helm repo add stable https://kubernetes-charts.storage.googleapis.com/
helm repo update
## C.2 Install metallb
Ikuti https://metallb.universe.tf/installation/
Krn cluster yg kubuat di vm local, dan gak punya dedicated LB, maka aku butuh metallb untuk assign IP ke service bertipe LoadBalancer
bash
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/namespace.yaml
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/metallb.yaml
kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"Bikin configmap untuk metallb, silakan merujuk ke https://metallb.universe.tf/configuration/#layer-2-configuration
nano metallb-configmap.yaml
# isinya sbb:
apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb-system
name: config
data:
config: |
address-pools:
- name: default
protocol: layer2
addresses:
- 192.168.100.30-192.168.100.40Yang perlu diperhatikan adalah 192.168.100.30-192.168.100.40, IP ini aku kasih range dari router, ya krn ini VM di local. Kalau di cloud/datacenter range ip tersebut harus ip public supaya service bisa dijangkau dari luar.
bash kubectl apply -f metallb-configmap.yaml
Verifikasi hasil install metallb
bash
root@master:~# kubectl -n metallb-system get po -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
controller-57f648cb96-b5llj 1/1 Running 0 33m 10.244.1.2 nodesatu <none> <none>
speaker-csgx5 1/1 Running 0 33m 192.168.100.199 nodesatu <none> <none>
speaker-mrv7x 1/1 Running 0 33m 192.168.100.198 master <none> <none>Ikuti https://kubernetes.github.io/ingress-nginx/deploy/#using-helm
Ingress controller ini membantu untuk expose service ke luar cluster, misalnya expose web service via domain.
Nginx Ingress Controller hanyalah satu dari sekian banyak Ingress Controller. Selain nginx ada Traefik, HAProxy, dll. Aku sendiri selama ini pakai Traefik v1.7.24 di production. Dah kali ini aku akan pakai Nginx
bash helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx helm install my-release ingress-nginx/ingress-nginx
Perintah diatas akan install nginx di namespace default, kita check:
bash
@master:~# kubectl -n default get po,svc -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/my-release-ingress-nginx-controller-7fcf9d598b-k68gs 1/1 Running 0 34m 10.244.1.4 nodesatu <none> <none>
NAME TYPE CLUSTER-IP PORT(S) AGE SELECTOR
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 85m <none>
service/my-release-ingress-nginx-controller LoadBalancer 10.110.208.65 192.168.100.30 80:30509/TCP,443:30954/TCP 34m app.kubernetes.io/component=controller,app.kubernetes.io/instance=my-release,app.kubernetes.io/name=ingress-nginx
service/my-release-ingress-nginx-controller-admission ClusterIP 10.104.254.133 <none> 443/TCP 34m app.kubernetes.io/component=controller,app.kubernetes.io/instance=my-release,app.kubernetes.io/name=ingress-nginxBerkat bantuan metallb, EXTERNAL-IP dari service/my-release-ingress-nginx-controller tidak pending. Dalam hal ini dia mendapatkan IP 192.168.100.30. IP tsb kita asumsikan IP public yg mana akan dimasukan ke DNS A Record
Untuk memastikan ingress controllernya berfungsi, aku akan test dari luar cluster yakni dari PC.
bash ➜ ~ curl 192.168.100.30
Kita coba deploy aplikasi whoareyou
Isi dari whoareyou.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: whoareyou-deployment
spec:
replicas: 2
selector:
matchLabels:
app: whoareyou
template:
metadata:
labels:
app: whoareyou
spec:
containers:
- name: whoareyou-container
image: containous/whoami
---
apiVersion: v1
kind: Service
metadata:
name: whoareyou-service
spec:
ports:
- name: http
targetPort: 80
port: 80
selector:
app: whoareyou
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: whoareyou-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: whoareyou.kvm.local
http:
paths:
- path: /
backend:
serviceName: whoareyou-service
servicePort: httpbash kubectl apply -f whoareyou.yaml
Verifikasi hasil deploy:
bash
root@master:~# kubectl -n default get po,svc,ingress
NAME READY STATUS RESTARTS AGE
pod/my-release-ingress-nginx-controller-7fcf9d598b-k68gs 1/1 Running 0 45m
pod/whoareyou-deployment-74bff9c468-mlnjd 1/1 Running 0 110s
pod/whoareyou-deployment-74bff9c468-p5bkp 1/1 Running 0 110s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 97m
service/my-release-ingress-nginx-controller LoadBalancer 10.110.208.65 192.168.100.30 80:30509/TCP,443:30954/TCP 45m
service/my-release-ingress-nginx-controller-admission ClusterIP 10.104.254.133 <none> 443/TCP 45m
service/whoareyou-service ClusterIP 10.108.79.85 <none> 80/TCP 110s
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.extensions/whoareyou-ingress <none> whoareyou.kvm.local 192.168.100.30 80 110sKita test hasil deploynya. Anggaplah whoareyou.kvm.local nama domain yg kita set dan 192.168.100.30 adalah public ip. Krn ini di local, cara testnya bisa kita masukin ke /etc/hosts komputer di luar cluster. Atau pakai curl dengan param header host. Disini aku test pakai curl
bash ➜ ~ curl -H “Host: whoareyou.kvm.local” 192.168.100.30 Hostname: whoareyou-deployment-74bff9c468-mlnjd IP: 127.0.0.1 IP: 10.244.1.7 RemoteAddr: 10.244.1.4:33058 GET / HTTP/1.1 Host: whoareyou.kvm.local User-Agent: curl/7.70.0 Accept: / X-Forwarded-For: 10.244.1.1 X-Forwarded-Host: whoareyou.kvm.local X-Forwarded-Port: 80 X-Forwarded-Proto: http X-Real-Ip: 10.244.1.1 X-Request-Id: bce405ea511051351247e01746104180 X-Scheme: http
Sip, sukses. Yg keluar adalah output dari pod whoareyou.
Bagaimana kalau kita pakai nama domain yg lain?
bash
➜ ~ curl -H "Host: sembarangan.com" 192.168.100.30
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.19.0</center>
</body>
</html>Outputnya adalah 404, karena ingress rule untuk domain sembarangan.com tidak ada.
Kesimpulannya adalah, kubeadm ini sangat memudahkan kita untuk setup kubernetes cluster. Banyak magic-nya. Aku senang dengan adanya kubeadm