728x90
SMALL
구성 요소 및 버전 (2025년 기준)
| 구성 요소 | Version |
| OS | Rocky Linux 8.10 |
| Kubernetes | v1.30.0 |
| containerd | v1.7.13 |
| crictl | v1.30.0 |
| runc | v1.1.12 |
| CNI 플러그인 | flannel v0.24.2 |
| 내부 Registry | Harbor v2.10.3 (포트 5000) |
✅ 환경 개요
- 개방망 환경: 인터넷 연결 가능, 설치 파일 및 이미지 수집
- 폐쇄망 환경: 인터넷 연결 불가, Harbor를 통한 내부 이미지 관리
- 구성: master 1대 + worker 3대 클러스터
- 전제조건: Harbor 설정 완료 (https://sungbin-park.tistory.com/154) / port 5000 설정
0. 설치 환경 변수 등록
cat <<EOF > /etc/profile.d/k8s-env.sh
export K8S_VERSION=v1.30.0
export CONTAINERD_VERSION=1.7.13
export RUNC_VERSION=1.1.12
export CRICTL_VERSION=1.30.0
export FLANNEL_VERSION=v0.24.2
export HARBOR_VERSION=v2.10.3
export HARBOR_HOST=harbor.local
export HARBOR_PORT=5000
export REGISTRY_ADDR="$HARBOR_HOST:$HARBOR_PORT"
export POD_SUBNET=10.244.0.0/16
EOF
source /etc/profile.d/k8s-env.sh1. 개방망에서 설치 번들 생성
1-1. 디렉토리 생성
mkdir -p ~/k8s-offline-bundle/{binaries,cri,images/docker-save,rpms,addons,scripts}1-2. Kubernetes 바이너리 수집
cd ~/k8s-offline-bundle/binaries
for cmd in kubeadm kubectl kubelet; do
curl -LO https://dl.k8s.io/release/${K8S_VERSION}/bin/linux/amd64/$cmd
chmod +x $cmd
done1-3. Container runtime 수집
# containerd, runc, crictl
cd ~/k8s-offline-bundle/cri
curl -LO https://github.com/containerd/containerd/releases/download/v${CONTAINERD_VERSION}/containerd-${CONTAINERD_VERSION}-linux-amd64.tar.gz
curl -LO https://github.com/opencontainers/runc/releases/download/v${RUNC_VERSION}/runc.amd64
curl -LO https://github.com/kubernetes-sigs/cri-tools/releases/download/v${CRICTL_VERSION}/crictl-v${CRICTL_VERSION}-linux-amd64.tar.gz1-4. 필수 RPM 패키지 수집
cd ~/k8s-offline-bundle/rpms
dnf install --downloadonly --downloaddir=. \
vim git wget curl net-tools bash-completion \
ipvsadm ipset sysstat conntrack \
yum-utils device-mapper-persistent-data lvm2 \
socat ebtables ethtool container-selinux epel-release1-5. 이미지 목록 작성 및 저장
cat <<EOF > ~/k8s-offline-bundle/images/image-list.txt
registry.k8s.io/kube-apiserver:${K8S_VERSION}
registry.k8s.io/kube-controller-manager:${K8S_VERSION}
registry.k8s.io/kube-scheduler:${K8S_VERSION}
registry.k8s.io/kube-proxy:${K8S_VERSION}
registry.k8s.io/pause:3.10
registry.k8s.io/etcd:3.5.10-0
registry.k8s.io/coredns/coredns:v1.11.1
docker.io/flannel/flannel:${FLANNEL_VERSION}
EOF
cd ~/k8s-offline-bundle/images
while read img; do
docker pull "$img"
tagged_img=${REGISTRY_ADDR}/$(echo "$img" | cut -d'/' -f2-)
docker tag "$img" "$tagged_img"
docker push "$tagged_img"
docker save "$tagged_img" -o "docker-save/$(echo "$tagged_img" | tr '/:' '_').tar"
done < image-list.txt1-6. CNI Plugin 다운로드. CNI Plugin 다운로드
cd ~/k8s-offline-bundle/addons
curl -LO https://raw.githubusercontent.com/flannel-io/flannel/${FLANNEL_VERSION}/manifests/kube-flannel.yml1-7. 번들 압축 및 전송
cd ~/k8s-offline-bundle
tar czvf k8s-offline-bundle.tar.gz *
[root@rocky84 k8s-offline-bundle]# tree -L 2
.
├── addons
│ └── kube-flannel.yml
├── binaries
│ ├── kubeadm
│ ├── kubectl
│ └── kubelet
├── cri
│ ├── containerd-1.7.13-linux-amd64.tar.gz
│ ├── crictl-v1.30.0-linux-amd64.tar.gz
│ └── runc.amd64
├── images
│ ├── docker-save
│ ├── image-list.txt
│ └── k8s_core_images.tar
├── k8s-offline-bundle.tar.gz
├── rpms
│ ├── conntrack-tools-1.4.4-11.el8.x86_64.rpm
│ ├── epel-release-8-18.el8.noarch.rpm
│ ├── gpm-libs-1.20.7-17.el8.x86_64.rpm
│ ├── ipvsadm-1.31-1.el8.x86_64.rpm
│ ├── libnetfilter_cthelper-1.0.0-15.el8.x86_64.rpm
│ ├── libnetfilter_cttimeout-1.0.0-11.el8.x86_64.rpm
│ ├── libnetfilter_queue-1.0.4-3.el8.x86_64.rpm
│ ├── lm_sensors-libs-3.4.0-23.20180522git70f7e08.el8.x86_64.rpm
│ ├── socat-1.7.4.1-1.el8.x86_64.rpm
│ ├── sysstat-11.7.3-13.el8_10.x86_64.rpm
│ ├── vim-common-8.0.1763-19.el8_6.4.x86_64.rpm
│ ├── vim-enhanced-8.0.1763-19.el8_6.4.x86_64.rpm
│ ├── vim-filesystem-8.0.1763-19.el8_6.4.noarch.rpm
│ └── yum-utils-4.0.21-25.el8.noarch.rpm
└── scripts
7 directories, 24 files
2. 폐쇄망에서 설치
2-1. 번들 압축 해제
mkdir -p /opt/k8s-offline
cd /opt/k8s-offline
tar xzvf /경로/k8s-offline-bundle.tar.gz2-2. 로컬 YUM 저장소 구성
cat <<EOF > /etc/yum.repos.d/sungbin_2025_0415.repo
[sungbin-offline]
name=Sungbin Offline Local Repo
baseurl=file:///opt/k8s-offline/rpms
enabled=1
gpgcheck=0
EOF
dnf clean all
# 인터넷 연결이 없으므로 외부 repo 비활성화 필요
dnf install -y *.rpm --disablerepo='*' --enablerepo='sungbin-offline'
##############################################################
저장소 ID 저장소 이름
appstream Rocky Linux 8 - AppStream
baseos Rocky Linux 8 - BaseOS
docker-ce-stable Docker CE Stable - x86_64
extras Rocky Linux 8 - Extras
sungbin-offline Sungbin Offline Local Repo2-3. 패키지 및 바이너리 설치
cp /opt/k8s-offline/binaries/kube* /usr/local/bin/
chmod +x /usr/local/bin/kube*2-4. containerd 설치 및 Harbor 설정
cd /opt/k8s-offline/cri
tar xzvf containerd-*.tar.gz -C /usr/local
install -m 755 runc.amd64 /usr/local/sbin/runc
tar xzvf crictl-*.tar.gz -C /usr/local/bin
mkdir -p /etc/containerd
containerd config default > /etc/containerd/config.toml
# Harbor Mirror 설정 (비밀번호 인증 시 user:pass@ 추가 가능)
sed -i '/\[plugins."io.containerd.grpc.v1.cri"\]/a \\t[plugins."io.containerd.grpc.v1.cri".registry.mirrors."harbor.local:5000"]\n\\t\\tendpoint = ["http://harbor.local:5000"]' /etc/containerd/config.toml
# ⚠️ 해당 mirror 설정은 containerd가 image를 pull할 때만 적용
# ctr로 import한 이미지만으로는 kubelet이 인식하지 못합니다.
# 반드시 kubeadm config 파일에 imageRepository를 명시
# kubelet이 참조하는 이미지 확인
crictl images
# 이미지 repository 설정 확인
cat /etc/kubeadm/config.yaml | grep imageRepository
systemctl daemon-reexec
systemctl daemon-reload
systemctl enable --now containerd2-5. 이미지 import
cd /opt/k8s-offline/images
ctr -n k8s.io images import k8s_core_images.tar
# 확인
ctr -n k8s.io images ls
# 예시
ctr -n k8s.io images ls
REF TYPE DIGEST SIZE PLATFORMS LABELS
docker.io/flannel/flannel:v0.24.2 application/vnd.oci.image.manifest.v1+json sha256:a9a85cffd6984f14b78f4689d73f60ea8b160605ff7a5be442c505bfba2c7f92 73.6 MiB linux/amd64 -
registry.k8s.io/coredns/coredns:v1.11.1 application/vnd.oci.image.manifest.v1+json sha256:1b36416706e3fde0bcc40d9b10d66a84dffc740fba37c9deb6e8df5455405cd1 58.4 MiB linux/amd64 -
registry.k8s.io/etcd:3.5.10-0 application/vnd.oci.image.manifest.v1+json sha256:0722a4ca9b42aeee73156b6d5936bf3fc22578b358a9403d069c174455f281a6 142.0 MiB linux/amd64 -
registry.k8s.io/kube-apiserver:v1.30.0 application/vnd.oci.image.manifest.v1+json sha256:5fa8706388c95f50f72d4f396fa27e0daf7e2d0e626b1362cb94014780338bf8 112.2 MiB linux/amd64 -
registry.k8s.io/kube-controller-manager:v1.30.0 application/vnd.oci.image.manifest.v1+json sha256:5ad3066d1a00472009788196eda2099ccae53f73b04f7cdbc6d238dd4db6bdef 107.0 MiB linux/amd64 -
registry.k8s.io/kube-proxy:v1.30.0 application/vnd.oci.image.manifest.v1+json sha256:15ae79def5c0ce0502924319d694c4908a3adade5ed427341690f817e3dbaa5c 81.9 MiB linux/amd64 -
registry.k8s.io/kube-scheduler:v1.30.0 application/vnd.oci.image.manifest.v1+json sha256:36b0a4ad4ddec29dff5bbf5165d8674826e153f00833ab2774de40d5642617dd 60.1 MiB linux/amd64 -
registry.k8s.io/pause:3.10 application/vnd.oci.image.manifest.v1+json sha256:a883b8d67f5fe8ae50f857fb4c11c789913d31edff664135b9d4df44d3cb85cb 721.8 KiB linux/amd64 -2-6. 시스템 준비
# SELinux 비활성화
setenforce 0
sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config
# swap 비활성화
swapoff -a
sed -i '/swap/d' /etc/fstab
# 설정 보강
modprobe br_netfilter
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system
# kubelet systemd 설정 강화
mkdir -p /etc/systemd/system/kubelet.service.d
cat <<EOF > /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
[Service]
Environment="KUBELET_EXTRA_ARGS=--cgroup-driver=systemd"
EOF
systemctl daemon-reexec
systemctl daemon-reload
systemctl restart kubelet
###########################################################################################
# /etc/hosts 에, master 와 worker 들 등록
cat <<EOF >> /etc/hosts
192.168.0.10 master
192.168.0.11 worker1
192.168.0.12 worker2
192.168.0.13 worker3
EOF2-7. kubeadm config.yaml 작성
mkdir -p /etc/kubeadm
cat <<EOF > /etc/kubeadm/config.yaml
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
kubernetesVersion: v1.30.0
imageRepository: harbor.local:5000/registry.k8s.io
networking:
podSubnet: 10.244.0.0/16
nodeRegistration:
kubeletExtraArgs:
cgroup-driver: systemd
EOF2-8. Kubernetes 마스터 초기화
kubeadm config images list --config=/etc/kubeadm/config.yaml
kubeadm init --config=/etc/kubeadm/config.yaml
mkdir -p $HOME/.kube
cp /etc/kubernetes/admin.conf $HOME/.kube/config
kubectl apply -f /opt/k8s-offline/addons/kube-flannel.yml2-9. 워커 노드 Join
# 마스터에서 출력된 join 명령 사용 또는 아래로 재생성
kubeadm token create --print-join-command
# 마스터 노드에서 출력된 join 명령어 사용
# kubeadm join <MASTER_IP>:6443 --token <TOKEN> \
# --discovery-token-ca-cert-hash sha256:<HASH>
.
LIST
'🔹Kubernetes' 카테고리의 다른 글
| Argo CD / Argo Rollouts / Argo Workflows UI 가이드 (0) | 2025.04.16 |
|---|---|
| [Rocky Linux 8.10] Harbor v2.10.3 설치 및 구축 가이드 (0) | 2025.04.15 |
| 쿠버네티스 워크로드 선택 기준 (0) | 2025.02.28 |
| taint와 toleration을 사용하는 이유 (0) | 2025.02.27 |
| Kubernetes 워크로드 리소스들의 특징 및 종속성 (0) | 2025.02.25 |