kubernetes业务容器化案例（nginx+tomcat）
kubernetes 业务容器化案例（nginx+tomcat）
业务容器化优势：
- 提高资源利用率、节约部署成本
- 提供部署效率，基于kubernetes实现微服务的快速部署与交付、容器的批量调度与秒级启动
- 实现横向扩容、灰度部署、回滚、链路追踪、服务治理等
- 可根据业务负载进行自动弹性伸缩
- 容器将环境和代码打包再镜像内，保证了测试与生产环境的一致性
- 紧跟云原生社区技术发展的步伐，不给公司遗留技术债，为后期技术升级夯实了基础
- 为个人储备前言技术
业务容器化案例之一：业务规划及镜像分层构建：
开发将代码push到gitlab，通过jenkins将代码拉取到服务器，然后通过maven等编译工具进行编译，将代码编程成jar包或war包，然后通过dockerfile打包镜像，把镜像传到harbor仓库，通过k8s启动镜像
镜像分层
业务容器化案例之二：Nginx+Tomcat+NFS实现动静分离
分为这几步：
- Nginx基础镜像制作
- Nginx业务镜像制作
- Nginx业务镜像测试
- 在kubernetes环境运行nginx
- JDK基础镜像制作
- tomcat基础镜像制作
- tomcat业务镜像app1制作
- 在kubernetes环境运行tomcat
- 在kubernetes中nginx+tomcat实现动静分离
- 基于NFS实现数据共享
- 在后端服务生成数据并验证访问
1、系统镜像制作
这里用centos系统作为最底层镜像
[root@master1 centos]# ll
总用量 31848
-rw-r--r-- 1 root root 162 5月 2 2022 build-command.sh
-rw-r--r-- 1 root root 420 5月 2 2022 Dockerfile
-rw-r--r-- 1 root root 32600353 5月 18 2021 filebeat-7.12.1-x86_64.rpm
[root@master1 centos]#
[root@master1 centos]# cat Dockerfile
#自定义Centos 基础镜像
FROM centos:7.9.2009
MAINTAINER dujie 871578547@qq.com
ADD filebeat-7.12.1-x86_64.rpm /tmp
RUN yum install -y /tmp/filebeat-7.12.1-x86_64.rpm vim wget tree lrzsz gcc gcc-c++ automake pcre pcre-devel zlib zlib-devel openssl openssl-devel iproute net-tools iotop && rm -rf /etc/localtime /tmp/filebeat-7.12.1-x86_64.rpm && ln -snf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
[root@master1 centos]# cat build-command.sh
#!/bin/bash
docker build -t harbor.dujie.com/baseimages/dujie-centos-base:7.9.2009 .
docker push harbor.dujie.com/baseimages/dujie-centos-base:7.9.2009
[root@master1 centos]# sh build-command.sh
Sending build context to Docker daemon 32.6MB
Step 1/4 : FROM centos:7.9.2009
---> eeb6ee3f44bd
Step 2/4 : MAINTAINER dujie 871578547@qq.com
---> Using cache
---> d8a48c2ee619
Step 3/4 : ADD filebeat-7.12.1-x86_64.rpm /tmp
---> Using cache
---> b8f9ae020d7a
Step 4/4 : RUN yum install -y /tmp/filebeat-7.12.1-x86_64.rpm vim wget tree lrzsz gcc gcc-c++ automake pcre pcre-devel zlib zlib-devel openssl openssl-devel iproute net-tools iotop && rm -rf /etc/localtime /tmp/filebeat-7.12.1-x86_64.rpm && ln -snf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
---> Using cache
---> 5780d37bc1ef
Successfully built 5780d37bc1ef
Successfully tagged harbor.dujie.com/baseimages/dujie-centos-base:7.9.2009
The push refers to repository [harbor.dujie.com/baseimages/dujie-centos-base]
a2e5c471115e: Mounted from baseimages/magedu-centos-base
4d47db4a287f: Mounted from baseimages/magedu-centos-base
174f56854903: Mounted from baseimages/magedu-centos-base
7.9.2009: digest: sha256:5b3ea8aa35dee887a27cc7933b09f35b90ee40a4a7d6095bfbac485e5285da2b size: 954此时基础的系统镜像就已经成功上传到harbor上了
2、jdk基础镜像制作
[root@master1 pub-images]# cd jdk-1.8.212/
[root@master1 jdk-1.8.212]# ll
总用量 190456
-rw-r--r-- 1 root root 146 5月 2 2022 build-command.sh
-rw-r--r-- 1 root root 393 5月 2 2022 Dockerfile
-rw-r--r-- 1 root root 195013152 6月 22 2021 jdk-8u212-linux-x64.tar.gz
-rw-r--r-- 1 root root 2105 6月 22 2021 profile
[root@master1 jdk-1.8.212]# cat profile
...
export JAVA_HOME=/usr/local/jdk
export TOMCAT_HOME=/apps/tomcat
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$TOMCAT_HOME/bin:$PATH
export CLASSPATH=.$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$JAVA_HOME/lib/tools.jar
[root@master1 jdk-1.8.212]# cat Dockerfile
#JDK Base Image
FROM harbor.dujie.com/baseimages/dujie-centos-base:7.9.2009
MAINTAINER dujie "xxxx@qq.com"
ADD jdk-8u212-linux-x64.tar.gz /usr/local/src/
RUN ln -sv /usr/local/src/jdk1.8.0_212 /usr/local/jdk
ADD profile /etc/profile
ENV JAVA_HOME /usr/local/jdk
ENV JRE_HOME $JAVA_HOME/jre
ENV CLASSPATH $JAVA_HOME/lib/:$JRE_HOME/lib/
ENV PATH $PATH:$JAVA_HOME/bin
[root@master1 jdk-1.8.212]# cat build-command.sh
#!/bin/bash
docker build -t harbor.dujie.com/pub-images/jdk-base:v8.212 .
sleep 1
docker push harbor.dujie.com/pub-images/jdk-base:v8.212
# 构建镜像并推送到harbor
[root@master1 jdk-1.8.212]# sh build-command.sh
3、tomcat基础镜像制作
[root@master1 tomcat-base-8.5.43]# ll
总用量 9500
-rw-r--r-- 1 root root 9717059 6月 22 2021 apache-tomcat-8.5.43.tar.gz
-rw-r--r-- 1 root root 148 5月 9 14:21 build-command.sh
-rw-r--r-- 1 root root 322 5月 9 14:21 Dockerfile
[root@master1 tomcat-base-8.5.43]#
[root@master1 tomcat-base-8.5.43]#
[root@master1 tomcat-base-8.5.43]# cat build-command.sh
#!/bin/bash
docker build -t harbor.dujie.com/pub-images/tomcat-base:v8.5.43 .
sleep 3
docker push harbor.dujie.com/pub-images/tomcat-base:v8.5.43
[root@master1 tomcat-base-8.5.43]# cat Dockerfile
#Tomcat 8.5.43基础镜像
FROM harbor.dujie.com/pub-images/jdk-base:v8.212
MAINTAINER dujie "xx@163.com"
RUN mkdir /apps /data/tomcat/webapps /data/tomcat/logs -pv
ADD apache-tomcat-8.5.43.tar.gz /apps
RUN useradd tomcat -u 2050 && ln -sv /apps/apache-tomcat-8.5.43 /apps/tomcat && chown -R tomcat.tomcat /apps /data
# 制作镜像并推送到harbor
[root@master1 tomcat-base-8.5.43]# sh build-command.sh 4、nginx基础镜像制作
[root@master1 nginx-base]# ll
总用量 1048
-rw-r--r-- 1 root root 146 5月 9 14:26 build-command.sh
-rw-r--r-- 1 root root 436 5月 9 14:26 Dockerfile
-rw-r--r-- 1 root root 1062124 11月 16 2021 nginx-1.20.2.tar.gz
[root@master1 nginx-base]#
[root@master1 nginx-base]# cat Dockerfile
#Nginx Base Image
FROM harbor.dujie.com/baseimages/dujie-centos-base:7.9.2009
RUN yum install -y vim wget tree lrzsz gcc gcc-c++ automake pcre pcre-devel zlib zlib-devel openssl openssl-devel iproute net-tools iotop
ADD nginx-1.20.2.tar.gz /usr/local/src/
RUN cd /usr/local/src/nginx-1.20.2 && ./configure && make && make install && ln -sv /usr/local/nginx/sbin/nginx /usr/sbin/nginx &&rm -rf /usr/local/src/nginx-1.20.2.tar.gz
[root@master1 nginx-base]#
[root@master1 nginx-base]# cat build-command.sh
#!/bin/bash
docker build -t harbor.dujie.com/pub-images/nginx-base:v1.20.2 .
sleep 1
docker push harbor.dujie.com/pub-images/nginx-base:v1.20.2
# 制作镜像并推送到harbor
[root@master1 tomcat-base-8.5.43]# sh build-command.sh 5、tomcat应用镜像创建
[root@master1 tomcat-app1]# ll
总用量 23580
-rw-r--r-- 1 root root 154 5月 2 2022 app1.tar.gz
-rwxr-xr-x 1 root root 139 5月 9 14:36 build-command.sh
-rwxr-xr-x 1 root root 23611 6月 22 2021 catalina.sh
-rw-r--r-- 1 root root 534 5月 9 14:35 Dockerfile
-rw-r--r-- 1 root root 24086235 6月 22 2021 filebeat-7.5.1-x86_64.rpm
-rw-r--r-- 1 root root 667 10月 24 2021 filebeat.yml
-rw-r--r-- 1 root root 63 10月 24 2021 index.html
drwxr-xr-x 2 root root 24 5月 2 2022 myapp
-rwxr-xr-x 1 root root 373 5月 2 2022 run_tomcat.sh
-rw-r--r-- 1 root root 6462 10月 10 2021 server.xml
# 打镜像
[root@master1 tomcat-app1]# cat Dockerfile
#tomcat web1
FROM harbor.dujie.com/pub-images/tomcat-base:v8.5.43
# 将 catalina 文件发送到容器中（增加启动参数时可以这样）
ADD catalina.sh /apps/tomcat/bin/catalina.sh
# tomcat主配置文件
ADD server.xml /apps/tomcat/conf/server.xml
#ADD myapp/* /data/tomcat/webapps/myapp/
# 应用代码
ADD app1.tar.gz /data/tomcat/webapps/myapp/
# 启动应用的脚本
ADD run_tomcat.sh /apps/tomcat/bin/run_tomcat.sh
#ADD filebeat.yml /etc/filebeat/filebeat.yml
RUN chown -R tomcat.tomcat /data/ /apps/
#ADD filebeat-7.5.1-x86_64.rpm /tmp/
#RUN cd /tmp && yum localinstall -y filebeat-7.5.1-amd64.deb
# 暴露端口
EXPOSE 8080 8443
# 容器中执行启动命令
CMD ["/apps/tomcat/bin/run_tomcat.sh"]
# 启动容器的脚本
[root@master1 tomcat-app1]# cat run_tomcat.sh
#!/bin/bash
#echo "nameserver 223.6.6.6" > /etc/resolv.conf
#echo "192.168.7.248 k8s-vip.example.com" >> /etc/hosts
#/usr/share/filebeat/bin/filebeat -e -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat -path.logs /var/log/filebeat &
su - tomcat -c "/apps/tomcat/bin/catalina.sh start"
tail -f /etc/hosts
# 构建镜像并推送到harbor
[root@master1 tomcat-app1]# cat build-command.sh
#!/bin/bash
TAG=$1
docker build -t harbor.dujie.com/app/tomcat-app1:${TAG} .
sleep 3
docker push harbor.dujie.com/app/tomcat-app1:${TAG}6、nginx应用镜像创建
[root@master1 nginx]# ll
总用量 20
-rw-r--r-- 1 root root 234 6月 22 2021 app1.tar.gz
-rwxr-xr-x 1 root root 220 5月 9 14:33 build-command.sh
-rw-r--r-- 1 root root 483 5月 9 14:32 Dockerfile
-rw-r--r-- 1 root root 35 5月 2 2022 index.html
-rw-r--r-- 1 root root 3147 5月 9 15:11 nginx.conf
drwxr-xr-x 2 root root 24 5月 2 2022 webapp
[root@master1 nginx]# cat Dockerfile
#Nginx 1.20.2
FROM harbor.dujie.com/pub-images/nginx-base:v1.20.2
RUN useradd tomcat -u 2050
ADD nginx.conf /usr/local/nginx/conf/nginx.conf
ADD app1.tar.gz /usr/local/nginx/html/webapp/
ADD index.html /usr/local/nginx/html/index.html
#静态资源挂载路径
RUN mkdir -p /usr/local/nginx/html/webapp/static /usr/local/nginx/html/webapp/images && chown tomcat.tomcat -R /usr/local/nginx/html/webapp/static /usr/local/nginx/html/webapp/images
EXPOSE 80 443
CMD ["nginx"]
[root@master1 nginx]# cat nginx.conf
user tomcat tomcat;
worker_processes auto;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#pid logs/nginx.pid;
daemon off;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
#access_log logs/access.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
upstream tomcat_webserver {
# 这里要写后端tomcat的svc名字
server magedu-tomcat-app1-service.magedu.svc.cluster.local:80;
}
server {
listen 80;
server_name localhost;
location / {
root html;
index index.html index.htm;
}
location /webapp {
root html;
index index.html index.htm;
}
location /myapp {
proxy_pass http://tomcat_webserver;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
[root@master1 nginx]# cat build-command.sh
#!/bin/bash
TAG=$1
docker build -t harbor.dujie.com/app/nginx-web1:${TAG} .
echo "镜像构建完成，即将上传到harbor"
sleep 1
docker push harbor.dujie.com/app/nginx-web1:${TAG}
echo "镜像上传到harbor完成"
部署到k8s
tomcat yaml文件
[root@master1 tomcat-app1]# cat tomcat-app1.yaml
kind: Deployment
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
metadata:
labels:
app: magedu-tomcat-app1-deployment-label
name: magedu-tomcat-app1-deployment
namespace: magedu
spec:
replicas: 2
selector:
matchLabels:
app: magedu-tomcat-app1-selector
template:
metadata:
labels:
app: magedu-tomcat-app1-selector
spec:
containers:
- name: magedu-tomcat-app1-container
image: harbor.dujie.com/app/tomcat-app1:v1
#command: ["/apps/tomcat/bin/run_tomcat.sh"]
imagePullPolicy: IfNotPresent
#imagePullPolicy: Always
ports:
- containerPort: 8080
protocol: TCP
name: http
env:
- name: "password"
value: "123456"
- name: "age"
value: "18"
resources:
limits:
cpu: 1
memory: "512Mi"
requests:
cpu: 500m
memory: "512Mi"
volumeMounts:
- name: dujie-images
mountPath: /usr/local/nginx/html/webapp/images
readOnly: false
- name: dujie-static
mountPath: /usr/local/nginx/html/webapp/static
readOnly: false
volumes:
- name: dujie-images
nfs:
server: 192.168.154.130
path: /data/k8sdata/dujie/images
- name: dujie-static
nfs:
server: 192.168.154.130
path: /data/k8sdata/dujie/static
# nodeSelector:
# project: magedu
# app: tomcat
---
kind: Service
apiVersion: v1
metadata:
labels:
app: magedu-tomcat-app1-service-label
name: magedu-tomcat-app1-service
namespace: magedu
spec:
#type: NodePort
ports:
- name: http
port: 80
protocol: TCP
targetPort: 8080
#nodePort: 30092
selector:
app: magedu-tomcat-app1-selectornginxyaml文件
[root@master1 nginx]# cat nginx.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
labels:
app: magedu-nginx-deployment-label
name: magedu-nginx-deployment
namespace: magedu
spec:
replicas: 1
selector:
matchLabels:
app: magedu-nginx-selector
template:
metadata:
labels:
app: magedu-nginx-selector
spec:
containers:
- name: magedu-nginx-container
image: harbor.dujie.com/app/nginx-web1:v1
#command: ["/apps/tomcat/bin/run_tomcat.sh"]
#imagePullPolicy: IfNotPresent
imagePullPolicy: Always
ports:
- containerPort: 80
protocol: TCP
name: http
- containerPort: 443
protocol: TCP
name: https
env:
- name: "password"
value: "123456"
- name: "age"
value: "20"
resources:
limits:
cpu: 500m
memory: 500Mi
requests:
cpu: 500m
memory: 500Mi
volumeMounts:
- name: magedu-images
mountPath: /usr/local/nginx/html/webapp/images
readOnly: false
- name: magedu-static
mountPath: /usr/local/nginx/html/webapp/static
readOnly: false
volumes:
- name: magedu-images
nfs:
server: 192.168.154.130
path: /data/k8sdata/dujie/images
- name: magedu-static
nfs:
server: 192.168.154.130
path: /data/k8sdata/dujie/static
#nodeSelector:
# group: magedu
---
kind: Service
apiVersion: v1
metadata:
labels:
app: magedu-nginx-service-label
name: magedu-nginx-service
namespace: magedu
spec:
type: NodePort
ports:
- name: http
port: 80
protocol: TCP
targetPort: 80
nodePort: 30090
- name: https
port: 443
protocol: TCP
targetPort: 443
nodePort: 30091
selector:
app: magedu-nginx-selector验证
[root@master1 nginx]# kubectl get pods -n magedu
NAME READY STATUS RESTARTS AGE
magedu-nginx-deployment-5476cbb84f-jq5ch 1/1 Running 0 25m
magedu-tomcat-app1-deployment-685d9fb5f4-bvhn2 1/1 Running 0 36m
magedu-tomcat-app1-deployment-685d9fb5f4-mq7z8 1/1 Running 0 34m
[root@master1 nginx]#
[root@master1 nginx]#
[root@master1 nginx]# kubectl exec -it ^C
[root@master1 nginx]# kubectl get svc -n magedu
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
magedu-nginx-service NodePort 10.100.229.4 <none> 80:30090/TCP,443:30091/TCP 25m
magedu-tomcat-app1-service ClusterIP 10.100.151.235 <none> 80/TCP 38m
[root@master1 nginx]# kubectl exec -it magedu-nginx-deployment-5476cbb84f-jq5ch bash -n magedu
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
[root@magedu-nginx-deployment-5476cbb84f-jq5ch /]#
[root@magedu-nginx-deployment-5476cbb84f-jq5ch /]#
[root@magedu-nginx-deployment-5476cbb84f-jq5ch /]#
[root@magedu-nginx-deployment-5476cbb84f-jq5ch /]#
[root@magedu-nginx-deployment-5476cbb84f-jq5ch /]# curl -Lv magedu-tomcat-app1-service.magedu.svc.cluster.local/myapp
* About to connect() to magedu-tomcat-app1-service.magedu.svc.cluster.local port 80 (#0)
* Trying 10.100.151.235...
* Connected to magedu-tomcat-app1-service.magedu.svc.cluster.local (10.100.151.235) port 80 (#0)
> GET /myapp HTTP/1.1
> User-Agent: curl/7.29.0
> Host: magedu-tomcat-app1-service.magedu.svc.cluster.local
> Accept: */*
>
< HTTP/1.1 302
< Location: /myapp/
< Transfer-Encoding: chunked
< Date: Tue, 09 May 2023 07:39:26 GMT
<
* Ignoring the response-body
* Connection #0 to host magedu-tomcat-app1-service.magedu.svc.cluster.local left intact
* Issue another request to this URL: 'HTTP://magedu-tomcat-app1-service.magedu.svc.cluster.local/myapp/'
* Found bundle for host magedu-tomcat-app1-service.magedu.svc.cluster.local: 0x1ded040
* Re-using existing connection! (#0) with host magedu-tomcat-app1-service.magedu.svc.cluster.local
* Connected to magedu-tomcat-app1-service.magedu.svc.cluster.local (10.100.151.235) port 80 (#0)
> GET /myapp/ HTTP/1.1
> User-Agent: curl/7.29.0
> Host: magedu-tomcat-app1-service.magedu.svc.cluster.local
> Accept: */*
>
< HTTP/1.1 200
< Accept-Ranges: bytes
< ETag: W/"35-1651482222000"
< Last-Modified: Mon, 02 May 2022 09:03:42 GMT
< Content-Type: text/html
< Content-Length: 35
< Date: Tue, 09 May 2023 07:39:26 GMT
<
<h1>tomcat app1 for linux n66</h1>
* Connection #0 to host magedu-tomcat-app1-service.magedu.svc.cluster.local left intact
[root@magedu-nginx-deployment-5476cbb84f-jq5ch /]# 本博客所有文章除特别声明外，均采用 CC BY-NC-SA 4.0 许可协议。转载请注明来自 J.のblog！
微信
支付宝
相关推荐
评论
