Kubernetes部署renren-fast开源项目

renren-fast是一个开源的前后端分离项目，前端使用vue框架，后端采用springboot。
在刚接触完Kubernetes后，使用Kubernetes部署renren-fast项目练练手。

获取renren-fast

#获取后端
git clone https://git.oschina.net/renrenio/renren-fast.git
#获取前端
git clone https://github.com/daxiongYang/renren-fast-vue.git

部署后端

修改mysql配置

修改application-dev.yml的mysql配置项，我主要修改了以下几项,此处headless-mysql是我的无头服务的名称，renren-fast是库名

url: jdbc:mysql://headless-mysql:3306/renren-fast?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai
username: root
password: 123456

修改redis配置

修改application.yml的redis配置项，我主要修改了以下几处，此处headless-redis是我的无头服务名称，password是客户端连接密码

host: headless-redis
password: 123456

打包

mvn clean install

部署nfs的storageclass

（1）安装nfs

# 安装nfs
yum -y install nfs-utils rpcbind
#创建nfs共享目录及设置权限
mkdir /nfs/renren -p
chmod 755 /nfs/renren -R
#配置nfs
cat /etc/exports
/nfs/renren *(async,insecure,no_root_squash,no_subtree_check,rw)

（2）启动nfs

systemctl start rpcbind
systemctl enable rpcbind
systemctl status rpcbind
systemctl start nfs
systemctl enable nfs
systemctl status nfs

（3）客户端配置
客户端在使用nfs时，需要安装nfs

#在客户端上安装启动nfs
yum -y install nfs-utils rpcbind
systemctl start rpcbind
systemctl enable rpcbind
systemctl start nfs
systemctl enable nfs
#查看nfs共享
showmount -e k8s-master

（4）创建基于nfs的storageclass

git clone https://github.com/kubernetes-retired/external-storage.git

主要部署下面这些资源清单，只有deployment.yaml需要修改nfs相关配置

# 部署deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  labels:
    app: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: quay.io/external_storage/nfs-client-provisioner:latest
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: fuseim.pri/ifs
            - name: NFS_SERVER
              value: 192.168.126.130  #修改nfs服务器地址
            - name: NFS_PATH
              value: /nfs/renren       #nfs目录
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.126.130  #修改nfs服务器地址
            path: /nfs/renren        #nfs目录

kubectl apply -f deployment.yaml
#部署rbac
kubectl apply -f rbac.yaml
#部署storageclass
kubectl apply -f class.yaml
#查看创建的资源
kubectl get pods

部署mysql

需要提前创建好namespace，我创建的是renren-fast
（1）创建pvc

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-mysql
  namespace: renren-fast
  annotations:
    volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
  volumeMode: Filesystem
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 5Gi

kubectl apply -f pvc-mysql.yaml

（2）创建secret

apiVersion: v1
kind: Secret
metadata:
  name: secret-mysql
  namespace: renren-fast
data:
  password: dGFuZzE2MTE=
stringData:
  username: root

kubectl apply -f secret-mysql.yaml

（3）创建configmap

mkdir -p /nfs/renren/conf_mysql
kubectl create configmap mysql-configmap --from-file=/nfs/renren/conf_mysql -n renren-fast

需要提前将mysql容器中的配置文件准备好

（4）创建headless

apiVersion: v1
kind: Service
metadata:
  name: headless-mysql
  namespace: renren-fast
spec:
  clusterIP: None
  selector:
    app: statefulset-mysql
  ports:
    - port: 3306
    
kubectl apply -f headless-mysql.yaml

（5）创建statefulset

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: statefulset-mysql
  namespace: renren-fast
spec:
  selector:
    matchLabels:
      app: statefulset-mysql
  serviceName: headless-mysql
  template:
    metadata:
      labels:
        app: statefulset-mysql
    spec:
      volumes:
        - name: mysql-data
          persistentVolumeClaim:
            claimName: pvc-mysql
        - name: mysql-conf
          configMap:
            name: mysql-configmap
            items:
              - key: mysql.cnf
                path: mysql.cnf
              - key: mysqld.cnf
                path: mysqld.cnf
              - key: mysqldump.cnf
                path: mysqldump.cnf
              - key: docker.cnf
                path: docker.cnf
      containers:
        - name: mysql
          image: mysql:5.6
          ports:
            - containerPort: 3306
          env:
            - name: MYSQL_ROOT_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: secret-mysql
                  key: password
          volumeMounts:
            - mountPath: /var/lib/mysql
              name: mysql-data
            - mountPath: /etc/mysql/conf.d/mysql.cnf
              subPath: mysql.cnf
              name: mysql-conf
            - mountPath: /etc/mysql/conf.d/mysqldump.cnf
              subPath: mysqldump.cnf
              name: mysql-conf
            - mountPath: /etc/mysql/conf.d/docker.cnf
              subPath: docker.cnf
              name: mysql-conf
            - mountPath: /etc/mysql/mysql.conf.d/mysqld.cnf
              subPath: mysqld.cnf
              name: mysql-conf
              
kubectl apply -f statefulset-mysql.yaml

部署完成mysql后，可以登录进去，将mysql.sql导入进去，库名是renren-fast，mysql.sql文件在renren-fast\db里面

部署redis

（1）创建pvc

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-redis
  namespace: renren-fast
  annotations:
    volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
  accessModes:
    - ReadWriteMany
  volumeMode: Filesystem
  resources:
    requests:
      storage: 2Gi
      
kubectl apply -f pvc-redis.yaml

（2）创建configmap

mkdir -p /nfs/renren/conf_redis
kubectl create configmap redis-configmap --from-file=redis.conf -n renren-fast

需要提前将redis.conf准备好，相关pid、log、rdb、aof相关的文件我都放到了容器的/data目录下，redis密码记得改成与项目配置文件的一致
（3）创建headless

apiVersion: v1
kind: Service
metadata:
  name: headless-redis
  namespace: renren-fast
spec:
  clusterIP: None
  selector:
    app: statefulset-redis
  ports:
    - port: 6379
    
kubectl apply -f headless-redis.yaml

（4）创建statefulset

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: statefulset-redis
  namespace: renren-fast
spec:
  selector:
    matchLabels:
      app: statefulset-redis
  serviceName: headless-redis
  template:
    metadata:
      labels:
        app: statefulset-redis
    spec:
      volumes:
        - name: redis-conf
          configMap:
            name: redis-configmap
        - name: redis-data
          persistentVolumeClaim:
            claimName: pvc-redis
      containers:
        - name: redis
          image: redis:5.0.5
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 6379
          volumeMounts:
            - mountPath: /data/redis.conf
              name: redis-conf
              subPath: redis.conf
            - mountPath: /data
              name: redis-data
          command:
            - redis-server
            - /data/redis.conf
              
kubectl apply -f statefulset-redis.yaml

部署java环境

（1）我这边制作了一个简单的dockerfile，将打包的renren-fast.jar与dockerfile放在同级目录

FROM openjdk:8
RUN mkdir -p /app
WORKDIR /app
COPY renren-fast.jar .
EXPOSE 8080
CMD java -jar renren-fast.jar >> renren.log

#build
docker build -t my-java-app .

（2）创建pvc
需要将打包好的renren-fast.jar放入这个pvc生成的目录下

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-java
  namespace: renren-fast
  annotations:
    volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
  accessModes:
    - ReadWriteMany
  volumeMode: Filesystem
  resources:
    requests:
      storage: 1Gi
      
kubectl apply -f pvc-java.yaml

（3）创建deployment-java

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-java
  namespace: renren-fast
spec:
  replicas: 2
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      volumes:
        - name: myapp-jar
          persistentVolumeClaim:
            claimName: pvc-java
      containers:
        - name: java
          image: my-java-app
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8080
          volumeMounts:
            - mountPath: /app
              name: myapp-jar
        
kubectl apply -f deployment-java.yaml

（4）部署service-java

apiVersion: v1
kind: Service
metadata:
  name: service-java
  namespace: renren-fast
spec:
  type: NodePort
  selector:
    app: myapp
  ports:
    - port: 8080
    
kubectl apply -f service-java.yaml
#获取svc,拿到nodeport暴露的端口
kubectl get svc -n renren-fast

测试后端部署是否成功

在浏览器输入 宿主机ip:nodeport/renren-fast/swagger/index.html,例如http://192.168.126.130:31375/renren-fast/swagger/index.html

部署前端

修改renren-fast-vue\static\config\index-prod.js，修改下面配置，将ip、port改成对应的即可

// api接口请求地址
window.SITE_CONFIG['baseUrl'] = 'http://192.168.126.130:31375/renren-fast';

#安装依赖，可能由于网络原因，部份依赖安装失败，可以尝试安装cnpm，然后使用cnpm安装依赖
npm install
#打包
npm run build

打包成功后会生成一个dist目录
（1）创建pvc
需要将打包出来的dist目录放入这个pvc生成的目录下

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-nginx
  namespace: renren-fast
  annotations:
    volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
  accessModes:
    - ReadWriteMany
  volumeMode: Filesystem
  resources:
    requests:
      storage: 1Gi
      
kubectl apply -f pvc-nginx.yaml

（2）创建configmap

mkdir -p /nfs/renren/conf_nginx
kubectl create configmap nginx-configmap --from-file=/nfs/renren/conf_nginx -n renren-fast

需要提前准备nginx配置文件，我这边就是要nginx.conf和default.conf，修改default.conf

（3）deployment-nginx

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-nginx
  namespace: renren-fast
spec:
  replicas: 2
  selector:
    matchLabels:
      app: deployment-nginx
  template:
    metadata:
      labels:
        app: deployment-nginx
    spec:
      volumes:
        - name: nginx-conf
          configMap:
            name: nginx-configmap
            items:
              - key: nginx.conf
                path: nginx.conf
              - key: default.conf
                path: default.conf
        - name: nginx-data
          persistentVolumeClaim:
            claimName: pvc-nginx
      containers:
        - name: nginx
          image: nginx
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 80
          volumeMounts:
            - mountPath: /etc/nginx/conf.d/default.conf
              name: nginx-conf
              subPath: default.conf
            - mountPath: /etc/nginx/nginx.conf
              name: nginx-conf
              subPath: nginx.conf
            - mountPath: /usr/share/nginx/html
              name: nginx-data
              
kubectl apply -f deployment-nginx.yaml

（4）创建service-nginx

apiVersion: v1
kind: Service
metadata:
  name: service-nginx
  namespace: renren-fast
spec:
  type: NodePort
  selector:
    app: deployment-nginx
  ports:
    - port: 80
    
kubectl apply -f service-nginx.yaml
#获取svc,拿到nodeport暴露的端口
kubectl get svc -n renren-fast

测试

到此整个项目都部署完成了，浏览器输入 ip:nodeport 即可访问

用户名 admin 密码 admin