{"id":1312,"date":"2023-02-26T11:04:39","date_gmt":"2023-02-26T07:34:39","guid":{"rendered":"https:\/\/mshaeri.com\/blog\/?p=1312"},"modified":"2023-03-21T14:24:42","modified_gmt":"2023-03-21T10:54:42","slug":"local-ci-di-pipeline-with-gitblit-jenkins-kubernetes-microservice-architecture-react-spring-boot-flask-part-1","status":"publish","type":"post","link":"https:\/\/mshaeri.com\/blog\/local-ci-di-pipeline-with-gitblit-jenkins-kubernetes-microservice-architecture-react-spring-boot-flask-part-1\/","title":{"rendered":"Local CI\/CD Pipeline With Gitblit + Jenkins + Kubernetes For Microservice Architecture React + Spring Boot + Flask Part 1"},"content":{"rendered":"\n

A local CI\/CD pipeline offers several benefits for software development teams. It can help improve the security of your code by keeping sensitive information on companies local machine. This reduces the risk of security breaches and other issues that can arise when code is stored on a remote server. Moreover, setting up a local CI\/CD pipeline can be more cost-effective than using a remote pipeline, especially for small projects. Using open-source tools and running the pipeline on a local server or virtual machine, you can save money on hosting and infrastructure costs.<\/p>\n\n\n\n

By the recent change in Docker hub account policy, if accounts do not upgrade to a paid plan before April 14, 2023, their organization's images may be deleted after 30 days. <\/pre>\n\n\n\n
In this article we go through following steps to setup a local CI\/CD pipeline for your small to medium sized team :<\/p>\n\n\n\n
  1. Setup Kubernetes<\/a><\/li>
  2. Setup local docker registry on Kubernetes<\/a><\/li>
  3. Setup Gitblit on Kubernetes<\/a><\/li>
  4. Setup Jenkins on Kubernetes<\/a><\/li>
  5. Configure Spring boot Backend pipeline in Jenkins<\/li>
  6. Configure Flask Backend pipeline in Jenkins<\/li>
  7. Configure React Frontend pipeline in Jenkins<\/li>
  8. Code in Frontend and Backend projects<\/li>
  9. Run Frontend and Backend pipelines<\/li><\/ol>\n\n\n\n
    We are going to setup all the CI\/CD nodes including Source code repository (Gitblit), Automation server (Jenkins) and Docker image repository on top of Kubernetes, this figure shows the final architecture of pipeline :<\/p>\n\n\n\n
    \"cidi<\/a>
    Local CI\/DI Pipeline For Microservices Architecture<\/figcaption><\/figure><\/div>\n\n\n\n
     Setup Kubernetes  <\/h2>\n\n\n\n
    Kubernetes works on top of Docker, it undertakes tasks including creating container , deployment and configuration. There exist several methods and tools to setup a K8s such as Docker Desktop<\/a>, minikube<\/a>, kind<\/a>, k3s<\/a>, kubeadm <\/a>and etc. In this article for the sake of simplicity I setup K8s using Docker Desktop.<\/p>\n\n\n\n
    If you haven’t already installed the Docker Desktop, download and install latest version of it which is compatible to your OS from the official Docker website. Docker is originally made for Linux, but thanks to Hyper-v and WSL 2 features you can also run it in windows. So, you need to enable\/install one of them in before Docker installation.<\/p>\n\n\n\n
    \"\"<\/a>
    Check the box for Hyper-V feature in the list<\/figcaption><\/figure><\/div>\n\n\n\n
    In the first installation step where the installer ask you to use WSL 2 instead of Hyper-V, you can check the box if you have WSL 2 installed and click OK<\/strong>. WSL 2 works on top of Hyper-V, but it doesn’t require Hyper-V to be enabled. <\/p>\n\n\n\n
    \"\"<\/a>
    Check the box if you want to use WSL 2, but make sure you have already installed it.<\/figcaption><\/figure><\/div>\n\n\n\n
    Docker Desktop comes with a graphical user interface that allows configuring and installing Kubernetes. Open the Docker Desktop application and go to the settings menu, under the Kubernetes<\/em> tab, check the Enable Kubernetes<\/strong> checkbox and click Apply & Restart<\/strong>.<\/p>\n\n\n\n
    \"\"<\/a>
    To enable Kubernetes, check the Enable Kubernetes<\/strong> checkbox and click Apply & Restart<\/strong> <\/figcaption><\/figure><\/div>\n\n\n\n
    After Docker Desktop restarts, use the following command to verify that the Kubernetes cluster is running :<\/p>\n\n\n\n
    kubectl cluster-info<\/code><\/pre>\n\n\n\n
    If the installation is completed successfully, the above command should display information about your Kubernetes cluster. <\/p>\n\n\n\n
    We need to install Kubernetes dashboard to have graphical interface in order to work with K8s and obtain an overview of the applications that are active on the cluster. It requires manual setup :<\/p>\n\n\n\n
    First execute below command :<\/p>\n\n\n\n
    kubectl apply -f https:\/\/raw.githubusercontent.com\/kubernetes\/dashboard\/v2.6.0\/aio\/deploy\/recommended.yaml<\/code><\/pre>\n\n\n\n
    Next,  create a proxy server between the client and the Kubernetes API server. This allows the user to interact with the Kubernetes API server : <\/p>\n\n\n\n
    kubectl proxy<\/code><\/pre>\n\n\n\n
    The Dashboard can be accessed at  http:\/\/localhost:8001\/api\/v1\/namespaces\/kubernetes-dashboard\/services\/https:kubernetes-dashboard:\/proxy\/<\/code><\/strong>. However, the login page needs you to provide token which you can create with following command :<\/p>\n\n\n\n
    kubectl -n kubernetes-dashboard create token admin-user<\/code><\/pre>\n\n\n\n
    \"Kubernetes<\/a>
    Kubernetes dashboard login page<\/figcaption><\/figure><\/div>\n\n\n\n
     Setup Local Docker Registry On Kubernetes<\/h2>\n\n\n\n
    Besides many advantages of using the dockerhub <\/a>, it comes with its own set of limitations. For example, anonymous users and members using free plan are limited to 100 and 200 container image pull requests per 6 hours. In such cases, private local repository would be a better choice. It enables us to have a image repository that in addition to unlimited pull and push requests, it gives us control over storage options and many other options per our needs. Let’s get to the point :<\/p>\n\n\n\n
    First of all let’s create a namespace for our CI\/CD pipeline in Kubernetes with the following command :<\/p>\n\n\n\n
    kubectl create namespace cidi-pipeline<\/code><\/pre>\n\n\n\n
    <\/p>\n\n\n\n
    Next, with openssl <\/strong>and htpasswd <\/strong>tools we build  TLS certificate and implement user authentication for our repository.<\/p>\n\n\n\n
    TLS certificate :<\/p>\n\n\n\n
    mkdir certs\nopenssl req -x509 -newkey rsa:4096 -days 365 -nodes -sha256 -keyout certs\/tls.key -out certs\/tls.crt -subj \"\/CN=docker-registry\" -addext \"subjectAltName = DNS:docker-registry\"<\/code><\/pre>\n\n\n\n
    <\/p>\n\n\n\n
    Adding authentication :<\/p>\n\n\n\n
    mkdir auth\ndocker run --rm --entrypoint htpasswd registry:2.6.2 -Bbn admin 1234@5 > auth\/htpasswd<\/code><\/pre>\n\n\n\n
    Let’s look at breakdown of the each parts of the above command. In the above command with “--rm\"<\/strong><\/code> flag we tell Docker to automatically remove the container when it’s done running. With “--entrypoint htpasswd\"<\/strong><\/code> we set the entrypoint for the container to be htpasswd<\/strong><\/code>, which is a tool used to manage user authentication files. “registry:2.6.2<\/strong><\/code>”  is the name of the Docker image that our container will be based on. “-Bbn ... ...<\/code>“<\/strong> is  the htpasswd<\/code> <\/strong>option to create an encrypted password for the user admin<\/strong><\/code> with the password 1234@5<\/strong><\/code>. “-Bbn<\/strong>” consists of three flags. The B<\/strong><\/code> flag tells the htpasswd <\/strong>to use bcrypt<\/strong> encryption algorithm for password and b<\/strong> flag get the password from the command line instead of prompting for it, and n <\/strong>flag shows the results on standard output instead of updating a file. “> auth\/htpasswd<\/code>“<\/strong> redirects the output of the htpasswd<\/code> <\/strong>command to a file named htpasswd<\/code> <\/strong>in the auth<\/code> <\/strong>directory which we created at first line. This file will contain the username and encrypted password that were just created.<\/p>\n\n\n\n
    Next, we generate two types of secrets – one TLS type secret and another Generic type secret – which we will use to mount our certificate and password, respectively.<\/p>\n\n\n\n
    kubectl create secret tls certs-secret --cert=certs\/tls.crt --key=certs\/tls.key -n cicd-pipeline\n\nkubectl create secret generic auth-secret --from-file=auth\/htpasswd -n cicd-pipeline<\/code><\/pre>\n\n\n\n
    Values of htpasswd<\/code> file will be saved in the Generic Secret we just created in Kubernetes.<\/p>\n\n\n\n
    The registry-volume.yaml<\/strong> config file creates the Persistent Volume for our local Docker registery:<\/p>\n\n\n\n
    apiVersion: v1\nkind: PersistentVolume\nmetadata:\n    name: docker-repo-pv\nspec:\n  capacity:\n    storage: 1Gi\n  accessModes:\n     - ReadWriteOnce\n  hostPath:\n    path: \/tmp\/repository\n---\napiVersion: v1\nkind: PersistentVolumeClaim\nmetadata:\n  name: docker-repo-pvc\nspec:\n  accessModes:\n    - ReadWriteOnce\n  resources:\n    requests:\n      storage: 1Gi<\/code><\/pre>\n\n\n\n
    accessModes<\/code> <\/strong>specify the ways in which the persistent volume can be accessed by the associated pod(s). Here, ReadWriteOnce<\/code> <\/strong>indicates that the our persistent volume can be mounted as read-write by a single node in the cluster at a time.<\/p>\n\n\n\n
    You can apply it to K8s cluster via Kubernetes dashboard or in command line. Here I’ll do it in command line:<\/p>\n\n\n\n
    kubectl create -f registry-volume.yaml -n cicd-pipeline<\/code><\/pre>\n\n\n\n
    After creating both Persistent Volume<\/strong> and a Persistent Volume Claim<\/strong>, we can create docker registry pod utilizing this volume.<\/p>\n\n\n\n
    Apply registry-pod-service.yaml config file to your Kubernetes cluster to create the pod and service in cicd-pipeline namespace  :<\/p>\n\n\n\n
    apiVersion: v1\nkind: Pod\nmetadata:\n  name: docker-registry-pod\n  labels:\n    app: registry\nspec:\n  containers:\n    - name: registry\n      image: registry:2.6.2\n      volumeMounts:\n        - name: repo-vol\n          mountPath: \"\/var\/lib\/registry\"\n        - name: certs-vol\n          mountPath: \"certs\"\n          readOnly: true\n        - name: auth-vol\n          mountPath: \"auth\"\n          readOnly: true\n      env:\n        - name: REGISTRY_AUTH\n          value: \"htpasswd\"\n        - name: REGISTRY_AUTH_HTPASSWD_REALM\n          value: \"Registry Realm\"\n        - name: REGISTRY_AUTH_HTPASSWD_PATH\n          value: \"auth\/htpasswd\"\n        - name: REGISTRY_HTTP_TLS_CERTIFICATE\n          value: \"certs\/tls.crt\"\n        - name: REGISTRY_HTTP_TLS_KEY\n          value: \"certs\/tls.key\"\n  volumes:\n        - name: repo-vol\n          persistentVolumeClaim:\n            claimName: docker-repo-pvc\n        - name: certs-vol\n          secret:\n            secretName: certs-secret\n        - name: auth-vol\n          secret:\n            secretName: auth-secret\n\n---\n\nkind: Service \napiVersion: v1 \nmetadata:\n  name: docker-registery\nspec:\n  # Expose the service on a static port on each node\n  # so that we can access the service from outside the cluster \n  type: NodePort\n  selector:\n    app: registry \n  ports:\n    # Three types of ports for a service\n    # nodePort - a static port assigned on each the node\n    # port - port exposed internally in the cluster\n    # targetPort - the container port to send requests to\n    - nodePort: 30001\n      port: 5000 \n      targetPort: 5000<\/code><\/pre>\n\n\n\n
    Use the option “-n cidi-pipeline”<\/strong> when applying the YAML config file to ensure they are created in cidi-pipeline :<\/p>\n\n\n\n
    kubectl create -f docker-registry-pod.yaml -n cicd-pipeline<\/code><\/pre>\n\n\n\n
    <\/p>\n\n\n\n
    To verify that everything is working properly, use the following command to log in to the local registry from outside the cluster.<\/p>\n\n\n\n
    docker login 127.0.0.1:30001 -u admin -p 123456<\/code><\/pre>\n\n\n\n
    If everything is OK, you should get “Login Succeeded” <\/strong>message.<\/p>\n\n\n\n
    Setup GitBlit on Kubernetes<\/h2>\n\n\n\n
    Gitblit is an open-source Git server that is designed to be easy to set up, configure and use. It provides a web GUI for managing repositories, which allows users to perform various Git operations, such as cloning, branching, merging, and pushing. It’s lightweight and easy to deploy, which makes it popular choice for small teams. Setting up GitBlit in Kubernetes involves a three steps:<\/p>\n\n\n\n
    1. Create PV and PVC for repositories storage<\/li>
    2. Create a Kubernetes Pod for GitBlit and attach volume <\/li>
    3. Create a service for the Pod<\/li><\/ol>\n\n\n\n
      Create PV and PVC using below config :<\/p>\n\n\n\n
      apiVersion: v1\n kind: PersistentVolume\n metadata:\n     name: gitblit-repo-pv\n spec:\n   capacity:\n     storage: 1Gi\n   accessModes:\n      - ReadWriteOnce\n   hostPath:\n     path: \/tmp\/gitblitrepository\n apiVersion: v1\n kind: PersistentVolumeClaim\n metadata:\n   name: gitblit-repo-pvc\n spec:\n   accessModes:\n     - ReadWriteOnce\n   resources:\n     requests:\n       storage: 2Gi<\/code><\/pre>\n\n\n\n
      Gitblit Kubernetes Pod:<\/p>\n\n\n\n
      apiVersion: v1\n kind: Pod\n metadata:\n   name: gitblit-pod\n   labels:\n     app: gitblit\n spec:\n   containers:\n   - name: gitblit\n     image: gitblit\/gitblit:latest \n     ports:\n     - containerPort: 8443\n     - containerPort: 8080\n     - containerPort: 9418\n     - containerPort: 29418\n     volumeMounts:\n     - name: gitblit-data\n       mountPath: \/opt\/gitblit-data\n   volumes:\n   - name: gitblit-data\n     persistentVolumeClaim:\n       claimName: gitblit-repo-pvc <\/code><\/pre>\n\n\n\n
      <\/p>\n\n\n\n
      Gitblit Kubernetes service :<\/p>\n\n\n\n
      apiVersion: v1\n kind: Service\n metadata:\n   name: gitblit-service\n spec:\n   selector:\n     app: gitblit\n   ports:\n     - name: https\n       protocol: TCP\n       port: 8443\n       targetPort: 8443\n       nodePort: 30443\n     - name: http\n       protocol: TCP\n       port: 8080\n       targetPort: 8080\n       nodePort: 30080\n     - name: git\n       protocol: TCP\n       port: 9418\n       targetPort: 9418\n       nodePort: 30418\n     - name: ssh\n       protocol: TCP\n       port: 29418\n       targetPort: 29418\n       nodePort: 30419\n   type: NodePort<\/code><\/pre>\n\n\n\n
      You can either apply all of above configs using K8s dashboard or apply them in command line as yml file like this :<\/p>\n\n\n\n
      \nkubectl create -f gitblit-volume.yaml -n cicd-pipeline\nkubectl create -f gitblit-pod.yaml -n cicd-pipeline\nkubectl create -f gitblit-service.yaml -n cicd-pipeline\n<\/code><\/pre>\n\n\n\n
      If everything is OK, after creating the service, by opening https:\/\/your-node-ip:30443 <\/a>you should see Gitblit home page, however, due to invalid cert, you’ll get insecure connection warning  :<\/p>\n\n\n\n
      \"Gitblit<\/a>
      Gitblit home page running as Kubernetes service<\/figcaption><\/figure>\n\n\n\n
      Setup Jenkins In Local Kubernetes<\/h2>\n\n\n\n
      To setup Jenkins in Kubernetes apply the following config in Kubernetes either using kubectl or Kubernetes dashboard in cicd-pipeline name space :<\/p>\n\n\n\n
      apiVersion: v1\n kind: PersistentVolume\n metadata:\n     name: jenkins-repo-pv\n spec:\n   capacity:\n     storage: 3Gi\n   accessModes:\n      - ReadWriteOnce\n   hostPath:\n     path: \/tmp\/jenkinsData\n apiVersion: v1\n kind: PersistentVolumeClaim\n metadata:\n   name: jenkins-repo-pvc\n spec:\n   accessModes:\n     - ReadWriteOnce\n   resources:\n     requests:\n       storage: 3Gi\n apiVersion: v1\n kind: Pod\n metadata:\n   name: jenkins\n   labels:\n     app: jenkins\n spec:\n   containers:\n     - name: jenkins\n       image: jenkins\/jenkins:lts\n       ports:\n         - containerPort: 8080\n       volumeMounts:\n         - name: jenkins-data\n           mountPath: \/var\/jenkins_home\n   volumes:\n     - name: jenkins-data\n       persistentVolumeClaim:\n         claimName: jenkins-repo-pvc\n \n apiVersion: v1\n kind: Service\n metadata:\n   name: jenkins\n   labels:\n     app: jenkins\n spec:\n   selector:\n     app: jenkins\n   type: NodePort\n   ports:\n     - port: 8080\n       targetPort: 8080\n       nodePort: 30088<\/code><\/pre>\n\n\n\n
      As you can see, this time, instead of applying separate config files, I put all steps of creating volume, creating pod and creating service in single config file. In case you want to  apply in command line, you can execute this command :<\/p>\n\n\n\n
      \nkubectl create -f jenkins.yaml -n cicd-pipeline\n<\/code><\/pre>\n\n\n\n
      After creating the service, you can reach Jenkins web UI at http:\/\/your-node-ip:30088. At first, Jenkins ask for a initial administrator password. <\/p>\n\n\n\n
      \"Jenkins<\/a>
      Jenkins Administrator Password<\/figcaption><\/figure><\/div>\n\n\n\n
      Use kubectl to get the password from initialAdminPassword file within the Jenkins pod :<\/p>\n\n\n\n
      kubectl exec -it jenkins-pod -n cidi-pipeline -- cat \/var\/jenkins_home\/secrets\/\ninitialAdminPassword<\/code><\/pre>\n\n\n\n
      Running above command will yield a password string, copy it and past in Jenkins and click on the button to start initial Jenkins setup. In the next step choose “Install suggested plugin”. It will install all required plugin that you need in CI\/CD process :<\/p>\n\n\n\n
      \"Installing<\/a>
      Installing suggested plugin in Jenkins<\/figcaption><\/figure><\/div>\n\n\n\n
      After completing plugin installation, you will be asked for choosing a user name password for admin. Fill the form and finish the setup:<\/p>\n\n\n\n
      \"\"<\/a>
      Jenkins form for admin username and password<\/figcaption><\/figure><\/div>\n\n\n\n
      The Jenkins setup is finished, now you can login in with user name and password you have chosen in previous step and start working  :<\/p>\n\n\n\n
      \"jenkins<\/a><\/figure><\/div>\n\n\n\n
      Now that we have installed 4 main pillars of the pipeline, we can shape and configure our pipeline to start the production. I will explain further steps in the next part of this article. Keep in touch!<\/p>\n","protected":false},"excerpt":{"rendered":"
      A local CI\/CD pipeline offers several benefits for software development teams. It can help improve the security of your code by keeping sensitive information on … <\/p>\n","protected":false},"author":1,"featured_media":1368,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[203,1,202,124,195,69,205,204,35,41,70],"tags":[189,206,115,188,9,207,208,39,209,63,71],"_links":{"self":[{"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/posts\/1312"}],"collection":[{"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/comments?post=1312"}],"version-history":[{"count":0,"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/posts\/1312\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/media\/1368"}],"wp:attachment":[{"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/media?parent=1312"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/categories?post=1312"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/tags?post=1312"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}