Kubernetes health checks

Read more about the article Deploying loklak Server on Kubernetes with External Elasticsearch

Deploying loklak Server on Kubernetes with External Elasticsearch

Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. - kubernetes.io Kubernetes is an awesome cloud platform, which ensures that cloud applications run reliably. It runs automated tests, flawless updates, smart roll out and rollbacks, simple scaling and a lot more. So as a part of GSoC, I worked on taking the loklak server to Kubernetes on Google Cloud Platform. In this blog post, I will be discussing the approach followed to deploy development branch of loklak on Kubernetes. New Docker Image Since Kubernetes deployments work on Docker images, we needed one for the loklak project. The existing image would not be up to the mark for Kubernetes as it contained the declaration of volumes and exposing of ports. So I wrote a new Docker image which could be used in Kubernetes. The image would simply clone loklak server, build the project and trigger the server as CMD - FROM alpine:latest ENV LANG=en_US.UTF-8 ENV JAVA_TOOL_OPTIONS=-Dfile.encoding=UTF8 WORKDIR /loklak_server RUN apk update && apk add openjdk8 git bash && \ git clone https://github.com/loklak/loklak_server.git /loklak_server && \ git checkout development && \ ./gradlew build -x test -x checkstyleTest -x checkstyleMain -x jacocoTestReport && \ # Some Configurations and Cleanups CMD ["bin/start.sh", "-Idn"] [SOURCE] This image wouldn’t have any volumes or exposed ports and we are now free to configure them in the configuration files (discussed in a later section). Building and Pushing Docker Image using Travis To automatically build and push on a commit to the master branch, Travis build is used. In the after_success section, a call to push Docker image is made. Travis environment variables hold the username and password for Docker hub and are used for logging in - docker login -u $DOCKER_USERNAME -p $DOCKER_PASSWORD [SOURCE] We needed checks there to ensure that we are on the right branch for the push and we are not handling a pull request - # Build and push Kubernetes Docker image KUBERNETES_BRANCH=loklak/loklak_server:latest-kubernetes-$TRAVIS_BRANCH KUBERNETES_COMMIT=loklak/loklak_server:kubernetes-$TRAVIS_COMMIT if [ "$TRAVIS_BRANCH" == "development" ]; then docker build -t loklak_server_kubernetes kubernetes/images/development docker tag loklak_server_kubernetes $KUBERNETES_BRANCH docker push $KUBERNETES_BRANCH docker tag $KUBERNETES_BRANCH $KUBERNETES_COMMIT docker push $KUBERNETES_COMMIT elif [ "$TRAVIS_BRANCH" == "master" ]; then # Build and push master else echo "Skipping Kubernetes image push for branch $TRAVIS_BRANCH" fi [SOURCE] Kubernetes Configurations for loklak Kubernetes cluster can completely be configured using configurations written in YAML format. The deployment of loklak uses the previously built image. Initially, the image tagged as latest-kubernetes-development is used - apiVersion: apps/v1beta1 kind: Deployment metadata: name: server namespace: web spec: replicas: 1 template: metadata: labels: app: server spec: containers: - name: server image: loklak/loklak_server:latest-kubernetes-development ... [SOURCE] Readiness and Liveness Probes Probes act as the top level tester for the health of a deployment in Kubernetes. The probes are performed periodically to ensure that things are working fine and appropriate steps are taken if they fail. When a new image is updated, the older pod still runs and servers the requests. It is replaced by the new ones only when the probes are…

Continue ReadingDeploying loklak Server on Kubernetes with External Elasticsearch
Read more about the article Utilizing Readiness Probes for loklak Dependencies in Kubernetes

Utilizing Readiness Probes for loklak Dependencies in Kubernetes

When we use any application and fail to connect to it, we do not give up and retry connecting to it again and again. But in the reality we often face this kind of obstacles like application that break instantly or when connecting to an API or database that is not ready yet, the app gets upset and refuses to continue to work. So, something similar to this was happening with api.loklak.org. In such cases we can’t really re-write the whole application again every time the problem occurs.So for this we need to define dependencies of some kind that can handle the situation rather than disappointing the users of loklak app. Solution: We will just wait until a dependent API or backend of loklak is ready and then only start the loklak app. For this to be done, we used Kubernetes Health Checks. Kubernetes health checks are divided into liveness and readiness probes. The purpose of liveness probes are to indicate that your application is running. Readiness probes are meant to check if your application is ready to serve traffic. The right combination of liveness and readiness probes used with Kubernetes deployments can: Enable zero downtime deploys Prevent deployment of broken images Ensure that failed containers are automatically restarted Pod Ready to be Used? A Pod with defined readiness probe won’t receive any traffic until a defined request can be successfully fulfilled. This health checks are defined through the Kubernetes, so we don’t need any changes to be made in our services (APIs). We just need to setup a readiness probe for the APIs that loklak server is depending on. Here you can see the relevant part of the container spec you need to add (in this example we want to know when loklak is ready): readinessProbe: httpGet: path: /api/status.json port: 80 initialDelaySeconds: 30 timeoutSeconds: 3   Readiness Probes Updating deployments of loklak when something pushed into development without readiness probes can result in downtime as old pods are replaced by new pods in case of Kubernetes deployment. If the new pods are misconfigured or somehow broken, that downtime extends until you detect the problem and rollback. With readiness probes, Kubernetes will not send traffic to a pod until the readiness probe is successful. When updating a loklak deployment, it will also leave old one’s running until probes have been successful on new copy. That means that if loklak server new pods are broken in some way, they’ll never see traffic, instead old pods of loklak server will continue to serve all traffic for the deployment. Conclusion Readiness probes is a simple solution to ensure that Pods with dependencies do not get started before their dependencies are ready (in this case for loklak server). This also works with more than one dependency. Resources Code in loklak server: https://github.com/loklak/loklak_server . More about readiness and liveness probes at: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/ Kubernetes Health checks: https://kubernetes-v1-4.github.io/docs/user-guide/production-pods/#liveness-and-readiness-probes-aka-health-checks Blog posts related to kubernetes: https://www.ianlewis.org/en/tag/kubernetes. Tutorial for creating and using Pods and probes: https://github.com/googlecodelabs/orchestrate-with-kubernetes/blob/master/labs/monitoring-and-health-checks.md

Continue ReadingUtilizing Readiness Probes for loklak Dependencies in Kubernetes