Speeding up the Travis Build to Decrease the Building Time

Meilix is the repository which uses build script to generate community version of lubuntu as LXQT Desktop. It usually takes around 25-26 to build and deploy the ISO as a Github Release on master branch.
Observing the build log we can see that there are most of the packages like debootstrap, squashfs-tool, etc which are being fetch and setup at the time of building which results in increasing build time.

The issue is to decrease the build time supplying the packages from Travis so that when we run build.sh we won’t be required to download them again and thus few minutes will get reduced.
We included list of packages to be pre-downloaded in .travis.yml

include:
  - os: linux
    addons:
      apt:
        sources:
          - ubuntu-toolchain-r-test
        packages:
          - debootstrap
          - genisoimage
          - p7zip-full
          - squashfs-tools
          - ubuntu-dev-tools
          - dpkg-dev
          - debhelper
          - fakeroot
          - devscripts

These are some important packages included in the build.sh  as devtools=”debootstrap genisoimage p7zip-full squashfs-tools ubuntu-dev-tools” which are always fetched, so we included it into .travis.yml and downloaded it before entering into the chroot environment. By specifying those packages in the .travis.yml, Travis provides those packages beforehand into the docker container so it will run our script. Since the scripts also include package then when the script runs apt-get it won’t download those packages again. They are specified outside the chroot environment because they are expected to be at the system the build.sh script is run to get the iso. By this way, we get a sharp decrease in build time as the internet in the Travis CI container is not so fast so the package download time can be avoided. Now the build is taking around 15-16 minutes to build and deploy.

One thing to note that we didn’t remove those packages off build.sh so that build.sh works outside Travis CI as well.

References:
Pull #176 by @abishekvashok
Speeding up Travis Build by Travis CI
Faster Build by atchai.com

Automatic Signing and Publishing of Android Apps from Travis

As I discussed about preparing the apps in Play Store for automatic deployment and Google App Signing in previous blogs, in this blog, I’ll talk about how to use Travis Ci to automatically sign and publish the apps using fastlane, as well as how to upload sensitive information like signing keys and publishing JSON to the Open Source repository. This method will be used to publish the following Android Apps:

Current Project Structure

The example project I have used to set up the process has the following structure:

It’s a normal Android Project with some .travis.yml and some additional bash scripts in scripts folder. The update-apk.sh file is standard app build and repo push file found in FOSSASIA projects. The process used to develop it is documented in previous blogs. First, we’ll see how to upload our keys to the repo after encrypting them.

Encrypting keys using Travis

Travis provides a very nice documentation on encrypting files containing sensitive information, but a crucial information is buried below the page. As you’d normally want to upload two things to the repo – the app signing key, and API JSON file for release manager API of Google Play for Fastlane, you can’t do it separately by using standard file encryption command for travis as it will override the previous encrypted file’s secret. In order to do so, you need to create a tarball of all the files that need to be encrypted and encrypt that tar instead. Along with this, before you need to use the file, you’ll have to decrypt in in the travis build and also uncompress it for use.

So, first install Travis CLI tool and login using travis login (You should have right access to the repo and Travis CI in order to encrypt the files for it)

Then add the signing key and fastlane json in the scripts folder. Let’s assume the names of the files are key.jks and fastlane.json

Then, go to scripts folder and run this command to create a tar of these files:

tar cvf secrets.tar fastlane.json key.jks

 

secrets.tar will be created in the folder. Now, run this command to encrypt the file

travis encrypt-file secrets.tar

 

A new file secrets.tar.enc will be created in the folder. Now delete the original files and secrets tar so they do not get added to the repo by mistake. The output log will show the the command for decryption of the file to be added to the .travis.yml file.

Decrypting keys using Travis

But if we add it there, the keys will be decrypted for each commit on each branch. We want it to happen only for master branch as we only require publishing from that branch. So, we’ll create a bash script prep-key.sh for the task with following content

#!/bin/sh
set -e

export DEPLOY_BRANCH=${DEPLOY_BRANCH:-master}

if [ "$TRAVIS_PULL_REQUEST" != "false" -o "$TRAVIS_REPO_SLUG" != "iamareebjamal/android-test-fastlane" -o "$TRAVIS_BRANCH" != "$DEPLOY_BRANCH" ]; then
    echo "We decrypt key only for pushes to the master branch and not PRs. So, skip."
    exit 0
fi

openssl aes-256-cbc -K $encrypted_4dd7_key -iv $encrypted_4dd7_iv -in ./scripts/secrets.tar.enc -out ./scripts/secrets.tar -d
tar xvf ./scripts/secrets.tar -C scripts/

 

Of course, you’ll have to change the commands and arguments according to your need and repo. Specially, the decryption command keys ID

The script checks if the repo and branch are correct, and the commit is not of a PR, then decrypts the file and extracts them in appropriate directory

Before signing the app, you’ll need to store the keystore password, alias and key password in Travis Environment Variables. Once you have done that, you can proceed to signing the app. I’ll assume the variable names to be $STORE_PASS, $ALIAS and $KEY_PASS respectively

Signing App

Now, come to the part in upload-apk.sh script where you have the unsigned release app built. Let’s assume its name is app-release-unsigned.apk.Then run this command to sign it

cp app-release-unsigned.apk app-release-unaligned.apk
jarsigner -verbose -tsa http://timestamp.comodoca.com/rfc3161 -sigalg SHA1withRSA -digestalg SHA1 -keystore ../scripts/key.jks -storepass $STORE_PASS -keypass $KEY_PASS app-release-unaligned.apk $ALIAS

 

Then run this command to zipalign the app

${ANDROID_HOME}/build-tools/25.0.2/zipalign -v -p 4 app-release-unaligned.apk app-release.apk

 

Remember that the build tools version should be the same as the one specified in .travis.yml

This will create an apk named app-release.apk

Publishing App

This is the easiest step. First install fastlane using this command

gem install fastlane

 

Then run this command to publish the app to alpha channel on Play Store

fastlane supply --apk app-release.apk --track alpha --json_key ../scripts/fastlane.json --package_name com.iamareebjamal.fastlane

 

You can always configure the arguments according to your need. Also notice that you have to provide the package name for Fastlane to know which app to update. This can also be stored as an environment variable.

This is all for this blog, you can read more about travis CLI, fastlane features and signing process in these links below:

Updating of Linux Standard Base (lsb) and Shorten of log of Meilix Build

Updating the Linux Standard Base of the Meilix

Originally Meilix uses the Ubuntu mirror to fetch the Kernel source for the building of the Operating System. Therefore whenever a user type lsb_release -a for fetching the required information, they get the Ubuntu build info. The task is to change the config file to update the Linux Base Information from Ubuntu to Meilix.

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 17.04
Release: 17.04
Codename: zesty

We need to patch a file in the location meilix-default-settings/etc/lsb-release which contains the information of the lsb release and this will overwrite the original configuration of Meilix.
This is how the lsb-release file looks like now:

DISTRIB_ID=Meilix
DISTRIB_RELEASE=17.04
DISTRIB_CODENAME=meilix
DISTRIB_DESCRIPTION="Meilix 17.04"

We made the required changes in the file as shown above and the output is as follows:

Shorten the log length of Meilix in Travis

We are facing issues while deployment of Meilix ISO in Travis and the error follows is ReadTimeout Error. Example log of one of fail build is:

This error gets solved automatically and the the ISO gets deployment after 1 or 2 restart of the build. But this time the error doesn’t get solved and after several attempts of restart of build, the ISO doesn’t get deployed.

Reason behind the error:

Travis is taking a lot of time to build the ISO. Travis logs are exceeding the time limit.

Proposed solution:

Reduce the time of build or shift to a new CI.
Reduce the log of the build so as to get the log within 9999 lines.

Solution Implemented

The best solution is to reduce the number of lines used in the log and this will also reduce the time of the build.
I tried concealing some command outputs by appending >/dev/null 2>&1 to some of the commands that has long outputs and adding -y to the commands like:

apt-get -qq -y --purge install file-roller unrar

References

Wiki of Linux Standard Base
Linux Foundation lsb
Ubuntu answer to reduce log

Setting up Travis Continuous Integration in Giggity

Travis is a continuous integration service that enables you to run tests against your latest Android builds. You can setup your projects to run both unit and integration tests, which can also include launching an emulator. I recently added Travis Continuous Integration Connfa, Giggity and Giraffe app. In this blog, I describe how to set up Travis Continuous Integration in an Android Project with reference to Giggity app.

  • Use your GitHub account, sign in to either to Travis CI .org for public repositories or Travis CI .com for private repositories
  • Accept the GitHub access permissions confirmation.
  • Once you’re signed in to Travis CI, and synchronized your GitHub repositories, go to your profile page and enable the repository you want to build:

  • Now you need to add a .travis.yml file into the root of your project. This file will tell how Travis handles the builds. You should check your .travis file on Travis Web Lint before committing any changes to it.
  • You can find the very basic instructions for building an Android project from the Travis documentation. But here we specify the .travis.yml build accordingly for Giggity’s continuous integration. Here, language shows that it is an Android project. We write “language: ruby” if it is a ruby project.  If you need a more customizable environment running in a virtual machine, use the Sudo Enabled infrastructure. Similarly, we define the API, play services and libraries defined as shown.
language: android
sudo: required
jdk: 
 - oraclejdk8
# Use the Travis Container-Based Infrastructure
android:
  components:
    - platform-tools
    - tools
    - build-tools-25.0.3
    - android-25
    
    # For Google APIs
    - addon-google_apis-google-$ANDROID_API_LEVEL
    # Google Play Services
    - extra-google-google_play_services
    # Support library
    - extra-android-support
    # Latest artifacts in local repository
    - extra-google-m2repository
    - extra-android-m2repository
    - android-sdk-license-.+
    - '.+'

before_script:
  - chmod +x gradlew    

script:
  - ./gradlew build

Now when you make a commit or pull request Travis check if all the defines checks pass and it is able to be merged. To be more advanced you can also define if you want to build APKs too with every build.

References:

  • Travis Continuous Integration Documentation – https://docs.travis-ci.com/user/getting-started/

Shorten the Travis build time of Meilix

Meilix is a lubuntu based script. It uses Travis to compile and deploy the iso as Github Release. Normally Travis took around 17-21 minutes to build the script and then to deploy the iso on the page. It is quite good that one gets the iso in such a small interval of time but if we would able to decrease this time even more then that will be better.

Meilix script consists of basically 2 tests and 1 deploy:

The idea behind reducing the time of Meilix building is to parallely run the tests which are independent to each other. So here we run the build.sh and aptRepoUpdater.sh parallely to reduce the time upto some extent.
This pictures denotes that both the tests are running parallely and Github Releases is waiting for them to get complete successfully to deploy the iso.

Let’s see the code through which this made possible:

jobs:
  include:
    - script: ./build.sh
    - script: 'if [ "$TRAVIS_PULL_REQUEST" = "false" ]; then bash ./scripts/aptRepoUpdater.sh; fi'
    - stage: GitHub Release
      script: echo "Deploying to GitHub releases ..."

Here we included a job section in which we wrote the test which Travis has to carry out parallely. This will run both the script at the same time and can help to reduce the time.
After both the script run successfully then Github Release the iso.

Here we can see that we are only able to save around 30-40 seconds and that much matters a lot in case if we have more than 1 build going on the same time.

Links to follow:
Travis guide to build stages

Heroku Deployment through Travis for Meilix-Generator

This article will tell the way to deploy the Meilix Generator on Heroku with the help of Travis. A successful deployment will help as a test for a good PR. Later in the article, we’ll see the one-button deployment on Heroku.

We will here deploy Meilix Generator on Heroku. The way to deploy the project on Heroku is that one should connect its Github account and deploy it on Heroku. The problem arises when one wants to deploy the project on each and every commit. This will help to test that the commits are passing or not. Here we will see that how to use Travis to deploy on Heroku on each and every commit. If the Travis test passed which means that the changes made in the commit are implemented.
We used the same idea to test the commits for Meilix Generator.

Idea behind it

Travis (.travis.yml) will be helpful to us to achieve this. We will use this deploy build to Heroku on each commit. If it gets successfully deployed then it proves that the commit made is working.

How to implement it

I will use Meilix Generator repository to tell the way to implement this. It is as simple as editing the .travis.yml file. We just have to add few lines to .travis.yml and hence it will get deployed.

deploy:
  provider: heroku
  api_key:
    secure: "YOUR ENCRYPTED API KEY"     # explained below
  app: meilix-generator                    # write the name of the app
  on:
    repo: fossasia/meilix-generator                # repo name
    branch: master                        # branch name

Way to generate the api key:
This is really a matter of concern since if this gets wrong then the deployment will not occur.

Steps:

  • cd into the repository which you want to deploy on Heroku.
  • Login Heroku CI and Travis CI into your terminal and type the following.

travis encrypt $(heroku auth:token) --add deploy.api_key

This will automatically provide the key inside the .travis.yml file.

You can also configure manually using

travis heroku setup

That it, you are done, test the build.

Things are still left:

But we are still left with the test of the PR.
For this we have to create a new app.json file as:

{
    "name": "Meilix-Generator",
    "description": "A webapp which generates iso for you",
    "repository": "https://github.com/fossasia/meilix-generator/",
    "logo": "https://github.com/fossasia/meilix-generator/blob/master/static/logo.png",
    "keywords": [
        "meilix-generator",
        "fossasia",
        "flask"
    ],
    "env": {
        "APP_SECRET_TOKEN": {
            "generator": "secret"
        },
        "ON_HEROKU": "true",
        "FORCE_SSL": "true",
        "INVITATION_CODE": {
            "generator": "secret"
        }
    },
    "buildpacks": {
            "url": "heroku/python"        # this is the only place of concern
        }
}

This code should be put in a file in the root of the repo with the name as app.json.
In the buildpacks : the url should be the one which contains the code base language used.

This can be helpful in 2 ways:

  1. Test the commit made and deploy it on Heroku
    2. One-click deployment button which will deploy the app on Heroku
  • Test the deployment through the URL:

https://heroku.com/deploy?template=https://github.com/user_name/repo_name/tree/master

  • Way to add the button:

[![Deploy](https://www.herokucdn.com/deploy/button.svg)](https://heroku.com/deploy)

How can this idea be helpful to a developer

A developer can use this to deploy its app on Heroku and test the commit automatically and view the quality and status of PR too.

Useful repositories and link which uses this:

I have used the same idea in my project. Do have a look:

https://github.com/fossasia/meilix-generator
deployment on Heroku
one-click deployment
app.json file schema   

Auto Deployment of SUSI Server using Kubernetes on Google Cloud Platform

Recently, we auto deployed SUSI Server on Google Cloud Platform using Kubernetes and Docker Images after each commit in the GitHub repo with the help of Travis Continuous Integration. So, basically, whenever a new commit is added to the repo, during the Travis build, we build the docker image of the server and then use it to deploy the server on Google Cloud Platform. We use Kubernetes for deployment since it is very easy to scale up the Project when traffic on the server is increased and Docker because using it we can easily build docker images which then can be used to update the deployment. This schematic will make things more clear what exactly is the procedure.

Prerequisites

  1. You must be signed in to your Google Cloud Console and have enabled billing and must have credits left in your account.
  2. You must have a docker account and a repo in it. If you don’t have one, make it now.
  3. You should have enabled Travis on your repo and have a Travis.yml file in your repo.
  4. You must already have a project in Google Cloud Console. Make a new one if you don’t have.

Pre Deployment Steps

You will be needed to do some work on Google Cloud Platform before actually starting the auto deployment process. Those are:

  1. Creating a new Cluster.
  2. Adding and Formatting Persistence Disk
  3. Adding a Persistent Volume CLaim (PVC)
  4. Labeling a node as primary.

Check out this documentation on how to do that. It may help.

Implementation

Img src: https://cloud.google.com/solutions/continuous-delivery-with-travis-ci

1. The first step is simply to add this line in Travis.yml file and create an empty deploy.sh, file mentioned below.

after_success:
- bash kubernetes/travis/deploy.sh

Now we’ll be moving line by line and adding commands in the empty deploy.sh file that we created in the previous step.

2. Next step is to remove obsolete Google Cloud files and install Google Cloud SDK and kubectl command. Use following lines to do that.

echo ">>> Removing obsolete gcoud files"
sudo rm -f /usr/bin/git-credential-gcloud.sh
sudo rm -f /usr/bin/bq
sudo rm -f /usr/bin/gsutil
sudo rm -f /usr/bin/gcloud

echo ">>> Installing new files"
curl https://sdk.cloud.google.com | bash;
source ~/.bashrc
gcloud components install kubectl

3. In this step you will be needed to download a JSON file which contains your Google Cloud Credentials, then copy that file to your repo and encrypt it using Travis encryption keys. Follow https://youtu.be/7U4jjRw_AJk this video to see how to do that.

4. So, now you have added your encrypted credentials.json files in your repo and now you need to use those credentials to login into your google cloud account. So, use below lines to do that.

echo ">>> Decrypting credentials and authenticating gcloud account"
# Decrypt the credentials we added to the repo using the key we added with the Travis command line tool
openssl aes-256-cbc -K $encrypted_YOUR_key -iv $encrypted_YOUR_iv -in ./kubernetes/travis/Credentials.json.enc -out Credentials.json -d
gcloud auth activate-service-account --key-file Credentials.json
export GOOGLE_APPLICATION_CREDENTIALS=$(pwd)/Credentials.json
#add gcoud project id
gcloud config set project YOUR_PROJECT_ID
gcloud container clusters get-credentials YOUR_CONTAINER

The above lines of code first decrypt your credentials, then login into your account and set the project you already created earlier.

5. Now, we have logged into Google Cloud, we need to build docker image from a dockerfile. Follow official docker docs to see how to write a dockerfile. Here is an example of dockerfile. You will need to add “$DOCKER_USERNAME” and “$DOCKER_PASSWORD” as environment variables in Travis settings of your repo.

echo ">>> Building Docker image"
cd kubernetes/images

docker build --no-cache -t YOUR_DOCKER_USERNAME/YOUR_DOCKER_REPO:$TRAVIS_COMMIT .
docker login -u="$DOCKER_USERNAME" -p="$DOCKER_PASSWORD"
docker tag YOUR_DOCKER_USERNAME/YOUR_DOCKER_REPO:$TRAVIS_COMMIT YOUR_DOCKER_USERNAME/YOUR_DOCKER_REPO:latest

6. Now, just push the docker image created in previous step and update the deployment.

echo ">>> Pushing docker image"
docker push YOUR_DOCKER_USERNAME/YOUR_DOCKER_REPO

echo ">>> Updating deployment"
kubectl set image deployment/YOUR_CONTAINER_NAME --namespace=default YOUR_CONTAINER_NAME=YOUR_DOCKER_USERNAME/YOUR_DOCKER_REPO:$TRAVIS_COMMIT

Summary

This blog was about how we have configured travis build and auto deployed SUSI Server on Google Cloud Platform using Kubernetes and Docker. You can do the same with your server too or if you are looking to contribute to SUSI Server, this may help you a little in understanding the code of the repo.

Resources

  1. The documentation for setting up your project on Google CLoud Console before starting auto deployment https://github.com/fossasia/susi_server/blob/afb00cd9c421876f5d640ce87941e502aa52e004/docs/installation/installation_kubernetes_gcloud.md
  2. The documentation for encrypting your google cloud credentials and adding them to your repo https://cloud.google.com/solutions/continuous-delivery-with-travis-ci
  3. Docs for Docker to get you started with Docker https://docs.docker.com/
  4. Travis Documentation on how to secure your credentials https://docs.travis-ci.com/user/encryption-keys/
  5. Travis Documentation on how to add environment variables in your repo settings https://docs.travis-ci.com/user/environment-variables/

Setting Environment Variables up in Travis and Heroku for Meilix and Meilix-Generator

Meilix Generator is a webapp whose task is to take input as a configuration and start the Meilix build. But an anonymous person cannot start the Meilix build of any user and deploy the release in the repository. There are ways which are used as authentication passes through environment variable to start the build. In this article, I show the way I used to trigger Meilix by setting up environment variables in Meilix Generator.

Environment variables are of great use when one has to supply personal token in an open-source project for accessing the repository. So down there, we will have ways to configure the variables in Heroku and Travis. There are so many wikis out there but this one is the blend of both Heroku and Travis.

Heroku

There are several ways to setup variables in Heroku. The way I’m going to describe below is used to access Travis build using Heroku.
Using the Heroku variable generated using Travis will help to trigger the build on Travis.

How:

Idea:
We will use Travis CLI to generate a token (unique and keep it secret). Then provide the token as a variable name to the Heroku.
Backdoor:
This Travis token will give access to the Heroku to trigger the build on that particular Travis account. We use variable to provide the token since in the script we will use this variable as an environment variable to fetch the token in the place of token like as $token.
Implementation:
Open your terminal and type the following:

sudo apt install ruby ruby-dev
sudo gem install travis                       	# install Travis CLI
travis login --org   					# login into Travis
travis token --org		# generate your secret personal access token

You will get a token, copy and paste it into your Heroku app’s settings config vars token. You have to use the `KEY` as the variable which is used in the script for triggering the build. Save it and you are done the setting of the token in the Heroku.

Travis

Now it’s time for Travis token.
It is used to deploy the build to that repository only.
We can use the token in two ways either paste it in the setting of that repository on Travis or pasting the encrypted form of that in the .travis.yml file in that repository. Both will work. But one thing to remember that you must have the write access to that repository.

How

,Idea:
It is used in .travis.yml file as an environment variable to successfully build and deploy the application as a Github release.
Backdoor:
The token gives the permission to Travis to deploy the build application in the GitHub release of the repo(if one using to deploy it there only).
Implementation:
Head up to Github and generate a personal access token with scope repo. Copy the generated token in a safe place.

Way 1:

Paste the token in the setting of the repo in Travis in Environment Variable option. Now it will access the Github repository since it has got the permission from the personal token generated from Github.

Way 2:
Open terminal:

cd repo_name				# cd into the cloned repo
travis encrypt secret_token	#replace secret_token with the token generated

or

travis encrypt secret_token -r user/repo 	#if you are not in the repo

Copy that encrypted token and paste it in proper format in the .travis.yml file. Now you enabled Travis giving permission to release the build.

How can this idea be helpful to a developer

A developer can use this to build the Github Release in its repository. One can secure its token using this technique. One can use it to trigger its personal project in Travis using Heroku.

Useful repositories and link which uses this:

I have used the same idea in my project. Do have a look https://github.com/fossasia/meilix-generator
about environment variable
encryption keys
triggering build

Building Meilix in Travis using Heroku

Suppose you have to trigger (start) Travis but not through making a commit but through clicking a button on the webapp of the Meilix Generator. Through the webapp of Meilix Generator, we can pass the tag of the build which will be initiated and can also get the build link which is built by Travis.
Heroku is the place where we have deployed our webapp and through a button on the webapp that we used to start the build on the Travis. We have the access to give a tag to the build and with the help of this, we can even predict the URL of the build beforehand. So one can use it for its own personal project in a number of ways. And how I used this feature in Meilix Generator using Meilix script is described below:

How I used this idea

FOSSASIA meilix repository consists the script of a Linux Operating System based on Lubuntu. It uses Travis to build that script to result in a release of an iso file.

Now we thought an idea of building an autonomous system to start this build and get the release and in the meanwhile also make some required changes to the script to get it into the OS. We came up with an idea of a webapp which ask user its email id and tag of the build and till now a picture from the user which will be set as a wallpaper. It means the user would be able to config its distro according to its need through the graphical interface without a single line to code from the user end.

Through the webapp, a build button is taken as an input to go to a build page which triggers the Travis with the same user configuration to build the iso and deploy it on Github page. The user gets the link to the build on the next page only.

How I implemented this idea

Thanks to Travis API without which our idea is impossible to implement. We used a shell script to outframe our idea. The script takes the input of the user’s, repository, and branch to decide to where the trigger to take place.

There are two files one as travis_token as:

fossasia meilix master    # in the format of user repo branch

And script.sh as:  

#!/bin/bash
cat travis_tokens | while read line;     # this lines takes input of the user, repo and branch
do
    array=(${line})
    user="${array[0]}"
    project="${array[1]}"
    len=${#array[@]}
    for ((i=2; i<len; i++)); do
        branch="${array[i]}"            # supplied each value as variable
        body="{\"request\":{
            \"branch\":\"${branch}\",
            \"config\":{
                \"env\":{
                    \"email\":\"${email}\",    # supplied email and travis tag as environment variable
                    \"TRAVIS_TAG\":\"${TRAVIS_TAG}\"
                }
            }
    }}"
    echo "This Link Will be ready in approx 20 minutes"
    echo "https://github.com/fossasia/meilix/releases/download/${TRAVIS_TAG}/meilix-zesty-`date +%Y%m%d`-i386.iso"                  # a pre-predication of the link, we provide tag from user and date from system.
        curl -s -X POST \           # sending an API POST request to Travis to trigger the build of most recent commit 
            -H "Content-Type: application/json" \
            -H "Accept: application/json" \
            -H "Travis-API-Version: 3" \
            -H "Authorization: token ${KEY}" \     # this is stored in Heroku as KEY as environment variable and supplied from there only
            -d "${body}" \
            "https://api.travis-ci.org/repo/${user}%2F${project}/requests"  #%2 is used to interpret user and repo name as a single URL segment.
    done
done

After the trigger, you will get email which consists of a downloadable link to the iso.

How can this idea be helpful to a developer

There are lots of ways a developer can use this idea out. If a developer wants their user to automatically trigger the build and get the release build directly.

One can use it to set even the commit message through the shell script and customizing build configuration like replace, merge or deep_merge a configuration with the original .travis.yml file present in source repo.

Useful repositories and link which uses this:

Know more about Travis API v3:
Triggering the build
API blog

Have a look at our webapp and generate your own iso:
https://melix-generator.herokuapp.com/

Source code here:
https://github.com/fossasia/meilix-generator
https://github.com/fossasia/meilix

How Meilix Generator sends Email Notifications with SendGrid

We wanted to notify the users once the build was ready for download. To solve this we attempted making an email server on Meilix Generator but that can send email when it starts but it would take around 20 minutes to get the build ready so we thought of checking the deploy link status and send email whenever the link status was available (200) but the problem with this method was that the link can be pre available if ISO is rebuilt for same event.

Then, we attempted sending mail by Travis CI but the problem in that was closed SMTP ports (they have a strict policy about that) then we thought that Travis CI can trigger the Sendgrid which can send email to the user with the help of API.

We will use this code so that once the deployment of ISO by Travis CI is done it can execute the email script which requests Sendgrid to send email to the user.

after_deploy:
  - ./mail.py

 

We can create code using code generation service of Sendgrid we are going to choose python as it is easier to manipulate strings in python and we are going to use email as an environment variable.

After generation of python 3 code from the sendgrid website we are going to edit the message and email and hide the API key as an environment variable and create an authorization string to be used there too.

The URL will be generated by the below script as the body of url remains same only two things will change the TRAVIS_TAG which is event name and date.

date = datetime.datetime.now().strftime('%Y%m%d')
url="https://github.com/xeon-zolt/meilix/releases/download/"+os.environ["TRAVIS_TAG"]+"/meilix-zesty-"+date+"-i386.iso"

 

We can use this to hide the api key and use it as an environment variable because if the api key is visible in logs anyone can use it to exploit it and use it for spamming purpose.

authorization = "Bearer " + os.environ["mail_api_key"]
headers = {
    'authorization': authorization,

 

The main thing left to edit in the script is the message which is in the payload and is a string type so we are going to use the email received by Meilix generator as an environment variable and concatenate it with the payload string the message sent is in the value which is in the HTML format and we add the generated URL in similar way we added email variable to string.

payload = "{\"personalizations\":[{\"to\":[{\"email\":\"" + os.environ["email"] + "\"}],\"subject\":\"Your ISO is Ready\"}],\"from\":{\"email\":\"[email protected]\",\"name\":\"Meilix Generator\"},\"reply_to\":{\"email\":\"[email protected]\",\"name\":\"Meilix Generator\"},\"subject\":\"Your ISO is ready\",\"content\":[{\"type\":\"text/html\",\"value\":\"<html><p>Hi,<br>Your ISO is ready<br>URL : "+url+"<br><br>Thank You,<br>Meilix Generator Team</p></html>\"}]}"

 

The sent email looks like this

References