Speeding up the Travis Build to Decrease the Building Time

Meilix is the repository which uses build script to generate community version of lubuntu as LXQT Desktop. It usually takes around 25-26 to build and deploy the ISO as a Github Release on master branch.
Observing the build log we can see that there are most of the packages like debootstrap, squashfs-tool, etc which are being fetch and setup at the time of building which results in increasing build time.

The issue is to decrease the build time supplying the packages from Travis so that when we run build.sh we won’t be required to download them again and thus few minutes will get reduced.
We included list of packages to be pre-downloaded in .travis.yml

include:
  - os: linux
    addons:
      apt:
        sources:
          - ubuntu-toolchain-r-test
        packages:
          - debootstrap
          - genisoimage
          - p7zip-full
          - squashfs-tools
          - ubuntu-dev-tools
          - dpkg-dev
          - debhelper
          - fakeroot
          - devscripts

These are some important packages included in the build.sh  as devtools=”debootstrap genisoimage p7zip-full squashfs-tools ubuntu-dev-tools” which are always fetched, so we included it into .travis.yml and downloaded it before entering into the chroot environment. By specifying those packages in the .travis.yml, Travis provides those packages beforehand into the docker container so it will run our script. Since the scripts also include package then when the script runs apt-get it won’t download those packages again. They are specified outside the chroot environment because they are expected to be at the system the build.sh script is run to get the iso. By this way, we get a sharp decrease in build time as the internet in the Travis CI container is not so fast so the package download time can be avoided. Now the build is taking around 15-16 minutes to build and deploy.

One thing to note that we didn’t remove those packages off build.sh so that build.sh works outside Travis CI as well.

References:
Pull #176 by @abishekvashok
Speeding up Travis Build by Travis CI
Faster Build by atchai.com

Working of Meilix Build Script

Meilix build script is divided into several parts the initial part consist of getting the build environment ready for build which includes installation of packages to build the ISO.

Iike debootstrap genisoimage p7zip-full squashfs-tools ubuntu-dev-tools.

The we have a debuild script which is ran by the build script in initially to rebuild the metapackages so that if any change is created it can be included in the build.

To make the change in metapackage like meilix-default-settings.deb we include the changes in the meilix-default-settings/ to test the meta package locally we can use the following script to build it for testing.

#!/bin/bash
rm meilix-default-settings_*                                    #removes the older meilix-default-settings packages
cd meilix-default-settings                                      #cd into the metapackage directory
debuild -uc -us                                                 #debuild the meilix-default-settings metapackage

 

After creation of packages we prepare chroot with the help of debbootstrap using ubuntu mirror for application installation after that we copy the files we created using debuild script and the files we require in the chroot.
We have a different file for working in the chroot which executes once the chroot is ready.

Now we copy the kernel from chroot to the build server /image for creation of live cd

sudo cp chroot/boot/vmlinuz-**-generic image/casper/vmlinuz sudo cp chroot/boot/initrd.img-**-generic image/casper/initrd.lz

Next step is to extract initrd for updating the uuid information for the above changes.

7z e image/casper/initrd.lz && \
  mkdir initrd_FILES/ && \
  mv initrd initrd_FILES/ && \
  cd initrd_FILES/ && \
  cpio -id < initrd && \
  cd .. && \
  cp initrd_FILES/conf/uuid.conf image/.disk/casper-uuid-generic && \

Now we pack the ISO using mkisofs but the problem here is we don’t have a boot.cat file and without that we will have errors in packing the ISO so to handle that we first create a temporary ISO to be able to extract boot.cat from it.

IMAGE_NAME=${IMAGE_NAME:-"Meilix ${release} $(date -u +%Y%m%d) - ${arch}"}
ISOFILE=meilix-${release}-$(date -u +%Y%m%d)-${arch}.iso
sudo mkisofs -r -V "$IMAGE_NAME" -cache-inodes -J -l \
  -b isolinux/isolinux.bin -c isolinux/boot.cat \
  -no-emul-boot -boot-load-size 4 -boot-info-table \
  --publisher "Meilix Packaging Team" \
  --volset "Ubuntu Linux http://www.ubuntu.com" \
  -p "${DEBFULLNAME:-$USER} <${DEBEMAIL:-on host $(hostname --fqdn)}>" \
  -A "$IMAGE_NAME" \
  -m filesystem.squashfs \
  -o ../$ISOFILE.tmp .

After getting a updated boot.cat file by mounting the temporary ISO and extracting boot.cat from it, we repeat the similar steps to create the ISO.

Chroot Script

Inside the chroot script we are going to install all the packages we require in the ISO and the metapackages we created for ISO. we can also add the custom changes we require like turning off screen dimming or setting plymouth after installing the package.

echo -ne "\033[9;0]" >> /etc/issue
setterm -blank 0 >> /etc/issue

At the end of chroot script we do the cleanup.

perl -i -nle 'print unless /^Package: language-(pack|support)/ .. /^$/;' /var/lib/apt/extended_states 
apt-get -qq clean
 rm -rf /tmp/* #rm /etc/resolv.conf 
rm meilix-default-settings_1.0_all.deb 
rm meilix-metapackage_1.0-1_all.deb 
rm systemlock_0.1-1_all.deb plymouth-meilix-logo_1.0-1_all.deb plymouth-meilix-text_1.0-1_all.deb skype-ubuntu_4.1.0.20-1_i386.deb rm meilix-imclient_*_all.deb
rm /sbin/initctl dpkg-divert --rename --remove /sbin/initctl

Resources

How to make changes in Meilix Without rebuilding the ISO

We were building Meilix from build scripts from webapp which was taking 20 minutes approx. So to reduce that time we had an idea of using a pre built ISO as it requires fewer resources and less time as compared to the building the ISO from build script and makes modifications in it which would take less time after testing it took approx 8 minutes. The following steps were followed to edit Meilix ISO.

We require following packages for unpacking and repacking the ISO.

  • squashfs-tools
  • Genisoimage

Let’s start by unpacking the ISO. For that, we first mount the ISO.

sudo mount -o loop meilix-zesty-20170611-i386.iso mnt/

 

Now we extract the content of the ISO into a directory extract-cd and extract the squash file system and move it to edit folder to prepare chroot.

sudo rsync --exclude=/casper/filesystem.squashfs -a mnt/ extract-cd
sudo unsquashfs mnt/casper/filesystem.squashfs
sudo mv squashfs-root edit

 

Now we can chroot and do the editing we require to do in the ISO.

sudo mount -o bind /run/ edit/run
sudo cp /etc/hosts edit/etc/
sudo mount --bind /dev/ edit/dev
sudo chroot edit

 

After doing the changes in chroot. For doing changes we can make a separate script to be executed inside the chroot.

exit
EOF
sudo umount edit/dev

 

After completing all the changes we required in the ISO the important part comes that is repacking the ISO with the applied changes.

Regenerate the manifest.

sudo chmod +w extract-cd/casper/filesystem.manifest
sudo su <<HERE
chroot edit dpkg-query -W --showformat='${Package} ${Version}\n' > extract-cd/casper/filesystem.manifest <<EOF
exit
EOF
HERE
sudo cp extract-cd/casper/filesystem.manifest extract-cd/casper/filesystem.manifest-desktop
sudo sed -i '/ubiquity/d' extract-cd/casper/filesystem.manifest-desktop
sudo sed -i '/casper/d' extract-cd/casper/filesystem.manifest-desktop

 

Now we compress the file system we have just edited.
For higher compression we can increase the block size or use xz but that will increase the cost of compression time so we didn’t choose it for Meilix as we required a faster method.

sudo mksquashfs edit extract-cd/casper/filesystem.squashfs -noappend

 

Now we are going to calculate the MD5 sums again for the changes and replace them with the older MD5 sums.

cd extract-cd/ && find . -type f -not -name md5sum.txt -not -path '*/isolinux/*' -print0 | xargs -0 -- md5sum > md5sum.txt

 

Last step is to go in the edit directory and generate the ISO.

mkisofs \
    -V "Custom Meilix" \
    -r -cache-inodes -J -l \
    -b isolinux/isolinux.bin \
    -c isolinux/boot.cat \
    -no-emul-boot -boot-load-size 4 -boot-info-table \
    -o ../meilix-i386-custom.iso .

 

This covers all the steps need to make changes in Meilix without rebuilding ISO.

Resources: