Meilix Build Process

Meilix is an operating system developed in FOSSASIA. It has the capability to be easily customize which makes it different from other operating systems.

It is of 32 bit right now.

It is all started with build.sh script.

And the basic idea is:

  • Declaring the mirror, version, and language to use.
  • Referring to the source file
  • Download and updating required packages
  • Create chroot
  • Using chroot to execute script

At the very beginning, the webapp Meilix Generator trigger Travis with the input provided in the form with the event name as the tag name of the Travis build. This will help to distinguish between the releases.

Then the travis script .travis.yml executes build.sh script after changing its permission.

  - chmod +x ./build.sh
  - ./build.sh |& tee log.txt

 

Let’s get into the build.sh script and look into it.

arch=${1:-i386}

 

First we select the bit version of the OS which we need to build.

datafiles="image-${arch}.tar.lzma sources.list"

 

This provides necessary data files required to build the OS. The tar.lzma contains exactly the same file which are there in an ISO of a OS. So it gives us the capability to make changes into the system files.

Then it starts to download the required packages which will help during the process.

Then runs a debuild script which debuilds the metapackages.

chmod +x ./scripts/debuild.sh
./scripts/debuild.sh

 

The debuild script actually repacks the meilix-default-setting metapackages with the required settings. Meilix-default-settings metapackage is fully explained here. Debuilding process is explained here.

Then Meilix goes on installing required packages and deleting the unrequired files.

Meilix build script is well commented to help to understand the whole process line by line.

Then Travis release a Meilix ISO in github release and mail the user with the log in the attachment.

Special Features:

Meilix has some special features which makes it fully customizable:

  • Meilix-default-setting metapackages, it contains the home folder and the changes made inside it can be seen in the OS.
  • Metapackages in the Meilix are very helpful in installing any required package.
  • Meilix System Lock helps to clean the system and get back to a fixed time.

Reference:

Flask Web Forms- Generator

Shell Scripting- Meilix

chroot – Debian Wiki

 

Adding Features into Meilix Generator Webapp

Meilix Generator is a webapp generated in FOSSASIA which takes input from user and send it to Meilix to trigger a build. Then a release is made whose link is emailed to the user. The webapp contains a form where there are fields for installing a particular packages, etc. In the following we will discuss about the ways to achieve the configurations in Meilix ISO without even touching the Meilix repo.

For adding an option in the webapp:

Editing the frontend

We need to edit this line in the index.html with this line:

<input name = "GENERATOR_package_vlc" type = "checkbox" value = "vlc" id = "vlc">

 

Making the script

Then we have to add a script in the script folder. This script is called by Meilix build script. This script contains the variable “GENERATOR_package_vlc”.

We name this file vlc-package.sh

Content:

#!/bin/bash
if echo "$GENERATOR_package_vlc" | grep -q vlc; then 
sudo apt-get install -q -y vlc; fi

 

Line 2 checks that the vlc is checked in the checkbox or not, if it is checked then the other next line gets executed otherwise not.

Calling the script from Meilix (needs to be done only once)

We will add a line in the Meililx build script to call those script present in the Meilix Generator repo.

SCRIPT_URL=https://www.github.com/fossasia/meilix-generator/archive/master.zip
wget -O $scripts.zip $SCRIPT_URL
unzip scripts.zip
SCRIPTS_FOLDER_IN_ZIP="meilix-generator-master/scripts"
ls $SCRIPTS_FOLDER_IN_ZIP; do
$SCRIPTS_FOLDER_IN_ZIP/script; done			#execute all scripts

 

Setting the URL via travis build config post to get all the values starting with GENERATOR_

GENERATOR_ = request.form['GENERATOR_']

 

So overall the abstract of the idea is:

  1. Getting the variables from html to travis as environment variable
  2. Cloning the meilix repo
  3. Executing all the scripts present.

References:

Request HTTP Python

Online Installation media

 

Speeding up the Travis Build to Decrease the Building Time

Meilix is the repository which uses build script to generate community version of lubuntu as LXQT Desktop. It usually takes around 25-26 to build and deploy the ISO as a Github Release on master branch.
Observing the build log we can see that there are most of the packages like debootstrap, squashfs-tool, etc which are being fetch and setup at the time of building which results in increasing build time.

The issue is to decrease the build time supplying the packages from Travis so that when we run build.sh we won’t be required to download them again and thus few minutes will get reduced.
We included list of packages to be pre-downloaded in .travis.yml

include:
  - os: linux
    addons:
      apt:
        sources:
          - ubuntu-toolchain-r-test
        packages:
          - debootstrap
          - genisoimage
          - p7zip-full
          - squashfs-tools
          - ubuntu-dev-tools
          - dpkg-dev
          - debhelper
          - fakeroot
          - devscripts

These are some important packages included in the build.sh  as devtools=”debootstrap genisoimage p7zip-full squashfs-tools ubuntu-dev-tools” which are always fetched, so we included it into .travis.yml and downloaded it before entering into the chroot environment. By specifying those packages in the .travis.yml, Travis provides those packages beforehand into the docker container so it will run our script. Since the scripts also include package then when the script runs apt-get it won’t download those packages again. They are specified outside the chroot environment because they are expected to be at the system the build.sh script is run to get the iso. By this way, we get a sharp decrease in build time as the internet in the Travis CI container is not so fast so the package download time can be avoided. Now the build is taking around 15-16 minutes to build and deploy.

One thing to note that we didn’t remove those packages off build.sh so that build.sh works outside Travis CI as well.

References:
Pull #176 by @abishekvashok
Speeding up Travis Build by Travis CI
Faster Build by atchai.com

Importing database with PHP script

In this post I would like to discuss how to import tables to database directly through PHP script.

For our Project Engelsystem we need to import the tables manually by the command line or PHPMYADMIN. Now the user need not worry about the importing table. They are directly imported through the script.

Initially we used to import the tables in a sql file by source command or mysql command.

$ mysql -u root -p

CREATE DATABASE engelsystem;

use engelsystem;

source db/install.sql;

source db/update.sql;

Script to import tables directly through PHP.

function import_tables() {
// get the database variables. 
global $DB_HOST, $DB_PASSWORD, $DB_NAME, $DB_USER;
// file names to import
 $import_install = '../db/install.sql';
 $import_update = '../db/update.sql';
 // command to import both the files.
 $command_install = 'mysql -h' .$DB_HOST .' -u' .$DB_USER .' -p' .$DB_PASSWORD .' ' .$DB_NAME .' < ' .$import_install;
 $command_update = 'mysql -h' .$DB_HOST .' -u' .$DB_USER .' -p' .$DB_PASSWORD .' ' .$DB_NAME .' < ' .$import_update;
 $output = array();
 // execute the command
 exec($command_install, $output, $worked_install);
 exec($command_update, $output, $worked_update);

// test whether they are imported successfully or not
switch ($worked_install && $worked_update) {
 case 0:
 return true;
 case 1:
 return false;
 }
}

Once we execute the above script the tables will be imported automatically. User need not import the tables manually.

In this way we can import tables using script. For more information visit about project please visit here.

Development: https://github.com/fossasia/engelsystem

Issues/Bugs:https://github.com/fossasia/engelsystem/issues

Writing Installation Script for Github Projects

I would like to discuss how to write a bash script to setup any github project and in particular PHP and mysql project. I would like take the example of engelsystem for this post.

First, we need to make a list of all environments and dependencies for our github project. For PHP and mysql project we need to install LAMP server.

The script to install all the dependencies for engelsystem are

echo "Update your package manager"
apt-get update

echo "Installing LAMP"
echo "Install Apache"
apt-get install apache2

echo "Install MySQL"
apt-get install mysql-server php5-mysql

echo "Install PHP"
apt-get install php5 libapache2-mod-php5 php5-mcrypt

echo "Install php cgi"
apt-cache search php5-cgi

We can even add echo statements with instructions.

Once we have installed LAMP. We need to clone the project from github. We need to install git and clone the repository. Script for installing git and cloning github repository are

echo "Install git"
apt-get install git
cd /var/www/html
echo "Cloning the github repository"
git clone --recursive https://github.com/fossasia/engelsystem.git
cd engelsystem

Now we are in the project directory. Now we need to set up the database and migrate the tables. We are creating a database engelsystem and migrating the tables in install.sql and update.sql.

echo "enter mysql root password"
# creating new database engelsystem
echo "create database engelsystem" | mysql -u root -p
echo "migrate the table to engelsystem database"
mysql -u root -p engelsystem < db/install.sql
mysql -u root -p engelsystem < db/update.sql

Once we are done with it. We need to copy config-sample.default.php to config.php and add the database and password for mysql.

echo "enter the database name username and password"
cp config/config-sample.default.php config/config.php

Now edit the config.php file. Once we have done this we need to restart apache then we can view the login page at localhost/engelsystem/public

echo "Restarting Apache"
service apache2 restart
echo "Engelsystem is successfully installed and can be viewed on local server localhost/engelsystem/public"

We need to add all these instructions in install.sh file.

Now steps to execute a bash file.

Change the permissions of install.sh file

$ chmod +x install.sh

Now run the file from your terminal by executing the following command

$ ./install.sh

Developers who are interested in contributing can work with us.

Development: https://github.com/fossasia/engelsystem

Issues/Bugs:https://github.com/fossasia/engelsystem/issues