Meilix Build Process

Meilix is an operating system developed in FOSSASIA. It has the capability to be easily customize which makes it different from other operating systems.

It is of 32 bit right now.

It is all started with build.sh script.

And the basic idea is:

  • Declaring the mirror, version, and language to use.
  • Referring to the source file
  • Download and updating required packages
  • Create chroot
  • Using chroot to execute script

At the very beginning, the webapp Meilix Generator trigger Travis with the input provided in the form with the event name as the tag name of the Travis build. This will help to distinguish between the releases.

Then the travis script .travis.yml executes build.sh script after changing its permission.

  - chmod +x ./build.sh
  - ./build.sh |& tee log.txt

 

Let’s get into the build.sh script and look into it.

arch=${1:-i386}

 

First we select the bit version of the OS which we need to build.

datafiles="image-${arch}.tar.lzma sources.list"

 

This provides necessary data files required to build the OS. The tar.lzma contains exactly the same file which are there in an ISO of a OS. So it gives us the capability to make changes into the system files.

Then it starts to download the required packages which will help during the process.

Then runs a debuild script which debuilds the metapackages.

chmod +x ./scripts/debuild.sh
./scripts/debuild.sh

 

The debuild script actually repacks the meilix-default-setting metapackages with the required settings. Meilix-default-settings metapackage is fully explained here. Debuilding process is explained here.

Then Meilix goes on installing required packages and deleting the unrequired files.

Meilix build script is well commented to help to understand the whole process line by line.

Then Travis release a Meilix ISO in github release and mail the user with the log in the attachment.

Special Features:

Meilix has some special features which makes it fully customizable:

  • Meilix-default-setting metapackages, it contains the home folder and the changes made inside it can be seen in the OS.
  • Metapackages in the Meilix are very helpful in installing any required package.
  • Meilix System Lock helps to clean the system and get back to a fixed time.

Reference:

Flask Web Forms- Generator

Shell Scripting- Meilix

chroot – Debian Wiki

 

Continue ReadingMeilix Build Process

Adding Features into Meilix Generator Webapp

Meilix Generator is a webapp generated in FOSSASIA which takes input from user and send it to Meilix to trigger a build. Then a release is made whose link is emailed to the user. The webapp contains a form where there are fields for installing a particular packages, etc. In the following we will discuss about the ways to achieve the configurations in Meilix ISO without even touching the Meilix repo.

For adding an option in the webapp:

Editing the frontend

We need to edit this line in the index.html with this line:

<input name = "GENERATOR_package_vlc" type = "checkbox" value = "vlc" id = "vlc">

 

Making the script

Then we have to add a script in the script folder. This script is called by Meilix build script. This script contains the variable “GENERATOR_package_vlc”.

We name this file vlc-package.sh

Content:

#!/bin/bash
if echo "$GENERATOR_package_vlc" | grep -q vlc; then 
sudo apt-get install -q -y vlc; fi

 

Line 2 checks that the vlc is checked in the checkbox or not, if it is checked then the other next line gets executed otherwise not.

Calling the script from Meilix (needs to be done only once)

We will add a line in the Meililx build script to call those script present in the Meilix Generator repo.

SCRIPT_URL=https://www.github.com/fossasia/meilix-generator/archive/master.zip
wget -O $scripts.zip $SCRIPT_URL
unzip scripts.zip
SCRIPTS_FOLDER_IN_ZIP="meilix-generator-master/scripts"
ls $SCRIPTS_FOLDER_IN_ZIP; do
$SCRIPTS_FOLDER_IN_ZIP/script; done			#execute all scripts

 

Setting the URL via travis build config post to get all the values starting with GENERATOR_

GENERATOR_ = request.form['GENERATOR_']

 

So overall the abstract of the idea is:

  1. Getting the variables from html to travis as environment variable
  2. Cloning the meilix repo
  3. Executing all the scripts present.

References:

Request HTTP Python

Online Installation media

 

Continue ReadingAdding Features into Meilix Generator Webapp

Implementation of Features in Generator UI

In the early stage of development, Meilix Generator only has wallpaper and event name customization. But today the webapp has bunch of customization and features list which enables an user to design its own customizable ISO.

Iteration in the form

Meilix Generator came across several changes in the form throughout the time.

At starting we only have an email part where the ISO get mailed, a name for the event so as to distinguish the ISO image and an image upload which will be set as the default desktop wallpaper in the ISO.

Then the user gets a link which get activated after 20 minutes. Till then user have to preserve the link to download the ISO.

Then we introduced a new field which contains event link and this link will be set as the homepage of the browser. And we change the basic UI of the webapp.

At the same we implemented SendGrid to send the user the email link in their mail. This decreases the burden of carrying the downloadable link till the ISO becomes ready.

Finally today Meilix Generator looks like this. It got some more customizable fields like providing default search engine, bookmark enabling or disabling and packages to include in the ISO.

It has a link on the footer from which the latest pre-build ISO can be downloaded instantly and another link which takes user to the releases page of Meilix.

Reference:

SendGrid Email Delivery Service

SendGrid Email API

Continue ReadingImplementation of Features in Generator UI

Using file.io for Meilix Deployment

Meilix is developed in FOSSASIA and is deployed as a release on Github, but the download speed on GitHub for large files is slow. Alternatively deployment can be done on 3rd party servers. transfer.sh offers a good service for a start, but they only have a reduced storage for heavy usage such as what is required for Meilix. file.io is a good alternative. file.io has API features which can be used in .travis.yml to deploy meilix. For the time being, we are deploying a generator branch to file.io

Changes made in the .travis.yml

We need to edit the .travis.yml to deploy the artifact on file.io

deploy:
provider: script
script: curl -F "file=@/home/travis/$(image_name)" https://file.io
on:
branch: generator

We need to edit the deploy attribute in the travis to get the deployment done in file.io. Query contains the address of the file which needs to be uploaded on file.io. Then the branch name is provided on which deployment needs to be done.

Output:

Travis executes the command to deploy the application.

{"success":true,"key":"5QBEry","link":"https://file.io/5QBEry","expiry":"14 days"}

success: true  the artifact is successfully deployed.

key and link: gives the link from where the ISO can be downloaded.

expiry: tell the number of days after which the ISO will be deleted and by default it is set to 14 days.

We can manually input expiry parameter to declare the expiry time.

script: curl -F "file=@/home/travis/$(image_name)" https://file.io/?expires=1w

This will set the expiry time to seven days. If you set it with w, it will be number of weeks, m will be for number of months, y will be number of years.

file.io solved the most important issue of meilix deployment and this approach can be use several different project of FOSSASIA for the deployment purpose.

References:

Continue ReadingUsing file.io for Meilix Deployment

The Road to Success in Google Summer of Code 2017

It’s the best time when GCI students can get the overview experience of GSoC and all the aspiring participant can get themselves into different projects of FOSSASIA.

I’m a Junior year undergraduate student pursuing B.Tech in Electrical Engineering from Indian Institute of Technology Patna. This summer, I spent coding in Google Summer of Code with FOSSASIA organization. It feels great to be an open-source enthusiast, and Google as a sponsor make it as icing on the cake. People can learn new things here and meet new people.

I came to know about GSoC through my senior colleagues who got selected in GSoC in the year 2016. It was around September 2016 and I was in 2nd year of my college. At that time, last year, result of GSoC was declared.

What is GSoC?

Consider GSoC as a big bowl which has lots of small balls and those small balls are open-source organizations. Google basically acts as a sponsor for the open-source organizations. A timeline is proposed according to the applied organization and then student select their favorite organization and start to contribute to it. Believe me, it’s not only computer science branch specific, anyone can take part in it and there is no minimum CPI requirement. I consider myself to be one of the examples who have an electrical branch with not so good academic performance yet successfully being part of GSoC 2017.

How to select an organization?

This is the most important step and it takes time. I wandered around 100 organizations to find where my interest actually lies. But now, I’ll describe how to sort this and find your organization a little quicker. Take a pen and paper (kindly don’t use notepad of pc) and write down your field of interest in computer science. Number every point in decreasing order of your interest. Then for each respective field write down its basic pre-requisites. Visit GSoC website, go to organization tab and there is a slide for searching working field of the organization. Select only one organization, dig out its website, see the previous project and its application. If nothing fits you, repeat the same with another organization. And if that organization interests you, then look for a project of that organization. First of all, look at that application of the project, and give that application a try and must give a feedback to the organization. Then try to find that what languages, modules, etc that project used to work and how the project works. Don’t worry if nothing goes into your mind. Find out the developers mailing list, their chat channel, their code base area. And ask developers out there for help.

First Love It:

Open-Source, it’s a different world which exists on Earth. All organizations are open-source and all their codes are open and free to view. Find things that interests you the most and start to love the work. If you don’t understand a code, learn things by doing and asking. Most of the times we don’t get favorable responses, in such times we need to carry on and have patience for the best to happen.

My Favourite part:

GSoC has been my dream since the day I came to know about it. It’s only through this that one gets a chance to explore open-source softwares, and organizations get a chance to hire on board developers. This is the great initiative taken by Google which brings hope for the developers to increase the use of open-source. This is one of the ways through which one can look into the codes of the developers and help them out and even also get helped.

GSoC is the platform through which one can implement lots of new things, meet new people, develop new softwares and see the world around in a different way. That’s what happened with me, it’s just at the end of the first phase, my love towards open-source increased exponentially. Now I see every problem in my life as a way to solve it through the open-source. Rather it’s part of arranging an event or designing an invitation, I am encouraged to use open-source tools to help me out. It becomes very easy to distribute data and convey information through open-source, so the people can reach to you much easier.

You always see a thing according to your perspective and it’s always the best but the open-source gives it a view through the perspective of the world and gets the best from them through a compilation of all the sources. One can give ideas, their views, find something that other can’t even see and increase its karma through contribution. And all these things have been made possible through GOOGLE only. I became such that I can donate the rest of my life working for open-source. GSoC is responsible for including the open-source contribution in my daily life. It made me feel really bad if my Github profile page has 0 contributions at the end of the day. Open Source opens door to another world.

Challenging part:

To conclude, I would say that GSoC made me love the challenge. I became such that the things that come easily to me don’t taste good to me at all. Specifically, GSoC’s most challenging part is to get into it that is to get selected. I still can’t believe that I was selected. Now onwards it’s just fun and learning. Each and every day, I encountered several issues, bugs, etc but just before going to bed at night, there were things which collectively made me feel that whether the bug has been solved or not, but I was able to break the upper most covering of that conch shell. And such things increases the motivation and light up the enthusiasm to tackle the problem. Open-Source not only taught me to control different snapshots of software but also of time. I learn to manage different works of day efficiently and it includes the contribution in open-source as part of my daily life.

Advice to students:

The only problem new developers have is to get started. I’ll advise them to close their eyes and dive into it without thinking whether they would be able to complete this task or not. Believe me, you will gradually find that whether the task is completed or not but you are much above the condition than you were at the time of beginning the task.

Just learn by doing the things.

Make mistakes and enlist them as “things that will not work” so one may read it and avoid it.

GSoC Project link: https://summerofcode.withgoogle.com/projects/#5560333780385792
Final Code Submission: https://gist.github.com/meets2tarun/270f151d539298831ce542be5f733c82

Continue ReadingThe Road to Success in Google Summer of Code 2017

Speeding up the Travis Build to Decrease the Building Time

Meilix is the repository which uses build script to generate community version of lubuntu as LXQT Desktop. It usually takes around 25-26 to build and deploy the ISO as a Github Release on master branch.
Observing the build log we can see that there are most of the packages like debootstrap, squashfs-tool, etc which are being fetch and setup at the time of building which results in increasing build time.

The issue is to decrease the build time supplying the packages from Travis so that when we run build.sh we won’t be required to download them again and thus few minutes will get reduced.
We included list of packages to be pre-downloaded in .travis.yml

include:
  - os: linux
    addons:
      apt:
        sources:
          - ubuntu-toolchain-r-test
        packages:
          - debootstrap
          - genisoimage
          - p7zip-full
          - squashfs-tools
          - ubuntu-dev-tools
          - dpkg-dev
          - debhelper
          - fakeroot
          - devscripts

These are some important packages included in the build.sh  as devtools=”debootstrap genisoimage p7zip-full squashfs-tools ubuntu-dev-tools” which are always fetched, so we included it into .travis.yml and downloaded it before entering into the chroot environment. By specifying those packages in the .travis.yml, Travis provides those packages beforehand into the docker container so it will run our script. Since the scripts also include package then when the script runs apt-get it won’t download those packages again. They are specified outside the chroot environment because they are expected to be at the system the build.sh script is run to get the iso. By this way, we get a sharp decrease in build time as the internet in the Travis CI container is not so fast so the package download time can be avoided. Now the build is taking around 15-16 minutes to build and deploy.

One thing to note that we didn’t remove those packages off build.sh so that build.sh works outside Travis CI as well.

References:
Pull #176 by @abishekvashok
Speeding up Travis Build by Travis CI
Faster Build by atchai.com

Continue ReadingSpeeding up the Travis Build to Decrease the Building Time

Functionality and Customization of the Meilix Metapackage meilix-default-settings

Meilix has is made of build file and metapackages. Build file is responsible for executing commands and successfully implementing the work of metapackages.

Metapackages in Meilix
Name of metapackages used in Meilix are: meilix-artwork, meilix-default-settings.

meilix-default-settings

meilix-default-settings have 3 major folders debian, etc and usr and a Makefile. We are only concerned with etc and usr folder here.
etc and usr folders are folders in which if changes are made that can be seen the ISO. One can assume this as two folders present in the root folder of a Linux Distro.

Its directory is like this:

meilix-artwork

meilix-artwork has 1 main folder named as usr which contain share folder in which plymouth configuration is made. One can make changes here and it will directly seen in the Linux Distro.

Its directory looks like this:

How these meta packages actually work?
To get the answer one has to jump into the debian folder of any of the metapackage. It contains a control file. This contains information of the metapackages.

Source: meilix-default-settings
Section: x11
Priority: extra
Maintainer: meilix <vanhonit@gmail.com>
Build-Depends: debhelper (>= 8.0.0)
Standards-Version: 3.9.2
Homepage: http://mbm.vn

Package: meilix-default-settings
Architecture: all
Depends: ${shlibs:Depends}, ${misc:Depends}
Description: default settings for meilix
 Various system settings tailored for meilix.

One can update the metapackage from here and tweak with its depends. One come to know about the maintainer of the metapackage which can contacted in case of any issue. We can also know for which architecture this metapackage is made and about its description.
The whole debian does the work but after making any changes in the metapackage, it needs to be rebuild which is performed by debuild.sh. This is how a metapackages in Meilix works.

References:
Linux MetapackagesMatthartley from linux.com
Creating a MetapackageAjmitch from askubuntu.com

Continue ReadingFunctionality and Customization of the Meilix Metapackage meilix-default-settings

Deleting Meilix Github Releases

Meilix is the repository which uses build script to generate community version of lubuntu as LXQT Desktop. Meilix-Generator is the webapp which uses Meilix to generate ISO and deploy it on Meilix Github Release. Then the webapp mail the link of the ISO to the user.
Increasing number of ISO will increase the number of releases which results in dirty looking of Meilix repository. So we need to delete older releases after certain interval of time to make the repository release page looks good and decrease unwanted space.
This releases_maintainer.sh script will do this work for us.

#!/usr/bin/env bash
set -e
echo "This is a script to delete obsolete meilix iso builds by Abishek V Ashok"
echo "You have to add an authorization token to make it functional."

# jq is the JSON parser we will be using
sudo apt-get -y install jq

# Storing the response to a variable for future usage
response=`curl https://api.github.com/repos/fossasia/meilix/releases | jq '.[] | .id, .published_at'`

index=1  # when index is odd, $i contains id and when it is even $i contains published_date
delete=0 # Should we delete the release?
current_year=`date +%Y`  # Current year eg) 2001
current_month=`date +%m` # Current month eg) 2
current_day=`date +%d`   # Current date eg) 24

for i in $response; do
    if [ $((index % 2)) -eq 0 ]; then # We get the published_date of the release as $i's value here
        published_year=${i:1:4}
        published_month=${i:6:2}
        published_day=${i:9:2}

        if [ $published_year -lt $current_year ]; then
             let "delete=1"
        else
            if [ $published_month -lt $current_month ]; then
                let "delete=1"
            else
                if [ $((current_day-$published_day)) -gt 10 ]; then
                    let "delete=1"
                fi
            fi
        fi
    else # We get the id of the release as $i`s value here
        if [ $delete -eq 1 ]; then
            curl -X DELETE -H "Authorization: token $KEY" https://api.github.com/repos/fossasia/meilix/releases/$i
            let "delete=0"
        fi
    fi
    let "index+=1"
done

This code uses Github API to curl the Meilix releases. Github API is very useful in providing lots of information but here we are only concerned with the release date and time of the build.
Then we setup a condition if that satisfies then the release will automatically will get deleted.

For taking care of the authentication, a token has been uploaded to the Travis settings of Meilix of FOSSASIA.

The personal token has been generated by a user with write access to the repository with repo scope token.

This sort out the issue of having bulk of releases in the Meilix repository of FOSSASIA.

References:
Users Github API  by REST API v3
Repo Github API   by REST API v3

Continue ReadingDeleting Meilix Github Releases

Updating of Linux Standard Base (lsb) and Shorten of log of Meilix Build

Updating the Linux Standard Base of the Meilix

Originally Meilix uses the Ubuntu mirror to fetch the Kernel source for the building of the Operating System. Therefore whenever a user type lsb_release -a for fetching the required information, they get the Ubuntu build info. The task is to change the config file to update the Linux Base Information from Ubuntu to Meilix.

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 17.04
Release: 17.04
Codename: zesty

We need to patch a file in the location meilix-default-settings/etc/lsb-release which contains the information of the lsb release and this will overwrite the original configuration of Meilix.
This is how the lsb-release file looks like now:

DISTRIB_ID=Meilix
DISTRIB_RELEASE=17.04
DISTRIB_CODENAME=meilix
DISTRIB_DESCRIPTION="Meilix 17.04"

We made the required changes in the file as shown above and the output is as follows:

Shorten the log length of Meilix in Travis

We are facing issues while deployment of Meilix ISO in Travis and the error follows is ReadTimeout Error. Example log of one of fail build is:

This error gets solved automatically and the the ISO gets deployment after 1 or 2 restart of the build. But this time the error doesn’t get solved and after several attempts of restart of build, the ISO doesn’t get deployed.

Reason behind the error:

Travis is taking a lot of time to build the ISO. Travis logs are exceeding the time limit.

Proposed solution:

Reduce the time of build or shift to a new CI.
Reduce the log of the build so as to get the log within 9999 lines.

Solution Implemented

The best solution is to reduce the number of lines used in the log and this will also reduce the time of the build.
I tried concealing some command outputs by appending >/dev/null 2>&1 to some of the commands that has long outputs and adding -y to the commands like:

apt-get -qq -y --purge install file-roller unrar

References

Wiki of Linux Standard Base
Linux Foundation lsb
Ubuntu answer to reduce log

Continue ReadingUpdating of Linux Standard Base (lsb) and Shorten of log of Meilix Build

Turning off Power Management and Blanking of Screen in Meilix

Meililx has the LXQt desktop environment which already have the lxqt-powermanagement package. We don’t need to configure the power management during an event. Every time when an ISO is booted a pop-up arrives which asks to configure power. On the one hand, the pop-ups are a disturbing element during a presentation, slide-show, etc. On the other hand, while a presentation is going on and the presenter is explaining a slide for a longer duration then there is a possibility of the screen to turn blank or screen saver to get started due to inactivity. In this post we will discuss to resolve the above two issues.

Disabling Power Management
The issue has been filed #133 to solve. We can follow two approaches to shutdown the power daemon which is responsible for starting the power-management everytime during startup.

In the beginning the issue looks as:

One is to copy the script to /etc/X11/xorg.conf.d/10-monitor.conf by editing the meilix-default-settings metapackage. 

1. [panel1]
2. alignment=-1
3. animation-duration=0
4. background-color=@Variant(\0\0\0\x43\0\xff\xff\0\0\0\0\0\0\0\0)
5. background-image=
6. desktop=0
7. font-color=@Variant(\0\0\0\x43\0\xff\xff\0\0\0\0\0\0\0\0)
8. hidable=true
9. iconSize=22
10. lineCount=1
11. lockPanel=false
12. opacity=100
13.panelSize=32
14. plugins=mainmenu, desktopswitch, quicklaunch, taskbar, tray, statusnotifier, mount, volume, clock, showdesktop
15. position=Bottom
16. show-delay=0
17. width=100
18. width-percent=true

Explanation of code:

In the Monitor section, DPMS (Desktop Power Management System) is turned off by setting the value to false
In the ServerLayout section, Blanktime, Offtime, SuspendTime, StandbyTime are set to “0”, so that they will be disabled.
Through this way we are suspending the power-management to use it feature by making changes in the script. This approach will make sure that this feature can’t be changed through the GUI, too.

And the other approach is the basic one but it’s also fulfilling the similar need. Adding a line to chroot.sh

apt-get purge lxqt-powermanagement

Both will solve the issue and the pop will not arrive at the startup of the OS.
Now LXQt desktop starts as:

Disabling Blanking of Screen in Meilix
A presenter never wants to turn his monitor blank or screensaver while presenting his presentation. Here we will edit the .profile file in the home folder to fulfill our requirement.
For get the same file in the ISO, we need to patch the file meilix-default-settings/etc/skel/.profile.
Skel folder has the property to transfer the tree inside the folder to the home folder of the new user created. For example here, the new user created will have .profile folder inside its home directory and it will apply the requested changes.

Important code which is responsible for turning the blanking of screen off:

xset s off && xset -dpms

 

Link to important pages:
Disabling Power Management – Arch Wiki
Code for turning Power Management (PR) – @meets2tarun (author)
Code for turning off blanking of screen (PR) – @meets2tarun (author)

Continue ReadingTurning off Power Management and Blanking of Screen in Meilix