Update of Python Runtime in Meilix

Meilix Generator is a webapp uses flask with Python, Werkzeug, and Jinja 2. It triggers Travis to release in an ISO. It is deployed on Heroku. An older python version caused the webapp to load very slowly and it also become unsupported so there was a need to update the python version. In this blog post we walk through the update of Python in the project.

We can specify an explicit version of Python to be used to run your application. For example, if you require Python 2, add the following to your Pipfile:

[requires]
Python_version = "2.7"

 

Then run $ pipnv lock to generate Pipfile.lock and push to Heroku.

Another way:

If we are using pip, we can supply a runtime.txt file.

$ cat runtime.txt
python-3.6.1

 

Building of the webapp (Example)

The webapp build in Heroku and provide us a log. The log presents the packages installed and its version. Log also shows if any newer version is present for the package.

While building webapp, we get this as a log:

-----> Python app detected
! The latest version of Python 3 is python-3.6.5 (you are using python-3.6.1, which is unsupported).
! We recommend upgrading by specifying the latest version (python-3.6.5).

 

This confirms that we need to update python version and so thus we edited the runtime.txt

Now building the same webapp, we get:

Python app detected
-----> Found python-3.6.1, removing
-----> Installing python-3.6.5
-----> Installing pip

 

It already using older python, so it need first to remove the older version and then it install the latest one.

The same implementation can be seen in the history.

Reference:

Flask – The Microframework

Heroku Python Runtime

Continue ReadingUpdate of Python Runtime in Meilix

Adding Features into Meilix Generator Webapp

Meilix Generator is a webapp generated in FOSSASIA which takes input from user and send it to Meilix to trigger a build. Then a release is made whose link is emailed to the user. The webapp contains a form where there are fields for installing a particular packages, etc. In the following we will discuss about the ways to achieve the configurations in Meilix ISO without even touching the Meilix repo.

For adding an option in the webapp:

Editing the frontend

We need to edit this line in the index.html with this line:

<input name = "GENERATOR_package_vlc" type = "checkbox" value = "vlc" id = "vlc">

 

Making the script

Then we have to add a script in the script folder. This script is called by Meilix build script. This script contains the variable “GENERATOR_package_vlc”.

We name this file vlc-package.sh

Content:

#!/bin/bash
if echo "$GENERATOR_package_vlc" | grep -q vlc; then 
sudo apt-get install -q -y vlc; fi

 

Line 2 checks that the vlc is checked in the checkbox or not, if it is checked then the other next line gets executed otherwise not.

Calling the script from Meilix (needs to be done only once)

We will add a line in the Meililx build script to call those script present in the Meilix Generator repo.

SCRIPT_URL=https://www.github.com/fossasia/meilix-generator/archive/master.zip
wget -O $scripts.zip $SCRIPT_URL
unzip scripts.zip
SCRIPTS_FOLDER_IN_ZIP="meilix-generator-master/scripts"
ls $SCRIPTS_FOLDER_IN_ZIP; do
$SCRIPTS_FOLDER_IN_ZIP/script; done			#execute all scripts

 

Setting the URL via travis build config post to get all the values starting with GENERATOR_

GENERATOR_ = request.form['GENERATOR_']

 

So overall the abstract of the idea is:

  1. Getting the variables from html to travis as environment variable
  2. Cloning the meilix repo
  3. Executing all the scripts present.

References:

Request HTTP Python

Online Installation media

 

Continue ReadingAdding Features into Meilix Generator Webapp

Implementation of Features in Generator UI

In the early stage of development, Meilix Generator only has wallpaper and event name customization. But today the webapp has bunch of customization and features list which enables an user to design its own customizable ISO.

Iteration in the form

Meilix Generator came across several changes in the form throughout the time.

At starting we only have an email part where the ISO get mailed, a name for the event so as to distinguish the ISO image and an image upload which will be set as the default desktop wallpaper in the ISO.

Then the user gets a link which get activated after 20 minutes. Till then user have to preserve the link to download the ISO.

Then we introduced a new field which contains event link and this link will be set as the homepage of the browser. And we change the basic UI of the webapp.

At the same we implemented SendGrid to send the user the email link in their mail. This decreases the burden of carrying the downloadable link till the ISO becomes ready.

Finally today Meilix Generator looks like this. It got some more customizable fields like providing default search engine, bookmark enabling or disabling and packages to include in the ISO.

It has a link on the footer from which the latest pre-build ISO can be downloaded instantly and another link which takes user to the releases page of Meilix.

Reference:

SendGrid Email Delivery Service

SendGrid Email API

Continue ReadingImplementation of Features in Generator UI

Implementing Search Functionality In Calendar Mode On Schedule Page In Open Event Webapp

f79b5a2b-b679-4095-8311-cf403193e0cc.png

Calendar Mode

fef75c16-da0a-4145-a2e5-620d63637194.png

The list mode of the page already supported the search feature. We needed to implement it in the calendar mode. The corresponding issue for this feature is here. The whole work can be seen here.

First, we see the basic structure of the page in the calendar mode.

<div class="{{slug}} calendar">
 <!-- slug represents the currently selected date -->
 <!-- This div contains all the sessions scheduled on the selected date -->
 <div class="rooms">
   <!-- This div contains all the rooms of an event -->
   <!-- Each particular room has a set of sessions associated with it on that particular date -->
   <div class="room">
     <!-- This div contains the list of session happening in a particular room -->
     <div class="session"> <!-- This div contains all the information about a session -->
       <div class="session-name"> {{title}} </div> <!-- Title of the session -->
       <h4 class="text"> {{{description}}} </h4> <!-- Description of the session -->
       <!-- This div contains the info of the speakers presenting the session -->
       <div class="session-speakers-list">
         <div class="speaker-name"><strong>{{{title}}}</div> <!-- Name of the speaker -->
           <div class="session-speakers-more"> {{position}} {{organisation}} </div> <!-- Position and organization of speaker-->
         </div>
       </div>
     </div>
   </div>
 </div>
</div>

The user will type the query in the search bar near the top of the page. The search bar has the class fossasia-filter.

c30a84c8-c456-4116-86fa-c5ffc382bdb3.png

We set up a keyup event listener on that element so that whenever the user will press and release a key, we will invoke the event handler function which will display only those elements which match the current query entered in the search bar. This way, we are able to change the results of the search dynamically on user input. Whenever a single key is pressed and lifted off, the event is fired which invokes the handler and the session elements are filtered accordingly.

Now the important part is how we actually display and hide the session elements. We actually compare few session attributes to the text entered in the search box. The text attributes that we look for are the title of the session, the name, position , and organization of the speaker(s) presenting the session. We check whether the text entered by the user in the search bar appears contiguously in any of the above-mentioned attributes or not. If it appears, then the session element is shown. Otherwise, its display is set to hidden. The checking is case insensitive. We also count the number of the visible sessions on the page and if it is equal to zero, display a message saying that no results were found.

For example:- Suppose the user enters the string ‘wel’ in the search bar, then we will iterate over all the different sessions and only those who have ‘wel’ in their title or in the name/ position/organization of the speakers will be visible. Rest all the sessions would be hidden.

Here is the excerpt from the code. The whole file can be seen here

$('.fossasia-filter').change(function() {
 var filterVal = $(this).val(); // Search query entered by user
 $('.session').each(function() { // Iterating through all the sessions. Check for the title of the session and the name of the
   // speaker and its position and organization
if ($(this).find('.session-name').text().toUpperCase().indexOf(filterVal.toUpperCase()) >= 0 ||
 $(this).find('.session-speakers-list a p span').text().toUpperCase().indexOf(filterVal.toUpperCase()) >= 0 || $(this).find('.speaker-name').text().toUpperCase().indexOf(filterVal.toUpperCase()) >= 0) {
     $(this).show(); // Matched so display the session
   } else {
     $(this).hide(); // Hide the Element
   }
 });
 var calFilterLength = $('.calendar:visible').length;
 if((isCalendarView && calFilterLength == 0)) { // No session elements found
   $('.search-filter:first').after('<p id="no-results">No matching results found.</p>');
 }
}).keyup(function() {
 $(this).change();
});

Below is the default view of the calendar mode on the schedule page

471d6b30-d8db-4dea-8593-df0ad68072a6.png

On entering ‘wel’ in the search bar, sessions get filtered

d6eec31e-fde1-4640-881b-a0216b68533d.png

 References:

Continue ReadingImplementing Search Functionality In Calendar Mode On Schedule Page In Open Event Webapp

Implementing Logging Functionality in Open Event Webapp

  • INFO: Info statements give information about the task currently being performed by the webapp
  • SUCCESS: Success statements give the information of a task being successfully completed
  • ERROR: Error statements give information about a task failing to complete. These statements also contain a detailed error log

Along with the type of the statement, the object also contains information about the task. For all types of statements, there is a field called smallMessage containing short information about the task. For the ERROR statements where more information is required to see what went wrong, the message object has an additional field called largeMessage which holds detailed information about the event.

We also create a new file called buildlogger.js and define a function for creating log statements about the tasks being performed by generator and export it. The function creates a message object from the arguments received and then return it to the client under the buildLog event via the socket IO.

exports.addLog = function(type, smallMessage, socket, largeMessage) {
 var obj = {'type' : type, 'smallMessage' : smallMessage, 'largeMessage': largeMessage};
 var emit = false;
 if (socket.constructor.name === 'Socket') {
   emit = true;
 }
 if (emit) {
   socket.emit('buildLog', obj);
 }
};

Most of the steps of the generation process are defined in the generator.js file. So, we include the logging file there and call the addLog function for sending logs messages to the client. All the different steps like cleaning temporary folders, copying assets, fetching JSONs, creating the website directory, resizing images etc have multiple log statements for their inception and successful/erroneous completion. Below is an excerpt from the cleaning step.

var logger = require('./buildlogger.js');
 async.series([
   (done) => {
     console.log('CLEANING TEMPORARY FOLDERS\n');
     logger.addLog('Info', 'Cleaning up the previously existing temporary folders', socket);
     fs.remove(distHelper.distPath + '/' + appFolder, (err) => {
       if(err !== null) {
         // Sending Error Message when the remove process failed
         logger.addLog('Error', 'Failed to clean up the previously existing temporary folders', socket, err);
       }
       // Success message denoting the completion of the step
       logger.addLog('Success', 'Successfully cleaned up the temporary folders', socket);
       done(null, 'clean');
     });
   }
 )]

But we have only done the server side work now. We also have to handle the message on the client side. We send the message object to the client under the event buildLog and set up a listener for that event to catch the sent message. After the message object is received on the client side, we extract the information out of that object and then display it on the website. We have a div having an id of buildLog for displaying the log information. The content of the message is dynamically added to it as soon as it is received from the server. All the client side logic is handled in the form.js file.

socket.on('buildLog', function(data) {
   var spanElem = $('<span></span>'); // will contain the info about type of statement
   var spanMess = $('<span></span>'); // will contain the actual message
   var aElem = $('<button></button>'); // Button to view the detailed error log
   var paragraph = $('<p></p>'); // Contain the whole statement
   var divElem; // Contain the detailed error log
   spanMess.text(data.smallMessage);
   spanElem.text(data.type.toUpperCase() + ':');
   paragraph.append(spanElem);
   paragraph.append(spanMess);
   divElem.text(data.largeMessage);
   paragraph.append(aElem);
   paragraph.append(divElem);
   $('#buildLog').append(paragraph); // Div containing all the log messages
};

This is how the logs look on the client side. They are loaded on the go in real time as and when the events occur.

image (1).jpg

Resources:

Continue ReadingImplementing Logging Functionality in Open Event Webapp

Building a showcase site to display sample events and auto deploying them on each PR merge in Open Event Webapp

Open Event Webapp generates static websites of the event fed to it in the form of JSON data. Earlier, we used to automatically deploy the FOSSASIA Summit event website on the Github pages of the repository on every merge of a pull request.  The presence of it helped in showcasing and testing purposes. But a single event was not enough. Events are diverse, taking place in a large number of rooms, h a variety of sessions and extending over several days. So, having multiple events on the showcase site and continuously showcasing and testing them was a really great idea.

Here is the feature request for it. I will be explaining the major portions of it. The whole code can be found here

First of all, we need to build the main index page which we will showcase our sample events. It will contain the small images of the event and on clicking them, the selected event will be opened. For displaying features and capability of the generator,  we have two separate sections of events: one having the single session page and the other having the expandable sessions page. There will be links to the other components of the Open Event Ecosystem like Android App generator,  Web App generatorOrganizer App, and Eventyay.

The contents of that page are kept in the overviewSite folder. It contains the event background images of the events and the main index file. The basic structure of the file is shown below. The whole file can be viewed here

<div class="container-fluid bg-3 text-center">
 <div class="row margin">
   <div class="col-md-6 col-sm-12 col-xs-12">
     <div class="row">
       <h2 class="margin"> Apps with expandable sessions page </h2>
       <div class="col-md-6 col-sm-6 col-xs-12 margin">
         <p><strong>Open Tech Summit 2017</strong></p>
         <a href='./OpenTechSummit/index.html' target='_blank'>
           <img src="./otssmall.jpg" class=""  alt="Image">
         </a>
       </div>
      </div>
    </div>
   <div class="col-md-6 col-sm-12 col-xs-12">
     <div class="row">
       <h2 class="margin"> Apps with single sessions page </h2>
       <div class="col-md-6 col-sm-6 col-xs-12 margin">
         <p><strong>Mozilla All Hands 2017</strong></p>
         <a href='./MozillaAllHands2017/index.html' target='_blank'>
           <img src="./mozilla_banner.jpg" class=""  alt="Image">
         </a>
       </div>
     </div>
   </div>
 </div>
</div>

But, this is just the front end of the page. We haven’t generated the sample events yet and neither have we made any changes to make them auto deploy on each PR merge. The test script of the app which contains unit, acceptance and selenium tests runs on each PR made against the repo when Travis CI build is triggered. So, it makes sense to write code for generating the event there itself. When the test script has finished executing, all of the events would have been generated and present inside a single folder named a@a.com (We needed something short and easy to remember and the name doesn’t really matter). We then copy the contents of the overviewSite folder into the above folder. It already contains the folder of different sample events.

Here is the related code. The full test script file can be found here

describe('generate', function() {
 describe('.create different event sites and copy assets of overview site', function() {
   // Sample events are generated inside the a@a.com folder
   it('should generate the Mozilla All Hands 2017', function(done) {
     var data = {};
     // API endpoint configuration of the All Hands 2017 event and the session style set to single pages
     data.body = {
       "email": "a@a.com",
       "name": "Open Event",
       "apiendpoint":    "https://raw.githubusercontent.com/fossasia/open-event/master/sample/MozillaAllHands17",
       "sessionMode": "single",
       "datasource": "eventapi",
       "assetmode" : "download"
     };
     generator.createDistDir(data, 'Socket', function(appFolder) {
       assert.equal(appFolder, "a@a.com/MozillaAllHands2017");
       done();
     });
   });
   // For copying the static files required for the showcase site
   it('should copy all the static files', function(done) {
     var staticPath = __dirname + '/../src/backend/overviewSite/';
     function copyStatic(fileName) {
       // Copy the static files in the overviewSite folder to the a@a.com folder
     }
   });
 });
});

Everything is almost done now. We then just make some changes in the deploy script to publish the whole folder containing the different event samples instead of a particular sample and everything works fine.

We navigate to the a@a.com folder inside the dist directory and initialize a git repository there. We then set the username and email and fetch the contents of the gh-pages branch of the official repo (using the Github token. It is defined in the Travis Settings as a private environment variable) and reset it. We then add all the files present in the current directory to the staging area, commit them and push it to the upstream repository. Here is the relevant code. The whole github deploy script file can be viewed here

eval cd dist/a@a.com/
git init
git config --global user.name "Travis CI"
git config --global user.email "noreply+travis@fossasia.org"
git remote add upstream "https://$GH_TOKEN@github.com/"${TRAVIS_REPO_SLUG}".git"
git fetch upstream
git reset upstream/gh-pages
touch .
git add -A .
git commit -m "rebuild pages at ${rev}"
git push -q upstream HEAD:gh-pages

Here is the screenshot showing the successful deployment of the sample events. You can view the whole log here. A screenshot of it is below. You can view the higher quality image by clicking on it.

29e53dc0-8841-4329-8a37-8d2e9f08756c.png

After all the work, this is how the showcase page looks like. Samples are re-generated and the site is auto-deployed on every Pull Request merged in the repository.

66804d19-6862-494f-80b6-e1749493e780.png

On clicking on any of the listed events, we jump to the index page of that particular event. Like, If we click on the Facebook Developer Conference, a new tab is opened and we are taken to the front page of that event!

6a47f623-82c6-49e2-af1c-4d152aaaf9b7.png

References

Continue ReadingBuilding a showcase site to display sample events and auto deploying them on each PR merge in Open Event Webapp

Implementing Single Session Pages in Open Event Webapp

Recently, we implemented a feature request in the Open Event Webapp which allows the organizers of an event to choose the style of the sessions displayed on the event web site. Before this feature of individual session pages was implemented, we had a single default style of displaying collapsible session elements on the page. A small preview of the session containing its title, type, name, position, and picture of the speaker(s) presenting it would be shown on the different pages. When the user will click on this session element, it would collapse showing detailed information about it.

Before Clicking

2c12db41-d999-423a-bdee-6b639cbdbb0b.png

After Clicking on the first session element

fe1a3362-ebad-42a0-ba9d-0909c2de21a9.png

While this default behavior of collapse of session element on click (shown above) works well in most of the cases, it might not be apt in situations where the session contains a large amount of detail and a higher number of speakers presenting it. Single session pages would work better in that case.

So, we provided an option to select it on the generator form itself. We provided an input field asking which session style does the organizer want? Is it the single session style or the expandable session? The organizer can then select one of them and the site will be generated according to that!!

2989ee1f-509c-400d-94cd-023a06a3601d.png

The whole work was huge and you can view all of it here. I will only be describing the major parts of it here.

The first challenge was to make a template (handlebars) file for the individual sessions. This template would take a single session object as an input. The object would contain all the details about the session and after compilation during generation, a unique individual page for that session would be created.

Here is the basic structure of the template. You can view the whole file here

<div class="container session-container">
 <!-- Contains all the information about the session -->
 <div class="row single-session" id="{{session_id}}">
   <!-- Displaying the date, start and end time of the session -->
   <h3> {{startDay}} </h3>
   <div class = "eventtime"><span class="time-track">{{start}} - {{end}}</span></div>
   <div class="session-content">
     <div class="sizeevent event" id="event-title">
       <!-- Display the title, type and the track of the session -->
     </div>
     <div>
       <!-- Short abstract of the session -->
       {{{description}}}
       <div class="session-speakers-list">
         <!-- Contains detailed information about the speakers of the session. Display their picture, show their position, short biography and social links links like Github, Twitter and LinkedIn -->
       </div>
     </div>
   </div>
 </div>
</div>

But the work is not completed yet. We have to check the style of the session selected by the organizer and if he/she has selected the single session option, then we have to pass all the sessions to this template file during the generation process and create the pages. If the mode selected is expandable, then we carry out the normal generation procedure. Else, we extract every session from the JSON data and feed into to the above template. Since the number of sessions can be quite large, we don’t generate them alongside the other pages like tracks, schedule, rooms, speakers and so on. Instead, we create a new folder named sessions and put in all of the new individual pages there in one place. It helps to keep the directory clean and modularized. Also, since we are placing it inside of a session folder, we will have to update the links to the main pages in the navbar section. Like, instead of track.html, it will be ../tracks.html now. The file is given a common format name of session_sessionId where sessionId is the id of that particular session.

Here is the related code. It is taken from generator.js file in the project

function templateGenerate() {
 if(mode == 'single') {
   function checkLinks() {
     // Made necessary modifications in the links to the main pages
   }
   //jsonData contains all the information about the event
   var trackArr = jsonData.tracks;
   for(var i = 0; i < trackArr.length; i++) {
     var sessionArr = trackArr[i].sessions;
     for(var j = 0; j < sessionArr.length; j++) {
       var sessionObj = JSON.parse(JSON.stringify(sessionArr[j]));
       // Do some modifications in the sessionObj to include the track background and font color and additional links
     } 
   }
   var data = {session: sessionObj};
   checkLinks();
   // Pass the session object to the template, compile and minify the HTML file and place into the sessions folder
   fs.writeFileSync(distHelper.distPath + '/' + appFolder + '/sessions/session_' + sessionId + '.html', minifyHtml(sessiontpl(data)));
 }
}

Ok, most of the things are done now. Just one simple step is missing. When the user clicks on the small session element on the tracks, rooms or the schedule page, then we collapse the session element or open up a brand new session page for showing detailed information about the session depending upon what option the organizer has selected. So, we will have to make little appropriate change for handling this as well. Below code is taken from the tracks page template file

<div class = "room-filter">
 {{#if ../../../mode}}
   <div class="sizeevent event">
 {{else}}
   <div class="sizeevent event" data-toggle="collapse" data-target="#desc-{{session_id}}, #desc2-{{session_id}}">
 {{/if}}
</div>

$('.room-filter').click(function () {
 var sessionMode = "{{mode}}";
 var id =  $(this).attr('id');
 if(sessionMode == 'single') {
   var curUrl = window.location.href;
   var newUrl = curUrl.substring(0, curUrl.lastIndexOf('/') + 1) + 'sessions/session_' + id + ".html";
   window.location.href = newUrl;
 }
});

We are simply checking the mode and if it is set, it means that the session style is a single page. So, we don’t include the bootstrap collapse classes in that case. We handle that click event in the javascript part and appropriate redirect the user to the unique page for that session.

So, after all this hard work, this is how it looks like.

Before clicking

0fdb3641-2579-4537-8f36-910c636cf172.png

After clicking on the first session element, a new page is opened

e5d2ed0d-c54a-44fd-b30c-af440936c8ed.png

Resources

Continue ReadingImplementing Single Session Pages in Open Event Webapp