Open Event Server – Pages API

This article illustrates how the Pages API has been designed and implemented on the server side, i.e., FOSSASIA‘s Open Event Server. Pages endpoint is used to create static pages such as “About Page” or any other page that doesn’t need to be updated frequently and only a specific content is to be shown.

Parameters

  1. name – This stores the name of the page.
      1. Type – String
      2. Required – Yes
  2. title – This stores the title of the page.
      1. Type – String
      2. Required – No
  3. url – This stores the url of the page.
      1. Type – String
      2. Required – Yes
  4. description – This stores the description of the page.
      1. Type – String
      2. Required – Yes
  5. language – This stores the language of the page.
      1. Type – String
      2. Required – No
  6. index – This stores the position of the page.
      1. Type – Integer
      2. Required – No
      3. Default – 0
  7. place – Location where the page will be placed.
      1. Type – String
      2. Required – No
      3. Accepted Values – ‘footer’ and ‘event’

These are the allowed parameters for the endpoint.

Model

Lets see how we model this API. The ORM looks like this :

__tablename__ = 'pages'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String, nullable=False)
title = db.Column(db.String)
url = db.Column(db.String, nullable=False)
description = db.Column(db.String)
place = db.Column(db.String)
language = db.Column(db.String)
index = db.Column(db.Integer, default=0)

As you can see, we created a table called “pages”. This table has 8 columns, 7 of which are the parameters that I have mentioned above. The column “id” is an Integer column and is the primary key column. This will help to differentiate between the various entries in the table.

The visualisation for this table looks as follows :

API

We support the following operations:

  1. GET all the pages in the database
  2. POST create a new page
  3. GET details of a single page as per id
  4. PATCH a single page by id
  5. DELETE a single page by id

To implement this we first add the routes in our python file as follows :

api.route(PageList, 'page_list', '/pages')
api.route(PageDetail, 'page_detail', '/pages/<int:id>')

Then we define these classes to handle the requests. The first route looks as follows:

class PageList(ResourceList):
   """
   List and create page
   """
   decorators = (api.has_permission('is_admin', methods="POST"),)
   schema = PageSchema
   data_layer = {'session': db.session,
                 'model': Page}

As can be seen above, this request requires the user to be an admin. It uses the Page model described above and handles a POST request.

The second route is:

class PageDetail(ResourceDetail):
   """
   Page detail by id
   """
   schema = PageSchema
   decorators = (api.has_permission('is_admin', methods="PATCH,DELETE"),)
   data_layer = {'session': db.session,
                 'model': Page}

This route also requires the user to be an admin. It uses the Page model and handles PATCH, DELETE requests.

To summarise our APIs are:

GET

/v1/pages{?sort,filter}

POST

/v1/pages{?sort,filter}

GET

/v1/pages/{page_id}

PATCH

/v1/pages/{page_id}

DELETE

/v1/pages/{page_id}

References

Continue ReadingOpen Event Server – Pages API

Open Event Frontend – Updating Ember Models Table from V1 to V2

FOSSASIA‘s Open Event Frontend uses the Ember Models Table for rendering all its tables. This provides features like easy sorting, pagination etc. Another major feature is that it can be modified to meet our styling needs. As we use Semantic UI for styling, we added the required CSS classes to our table.

In version 1 this was done by overriding the classes, as shown below :

const defaultMessages = {
  searchLabel            : 'Search:',
  searchPlaceholder      : 'Search',


  ..... more to follow 
};

const defaultIcons = {
  sortAsc         : 'caret down icon',
  sortDesc        : 'caret up icon',
  columnVisible   : 'checkmark box icon',
  
  ..... more to follow  
};

const defaultCssClasses = {
  outerTableWrapper              : 'ui ui-table',
  innerTableWrapper              : 'ui segment column sixteen wide inner-table-wrapper',
  table                          : 'ui tablet stackable very basic table',
  globalFilterWrapper            : 'ui row',

 ... more to follow
};

const assign = Object.assign || assign;

export default TableComponent.extend({
  layout,

  _setupMessages: observer('customMessages', function() {
    const customIcons = getWithDefault(this, 'customMessages', {});
    let newMessages = {};
    assign(newMessages, defaultMessages, customIcons);
    set(this, 'messages', O.create(newMessages));
  }),

  _setupIcons() {
    const customIcons = getWithDefault(this, 'customIcons', {});
    let newIcons = {};
    assign(newIcons, defaultIcons, customIcons);
    set(this, 'icons', O.create(newIcons));
  },

  _setupClasses() {
    const customClasses = getWithDefault(this, 'customClasses', {});
    let newClasses = {};
    assign(newClasses, defaultCssClasses, customClasses);
    set(this, 'classes', O.create(newClasses));
  },

  simplePaginationTemplate: 'components/ui-table/simple-pagination',

  ........
});

And was used in the template as follows:

<div class="{{classes.outerTableWrapper}}">
  <div class="{{classes.globalFilterDropdownWrapper}}">

But in version 2, some major changes were introduced as follows:

  1. All partials inside a models-table were replaced with components
  2. models-table can now be used with block content
  3. New themes mechanism introduced for styling

Here, I will talk about how the theming mechanism has been changed. As I mentioned above, in version 1 we used custom classes and icons. In version 2 the idea itself has changed. A new type called Theme was added. It provides four themes out of the box – SemanticUI, Bootstrap4, Bootstrap3, Default.

We can create our custom theme based on any of the predefined themes. To suit our requirements we decided to modify the SemanticUI theme. We created a separate file to keep our custom theme so that code remains clean and short.

import Default from 'ember-models-table/themes/semanticui';

export default Default.extend({
 components: {
   'pagination-simple'    : 'components/ui-table/simple-pagination',
   'numericPagination'    : 'components/ui-table/numeric-pagination',
   .....  
 },

 classes: {
   outerTableWrapper              : 'ui ui-table',
   innerTableWrapper              : 'ui segment column sixteen wide inner-table-wrapper',
   .....
 },

 icons: {
   sortAsc         : 'caret down icon',
   sortDesc        : 'caret up icon',
   ......
 },

 messages: {
   searchLabel            : 'Search:',
   .....
 }
});

So a theme mostly consists of four main parts:

  • Components
  • Classes
  • Icons
  • Messages

The last three are same as customClasses and customIcons and customMessages in version 1. Components is the map for components used internally in the models-table. In case you need to use a custom component, that can be done as follows:

Make a new JavaScript file and provide its path in your theme file.

import DefaultDropdown from '../../columns-dropdown';
import layout from 'your layout file path';
export default DefaultDropdown.extend({
  layout
});

Now just create the theme file object and pass it to themeInstance in the ui-table file (can also be passed in the template and the controller, but this has to be done for each table individually).

import TableComponent from 'ember-models-table/components/models-table';
import layout from 'open-event-frontend/templates/components/ui-table';
import Semantic from 'open-event-frontend/themes/semantic';

export default TableComponent.extend({
 layout,

 themeInstance: Semantic.create()
});

Hence, version 2 introduces many new styling options and requires some refactoring for those who were using version 1. It is totally worth it though considering how easy and well managed it is now.

References

Continue ReadingOpen Event Frontend – Updating Ember Models Table from V1 to V2

Implementing Sponsors API for Events and Using Image Upload Widget in Open Event Frontend

 

This blog article will talk about how sponsors API has been implemented in events edit dashboard of Open Event Frontend. This discussion involves the /events/{event_identifier}/edit/sponsors route. Primary API endpoint of Open Event Server for fetching sponsors of an event are

GET         /v1/events/{event_identifier}/sponsors{?sort,filter}

GET        /v1/sponsors/{sponsor_id}

Next, we define the corresponding models according to the type of response returned by the server. This model extends the Base model.

import attr from 'ember-data/attr';
import ModelBase from 'open-event-frontend/models/base';
import { belongsTo } from 'ember-data/relationships';
import { computedSegmentedLink } from 'open-event-frontend/utils/computed-helpers';

export default ModelBase.extend({
 name        : attr('string'),
 level       : attr('number'),
 type        : attr('string'),
 url         : attr('string'),
 description : attr('string'),
 logoUrl     : attr('string'),

 event: belongsTo('event'),

 segmentedLink: computedSegmentedLink.bind(this)('url')
});

 

Here all the field attributes clearly signify what they mean. Event field has a relationship to events model, hence it is bound to event model through

belongsTo( ).

Next we fetch the data from the API and feed it into sponsor edit form available at event/{event_identifier}/edit/sponsor .

import Route from '@ember/routing/route';
import EventWizardMixin from 'open-event-frontend/mixins/event-wizard';

export default Route.extend(EventWizardMixin, {
 titleToken() {
   return this.get('l10n').t('Sponsors');
 },
 async model() {
   let data = this.modelFor('events.view.edit');
   data.sponsors = await data.event.get('sponsors');
   return data;
 }
});

 

We have defined model() asynchronously and return the fetched data to template. This data passes into sponsor form of event wizard located here.

We see that this form contains many widgets for handling form validation and its structure. Here we will explore the image upload widget of open event frontend that helps us adding image upload option to many forms across open event frontend.

Image upload component is like any other component of ember with many functions. This widget mainly includes processFiles() and uploadFiles(). Let us look at their working one by one. Full code of image-upload.js can be seen here.

processFiles(files) {
   if (files && files[0]) {
     isFileValid(files[0], this.maxSizeInKb, ['image/jpeg', 'image/png']).then(() => {
       const reader = new FileReader();
       reader.onload = e => {
         const untouchedImageData = e.target.result;
         if (this.get('needsCropper')) {
           this.set('imgData', untouchedImageData);
           this.set('cropperModalIsShown', true);
         } else {
           this.uploadImage(untouchedImageData);
         }
       };
       reader.readAsDataURL(files[0]);

     }).catch(error => {
       this.notify.error(error);
     });
   } else {
     this.notify.error(this.get('l10n').t('No FileReader support. Please use a more latest browser'));
   }

 },

 

This function accepts file input and processes them. It first passes it to a isFileValid() function with all arguments, which returns true if the files are in correct format. It then checks the dimensions and figures out if the image needs a cropper for cropping image. If yes, it opens a cropper modal and lets user crop the image to perfect size.

It then calls uploadImage() function to upload the cropped image. UploadImage() function looks something like this:

uploadImage(imageData) {
   this.set('selectedImage', imageData);
   this.set('needsConfirmation', false);
   this.set('uploadingImage', true);
   this.get('loader')
     .post('/upload/image', {
       data: imageData
     })
     .then(image => {
       this.set('uploadingImage', false);
       this.set('imageUrl', image.url);
     })
     .catch(() => {
       this.set('uploadingImage', false);
       this.set('errorMessage', this.i18n.t('An unexpected error occurred.'));
     });
 },

 

This function on receiving image data send a post request to the server for uploading the image to requested directory. If the server does not behave properly then it exits with an error.

Widgets such as image-upload help us to maintain our code in a better way by allowing re-usability of code. Ember provides good support for such controller, adapters, widgets to be used in the app.

Resources:

Continue ReadingImplementing Sponsors API for Events and Using Image Upload Widget in Open Event Frontend

Open Event Server – Change a Column from NULL to NOT NULL

FOSSASIA‘s Open Event Server uses alembic migration files to handle all database operations and updating. Whenever the database is changed a corresponding migration python script is made so that the database will migrate accordingly for other developers as well. But often we forget that the automatically generated script usually just add/deletes columns or alters the column properties. It does not handle the migration of existing data in that column. This can lead to huge data loss or error in migration as well.

For example :

def upgrade():
    # ### commands auto generated by Alembic - please adjust! ###
    op.alter_column('ticket_holders', 'lastname',
                    existing_type=sa.VARCHAR(),
                    nullable=False)
    # ### end Alembic commands ###

Here, the goal was to change the column “ticket_holders” from nullable to not nullable. The script that alembic autogenerated just uses op.alter_column().

It does not count for the already existing data. So, if the column has any entries which are null, this migration will lead to an error saying that the column contains null entries and hence cannot be “NOT NULL”.

How to Handle This?

Before altering the column definition we can follow the following steps :

  1. Look for all the null entries in the column
  2. Give some arbitrary default value to those
  3. Now we can safely alter the column definition

Let’s see how we can achieve this. For connecting with the database we will use SQLAlchemy. First, we get a reference to the table and the corresponding column that we wish to alter.

ticket_holders_table = sa.sql.table('ticket_holders',
                                        sa.Column('lastname', sa.VARCHAR()))

 

Since we need the “last_name” column from the table “ticket_holders”, we specify it in the method argument.

Now, we will give an arbitrary default value to all the originally null entries in the column. In this case, I chose to use a space character.

op.execute(ticket_holders_table.update()
               .where(ticket_holders_table.c.lastname.is_(None))
               .values({'lastname': op.inline_literal(' ')}))

op.execute() can execute direct SQL commands as well but we chose to go with SQLAlchemy which builds an optimal SQL command from our modular input. One such example of a complex SQL command being directly executed is :

op.execute('INSERT INTO event_types(name, slug) SELECT DISTINCT event_type_id, lower(replace(regexp_replace(event_type_id, \'& |,\', \'\', \'g\'), \' \', \'-\')) FROM events where not exists (SELECT 1 FROM event_types where event_types.name=events.event_type_id) and event_type_id is not null;'))

Now that we have handled all the null data, it is safe to alter the column definition. So we proceed to execute the final statement –

op.alter_column('ticket_holders', 'lastname',
                    existing_type=sa.VARCHAR(),
                    nullable=False)

Now the entire migration script will run without any error. The final outcome would be –

  1. All the null “last_name” entries would be replaced by a space character
  2. The “last_name” column would now be a NOT NULL column.

References

Continue ReadingOpen Event Server – Change a Column from NULL to NOT NULL

Open Event Web App – A PWA

Introduction

Progressive Web App (PWA) are web applications that are regular web pages or websites but can appear to the user like traditional applications or native mobile applications. The application type attempts to combine features offered by most modern browsers with the benefits of mobile experience. Open Event web app is a web application generator which has now introduced this new feature in its generated applications.

 

Why Progressive Web Apps?

The reasons why we enabled this functionality are that PWAs are –

  • Reliable – Load instantly and never show the downasaur, even in uncertain network conditions.
  • Fast – Respond quickly to user interactions with silky smooth animations and no janky scrolling.
  • Engaging – Feel like a natural app on the device, with an immersive user experience.

Thus where Open Event Web app generated applications are informative and only requires one time loading with functionalities like bookmarks depending on local storage of browser, we found Progressive web apps perfect to explain and demonstrate these applications as a whole.

How PWAs work?

The components associated with a progressive web application are :

Manifest: The web app manifest is a W3C specification defining a JSON-based manifest to provide developers a centralized place to put metadata associated with a web application.

Service Workers: Service Workers provide a scriptable network proxy in the web browser to manage the web/HTTP requests programmatically. The Service Workers lie between the network and device to supply the content. They are capable of using the cache mechanisms efficiently and allow error-free behavior during offline periods.

How we turned Open event Web app to a PWA?

Adding manifest.json

 {
"icons": [
    {
      "src": "./images/logo.png",
      "type": "image/png",
      "sizes": "96x96"
    }
  ],
  "start_url": "index.html",
  "scope": ".",
  "display": "standalone",
  "orientation": "portrait-primary",
  "background_color": "#fff",
  "theme_color": "#3f51b5",
  "description": "Open Event Web Application Generator",
  "dir": "ltr",
  "lang": "en-US"
}

 

Adding service workers

The initialization of service workers is done by calling an event listener namely ‘install’ :

var urlsToCache = [
 './css/bootstrap.min.css',
 './offline.html',
 './images/avatar.png'
];

self.addEventListener('install', function(event) {
 event.waitUntil(
   caches.open(CACHE_NAME).then(function(cache) {
     return cache.addAll(urlsToCache);
   })
 );
});

The service workers fetch the data from the cache when event listener ‘fetch’ is triggered. When a cache hit occurs the response data  is sent to the client from there otherwise it tries to fetch the data by making a request to the network. In case when network does not send response status code ‘200’, it sends an error response otherwise caches the data received.

self.addEventListener('fetch', function(event) {
 event.respondWith(
   caches.match(event.request).then(function(response) {
     // Cache hit - return response
     if (response) {
       return response;
     }

     var fetchRequest = event.request.clone();

     return fetch(fetchRequest)
       .then(function(response) {
         if (
           !response ||
           response.status !== 200 ||
           response.type !== 'basic'
         ) {
           return response;
         }
         var responseToCache = response.clone();
         caches.open(CACHE_NAME).then(function(cache) {
           cache.put(event.request, responseToCache);
         });
         return response;
       })
       .catch(function(err) {
         if (event.request.headers.get('Accept').indexOf('text/html') !== -1) {
           return caches.match('./offline.html');
         } else if (event.request.headers.get('Accept').indexOf('image') !== -1) {
           return caches.match('./images/avatar.png');
         } else {
           console.log(err);
         }
       });
   })
 );
});

The service workers are activated through the event listener namely ‘activate’ :

self.addEventListener('activate', function(event) {
 event.waitUntil(
   caches.keys().then(function(cacheNames) {
     return Promise.all(
       cacheNames.map(function(cacheName) {
         if (cacheName !== CACHE_NAME) {
           console.log('Deleting cache ' + cacheName);
           return caches.delete(cacheName);
         }
       })
     );
   })
 );
});

Adding service workers and manifest to the generator

Since we need to add the service workers and manifest to every web application generated through app generator, we copy the files ‘sw.js’ and ‘manifest.json’ in the directory structure of that particular web app using filestream module with the help of two abstract functions ‘copyServiceWorker’ and ‘copyManifestFile’ present in ‘distHelper.js’ code file.

 

distHelper.copyServiceWorker(appFolder, hashObj['hash'], function (err) {
 if (err) {
   console.log(err);
   logger.addLog('Error', 'Error occurred while copying service worker file', socket, err);
   return done(err);
 }
 return done(null);
});

distHelper.copyManifestFile(appFolder, eventName, function(err) {
 if (err) {
   console.log(err);
   logger.addLog('Error', 'Error occured while copying manifest file', socket, err);
   return done(err);
 }
 return done(null);
});

 

Further Improvements

Enabling push notifications for the bookmarked tracks and sessions. The user would be notified about the upcoming events through the notifications in the way the native mobile applications do.

Resources

 

Continue ReadingOpen Event Web App – A PWA

Making Shapes with PSLab Oscilloscope

Looking back to history, the first ever video game was ‘Pong’ which was played on an analog oscilloscope with a very small screen. Oscilloscopes are not made to play video games, but by just tinkering around its basic functionality which is display waveforms, we can do plenty of cool things. PSLab device also has an oscilloscope; in fact it’s a four channel oscilloscope.

This blog post will show you how the oscilloscope in PSLab is not just a cheap oscilloscope but it has lots of functionalities an industry grade oscilloscope has (except for the bandwidth limitation to a maximum of 2 MHz)

To produce shapes like above figures, we are using another instrument available in PSLab. That is ‘Waveform Generator’. PSLab Waveform Generator can generate three different waveforms namely Sine waves, Triangular waves and Square waves ranging from 5 Hz to 5 kHz.

To get started, first connect two jumper wires between SI1-CH1 and SI2-CH2 pins. We needn’t worry about ground pins as they are already internally connected. Now it’s time to open up the PSLab oscilloscope. Here we are going to utilize two channels for this activity and they will be CH1 and CH2. Check the tick boxes in front of ‘Chan 1’ and ‘Chan 2’ and set ‘Range’ to “+/-4V” to have the maximum visibility filling the whole screen with the waveform.

The shapes are drawn using a special mode called ‘X-Y Mode’ in PSLab oscilloscope. In this mode, two channels will be plotted against their amplitudes at every point in time.

As it is already mentioned that PSLab can generate many waveform types and also they can have different phase angles relative to each other. They can have different independent frequencies. With all these combinations, we can tweak the settings in Waveform Generator to produce different cool shapes in PSLab oscilloscope.

These shapes can vary from basic geometric shapes such as circle, square, rectangle to complicated shapes such as rhombus, ellipse and polynomial curves.

Circle

A circular shape can be made by generating two sine waves having the same frequency but with a phase difference of 90 degrees or 270 degrees between the two wave forms.

 

 
 

 


Square

Square shape can be made by generating two triangular waveforms again having the same frequency but with a phase difference of either 90 degrees or 270 degrees between the two.

 

 

 
 


Rectangle

Similar to creating a Square, by having the same frequency for both triangular waveforms but a different phase angle greater than or less than 90 degree will do the trick.

 

 

 
 


Rhombus

Keeping the waveform settings same for the rectangle, by changing the amplitude of the SI1 waveform using the knob we can generate a rhombic shape on the XY graph plot.

 

 

 
 


Ellipse

Generating ellipse is also similar to creating a rhombus. But here we are using sine waves instead of triangular waves. By changing the amplitude of SI1 using the knob we can change the curvature.

 

 

 


Helix

Helix or spiral shape can be generated using two sine waves having same phase but two different frequencies. Frequencies better be integer multiples of the smaller frequency to  have a steady shape.

 

 

 


Parabola

Parabolic shapes can be generated by mixing up triangular waves with sine waves with different phase angles.

 

 

 

 
 

More random shapes


References:

https://www.aps.org/publications/apsnews/200810/physicshistory.cfm

Continue ReadingMaking Shapes with PSLab Oscilloscope