Open-Event Permissions System and integrating it with decorators

All the large scale applications require a permissions system. Thus we also implemented a permissions system in our open-event organization server. It consists of certain pre-decided roles:

  1. Super-Admin
  2. Admin
  3. Organizer
  4. Co organizer
  5. Track organizer
  6. Anonymous user

Now we had to decide the permissions which each role would have. Hence we created a documentation regarding what URLs can be accessed by each role. We developed a list of services which the roles could use their permissions to access:

  1. Tracks
  2. Microlocations
  3. Speakers
  4. Sessions
  5. Sponsors

Thus the final step was to implement the permissions system to the appropriate views or URLs. Here comes the power of Flask decorators . I created a individual decorators @is_organizer, @is_admin, @is_super_admin etc… to check the respective roles. I created one main decorator @can_access to see whether the role can access the particular URL or view function

decorator

So in the above decorator I have simply take in the url and check whether it has ‘create’, ‘edit’ or ‘delete’ words in it. Depending on that the control goes in the particular IF statement. Now once it is decided what operation is being performed it checks what service is being accessed by the user. For example: if the operation is edit then it will check whether the service being edited is an event, session, sponsor etc…

Similar checks are performed by each operation. A check is performed of the request.url to see whether the string for that service is present in it. After it knows what service is being accessed its just a matter of using the CRUD functions of user table to check if the role accessing the resource has the requested permission using the functions:

  1. user.can_create()
  2. user.can_read()
  3. user.can_update()
  4. user.can_delete()

After this its just a matter of adding the decorator to each of the view functions and the system is implemented.  🙂

Continue ReadingOpen-Event Permissions System and integrating it with decorators

Testing with Jasmine-Node

This week I have started testing my own code with Jasmine-Node. Testing seems beneficial only when we do it with a right approach. I have gone through various articles and find out Jasmine as one of the best frameworks for testing Javascript applications.

For specifically NodeJS, a better option will be to use Jasmine-Node.

How to start with Jasmine-Node ?

For a quick-start, you need to have Jasmine-Node installed with npm.

npm install jasmine-node -g

After the installation, make a folder say spec/ in the root directory of the project. This will contain all the tests for the node app.

We can include the command  to run the tests in package.json file of the project.

./node_modules/.bin/jasmine-node spec

 

script

Like here it is included with the key tester. Now, we are ready to write our first test file.

So, in the Open-event-webapp spec/ folder I have created a file serverSpec.js. The file will test a basic function written in fold.js which is the file that includes all the functions of Open-event-webapp.

//fold.js

'use strict';


var exports = module.exports = {};

const fs = require('fs-extra');
const moment = require('moment');
const handlebars = require('handlebars');
const async = require('async');
const archiver = require('archiver');

//function to test

function speakerNameWithOrg(speaker) {
 return speaker.organisation ?
 `${speaker.name} (${speaker.organisation})` :
 speaker.name;
}

To test the above function, I have written the test like this

//serverSpec.js

'use strict'

const request = require("request");
const fold = require(__dirname +'/../src/generator/backend/fold.js');
const port = process.env.PORT || 5000;
const base_url = "http://localhost:"+ port +"/"

describe("Open Event Server", function() {
  describe("GET /", function() {
    it("returns status code 200", function(done) {
      request.get(base_url, function(error, response, body) {
      expect(response.statusCode).toBe(200);
      done();
    });
   });
 }); 
}); 

describe("Testing Fold.js",function() {
  describe("Testing SpeakerWithOrg function",function(){
    it("it will test speakername", function(){
       let speaker= {
          name: "speakername",
        }
    expect(fold.speakerNameWithOrg(speaker)).toEqual("speakername");
   });

    it("it will test speakername and organisation", function(){
       let speakerWithOrg= {
          name:"speakername",
          organisation:"Organisation",
        }
       expect(fold.speakerNameWithOrg(speakerWithOrg)).toEqual("speakername (Organisation)");
    });
 });
 
 });

There are two describe blocks here one to test if the server is running properly and the second block includes two ‘it’ statements which are used to test the function behavior.

The describe blocks are nested which is a nice feature provided by Node-Jasmine.

To run the above test in Open-event-webapp , use the command:

1. npm run start
2. npm run tester

On success, The output will be something like this

test

Other Sources

  1. Jasmine Presentation
  2. HTML goodies

 

 

Continue ReadingTesting with Jasmine-Node

How do we work? Agile

It’s not typical team. We don’t meet face-to-face each other on daily basis, one trouble is that we have to work from different time zones, and we have different operating system, the way of working, or even culture. But we are FOSSASIA Open Event team of six amazing developers so it does not discourage us to achieve our goals.

But even if we experience all these things we have to learn how to work together successfully. We are trying, and I think it is becoming better and better every day, the Agile methodology.

So first of all, before coding started, we had prepared user stories in issues. Every issue had been very well documented, and we had divided each issue to smaller ones. It brings us a lot of benefits, because it doesn’t matter where you work. If sub issue is empty you can take it. Due to the fact that issue is very well documented you don’t need to even ask someone what to do. This case saves a lot of time. I know that writing clear issues is very boring and It could seemed not to be beneficial, but it’s. We are still trying to improve our performance and we are looking for new opportunities where and how we can improve our work, without lost quality of our development process.

Then, we have one week sprint (milestones). It helps us to control our work and we know where are we, what have we done, and what is still to do. We know if our speed is enough or we have to move faster. And it also enables us to see our progress.Screen Shot 2016-06-25 at 11.27.08.png

Moreover, we have daily scrums. We answer three following questions – what did you do yesterday, what do you plan to do today and what is preventing you from achieving your goals. Furthermore, we follow continues integration system. We push code to our common Open Event repository on daily basis. It helps us to control our code and fix bugs quickly. I am sure we will continue along this path, and successfully finish to code an amazing Open Event web app.

Continue ReadingHow do we work? Agile

Organizer Server Permissions System

This post discusses about the Organizer Server Permissions System and how it has been implemented using database models.

The Organizer Server Permissions System includes Roles, and Services that these roles can access. Roles can broadly be classified into Event-specific Roles and System-wide Roles.

System-Wide Roles

System-wide roles can be considered a part of the staff maintaining the platform. We define two such roles: Admin and Super Admin. The Super Admin is the highest level user with permissions to access system logs, manage event-specific roles, etc.

System-wide roles are the easiest to implement. Since they’re directly related to a user we can add them as class variables in the User models.

class User(db.Model):
    # other stuff

    is_admin = db.Column(db.Boolean, default=False)
    is_super_admin = db.Column(db.Boolean, default=False)

    @property
    def is_staff(self):
        return self.is_admin or self.is_super_admin

Staff groups Admin and Super Admin roles. So to check if a user is a part of the staff (an admin or a super admin), the is_staff property can directly be used.

user.is_staff

Event-Specific Roles

An Event itself can contain many entities like Tracks, Sponsors, Sessions, etc. Our goal was to define permissions for Roles to access these entities. We grouped these entities and put them under “Services”. Services are nothing but database models associated with an event, that need to have restricted access for the Roles.

We define the following Services for an Event:

  • Track
  • Session
  • Speaker
  • Sponsor
  • Microlocation

Each of these services can either be created, read, updated or deleted. And depending on the Role a user has been assigned for a particular event, he or she can perform such operations.

There are four Event-Specific Roles:

  • Organizer
  • Co-organizer
  • Track Organizer
  • Moderator

As soon as the user creates an event, he is assigned the role of an Organizer, giving him access to all the services for that event. An Organizer can perform any operation on any of the services. A Co-organizer also has access to all the services but can only update them. A Track Organizer can just read and update already created Tracks. The Moderator can only read Tracks.

Although the initial distribution of permissions is kept as above, the Super Admin can (has permissions to) edit them later.

Screenshot from 2016-06-24 02:41:47

To implement permissions for Event specific roles, three new database models were required: Role, Service and Permission.

Role and Service would contain the above mentioned Roles and Services respectively. Permission would contain a Role column, a Service column and four other columns specifying what operation (create/read/update/delete) that Role is allowed to perform on the Service.

The final objective was to define these methods for the User class:

user.can_create(service, event_id)
user.can_read(service, event_id)
user.can_update(service, event_id)
user.can_delete(service, event_id)

Before this we needed a table specifying what Event Roles have been created for an event, and which users have been assigned these roles. The `UsersEventsRoles` model maintained this relationship.

We also needed to check if a user has been assigned a particular role for an event. For this I created a method for each of the roles.

# Event-specific Roles
ORGANIZER = 'organizer'
COORGANIZER = 'coorganizer'
TRACK_ORGANIZER = 'track_organizer'
MODERATOR = 'moderator'

class User(db.Model):
    # other stuff

    def _is_role(self, role_name, event_id):
        role = Role.query.filter_by(name=role_name).first()
        uer = UsersEventsRoles.query.filter_by(user=self,
                                               event_id=event_id,
                                               role=role).first()
        if not uer:
            return False
        else:
            return True

    def is_organizer(self, event_id):
        return self._is_role(ORGANIZER, event_id)

    def is_coorganizer(self, event_id):
        return self._is_role(COORGANIZER, event_id)

    def is_track_organizer(self, event_id):
        return self._is_role(TRACK_ORGANIZER, event_id)

    def is_moderator(self, event_id):
        return self._is_role(MODERATOR, event_id)

Here _is_role helps reduce code redundancy.

Like I said, our final objective was to create methods that determine if a user has permission to perform a particular operation on a service based on the role, I defined the following methods:

    # ...`User` class

    def _has_perm(self, operation, service_class, event_id):
        # Operation names and their corresponding permission in `Permissions`
        operations = {
            'create': 'can_create',
            'read': 'can_read',
            'update': 'can_update',
            'delete': 'can_delete',
        }
        if operation not in operations.keys():
            raise ValueError('No such operation defined')

        try:
            service_name = service_class.get_service_name()
        except AttributeError:
            # If `service_class` does not have `get_service_name()`
            return False

        service = Service.query.filter_by(name=service_name).first()

        uer_querylist = UsersEventsRoles.query.filter_by(user=self,
                                                         event_id=event_id)
        for uer in uer_querylist:
            role = uer.role
            perm = Permission.query.filter_by(role=role,
                                              service=service).first()
            if getattr(perm, operations[operation]):
                return True

        return False

    def can_create(self, service_class, event_id):
        return self._has_perm('create', service_class, event_id)

    def can_read(self, service_class, event_id):
        return self._has_perm('read', service_class, event_id)

    def can_update(self, service_class, event_id):
        return self._has_perm('update', service_class, event_id)

    def can_delete(self, service_class, event_id):
        return self._has_perm('delete', service_class, event_id)

The can_create, can_read, etc. defined in operations are the four columns in the Permission db model. Like _is_role, _has_perm method helps implementing the DRY philosophy.

Continue ReadingOrganizer Server Permissions System

Responsive UI: Modifying Designs with Device Width

An important feature of websites these days with the advancement of smartphones is being responsive with device size. We nowadays not only worry about the various width of laptop or desktop, which don’t vary by a huge amount but also need to worry about tablets and phones which have a much lesser width. The website’s UI should not break and should be as easy to use on phones as it is on desktops/laptops. Using frameworks like bootstraps, Semantic-UI solves this problem to a large extent. But what if we need to modify certain parts by our own in case of mobile devices? How do we do that?

Continue ReadingResponsive UI: Modifying Designs with Device Width

Handling file uploading in HTML5 and Express

The Open Event Webapp Generator has a pure HTML front end form, and Express based backend.

 

We needed an option where users can upload their own JSON files to create a schedule page for it.

On the HTML form that’s easy. As you can see here we simple add input tags with type=”file”


<input type="file" name="sponsorfile" id="sponsorfile">

In our express app, we need to use multer to be able to handle file uploads.

We create a middleware called uploadedFiles, and pass the middleware to the get() block

 

var express = require('express');
var multer = require('multer');
var app = express();

var upload = multer({dest: 'uploads/'});

var uploadedFiles = upload.fields([
  {name: 'speakerfile', maxCount: 1},
  {name: 'sessionfile', maxCount: 1},
  {name: 'trackfile', maxCount: 1},
  {name: 'sponsorfile', maxCount: 1},
  {name: 'eventfile', maxCount: 1},
  {name: 'locationfile', maxCount: 1}
]);

app.post('/live', uploadedFiles, function(req, res) {
         // req.files has the files
         // req.body has the POST body params
  });
});

Now we can access the files inside req.files (that will have a path to the temporary location of the file, you’ll have to use some filesystem module to read the files)

Continue ReadingHandling file uploading in HTML5 and Express

Paginated APIs in Flask

Week 2 of GSoC I had the task of implementing paginated APIs in Open Event project. I was aware that DRF provided such feature in Django so I looked through the Internet to find some library for Flask. Luckily, I didn’t find any so I decided to make my own.

A paginated API is page-based API. This approach is used as the API data can be very large sometimes and pagination can help to break it into small chunks. The Paginated API built in the Open Event project looks like this –

{
    "start": 41,
    "limit": 20,
    "count": 128,
    "next": "/api/v2/events/page?start=61&limit=20",
    "previous": "/api/v2/events/page?start=21&limit=20",
    "results": [
    	{
    		"data": "data"
    	},
    	{
    		"data": "data"
    	}
    ]
}

Let me explain what the keys in this JSON mean –

  1. start – It is the position from which we want the data to be returned.
  2. limit – It is the max number of items to return from that position.
  3. next – It is the url for the next page of the query assuming current value of limit
  4. previous – It is the url for the previous page of the query assuming current value of limit
  5. count – It is the total count of results available in the dataset. Here as the ‘count’ is 128, that means you can go maximum till start=121 keeping limit as 20. Also when you get the page with start=121 and limit=20, 8 items will be returned.
  6. results – This is the list of results whose position lies within the bounds specified by the request.

Now let’s see how to implement it. I have simplified the code to make it easier to understand.

from flask import Flask, abort, request, jsonify
from models import Event

app = Flask(__name__)

@app.route('/api/v2/events/page')
def view():
	return jsonify(get_paginated_list(
		Event, 
		'/api/v2/events/page', 
		start=request.args.get('start', 1), 
		limit=request.args.get('limit', 20)
	))

def get_paginated_list(klass, url, start, limit):
    # check if page exists
    results = klass.query.all()
    count = len(results)
    if (count < start):
        abort(404)
    # make response
    obj = {}
    obj['start'] = start
    obj['limit'] = limit
    obj['count'] = count
    # make URLs
    # make previous url
    if start == 1:
        obj['previous'] = ''
    else:
        start_copy = max(1, start - limit)
        limit_copy = start - 1
        obj['previous'] = url + '?start=%d&limit=%d' % (start_copy, limit_copy)
    # make next url
    if start + limit > count:
        obj['next'] = ''
    else:
        start_copy = start + limit
        obj['next'] = url + '?start=%d&limit=%d' % (start_copy, limit)
    # finally extract result according to bounds
    obj['results'] = results[(start - 1):(start - 1 + limit)]
    return obj

Just to be clear, here I am assuming you are using SQLAlchemy for the database. The klass parameter in the above code is the SqlAlchemy db.Model class on which you want to query upon for the results. The url is the base url of the request, here ‘/api/v2/events/page’ and it used in setting the previous and next urls. Other things should be clear from the code.

So this was how to implement your very own Paginated API framework in Flask (should say Python). I hope you found this post interesting.

Until next time.

Ciao

 

{{ Repost from my personal blog http://aviaryan.in/blog/gsoc/paginated-apis-flask.html }}

Continue ReadingPaginated APIs in Flask

Getting Signed Release apk’s from the command line

 

View at Medium.com

If anyone of you has deployed an application on the play store, you may have most probably used Android Studio’s built in Generate signed apkoption.

The generate apk option in android studio

Recently while making the Open Event Apk generator, I had to make release apk’s, so that they could be used by an event organiser to publish their app, plus apk’s had to be signed because if they were not signed, it would be impossible to upload due to checks by Google.

Error shown on the developers console

So since I was building the app using the terminal and I didn’t have the luxury of signing the app using Android studio and I had to look for alternatives. Luckily I found two of them :

  1. Using the Signing configs offered by gradle
  2. Using the Oracle sun jarsigner

First of all the signing configs in gradle is a great way to do this. Most Open source apps use this as a way to put their code out for everyone to view and sucessfully hide any private keys and password.

You just need to add few lines of code in your app level build.gradle file and create a file called keystore.properties

In your keystore.properties, we just need to store the sensitive info and this file will be accessible only to people who are part of the project.

storePassword=myStorePassword
keyPassword=mykeyPassword
keyAlias=myKeyAlias
storeFile=myStoreFileLocation

Next we go to the build.gradle and add these lines to read the keystore.properties file and it’s variables

// Create a variable called keystorePropertiesFile, and initialize it to your
// keystore.properties file, in the rootProject folder.
def keystorePropertiesFile = rootProject.file("keystore.properties")

// Initialize a new Properties() object called keystoreProperties.
def keystoreProperties = new Properties()

// Load your keystore.properties file into the keystoreProperties object.
keystoreProperties.load(new FileInputStream(keystorePropertiesFile))

Next we can add the signingConfigs task and reference the values we got above over there

android {
    signingConfigs {
        config {
            keyAlias keystoreProperties['keyAlias']
            keyPassword keystoreProperties['keyPassword']
            storeFile file(keystoreProperties['storeFile'])
            storePassword keystoreProperties['storePassword']
        }
    }
    ...
  }

So As you see this is as simple as this but according to my requirements this seemed a bit tedious since a person setting up the apk generator had to make a keystore file, then find the build.gradle and change the path of the keystore file according to the server directories. So this does the trick but this can be so tedious for someone with no technical experience, so I researched on other solutions and then I got it : Jarsigner and Zipalign

First of all,the jarsigner and zipalign are 2 great tools and the best part about them is that both of them work perfectly with a just one line commands. For signing the app :

jarsigner -keystore <keystore_file> -storepass <storepassword> <apknameTosigned> <alias>

and then zipaligning :

zipalign -v 4 <unaligned-apk-location> <path-to-generated-aligned-apk>

So this is it, we finally used these 2 commands to sign and zipalign an apk and it works perfectly fine. Please test and share comments of the demo live @ http://192.241.232.231. Ciao !

Continue ReadingGetting Signed Release apk’s from the command line

Migrations make us crazy!

Our Open Event team is not small, so problems with migrations occur very often. I’d like to describe how to solve these common problems and avoid Contributors frustration. Because more of us didn’t know how to solve this problems at the beginning so we wasted a lot of time to find a source of problem.

The most common mistake is that we forget run migration on Heroku. Developer is sometimes surprised because something works for him but he forgets to run migration on server. These small mistakes lead to huge problems and at that time our app throws a lots of bugs related to database. We can often see “Internal server error”.Screen Shot 2016-06-17 at 22.55.25.png So if developer changes table he has to run migration!

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python manage.py db migrate

But above command Quite often doesn’t solve our problem, because We face other problems while updating DB for example

alembic.util.exc.CommandError: Multiple head revisions are present for given argument ‘head’; please specify a specific target revision, ‘<branchname>@head’ to narrow to a specific head, or ‘heads’ for all heads

this problem is caused by two developers which push code to Github with migrations without merging two heads to achieve one head.

So to solve this problem you only have to know ids of two heads

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python manage.py db heads

e38935822969 (head)
f55fde3d62b1 (head)

Then you have to merge it

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python manage.py db merge e38935822969 f55fde3d62b1

Generating /vagrant/migrations/versions/ecb671d1eb4b_.py … done

Upgrade DB

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python manage.py db upgrade

INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.runtime.migration] Running upgrade 00ea66754d06 -> d84f976530e1, empty message
INFO [alembic.runtime.migration] Running upgrade d84f976530e1 -> 1b3e4f5f56bd, empty message
INFO [alembic.runtime.migration] Running upgrade 1b3e4f5f56bd -> e38935822969, empty message
INFO [alembic.runtime.migration] Running upgrade e38935822969, f55fde3d62b1 -> ecb671d1eb4b, empty message

And finally run migrations

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python manage.py db migrate
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.ddl.postgresql] Detected sequence named ‘role_id_seq’ as owned by integer column ‘role(id)’, assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named ‘microlocation_id_seq’ as owned by integer column ‘microlocation(id)’, assuming SERIAL and omitting….

Continue ReadingMigrations make us crazy!

Open Event Organizer’s Server: Custom Forms

As we know, open event organizer’s server provides a means of creating events. And an important part of events is sessions and speakers of sessions. But different events have different details required for sessions and speakers of sessions. Some might feel that only Name and Email-ID of speaker is enough, while others may require their Github, Facebook, etc. links. So the best way to solve this is to make a custom form so that the organizers get to select what all fields they want in the forms. But how to do this? Well, thanks to JSON

In the Step-5 of Event Creation Wizards, we have option (or switches) to include and require various fields available for the session form and speaker form for applying in a particular Session Form. Here is how it looks:
custom_form.png

As we can see, each field has 2 options – Include and Require. On clicking the Include switch the Require switch is enabled through jQuery. The Include switch means that the field is included in the form while the Require switch signifies that the field will be a compulsory field in the form.

Storing Options

We create a javascript array containing a single object. This object in turn is made up of objects with field names as keys and another object as value. This object in turn contains the keys include and require. The value of include and require is 0 by default. On selecting, their value is changed to 1. An example structure of the array is like:

[{
    "title": {
        "include": 1,
        "require": 1
    },
    "subtitle": {
        "include": 0,
        "require": 0
    },
    "short_abstract": {
        "include": 1,
        "require": 0
    },
    "long_abstract": {
        "include": 0,
        "require": 0
    },
    "comments": {
        "include": 1,
        "require": 0
    },
    "track": {
        "include": 0,
        "require": 0
    },
    "session_type": {
        "include": 0,
        "require": 0
    },
    "language": {
        "include": 0,
        "require": 0
    },
    "slides": {
        "include": 1,
        "require": 0
    },
    "video": {
        "include": 0,
        "require": 0
    },
    "audio": {
        "include": 0,
        "require": 0
    }
}]

This array is then converted to string format using JSON.stringify, submitted through form and stored in database in the form of a string.

Rendering Forms

Now, we have already stored the options as selected by the organizer. But now we need to render the forms based on the JSON string stored in the database. Firstly, in the server side we convert the string to JSON object in python using json.loads . Then we send these JSON objects to the templates.

In the templates, we create form elements based on the values of the include and require of each form element. A sample element HTML is like this:

Screenshot from 2016-06-17 22:38:40

We use a loop to iterate over the JSON object and thus add the elements which have  “include = 1”  . The final form looks something like this:

session-form

Continue ReadingOpen Event Organizer’s Server: Custom Forms