Event Management

Adding Client Side validation to Login and Registration Forms

Its very important to have a client side validation apart from a server side validation. The server side validation only helps the developers but not the clients. Thus it was necessary to add a client side validation to the Login page and its other components so that the users  would be comfortable in this. I had never before done this and was looking at how to achieve this using jQuery or JavaScript when I cam to know that we were already using an amazing validation tool : Bootstrap Validator It just involves wrapping the form in the html with the validator plugin and all the checks are automatically carried out by it.   As you can see in the above image, we have just added a data-toggle="validator" line to the form which automatically wraps the form with the plugin. The above form is the Create New Password form which checks whether the new password and the password entered again are matching or not. If not then the line , data-match="#new_password" checks it with the password in the new_password field and gives an error on the page to the client defined by the following, data-error="Passwords do not match Thus adding such checks to the form becomes very simple rather than using jQuery for it. Similarly it was important to add validation to the Register page. <input type="email" name="email" class="form-control" id="email" placeholder="Email" data-remote="{{ url_for('admin.check_duplicate_email') }}" data-remote-error="Email Address already exists" required=""> This is the validation for the email field. Apart from checking whether the text entered by the user is an email id or not it was also important to check whether the email entered by the user exists in the database or not. data-remote="{{ url_for('admin.check_duplicate_email') This line calls a view function to check whether the email entered by the user is duplicate or unique. Here is the function This function takes the email value from the request.args and then performs a simple check in the db to check for duplicate email. If the user doesnt exist then the validator receives a simple string "200 OK". If the error is 404 then it gives the error defined by, data-remote-error="Email Address already exists" However this error only for the data-remote part. If the error is something else then it is handled by , class="help-block with-errors" Thus without the hassle of using Ajax and JQuery we can easily add validation to the forms using Bootstrap Validator.

Continue ReadingAdding Client Side validation to Login and Registration Forms

How to parse json assets with gson

So most of us have json assets in our app which we parse on runtime to get the data and use it accordingly but what I have seen is that most of the people create a JSONObject or JSONArray after reading the json into an inputstream but then handling it becomes difficult since we have to manually extract every entity in each array which makes it bound to a lot of errors. A better approach to using it is making use of gson : an open source library by Google to serialise and deserialise Java object to (and from) Json. It’s pretty easy to use and makes the development process easy. For those of you not still convinced on using gson, I’d like to demonstrate the code we had to write to without using gson and the one using gson as well. So to start with lets see the json file we’ll be using. It’s the events.json file from the open event project. { "events": [ { "color": "#fdfdfd", "email": "dev@fossasia.org", "end_time": "2015-07-14T00:00:00", "id": 4, "latitude": 37.783839, "location_name": "Moscone centre", "logo": "http://mysecureshell.readthedocs.org/en/latest/_images/logo_redhat.png", "longitude": -122.400546, "name": "FOSSASIA", "slogan": "Fossasia", "start_time": "2015-05-28T13:00:00", "url": "www.google.com" } ] } As you can see it has an object that has an array of event objects. So what we’ll first do is that we’ll get the whole json as a string by openeing an inputstream and then directing it to a buffer. Then we convert the buffer array to a string object. String json = null; try { InputStream inputStream = getAssets().open("events.json"); int size = inputStream.available(); byte[] buffer = new byte[size]; inputStream.read(buffer); inputStream.close(); json = new String(buffer, "UTF-8"); } catch (IOException e) { e.printStackTrace(); } Now we have the json as a string which we can now parse it using a combination of JSONObject and JSONArray. First we’ll access data in the outer json object i.e. “events”. That’ll be done by JSONObject jsonObject = new JSONObject(json); JSONArray events = jsonObject.getJSONArray("events"); Now that we have the array, we can traverse it to get the objects inside events array for (int j=0; j < events.length(); j++){ JSONObject cit = events.getJSONObject(j); String color = jsonObject.getString("color"); String email = jsonObject.getString("email"); String endTime = jsonObject.getString("end_time"); String id = jsonObject.getString("id"); String latitude = jsonObject.getString("latitude"); String locationName = jsonObject.getString("location_name"); String logo = jsonObject.getString("logo"); String longitude = jsonObject.getString("longitude"); String name = jsonObject.getString("name"); String slogan = jsonObject.getString("slogan"); String startTime = jsonObject.getString("start_time"); String url = jsonObject.getString("url"); } This is how we go about it. Now for the exiting part. We already have an Event data class which has the constructor, getters and setters etc. public class Event { int id; String name; String email; String color; String logo; @SerializedName("start_time") String start; @SerializedName("end_time") String end; float latitude; float longitude; @SerializedName("location_name") String locationName; String url; String slogan; public Event(int id, String name, String email, String color, String logo, String start, String end, float latitude, float longitude, String locationName, String url, String slogan) { this.id = id; this.name = name; this.email = email; this.color = color; this.logo = logo; this.start =…

Continue ReadingHow to parse json assets with gson

Optimizing page load time

The average size of a web page has been growing at an accelerating rate over the last few years. Last week, when the open-event webapp was tested, it was very slow to load because of loading 67.2 MB data ( which contained audio and image files ) on the web page.I have taken following steps which are good to read to make any web application load faster. 1. Stop preloading of audio files The HTML5 audio player loads all audio files by default on the page. To stop this we have to change the code as <audio controls preload="none"> <source src="{{audio}}" type="audio/mpeg"> </audio> Setting preload option to none help us to load audio only when it is clicked. Hence it decreases the HTTP calls and decreases the page loading time. Previously Network tab with Audio files takes 68.0MB  load and 13.6 minute loading time. Now Network tab with option <audio controls preloaded="none"> takes 1.1 MB load and  1 minute loading time. This was a huge improvement but more optimizations can be done. 2. Using Compression Middleware Express compression middleware works like a charm by compressing the response coming to the web page. It is too simple to add. $ npm install compression var compression = require('compression') var express = require('express') var app = express() // compress all requests app.use(compression()) // add all routes This has compressed the responses and decreased the page load time. 3.  Reduce the number of HTTP requests Another great way of speeding up your web pages is to simply reduce the number of files that need to be loaded. Combine files Combining multiple stylesheets into one file is a really useful way of eliminating extra HTTP requests. This strategy can also be adopted for your JavaScript files. A single larger CSS or JavaScript file will often load quicker because more time can be spent downloading the data rather than establishing multiple connections to a server. After such optimizations, the webapp loads now in 8-15 seconds with DOM loaded in 2 seconds. The result of these optimizations are awesome. We can test the page speed by using various tools like pagespeed, speedtracer etc.

Continue ReadingOptimizing page load time
Read more about the article Update Fields with Array Input

Update Fields with Array Input

Screenshot from 2016-07-15 17:57:58.png

There are certain fields in form where instead of a single value, we need an array of values to be stored. This are fields under the same category but having multiple elements, e.g., Tracks, Session Types, Microlocations, Sponsors, Social Links and similar such fields. Now as we know the way of doing this using simple html is provide the <input> tag with property “name” as “<field_name>[]”. So suppose we want to save the names of the mutliple tracks, we will have something like this
Screenshot from 2016-07-15 18:02:09.png

But the problem begins when you want to update name of a particular element (in this case Track). How to do it? Should we delete and create entries again? That doesn’t sound too good, does it? So what should we do? Let’s see….

(more…)

Continue ReadingUpdate Fields with Array Input

Setting up Celery with Flask

In this article, I will explain how to use Celery with a Flask application. Celery requires a broker to run. The most famous of the brokers is Redis. So to start using Celery with Flask, first we will have to setup the Redis broker. Redis can be downloaded from their site http://redis.io. I wrote a script that simplifies downloading, building and running the redis server. #!/bin/bash # This script downloads and runs redis-server. # If redis has been already downloaded, it just runs it if [ ! -d redis-3.2.1/src ]; then wget http://download.redis.io/releases/redis-3.2.1.tar.gz tar xzf redis-3.2.1.tar.gz rm redis-3.2.1.tar.gz cd redis-3.2.1 make else cd redis-3.2.1 fi src/redis-server When the above script is ran from the first time, the redis folder doesn't exist so it downloads the same, builds it and then runs it. In subsequent runs, it will skip the downloading and building part and just run the server. Now that the redis server is running, we will have to install its Python counterpart. pip install redis After the redis broker is set, now its time to setup the celery extension. First install celery by using pip install celery. Then we need to setup celery in the flask app definition. # in app.py def make_celery(app): # set redis url vars app.config['CELERY_BROKER_URL'] = environ.get('REDIS_URL', 'redis://localhost:6379/0') app.config['CELERY_RESULT_BACKEND'] = app.config['CELERY_BROKER_URL'] # create context tasks in celery celery = Celery(app.import_name, broker=app.config['CELERY_BROKER_URL']) celery.conf.update(app.config) TaskBase = celery.Task class ContextTask(TaskBase): abstract = True def __call__(self, *args, **kwargs): with app.app_context(): return TaskBase.__call__(self, *args, **kwargs) celery.Task = ContextTask return celery celery = make_celery(current_app) Now that Celery is setup on our project, let’s define a sample task. @app.route('/task') def view(): background_task.delay(*args, **kwargs) return 'OK' @celery.task def background_task(*args, **kwargs): # code # more code Now to run the celery workers, execute celery worker -A app.celery That should be all. Now to run our little project, we can execute the following script. bash run_redis.sh & # to run redis celery worker -A app.celery & # to run celery workers python app.py If you are wondering how to run the same on Heroku, just use the free heroku-redis extension. It will start the redis server on heroku. Then to run the workers and app, set the Procfile as - web: sh heroku.sh Then set the heroku.sh as - #!/bin/bash celery worker -A app.celery & gunicorn app:app That’s a basic guide on how to run a Flask app with Celery and Redis. If you want more information on this topic, please see my post Ideas on Using Celery in Flask for background tasks.

Continue ReadingSetting up Celery with Flask

Ideas on using Celery with Flask for background tasks

Simply put, Celery is a background task runner. It can run time-intensive tasks in the background so that your application can focus on the stuff that matters the most. In context of a Flask application, the stuff that matters the most is listening to HTTP requests and returning response. By default, Flask runs on a single-thread. Now if a request is executed that takes several seconds to run, then it will block all other incoming requests as it is single-threaded. This will be a very bad-experience for the user who is using the product. So here we can use Celery to move time-hogging part of that request to the background. I would like to let you know that by “background”, Celery means another process. Celery starts worker processes for the running application and these workers receive work from the main application. Celery requires a broker to be used. Broker is nothing but a database that stores results of a celery task and provides a shared interface between main process and worker processes. The output of the work done by the workers is stored in the Broker. The main application can then access these results from the Broker. Using Celery to set background tasks in your application is as simple as follows - @celery.task def background_task(*args, **kwargs): # do stuff # more stuff Now the function background_task becomes function-able as a background task. To execute it as a background task, run - task = background_task.delay(*args, **kwargs) print task.state # task current state (PENDING, SUCCESS, FAILURE) Till now this may look nice and easy but it can cause lots of problems. This is because the background tasks run in different processes than the main application. So the state of the worker application differs from the real application. One common problem because of this is the lack of request context. Since a celery task runs in a different process, so the request context is not available. Therefore the request headers, cookies and everything else is not available when the task actually runs. I too faced this problem and solved it using an excellent snippet I found on the Internet. """ Celery task wrapper to set request context vars and global vars when a task is executed Based on http://xion.io/post/code/celery-include-flask-request-context.html """ from celery import Task from flask import has_request_context, make_response, request, g from app import app # the flask app __all__ = ['RequestContextTask'] class RequestContextTask(Task): """Base class for tasks that originate from Flask request handlers and carry over most of the request context data. This has an advantage of being able to access all the usual information that the HTTP request has and use them within the task. Pontential use cases include e.g. formatting URLs for external use in emails sent by tasks. """ abstract = True #: Name of the additional parameter passed to tasks #: that contains information about the original Flask request context. CONTEXT_ARG_NAME = '_flask_request_context' GLOBALS_ARG_NAME = '_flask_global_proxy' GLOBAL_KEYS = ['user'] def __call__(self, *args, **kwargs): """Execute task code with given…

Continue ReadingIdeas on using Celery with Flask for background tasks

Using compression middleware in Node/Express

If you're using ExpressJS to host your server, there's a small performance tweak that you can do (works on universally on any kind of server - HTML templates to JSON REST APIs to image servers) - which is to enable GZIP compression. The changed required to do it should be very simple. First install the compression npm package npm install --save compression In you app.js (or wherever you start your express server), modify the code to include the compression middleware like this var express = require('express); var app = express(); var compression = require('compression'); app.use(compression()); // Rest of your app code // app.get (endpoint, callback) // app.listen(port, callback) This should easily reduce page loads time to the order of 15-20%. For example, page load before compression middleware added   And Page load time after compression is added

Continue ReadingUsing compression middleware in Node/Express

Permission Decorators

A follow-up to one of my previous posts: Organizer Server Permissions System. I recently had a requirement to create permission decorators for use in our REST APIs. There had to be separate decorators for Event and Services. Event Permission Decorators Understanding Event permissions is simple: Any user can create an event. But access to an event is restricted to users that have Event specific Roles (e.g. Organizer, Co-organizer, etc) for that event. The creator of an event is its Organizer, so he immediately gets access to that event. You can read about these roles in the aforementioned post. So for Events, create operation does not require any permissions, but read/update/delete operations needed a decorator. This decorator would restrict access to users with event roles. def can_access(func): """Check if User can Read/Update/Delete an Event. This is done by checking if the User has a Role in an Event. """ @wraps(func) def wrapper(*args, **kwargs): user = UserModel.query.get(login.current_user.id) event_id = kwargs.get('event_id') if not event_id: raise ServerError() # Check if event exists get_object_or_404(EventModel, event_id) if user.has_role(event_id): return func(*args, **kwargs) else: raise PermissionDeniedError() return wrapper The has_role(event_id) method of the User class determines if the user has a Role in an event. # User Model class def has_role(self, event_id): """Checks if user has any of the Roles at an Event. """ uer = UsersEventsRoles.query.filter_by(user=self, event_id=event_id).first() if uer is None: return False else: return True Reading one particular event (/events/:id [GET]) can be restricted to users, but a GET request to fetch all the events (/events [GET]) should only be available to staff (Admin and Super Admin). So a separate decorator to restrict access to Staff members was needed. def staff_only(func): @wraps(func) def wrapper(*args, **kwargs): user = UserModel.query.get(login.current_user.id) if user.is_staff: return func(*args, **kwargs) else: raise PermissionDeniedError() return wrapper Service Permission Decorators Service Permissions for a user are defined using Event Roles. What Role a user has in an Event determines what Services he has access to in that Event. Access here means permission to Create, Read, Update and Delete services. The User model class has four methods to determine the permissions for a Service in an event. user.can_create(service, event_id) user.can_read(service, event_id) user.can_update(service, event_id) user.can_delete(service, event_id) So four decorators were needed to put alongside POST, GET, PUT and DELETE method handlers. I've pasted snippet for the can_update decorator. The rest are similar but with their respective permission methods for User class object. def can_update(DAO): def decorator(func): @wraps(func) def wrapper(*args, **kwargs): user = UserModel.query.get(login.current_user.id) event_id = kwargs.get('event_id') if not event_id: raise ServerError() # Check if event exists get_object_or_404(EventModel, event_id) service_class = DAO.model if user.can_update(service_class, event_id): return func(*args, **kwargs) else: raise PermissionDeniedError() return wrapper return decorator This decorator is a little different than can_access event decorator in a way that it takes an argument, DAO. DAO is Data Access Object. A DAO includes a database Model and methods to create, read, update and delete object of that model. The db model for a DAO would be the Service class for the object. You can look that the model class…

Continue ReadingPermission Decorators

Building interactive elements with HTML and javascript: Interact.js + drag-drop

{ Repost from my personal blog @ https://blog.codezero.xyz/building-interactive-elements-with-html-and-javascript-interact-js-drag-drop } In a few of the past blog posts, we saw about implementing drag-drop andresizing with HTML and javascript. The functionality was pretty basic with a simple drag-and-drop and resizing. That is where, a javascript library called as interact.js comes in. interact.js is a lightweight, standalone JavaScript module for handling single-pointer and multi-touch drags and gestures with powerful features including inertia and snapping. With Interact.js, building interactive elements is like a eating a piece of your favorite cake - that easy ! Getting started with Interact.js You have multiple option to include the library in your project. You can use bower to install (bower install interact) (or) npm (npm install interact.js) (or) You could directly include the library from a CDN (https://cdnjs.cloudflare.com/ajax/libs/interact.js/1.2.6/interact.min.js). Implementing a simple draggable Let's start with some basic markup. We'll be using the draggable class to enable interact.js on this element. <div id="box-one" class="draggable"> <p> I am the first Box </p> </div> <div id="box-two" class="draggable"> <p> I am the second Box </p> </div> The first step in using interact.js is to create an interact instance. Which you can create by using interact('<the selector>'). Once the instance is created, you'll have to call the draggable method on it to enable drag. Draggable accepts a javascript object with some configuration options and some pretty useful callbacks. // target elements with the "draggable" class interact('.draggable') .draggable({ // enable inertial throwing inertia: true, // keep the element within the area of it's parent restrict: { restriction: "parent", endOnly: true, elementRect: { top: 0, left: 0, bottom: 1, right: 1 } }, // enable autoScroll autoScroll: true, // call this function on every dragmove event onmove: dragMoveListener, }); function dragMoveListener (event) { var target = event.target, // keep the dragged position in the data-x/data-y attributes x = (parseFloat(target.getAttribute('data-x')) || 0) + event.dx, y = (parseFloat(target.getAttribute('data-y')) || 0) + event.dy; // translate the element target.style.webkitTransform = target.style.transform = 'translate(' + x + 'px, ' + y + 'px)'; // update the posiion attributes target.setAttribute('data-x', x); target.setAttribute('data-y', y); } Here we use the onmove event to move the box according to the dx and dyprovided by interact when the element is dragged. Implementing a simple drag-drop Now to the above draggable, we'll add a drop zone into which the two draggable boxes can be dropped. <div id="dropzone" class="dropzone">You can drop the boxes here</div> Similar to a draggable, we first create an interact instance. Then we call the dropzone method on to tell interact that, that div is to be considered as a dropzone. The dropzone method accepts a json object with configuration options and callbacks. // enable draggables to be dropped into this interact('.dropzone').dropzone({ // Require a 50% element overlap for a drop to be possible overlap: 0.50, // listen for drop related events: ondropactivate: function (event) { // add active dropzone feedback event.target.classList.add('drop-active'); }, ondragenter: function (event) { var draggableElement = event.relatedTarget, dropzoneElement = event.target; // feedback the possibility of a drop dropzoneElement.classList.add('drop-target'); }, ondragleave: function…

Continue ReadingBuilding interactive elements with HTML and javascript: Interact.js + drag-drop

Shell hacks

Custom Shell Working with database models needs a lot of use of the flask shell. You can access it with: python manage.py shell The default shell is quite unintuitive. It doesn't pretty print the outputs and has no support for auto-completion. I've installed IPython that takes care of that. But still working with models means writing a lot of import statements. Plus if there was some change in the code related to the app, then the shell had to restarted again so the changes could be loaded. Meaning writing the import statements again. We were using Flask-Script and I wanted to run a custom shell that imports all the required modules, models and helper functions, so I don't have to write them over and over. Some of them were as long as: from open_event.models.users_events_roles import UsersEventsRoles So I created a custom shell command with different context that overrides the default shell provided by the Flask-Script Manager. It was pretty easy with Flask-Script. One thing I had to keep in mind is that it needed to be in a different file than manage.py. Since manage.py was committed to source repo, changes to it would be tracked. So I needed a different file that could be excluded from the source repo. I created an smg.py that imported the Manager from the open_event module and overrides the shell command. from flask_script import Shell from open_event import manager from open_event.models.user import User from open_event.models.event import Event from open_event.helpers.data import save_to_db, delete_from_db def _make_context(): return dict( uq=User.query, eq=Event.query, su=User.query.get(1), User=User, Event=Event, savetodb=save_to_db, deletefromdb=delete_from_db ) if __name__ == "__main__": manager.add_command('shell', Shell(make_context=_make_context)) manager.run() Place this smg.py file in the same directory as manage.py, so you can access it with python smg.py shell. The code is pretty simple to understand. We import the Shell class from flask_script, create its object with our context and then add it to the manager as a command. _make_context contains what I usually like to have in my shell. It must always return a dictionary. The keys of this dictionary would be available as statements inside the shell with their values specified here. This helps a lot. Most of the time I would be working with the super_admin user, and I would need its User object from time to time. The super_admin user is always going to be the first user (User object with id 1). So instead of from open_event.models.user import User; su = User.query.get(1) I could just use the su variable. Models like User and Event are also readily available (so are their base queries). This is the default context that I always keep, but many times you need more models than the ones specified here. Like when I was working with the permissions system. from open_event.models.users_events_roles import UsersEventsRoles from open_event.models.service import Service from open_event.models.role import Role def _make_context(): return dict( # usual stuff UER=UsersEventsRoles, Service=Service, Role=Role ) You can even write a script that fetches all the database models (instance of sqlalchemy Model class) and then add them to the _make_context…

Continue ReadingShell hacks