Using Flask-REST-JSONAPI’s Resource Manager In Open Event API Server

For the nextgen Open Event API Server, we are using flask-rest-jsonapi to write all the API endpoints. The flask-rest-jsonapi is based on JSON API 1.0 Specifications for JSON object responses.

In this blog post, I describe how I wrote API schema and endpoints for an already existing database model in the Open Event API Server. Following this blog post, you can learn how to write similar classes for your database models. Let’s dive into how the API Schema is defined for any Resource in the Open Event API Server. Resource, here, is an object based on a database model. It provides a link between the data layer and your logical data abstraction. This ResourceManager has three classes.

  1. Resource List
  2. Resource Detail
  3. Resource Relationship

(We’ll take a look at the Speakers API.)
First, we see the already implemented Speaker Model
 :

class Speaker(db.Model):

   """Speaker model class"""

   __tablename__ = 'speaker'

   id = db.Column(db.Integer, primary_key=True)

   name = db.Column(db.String, nullable=False)

   photo = db.Column(db.String)

   website = db.Column(db.String)

   organisation = db.Column(db.String)

   is_featured = db.Column(db.Boolean, default=False)

   sponsorship_required = db.Column(db.Text)

   def __init__(self,

                name=None,

                photo_url=None,

                website=None,

                organisation=None,

                is_featured=False,

                sponsorship_required=None):

      self.name = name

      self.photo = photo_url

      self.website = website

      self.organisation = organisation

      self.is_featured = is_featured

      self.sponsorship_required = sponsorship_required

 

Here’s the Speaker API Schema:

class SpeakerSchema(Schema):
   class Meta:
       type_ = 'speaker'
       self_view = 'v1.speaker_detail'
       self_view_kwargs = {'id': '<id>'}
   id = fields.Str(dump_only=True)
   name = fields.Str(required=True)
   photo_url = fields.Url(allow_none=True)
   website = fields.Url(allow_none=True)
   organisation = fields.Str(allow_none=True)
   is_featured = fields.Boolean(default=False)
   sponsorship_required = fields.Str(allow_none=True)
class SpeakerList(ResourceList):  
   schema = SpeakerSchema
   data_layer = {'session': db.session,
                 'model': Speaker}
class SpeakerDetail(ResourceDetail):
   schema = SpeakerSchema
   data_layer = {'session': db.session,
                 'model': Speaker}
class SpeakerRelationship(ResourceRelationship):
   schema = SpeakerSchema
   data_layer = {'session': db.session,
                 'model': Speaker}

 

Last piece of code is listing the actual endpoints in __init__ file for flask-rest-jsonapi

api.route(SpeakerList, 'speaker_list', '/events/<int:event_id>/speakers', '/sessions/<int:session_id>/speakers', '/users/<int:user_id>/speakers')
api.route(SpeakerDetail, 'speaker_detail', '/speakers/<int:id>')
api.route(SpeakerRelationship, 'speaker_event', '/speakers/<int:id>/relationships/event')
api.route(SpeakerRelationship, 'speaker_user', '/speakers/<int:id>/relationships/user')
api.route(SpeakerRelationship, 'speaker_session', '/speakers/<int:id>/relationships/sessions')

 

How to write API schema from database model?

Each column of the database model is a field in the API schema. These are marshmallow fields and can be of several data types – String, Integer, Float, DateTime, Url.

Three class definitions follow the Schema class.

  • List:
    SpeakerList class is the basis of endpoints:
api.route(SpeakerList, 'speaker_list', '/events/<int:event_id>/speakers',
          '/sessions/<int:session_id>/speakers',                                                             
          '/users/<int:user_id>/speakers')

This class will contain methods that generate a list of speakers based on the id that is passed in view_kwargs. Let’s say that ‘/sessions/<int:session_id>/speakers’ is requested. As the view_kwargs here contains sesssion_id, the query methods in SpeakerList class will fetch a list of speaker profiles related to  the sessions identified by session_id.

The flask-rest-jsonapi allows GET and POST methods for ResourceList. When using these endpoints for POST, the before_create_object and before_post methods can be written. These methods are overridden from the base ResourceList class in flask-rest-jsonapi/resource.py when they are defined in Speaker’s class.

  • Detail: 

SpeakerDetail class provides these endpoints:

 api.route(SpeakerDetail, 'speaker_detail', '/speakers/<int:id>')

The Resource Detail provides methods to facilitate GET, PATCH and DELETE requests provided for the endpoints. Methods like: before_get_object, before_update_object, after_update_object are derived from ResourceDetail class. The endpoints return an object of the resource based on the view_kwargs in a JSON response.

  • Relationship:

SpeakerRelationship class, as you might have guesses, provides:

api.route(SpeakerRelationship, 'speaker_event', '/speakers/<int:id>/relationships/event')

api.route(SpeakerRelationship, 'speaker_user', '/speakers/<int:id>/relationships/user')

api.route(SpeakerRelationship, 'speaker_session', '/speakers/<int:id>/relationships/sessions')

SpeakerRelationship class provides methods to create, update and delete relationships between the speaker and related resources – events, users and sessions in this case.

The above is a bare bone API schema example. The actual implementation in Open Event Server has lots of helper methods too to cater to our specific needs.

Additional Resources:

Continue ReadingUsing Flask-REST-JSONAPI’s Resource Manager In Open Event API Server

Documenting Open Event API Using API-Blueprint

FOSSASIA‘s Open Event Server API documentation is done using an api-blueprint. The API Blueprint language is a format used to describe API in an API blueprint file, where a blueprint file (or a set of files) is such that describes an API using the API Blueprint language. To follow up with the blueprint, an apiary editor is used. This editor is responsible for rendering the API blueprint and printing the result in user readable API documented format. We create the API blueprint manually.

Using API Blueprint:-
We create the API blueprint by first adding the name and metadata for the API we aim to design. This step looks like this :-

FORMAT: V1
HOST: https://api.eventyay.com

# Open Event API Server

The Open Event API Server

# Group Authentication

The API uses JWT Authentication to authenticate users to the server. For authentication, you need to be a registered user. Once you have registered yourself as an user, you can send a request to get the access_token.This access_token you need to then use in Authorization header while sending a request in the following manner: `Authorization: JWT <access_token>`


API blueprint starts with the metadata, here FORMAT and HOST are defined metadata. FORMAT keyword specifies the version of API Blueprint . HOST defines the host for the API.

The heading starts with # and the first heading is regarded as the name of the API.

NOTE – Also all the heading starts with one or more # symbol. Each symbol indicates the level of the heading. One # symbol followed by heading serves as the top level i.e. one # = Top Level. Similarly for  ## = second level and so on. This is in compliance with normal markdown format.
        Following the heading section comes the description of the API. Further, headings are used to break up the description section.

Resource Groups:
—————————–
    By using group keyword at the starting of a heading , we create a group of related resources. Just like in below screenshot we have created a Group Users.

# Group Users

For using the API you need(mostly) to register as an user. Registering gives you access to all non admin API endpoints. After registration, you need to create your JWT access token to send requests to the API endpoints.


| Parameter | Description | Type | Required |
|:----------|-------------|------|----------|
| `name`  | Name of the user | string | - |
| `password` | Password of the user | string | **yes** |
| `email` | Email of the user | string | **yes** |

 

Resources:
——————
    In the Group Users we have created a resource Users Collection. The heading specifies the URI used to access the resource inside of the square brackets after the heading. We have used here parameters for the resource URI which takes us into how to add parameters to the URI. Below code shows us how to add parameters to the resource URI.

## Users Collection [/v1/users{?page%5bsize%5d,page%5bnumber%5d,sort,filter}]
+ Parameters
    + page%5bsize%5d (optional, integer, `10`) - Maximum number of resources in a single paginated response.
    + page%5bnumber%5d (optional, integer, `2`) - Page number to fetchedfor the paginated response.
    + sort (optional, string, `email`) - Sort the resources according to the given attribute in ascending order. Append '-' to sort in descending order.
    + filter(optional, string, ``) - Filter according to the flask-rest-jsonapi filtering system. Please refer: http://flask-rest-jsonapi.readthedocs.io/en/latest/filtering.html for more.

 

Actions:
————–
    An action is specified with a sub-heading inside of  a resource as the name of Action, followed by HTTP method inside the square brackets.
    Before we get on further, let us discuss what a payload is. A payload is an HTTP transaction message including its discussion and any additional assets such as entity-body validation schema.

There are two payloads inside an Action:

  1. Request: It is a payload containing one specific HTTP Request, with Headers and an optional body.
  2. Response: It is a payload containing one HTTP Response.

A payload may have an identifier-a string for a request payload or an HTTP status code for a response payload.

+ Request

    + Headers

            Accept: application/vnd.api+json

            Authorization: JWT <Auth Key>

+ Response 200 (application/vnd.api+json)


Types of HTTP methods for Actions:

  • GET – In this action, we simply send the header data like Accept and Authorization and no body. Along with it we can send some GET parameters like page[size]. There are two cases for GET: List and Detail. For example, if we consider users, a GET for List helps us retrieve information about all users in the response, while Details, helps us retrieve information about a particular user.

The API Blueprint examples implementation of both GET list and detail request and response are as follows.

### List All Users [GET]
Get a list of Users.

+ Request

    + Headers

            Accept: application/vnd.api+json

            Authorization: JWT <Auth Key>

+ Response 200 (application/vnd.api+json)

        {
            "meta": {
                "count": 2
            },
            "data": [
                {
                    "attributes": {
                        "is-admin": true,
                        "last-name": null,
                        "instagram-url": null,

 

### Get Details [GET]
Get a single user.

+ Request

    + Headers

            Accept: application/vnd.api+json

            Authorization: JWT <Auth Key>

+ Response 200 (application/vnd.api+json)

        {
            "data": {
                "attributes": {
                    "is-admin": false,
                    "last-name": "Doe",
                    "instagram-url": "http://instagram.com/instagram",

 

  • POST – In this action, apart from the header information, we also need to send a data. The data must be correct with jsonapi specifications. A POST body data for an users API would look something like this:
### Create User [POST]
Create a new user using an email, password and an optional name.

+ Request (application/vnd.api+json)

    + Headers

            Authorization: JWT <Auth Key>

    + Body

            {
              "data":
              {
                "attributes":
                {
                  "email": "example@example.com",
                  "password": "password",


A POST request with this data, would create a new entry in the table and then return in jsonapi format the particular entry that was made into the table along with the id assigned to this new entry.

  • PATCH – In this action, we change or update an already existing entry in the database. So It has a header data like all other requests and a body data which is almost similar to POST except that it also needs to mention the id of the entry that we are trying to modify.
### Update User [PATCH]
+ `id` (integer) - ID of the record to update **(required)**

Update a single user by setting the email, email and/or name.

Authorized user should be same as user in request body or must be admin.

+ Request (application/vnd.api+json)

    + Headers

            Authorization: JWT <Auth Key>

    + Body

            {
              "data": {
                "attributes": {
                  "password": "password1",
                  "avatar_url": "http://example1.com/example1.png",
                  "first-name": "Jane",
                  "last-name": "Dough",
                  "details": "example1",
                  "contact": "example1",
                  "facebook-url": "http://facebook.com/facebook1",
                  "twitter-url": "http://twitter.com/twitter1",
                  "instagram-url": "http://instagram.com/instagram1",
                  "google-plus-url": "http://plus.google.com/plus.google1",
                  "thumbnail-image-url": "http://example1.com/example1.png",
                  "small-image-url": "http://example1.com/example1.png",
                  "icon-image-url": "http://example1.com/example1.png"
                },
                "type": "user",
                "id": "2"
              }
            }

Just like in POST, after we have updated our entry, we get back as response the new updated entry in the database.

  • DELETE – In this action, we delete an entry from the database. The entry in our case is soft deleted by default. Which means that instead of deleting it from the database, we set the deleted_at column with the time of deletion. For deleting we just need to send header data and send a DELETE request to the proper endpoint. If deleted successfully, we get a response as “Object successfully deleted”.
### Delete User [DELETE]
Delete a single user.

+ Request

    + Headers

            Accept: application/vnd.api+json

            Authorization: JWT <Auth Key>

+ Response 200 (application/vnd.api+json)

        {
          "meta": {
            "message": "Object successfully deleted"
          },
          "jsonapi": {
            "version": "1.0"
          }
        }


How to check after manually entering all these? We can use the
apiary website to render it, or simply use different renderer to do it. How? Checkout for my next blog on apiary and aglio.

Learn more about api blueprint here: https://apiblueprint.org/

Continue ReadingDocumenting Open Event API Using API-Blueprint

Data Scraping using Python with BeautifulSoup4 for Open Event samples

was creating samples for Open Event Android and Open Event Webapp when the idea of web scraping through scripts stuck me.

I chose Python for web scraping. But what’s more to using Python other than it’s user-friendliness, easy syntax and speed? It’s the wide variety of open-source libraries that come along with it!

One of them is the BeautifulSoup library for extracting data from XML and HTML tags and files. It provides with ways to search and sort through webpages, find specific elements that you need and extract them in objects of your preference.

Apart from BeautifulSoup, another module that can be easily neglected but is of great use is:- urllib2. This module is to open URLs that are fed to it.

Hence, with a combination of the above 2 Python modules, I was able to scrape off the main sample data like speakers, sessions, tracks, sponsors and so on for PyCon 2017 (sample that I created recently) in a quicker and more efficient way.

I will now talk about these modules in greater detail and their use in my web scraping scripts.

Web Scraping with BeautifulSoup

Start by installing Beautifulsoup module in your Python environment. I believe that it will be best to install it using  pip.  Run the following command to install Beautifulsoup:-

pip install beautifulsoup4

Next up, you need to import the module to your script by adding the following line:-

from bs4 import BeautifulSoup

Now you need to open a webpage using the module in the following way. For that we first have to open the URL of the page that is being scraped. We use urllib2 to do that.

import urllib2
site=urllib2.urlopen("https://us.pycon.org/2017/sponsors/")

Next we use BeautifulSoup to get the contents of the webpage in the following way:-

website=BeautifulSoup(site)

Suppose that, next I want to obtain the data inside all the <div> tags in the webpage that have their class=”sponsor-level”.

I will use BeautifulSoup in the following way along with the ‘find_all’ method to do so :-

divs=website.find_all("div",{'class':'sponsor-level'})

Here, divs will now be a special <list> type element containing all the HTML in the div tags of class=’sponsor-level’. Now I can do whatever with I want with this list. Parse it in any way, store it any kind of data type without any difficulty.

BeautifulSoup also helps in accessing the children tags or elements of different HTML or XML tags using the ‘dot’ (.) operator. Following example will make it more clear:-

tag = soup.b

Here, ‘tag’ will contain all the  <b> elements of the parent element that is stored in ‘soup’ variable.

To access attributes of HTML or XML tags, there is a special way as demonstrated below:-

To access the ‘href’ attribute of the first <a> tag in the webpage:-

soup.find_all(“a”)[0][‘href’]

A sample script…

Following is a script that I used to obtain sponsors from the PyCon 2017 site in json format.

import urllib2

site=urllib2.urlopen("https://us.pycon.org/2017/sponsors/")

from bs4 import BeautifulSoup

s=BeautifulSoup(site)

divs=s.find_all("div",{'class':'sponsor-level'})

spons=[]

for level in divs:

levelname=level.find_all("h2")[0].string

level_sponsors=level.find_all("div",{'class':'sponsor'})

for level_sponsor in level_sponsors:

anchor=level_sponsor.find_all("a")

dictionary={}

dictionary['id']=""

dictionary['name']=anchor[1].string

dictionary['level']=""

dictionary['sponsor_Type']=levelname

dictionary['url']=anchor[0]['href']

dictionary['description']=level_sponsor.find_all("p")[1].string

spons.append(dictionary)

print spons

There are still a lot of other functionalities of BeautifulSoup that are available and can make your life easier. These can be found here in the official BeautifulSoup documentation.

Continue ReadingData Scraping using Python with BeautifulSoup4 for Open Event samples

Open Event Server: Working with Migration Files

FOSSASIA‘s Open Event Server uses alembic migration files to handle all database operations and updations.  From creating tables to updating tables and database, all works with help of the migration files.
However, many a times we tend to miss out that automatically generated migration files mainly drops and adds columns rather than just changing them. One example of this would be:

def upgrade():
    ### commands auto generated by Alembic - please adjust! ###
    op.add_column('session', sa.Column('submission_date', sa.DateTime(), nullable=True))
    op.drop_column('session', 'date_of_submission')

Here, the idea was to change the has_session_speakers(string) to is_session_speakers_enabled (boolean), which resulted in the whole dropping of the column and creation of a new boolean column. We realize that, on doing so we have the whole data under  has_session_speakers lost.

How to solve that? Here are two ways we can follow up:

  • op.alter_column:
    ———————————-

When update is as simple as changing the column names, then we can use this. As discussed above, usually if we migrate directly after changing a column in our model, then the automatic migration created would drop the old column and create a new column with the changes. But on doing this in the production will cause huge loss of data which we don’t want. Suppose we want to just change the name of the column of start_time to starts_at. We don’t want the entire column to be dropped. So an alternative to this is using op.alter_column. The two main necessary parameters of the op.alter_column is the table name and the column which you are willing to alter. The other parameters include the new changes. Some of the commonly used parameters are:

  1. nullable Optional: specify True or False to alter the column’s nullability.
  2. new_column_name – Optional; specify a string name here to indicate the new name within a column rename operation.
  3. type_Optional: a TypeEngine type object to specify a change to the column’s type. For SQLAlchemy types that also indicate a constraint (i.e. Boolean, Enum), the constraint is also generated.
  4. autoincrement –  Optional: set the AUTO_INCREMENT flag of the column; currently understood by the MySQL dialect.
  5. existing_typeOptional: a TypeEngine type object to specify the previous type. This is required for all column alter operations that don’t otherwise specify a new type, as well as for when nullability is being changed on a column.

    So, for example, if you want to change a column name from “start_time” to “starts_at” in events table you would write:
    op.alter_column(‘events’, ‘start_time’, new_column_name=’starts_at’)
def upgrade():
    ### commands auto generated by Alembic - please adjust! ###
    op.alter_column('sessions_version', 'end_time', new_column_name='ends_at')
    op.alter_column('sessions_version', 'start_time', new_column_name='starts_at')
    op.alter_column('events_version', 'end_time', new_column_name='ends_at')
    op.alter_column('events_version', 'start_time', new_column_name='starts_at')


Here,
session_version and events_version are the tables name altering columns start_time to starts_at and end_time to ends_at with the op_alter_column parameter new_column_name.

  • op.execute:
    ——————–

Now with alter_column, most of the alteration in the column name or constraints or types is achievable. But there can be a separate scenario for changing the column properties. Suppose I change a table with column “aspect_ratio” which was a string column and had values “on” and “off” and want to convert the type to Boolean True/False. Just changing the column type using alte_column() function won’t work since we need to also modify the whole data. So, sometimes we need to execute raw SQL commands. To do that, we can use the op.execute() function.
The way it is done:

def upgrade():
    ### commands auto generated by Alembic - please adjust! ###
    op.execute("ALTER TABLE image_sizes ALTER full_aspect TYPE boolean USING CASE 
            full_aspect WHEN 'on' THEN TRUE ELSE FALSE END", execution_options=None)

    op.execute("ALTER TABLE image_sizes ALTER icon_aspect TYPE boolean USING CASE 
            icon_aspect WHEN 'on' THEN TRUE ELSE FALSE END", execution_options=None)

    op.execute("ALTER TABLE image_sizes ALTER thumbnail_aspect TYPE boolean USING CASE 
            thumbnail_aspect WHEN 'on' THEN TRUE ELSE FALSE END"execution_options=None)

For a little more advanced use of op.execute() command will be:

op.alter_column('events', 'type', new_column_name='event_type_id')
    op.alter_column('events_version', 'type', new_column_name='event_type_id')
    op.execute('INSERT INTO event_types(name, slug) SELECT DISTINCT event_type_id, 
                lower(replace(regexp_replace(event_type_id, \'& |,\', \'\', \'g\'),
                \' \', \'-\')) FROM events where not exists (SELECT 1 FROM event_types 
                where event_types.name=events.event_type_id) and event_type_id is not
                null;')
    op.execute('UPDATE events SET event_type_id = (SELECT id FROM event_types WHERE 
                event_types.name=events.event_type_id)')
    op.execute('ALTER TABLE events ALTER COLUMN event_type_id TYPE integer USING 
                event_type_id::integer')

In this example:

  • op.alter_column() renames the column type to event_type_id of events table
  • op.execute() does the following:
  • Inserts into column name of event_types table the value of event_type_idN (which previously contained the name of the event_type) from events table, and
  • Inserts into slug column of event_types table the value of event_type_id where all letters are changed to lowercase; “& ” and “,” to “”; and spaces to “-”.
    1. Checks whether a type with that name already exists so as to disallow any duplicate entries in the event_types table.
    2. Checks whether the event_type_id is null because name of event_types table cannot be null.

You can learn more on Alembic migrations here: http://alembic.zzzcomputing.com/en/latest/ops.html

Continue ReadingOpen Event Server: Working with Migration Files

Open Event Server: Dealing with environment variables

FOSSASIA‘s Open Event Server uses some environment variables to maintain different configuration for different deployment. However one con of it was, for local development, developers have to run many export commands.

It all started with the “It is not convenient, we humans are lazy.” phrase. A recent suggestion was to read this environment variables with files rather than running commands every time.
This few example environment variable commands were:

export DATABASE_URL=postgresql://open_event_user:test@127.0.0.1:5432/test

and

export INTEGRATE_SOCKETIO=false

 

Open Event Server now uses .env file and envparse to handle this.

The changes were:

  1. We store the variables in .env file, let it get ignored by git, and
  2. Use envparse to parse the file.

What is envpase:
       envparse is a simple utility to parse environment variables. It aims to eliminate duplicated, often inconsistent parsing code and instead provide a single, easy-to-use wrapper.

So instead of reading the string from the command line everytime:

export DATABASE_URL=postgresql://open_event_user:password@127.0.0.1:5432/open_event_db

We read it from the file:

To read from command line we had:

SQLALCHEMY_DATABASE_URI = os.environ.get('DATABASE_URL', None)

Which now was parsed using envparse as:

SQLALCHEMY_DATABASE_URI =  env('DATABASE_URL', default=None)

In the above example, DATABASE_URL is a string and we store this string in SQLALCHEMY, setting the default by the default parameter(None in this case).

The type of environment variable is originally string, we can simply use env to parse these strings. For simplicity and understanding we can also use env.str. For different types, we can simply type cast them like env.bool (for boolean), env.int (for integer) etc. One good advantage of this envparse feature is the direct use of these type cast rather than checking for strings (i.e. reading the string from command line).

For example, earlier we had something like this:

socketio_integration = os.environ.get('INTEGRATE_SOCKETIO’)
if socketio_integration == true’:
INTEGRATE_SOCKETIO =True

Using envparse, we can have something like this:

INTEGRATE_SOCKETIO = env.bool('INTEGRATE_SOCKETIO’, default=False)

This helps in parsing as boolean instead of string. So, if we have INTEGRATE_SOCKETIO=true in .env file, we can write INTEGRATE_SOCKETIO = env.bool(‘INTEGRATE_SOCKETIO’, default=False). This automatically converts the true to boolean True in python.

In order to use envparse we install envparse with:

$ pip install envparse


In the case of
Open Event Server project, you only have to install the requirements.txt and copy the .env.example already mentioned in installation doc.

This is how we finally remove the headache of writing big exports again and again and fairly helps beginners to start off with Open Event Server  project much more easily and quickly.

Continue ReadingOpen Event Server: Dealing with environment variables

Step by step guide for Beginners in Web Development for Open Event Frontend

Originally the frontend and backend of the Open Event Server project were handled by FLASK with jinja2 being used for rendering templates. As the size of the project grew, it became difficult to keep track of all the modifications made on the frontend side. It also increased the complexity of the code. As a result of this, a new project Open Event Frontend was developed by decoupling the backend and frontend of the Open Event Orga Server. Now the server is being converted fully into functional API and database and the open event frontend project is primarily the frontend for the Open event server API where organisers, speakers and attendees can sign-up and perform various functions.     

The Open Event Frontend project is built on JavaScript web application framework, “Ember.js”. To communicate with the server API Ember.js user Ember data which is a data persistence module via the exposed endpoints. Suppose if you’re coming from the Android background, soon after diving into the web development you can relate that the web ecosystem is much larger than the mobile one and for the first timers it can be difficult to adopt with it because of the reason that in web there are multiple ways to perform a task which can be restricted to very few in the case of Android. For web applications, one can find that much more components are involved in setting up the project while in android one can easily start contributing to project soon after compiling it in Android Studio. One thing which is relatable for both the android and web development is that in the case of android one has to deal with the varying screen sizes and compatibility issue while in the web one has to deal with adding support for different browsers and versions which can be really annoying.

Now let’s see how one can start contributing to the Open event frontend project while having no or a little knowledge of web development. In case if you’ve previous knowledge of JavaScript then you can skip the step 1 and move directly to another step which is learning the framework.

(Here all the steps have been explained in reference if you’re switching from Android  to Web development.)

Step 1. Learning the Language – JavaScript

Now that when you’ve already put your feet into the web development it’s high time to get acquainted with the JavaScript. Essentially in the case of Ember which is easy to comprehend, you can though start with learning the framework itself but the executables file are written in JavaScript so to write them you must have basic knowledge of the concepts in the language. Understanding of JavaScript will help in letting know where the language ends and where the framework starts. It will also help in better understanding of the framework. In my opinion, the basic knowledge of JavaScript like the scope of variables, functions, looping, conditional statements, modifying array and dictionaries, ‘this’ keyword etc. helps in writing and understanding the .js files easily. Also, sometimes in JavaScript, an error might not be thrown as an exception while compiling but it may evaluate the program to undefined, knowledge of the language will help in debugging the code.

Step 2. Learning the Framework – Ember

Ember is a JavaScript Framework which works on Model-View-Controller(MVC) approach. The Ember is a battery included framework which generates all the boilerplate code including components, routes. Templates etc.  required for building an application’s frontend. It is very easy to understand and comprehend. In Ember, we can easily define the data models and relationships and ember will automatically guess the correct API endpoints. Apart from this, the documentation of the ember on its website is very much sufficient to start with. One can easily start developing applications after going through the tutorial mentioned on the ember’s website.  

Step 3. Testing

In the case of Android application development to write test we use android libraries like Mockito and Robolectric. Also, testing is a bit more difficult in Android app development because we have to explicitly write the test but it is a lot easier in the case of web development. In the case of Ember, it provides an ease of testing which no other framework and libraries provide. While generating a component or template ember itself generates the test files for them and all we have to do is to change them according to our requirement. Ember generates unit, acceptance and integration tests by making testing easier. So we don’t have to write the test explicitly we only have to modify the test files generated by ember.    

Step 4. Styling

In Android we have colors.xml, styles.xml, drawables, gradients, shapes etc. for styling our application but in the case of Web, we have Cascading Style Sheets (CSS) for styling our application. Simply using pure CSS make design complicated and difficult to understand, so to make it easier we combine a bunch of design elements with a style file and use Syntactically Awesome Style Sheets (Saas) with mixins to do that which makes creating styles a lot easier and more straightforward. So for styling, our web application one should have the knowledge of HTML as well as CSS.

In conclusion, I can say that learning web development requires learning a few things in parallel which includes learning a language, learning a framework, how to perform testing and different styling skills to make an application beautiful. Due to dynamic nature of the JavaScript and the sheer number of packages and components involved, as opposed to the safe environment that Android Studio provides, it can be sometimes really frustrating.  However, once learned the basics, the knowledge and skills can be easily transferred and applied over and over again.    

Continue ReadingStep by step guide for Beginners in Web Development for Open Event Frontend

Using Ember.js Components in Open Event Frontend

Ember.js is a comprehensive JavaScript framework for building highly ambitious web applications. The basic tenet of Ember.js is convention over configuration which means that it understands that a large part of the code, as well as development process, is common to most of the web applications. Talking about the components which are nothing but the elements whose role remain same with same properties and functions within the entire project. Components allow developers to bundle up HTML elements and styles into reusable custom elements which can be called anywhere within the project.

In Ember, the components consist of two parts: some JavaScript code and an HTMLBars template. The JavaScript component file defines the behaviour and properties of the component. The behaviours of the component are typically defined using actions. The HTMLBars file defines the markup for the component’s UI. By default, the component will be rendered into a ‘div’ tag element, but a different element can be defined if required. A great thing about templates in Ember is that other components can be called inside of a component’s template. To call a component in an Ember app, we must use {{curly-brace-syntax}}. By design, components are completely isolated which means that they are not directly affected by any surrounding CSS or JavaScript.

Let’s demonstrate a basic Ember component in reference to Open Event Frontend Project for displaying the text as a popup. The component will render a simple text view which will display the entire text. The component is designed with the purpose that many times due to unavailability of space we’re unable to show the complete text so such cases the component will compare the available space with the space required by the whole text view to display the text. If in case the available space is not sufficient then the text will be ellipsized and on hovering the text a popup will appear where the complete text can be seen.

Generating the component

The component can be generated using the following command:

$ ember g component smart-overflow

Note: The components name needs to include a hyphen. This is an Ember convention, but it is an important one as it’ll ensure there are no naming collisions with future HTML elements.This will create the required .js and .hbs files needed to define the component, as well as an Ember integration test.

The Component Template

In the app/templates/components/smart-overflow.hbs file we can create some basic markup to display the text when the component is called.

<span> {{yield}} </span>

The {{yield}} is handlebars expressions which will be helpful in rendering the data to display when the component is called.

The JavaScript Code

In the app/components/smart-overflow.js file, we will define the how the component will work when it is called.

import Ember from 'ember';

const { Component } = Ember;

export default Component.extend({
  classNames: ['smart-overflow'],
  didInsertElement() {
    this._super(...arguments);
    var $headerSpan = this.$('span');
    var $header = this.$();
    $header.attr('data-content', $headerSpan.text());
    $header.attr('data-variation', 'tiny');
    while ($headerSpan.outerHeight() > $header.height()) {
      $headerSpan.text((index, text) => {
        return text.replace(/\W*\s(\S)*$/, '...');
      });
      $header.popup({
        position: 'top center'
      });
      this.set('$header', $header);
    }
  },
  willDestroyElement() {
    this._super(...arguments);
    if (this.get('$header')) {
      this.get('$header').popup('destroy');
    }
  }
});

 

In the above piece of code, we have first taken the size of the available space in header variable and then taken the size of the content in header span variable. After that, we’re comparing both the sizes to check if the content is greater than the available space then we are ellipsizing the content and create a popup to display the complete text to produce good user experience.

Passing data to the component

To allow the component to display the data properly, we need to pass it in.

In the app/templates/components/event-card.hbs file we can call the component as many times as desired and pass in relevant data for each attribute.

<div class="ui card {{unless isWide 'event fluid' 'thirteen wide computer ten wide tablet sixteen wide mobile column'}}">
    {{#unless isWide}}
      <a class="image" href="{{href-to 'public' event.identifier}}">
        {{widgets/safe-image src=(if event.large event.large event.placeholderUrl)}}
      </a>
    {{/unless}}
    <a class="main content" href="{{href-to 'public' event.identifier}}">
      {{#smart-overflow class='header'}}
        {{event.name}}
      {{/smart-overflow}}
      <div class="meta">
        <span class="date">
          {{moment-format event.startTime 'ddd, MMM DD HH:mm A'}}
        </span>
      </div>
      {{#smart-overflow class='description'}}
        {{event.shortLocationName}}
      {{/smart-overflow}}
    </a>
    <div class="extra content small text">
      <span class="right floated">
        <i role="button" class="share alternate link icon" {{action shareEvent event}}></i>
      </span>
      <span>
        {{#if hasBlock}}
          {{yield}}
        {{else}}
          {{#each tags as |tag|}}
            <a>{{tag}}</a>
          {{/each}}
        {{/if}}
      </span>
    </div>
  </div>

 

Now if you view the app in the browser at localhost:4200, you will see something like this.

Fig. 1

In the end, we can say that with the components, the code remains much clear and readable. It makes more sense to the developers who happen upon them. The best part about them is their reusability across the application making the development process faster and much more efficient.

Reference: The Complete source for the smart overflow can be found here.

Continue ReadingUsing Ember.js Components in Open Event Frontend

Forms and their validation using Semantic UI in Open Event Frontend

A web form acts as a communication bridge that allows a user to communicate with the organisation and vice versa. In the Open Event project, we need forms so users can contact the organisation, to register themselves, to log into the website, to order a ticket or to query for some information. Here are a few things which were kept in mind before we designed forms in the Open Event Frontend Project:

  • The forms were designed on the principle of keeping it simple which means that it should ask only for the relevant information which is required in actual.
  • They contained the relevant fields ordered in a logical way according to their importance.
  • They offered clear error messages instantly to give direct feedback and allow users to make instant corrections.
  • The clear examples were shown in the front of the field.
  • Proper spacing among the fields was maintained to display proper error messages to the respective form fields.
  • The mandatory fields are highlighted using ‘*’ to avoid confusion.
  • Proper colour combinations have been used to inform the user about the progress while filling the form. For eg. red for any ‘error or incomplete’ information while green signifies ‘correct’.
  • Saving the current data in case the user has to go back to make any corrections later.
  • Allowing to toggle through the form using the keyboard.

The above designing principles helped in avoiding the negative user experience while using the forms.

Let’s take a closer look at the form and the form validation in case of purchase a new ticket form on the Orders page in Open Event Front-end application.

Creating a form

Let’s start by writing some HTML for the form:

<form class="ui form" {{action 'submit' on='submit' }}>
  <div class="ui padded segment">
    <h4 class="ui horizontal divider header">
      <i class="ticket icon"></i>
      {{t 'Ticket Buyer'}}
    </h4>
    <div class="field">
      <label class="required" for="firstname">{{t 'First Name'}}</label>
      {{input type='text' name='first_name' value=buyer.firstName}}
    </div>
    <div class="field">
      <label class="required" for="lastname">{{t 'Last Name'}}</label>
      {{input type='text' name='last_name' value=buyer.lastName}}
    </div>
    <div class="field">
      <label class="required" for="email">{{t 'Email'}}</label>
      {{input type='text' name='email' value=buyer.email}}
    </div>
    <h4 class="ui horizontal divider header">
        <i class="ticket icon"></i>
        {{t 'Ticket Holder\'s Information'}}
    </h4>
    {{#each holders as |holder index|}}
      <div class="inline field">
        <i class="user icon"></i>
         <label>{{t 'Ticket Holder '}}{{inc index}}</label>
      </div>
      <div class="field">
        <label class="required" for="firstname">{{t 'First Name'}}</label>
        {{input type='text' name=(concat 'first_name_' index) value=holder.firstName}}
      </div>
      <div class="field">
        <label class="required" for="lastname">{{t 'Last Name'}}</label>
        {{input type='text' name=(concat 'last_name_' index) value=holder.lastName}}
      </div>
      <div class="field">
        <label class="required" for="email">{{t 'Email'}}</label>
        {{input type='text' name=(concat 'email_' index) value=holder.email}}
      </div>
      <div class="field">
        {{ui-checkbox label=(t 'Same as Ticket Buyer') checked=holder.sameAsBuyer onChange=(action 'fillHolderData' holder)}}
      </div>
    {{/each}}
    <p>
      {{t 'By clicking "Pay Now", I acknowledge that I have read and agree with the Open Event terms of services and privacy policy.'}}
    </p>
    <div class="center aligned">
      <button type="submit" class="ui teal submit button">{{t 'Pay Now'}}</button>
    </div>
  </div>
</form>

 

The complete code for the form can be seen here.

In the above code, we have used Semantic UI elements like button, input, label, icon, header and modules like dropdown, checkbox to create the basic structure of the form.

The form is created using the Semantic markup. Along with semantic UI collection “form”, the segment element has been used to create the grouping of similar content like we have a timer and its related description that after 10 minutes the reservation will no longer be held are put together in a segment where they are arranged using semantic UI view “statistic”. Due to the vastness of semantic UI, all the styling has been done using it like fields inlining, button styling, segment background coloring etc.

The form is created using the Semantic markup. Along with semantic UI collection “form”, the segment element has been used to create the grouping of similar content like we have a timer and its related description that after 10 minutes the reservation will no longer be held are put together in a segment where they are arranged using semantic UI view “statistic”. Semantic UI elements like button, input, label, icon, header and modules like dropdown, checkbox have been used. Due to the vastness of semantic UI, all the styling has been done using it like fields inlining, button styling, segment background colouring etc.

The page for the above HTML code looks like this:

Image for the order form

Fig. 1: Order Form to purchase a ticket

The complete form can be seen on this link.

Adding form validations

We can also add validation in HTML format but writing the validation in JavaScript file is considered good practice.

Let’s see how we can add validation to fields in Javascript.

  getValidationRules() {
    let firstNameValidation = {
      rules: [
        {
          type   : 'empty',
          prompt : this.i18n.t('Please enter your first name')
        }
      ]
    };
    let lastNameValidation = {
      rules: [
        {
          type   : 'empty',
          prompt : this.i18n.t('Please enter your last name')
        }
      ]
    };
    let emailValidation = {
      rules: [
        {
          type   : 'empty',
          prompt : this.i18n.t('Please enter your email')
        }
      ]
    };
    let validationRules = {
      inline : true,
      delay  : false,
      on     : 'blur',
      fields : {
        firstName: {
          identifier : 'first_name',
          rules      : [
            {
              type   : 'empty',
              prompt : this.i18n.t('Please enter your first name')
            }
          ]
        },
        lastName: {
          identifier : 'last_name',
          rules      : [
            {
              type   : 'empty',
              prompt : this.i18n.t('Please enter your last name')
            }
          ]
        },
        email: {
          identifier : 'email',
          rules      : [
            {
              type   : 'email',
              prompt : this.i18n.t('Please enter a valid email address')
            }
          ]
        },
        zipCode: {
          identifier : 'zip_code',
          rules   : [
            {
              type   : 'empty',
              prompt : this.i18n.t('Please enter your zip code')
            }
          ]
        } 
    };

Let’s break this up, first, we have an array of validation rules.

  zipCode: {
     identifier : 'zip_code',
        rules   : [
        {
          type   : 'empty',
          prompt : this.i18n.t('Please enter your zip code')
         }
      ]
   }

The first part zipcode is the identifier in Semantic.

The next bit of the identifier, this can match against either id, name or data-validate attributes on the element. We have here picked up the name which we’re using on our labels.

Next bit of the rules, which is an array of objects defining the type of validation, and the message to prompt the user with.

The second part is the settings:

inline : true,

delay : false,

on : 'blur',

This part says we want validation to occur on blur, delayed and to be inline. This gives us the following effect:

Fig. 2: Order Form after validation

To summarise the post, one can say we have seen here how the form to purchase the event ticket has been designed, coded and styled. The complete form can be seen on this link and the complete code can be seen here. The entire form has been designed in such a way to keep it simple, clear and trustworthy without losing the user interaction.

References:

Continue ReadingForms and their validation using Semantic UI in Open Event Frontend

ButterKnife for Open Event Android

Open Event Android, by FOSSASIA has been coded in a very clean and systematic manner. One of the great things used in it is the ButterKnife library. It has made increased the readability and understand ability of the app’s code.

ButterKnife, developed and maintained by Jake Wharton(Square Inc.) is an Android library designed to make Java code less complex and efficient. The library operates with the use of annotations and hence, binds classes to the relevant annotations in Java and do the job.

First off, here’s the very first thing that you want to do before starting to use ButterKnife — Adding it’s dependency in the build.gradle file of your project.

compile 'com.jakewharton:butterknife:8.6.0
annotationProcessor 'com.jakewharton:butterknife-compiler:8.6.0'

At FOSSASIA, we have extensively used this amazing library to simplify the code for our Open Event Android application. I hope this blog will help newcomers in understanding our code base in a better way.

We have the famous @Bindview annotation by ButterKnife to bind views to Android components without having to call R.FindViewById() method for every view. The following is a code sample from Open Event Android about the same.

@BindView(R.id.toolbar) Toolbar toolbar;

@BindView(R.id.nav_view) NavigationView navigationView;

@BindView(R.id.progress) ProgressBar downloadProgress;

@BindView(R.id.layout_main) CoordinatorLayout mainFrame;

@BindView(R.id.drawer) DrawerLayout drawerLayout;

@BindView(R.id.appbar) AppBarLayout appBarLayout;

Similarly, the ButterKnife library also handles events in an Android application and has annotations like @onClick, @onLongClick among others. So while using ButterKnife for an event, choose the appropriate event and the corresponding annotation followed by the Java method that is to be executed. Here is an example for the same.

@OnClick(R.id.submit)

public void submit(View view) {

// TODO submit data to server...

}

The ButterKnife library also helps to bind resources in your project.

It simplifies the View holder pattern inside a list adapter as illustrated in the following example.

public class MyAdapter extends BaseAdapter {

@Override public View getView(int position, View view, ViewGroup parent) {

   ViewHolder holder;

   if (view != null) {

     holder = (ViewHolder) view.getTag();

   } else {

     view = inflater.inflate(R.layout.whatever, parent, false);

     holder = new ViewHolder(view);

     view.setTag(holder);

   }

   holder.name.setText("John Doe");

   // etc...

   return view;

 }

static class ViewHolder {

   @BindView(R.id.title) TextView name;

   @BindView(R.id.job_title) TextView jobTitle;

   public ViewHolder(View view) {

     ButterKnife.bind(this, view);

   }

 }

}

The above code sample has been taken from http://jakewharton.github.io/butterknife/ and you can visit the site for further in-depth knowledge on ButterKnife.

How ButterKnife works?

I will be talking a bit about the working of the ButterKnife library. Firstly, I would like to introduce annotations concept in Java briefly before proceeding.

“In the Java, computer programming language, an annotation is a form of syntactic metadata that can be added to Javasource code. Classes, methods, variables, parameters and packages may be annotated.” As it is rightly said, Annotations can be of various uses like Getting information for the compiler, Compile-time and deployment-time processing, Runtime processing. However, please note that annotations can NOT modify and edit the existing classes. They can simply make more classes.

With this, I’m gonna continue on the working of ButterKnife. From the above discussion it must be clear now that ButterKnife is not changing any of our classes. It makes new classes to handle the code.

ButterKnife goes through all the files in your project and identifies the ButterKnife annotations used, if any. It then creates new classes for every annotation encountered, based on the annotations used. These new classes contain the general methods that would have been used in a project without ButterKnife. When we finally call ButterKnife.inject(this) , the inject()of all the new classes made by ButterKnife are called which then perform the desired function during execution.

Finally, try to go through the complete documentation for Butterknife and Java annotations for more information. 

Continue ReadingButterKnife for Open Event Android

Diving into the codebase of the Open Event Front-end Project

This post aims to help any new contributor to get acquainted with the code base of the Open Event Front-end project.

The open event front-end is primarily the front-end for the Open Event API server. The project provides the functionality of signing up, add, update and view events details and many other functions to the organisers, speakers and attendees of the event which can be concert, conference, summit or a meetup. The open event front-end project is built on a JavaScript web application framework Ember.js. Ember uses a library Ember data for managing the data in the application and for communicating with server API via endpoints. Ember is a battery included framework which means that it creates all the boilerplate code required to set up the project and create a working web application which can be modified according to our needs.

The open event front-end is primarily the front-end for the Open Event API server. The project provides the functionality of signing up, add, update and view events details and many other functions to the organisers, speakers and attendees of the event which can be concert, conference, summit or a meetup. The open event front-end project is built on a JavaScript web application framework Ember.js. Ember uses a library Ember data for managing the data in the application and for communicating with server API via endpoints. Ember is a battery included framework which means that it creates all the boilerplate code required to set up the project and create a working web application which can be modified according to our needs.

For example: When I created a new project using the command:

$ ember new open-event-frontend

It created a new project open-event-frontend with all the boilerplate code required to set up the project which can be seen below.

create .editorconfig
create .ember-cli
create .eslintrc.js
create .travis.yml
create .watchmanconfig
create README.md
create app/app.js
create app/components/.gitkeep
Installing app
create app/controllers/.gitkeep
create app/helpers/.gitkeep
create app/index.html
create app/models/.gitkeep
create app/resolver.js
create app/router.js
create app/routes/.gitkeep
create app/styles/app.css
create app/templates/application.hbs
create app/templates/components/.gitkeep
create config/environment.js
create config/targets.js
create ember-cli-build.js
create .gitignore
create package.json
create public/crossdomain.xml
create public/robots.txt
create testem.js
create tests/.eslintrc.js
create tests/helpers/destroy-app.js
create tests/helpers/module-for-acceptance.js
create tests/helpers/resolver.js
create tests/helpers/start-app.js
create tests/index.html
create tests/integration/.gitkeep
create tests/test-helper.js
create tests/unit/.gitkeep
create vendor/.gitkeep
NPM: Installed dependencies
Successfully initialized git.

Now if we go inside the project directory we can see the following files and folders have been generated by the Ember-CLI which is nothing but a toolkit to create, develop and build ember application. Ember has a runtime resolver which automatically resolves the code if it’s placed at the conventional location which ember knows.

➜ ~ cd open-event-frontend
➜ open-event-frontend git:(master) ls
app  config  ember-cli-build.js  node_modules  package.json  public  README.md  testem.js  tests  vendor

What do these files and folders contain and what is their role in reference to the project, “Open event front-end”?

Directory structure of open event frontend project

Fig 1: Directory structure of the Open Event Front-end project

Let’s take a look at the folders and files Ember CLI generates.

App: This is the heart of the project. It is responsible for deciding how our application will look and work. This is the place where folders and files for models, components, routes, templates and styles are stored. The majority of our application’s code is written in this folder. The adapter directory contains the application.js file which is responsible for authorising all outgoing API requests. Since our application is an event application so it has various screens which display the list of events, sessions, schedule, speakers, functionality to add, delete, modify the event details etc. Most of the screens have some features in common which can be reused everywhere inside the application which is kept in components directory. The controllers directory contains the files which will be responsible for the front-end behaviour which is not related to the models of our application. For example actions like sharing the event details. The data of the event is fetched by API but when the share functionality has to work is decided by the controller. Models directory helps in mapping the format of data i.e. in which format the data is fetched from the API. As we interact with the application, it moves through many different states/screens using different URLs. The routes directory decides which template will be rendered and which model will be loaded to display the data in the current template. The services directory contains the services which kept on running in the application like authentication service to check whether the current user has logged in or not, translation service to translate the details into some other language other than English which is supported by the app if required. The styles directory contains .scss files for styling of layouts. The template directory contains the layouts for the user interface of the application in form of Handlebar Templates which contains the HTML code which will be rendered on the screen when the template will be called via routes. The utils directory contains all the utility files like the format of date and time, demographic details, event type, type of topics etc.   

Directory structure of App folder

Fig 2: Directory structure of the app folder

Config: It contains files where we can we configure the settings of our application. The environment.js file contains the details like application name and API endpoints and other details required to configure the application. The deploy.js file contains the configurational details about the deployment.

Directory structure of config folder

Fig 3: Directory structure of config folder

Mirage: It contains the files which are required to mock the API server. Since we need to fetch the data from the server so whenever we make a request to get and post the data, mirage will respond to that data. It also contains the file which helps in seeding fake data for the application. Here mocking means creating objects that simulate the behaviour of real objects. The default.js application handles the seeding the fake data but currently, we don’t require that hence it’s empty. The config.js file has the method which sets the API host and namespace of the application.

Directory structure of mirage folder

Fig 4: Directory structure of mirage folder

Node_modules: As we know that Ember is built with Node and uses various node.js modules for functioning. So this directory contains all the files generated from the running the npm package manager which we run at starting while installing and setting up the project on the local device. The package.json file contains all the file maintains the list of npm dependencies for our app.

Public: This directory contains all the assets such as images and fonts related to the application. In the case of our application, to add images apart from the ones that are fetched from the API, they have to be placed here.

Directory structure of public folder

Fig 5: Directory structure of public folder

Translations: Since our application provides various languages support so this directory contains the files to translate the messages.

Directory structure of translations folder

Fig 6: Directory structure of translations folder

Tests: This is the best feature which Ember has provided. We don’t have to generate test they are automatically generated by the Ember-CLI which can be modified according to our requirements. Ember generates three types of test which are Acceptance, Integration and Unit tests. Other than that Ember offers testing for helpers, controllers, routes and models. Test runner of Ember-CLI testem is configured in testem.js. All three types of test have been used in our application.

Directory structure of test folder

Fig 7: Directory structure of test folder

ember-cli-build.js: This file contains the detail how Ember CLI should build our app. In the case of our project, it contains all the metadata such as app imports and other details to build the app.   

Vendor: This directory usually contains all the front-end dependencies (such as JavaScript or CSS) which are not managed by Bower go (which is a package manager for managing components that contain HTML, CSS, JavaScript, fonts). But in the case of our project, we have only kept .gitkeep inside this since we have not included any dependency which Bower doesn’t maintain.

To summarise, this post explains about the content and the role of different files and folders in the directory structure of the Open Event Front-end Project which may help a new contributor to get started by getting a basic understanding of the project.

Continue ReadingDiving into the codebase of the Open Event Front-end Project