magdalenesuo | blog.fossasia.org

Open Event Server Ticket PDF: Where and Where-not to use static frame in xhtml2pdf

Post author:magdalenesuo
Post published:July 17, 2017
Post category:Event Event Management FOSSASIA GSoC Open Event
Post comments:0 Comments

One among the very important features of Open Event Server project is the tickets sales feature, where a user can buy a number of different tickets for a number of people after which he is given a link to download the ticket pdf. However, an issue concerning our Open Event Server project was that if a buyer bought different tickets at a time with different individual ticket holders, all tickets contained the same name, type and QR-Code, which in no way was acceptable since tickets and holders were different.

We use the xhtml2pdf facility in order to convert an html to pdf. An xhtml2pdf facility helps us in converting HTML contents into PDF without the use of browser ‘print’ facility. In order to do this, we use the help of pages and frames, pages being the page of the PDF document, while a frame being that part of area within the page where the contents get stored.

What is a PDF and HTML?

The basic understanding of a PDF and an HTML is that, a PDF or Portable Document Format has layout such that it is measured in terms of specific width and height. However, for an HTML or Hyper Text Markup Language, we do no not have those specific widths and heights. Rather an HTML’s width depends on a person’s device of view and height can be infinitely long as desired.

In terms of xhtml2pdf relation with pages and frames layout, we can identify with this diagram:

+-page———————+
|                      |
| +-content_frame-+ |
| |              | |
| |                | |
| |                | |
| |                | |
| +———————+   |
|                     |
+—————————-+

The stated issue was due to static frame in xhtml2pdf.

What is that you ask?

Static frames vs Content frames
xhtml2pdf uses the concept of Static Frames to define content that remains the same across different pages (like headers and footers), and uses Content Frames to position the to-be-converted HTML content.

Static Frames are defined through use of the @frame property -pdf-frame-content. Regular HTML content will not flow through Static Frames.

Content Frames are @frame objects without this property defined. Regular HTML content will flow through Content Frames.(xhtml2pdf documentation)

So, the basic idea of the use of -pdf-frame-content was to make the contents of the pdf static i.e. without having to continuously change alignments of the page. It made the whole pdf stay static.

This actually caused the whole pdf content to stay constant, event while the loop was going over different name, QR-code and ticket-name values. That means the first loop content stayed over even with changes in values.

Just a simple fix of not making the frame static was the solution.

Here is a few insight of what we had before that was causing the issue:

 <style>
        @page {
            size: a4 portrait;
            background-image: url('{{ base_dir }}/static/data/ticket-trans-
                                  notext.png');
            margin: 0;

            @frame col1_frame {             /* Content frame 1 */
                left: 80pt;  top: 30pt;
                height: 250pt; width: 300pt;
                -pdf-frame-content: main_content;
            }

            @frame qrcode {
                left: 455pt;  top: 50pt;
                width: 100pt; height: 120pt;
                -pdf-frame-content: qr_code;
            }
            @frame number_frame {
                 left: 455pt;  top: 25pt;
                width: 100pt; height: 30pt;
                -pdf-frame-content: number_content;
            }


             @frame col2_frame {
                left: 440pt; top: 170pt;
                -pdf-frame-content: personal_content;
            }
        }

Here, main_content, qrcode, personal_content and number_content? are simply the class you want to make static.
The following was done to fix it:

 <style>
        @page {
            size: a4 portrait;
            background-image: url('{{ base_dir }}/static/data/ticket-trans-
                                  notext.png');
            margin: 0;

            @frame col1_frame {             /* Content frame 1 */
                left: 80pt;  top: 30pt;
                height: 250pt; width: 300pt;
            }

            @frame qrcode {
                left: 455pt;  top: 50pt;
                width: 100pt; height: 120pt;
            }
            @frame number_frame {
                 left: 455pt;  top: 25pt;
                width: 100pt; height: 30pt;
                -pdf-frame-content: number_content;
            }


             @frame col2_frame {
                left: 440pt; top: 170pt;
            }
        }

But the main question was, ‘where and where-not to use static frame’.
So the simple answer to that is, when you want the contents to stay exactly the same over the different pages of the pdf (for example the company logo or the administrator signature, etc), then you can happily get on with -pdf-frame-content. i.e:

          -pdf-frame-content: <class_name>

If not, or you need to loop over different values, then use Content frames.

Also, for more detailed understanding of PDF and HTML pages and frames, we have this good documentation written about it here: https://github.com/xhtml2pdf/xhtml2pdf/blob/master/doc/source/format_html.rst#static-frames-vs-content-frames

Documenting Open Event API Using API-Blueprint

Post author:magdalenesuo
Post published:July 8, 2017
Post category:Event Event Management FOSSASIA GSoC Open Event
Post comments:0 Comments

FOSSASIA‘s Open Event Server API documentation is done using an api-blueprint. The API Blueprint language is a format used to describe API in an API blueprint file, where a blueprint file (or a set of files) is such that describes an API using the API Blueprint language. To follow up with the blueprint, an apiary editor is used. This editor is responsible for rendering the API blueprint and printing the result in user readable API documented format. We create the API blueprint manually.

Using API Blueprint:-
We create the API blueprint by first adding the name and metadata for the API we aim to design. This step looks like this :-

FORMAT: V1
HOST: https://api.eventyay.com

# Open Event API Server

The Open Event API Server

# Group Authentication

The API uses JWT Authentication to authenticate users to the server. For authentication, you need to be a registered user. Once you have registered yourself as an user, you can send a request to get the access_token.This access_token you need to then use in Authorization header while sending a request in the following manner: `Authorization: JWT <access_token>`

API blueprint starts with the metadata, here FORMAT and HOST are defined metadata. FORMAT keyword specifies the version of API Blueprint . HOST defines the host for the API.

The heading starts with # and the first heading is regarded as the name of the API.

NOTE – Also all the heading starts with one or more # symbol. Each symbol indicates the level of the heading. One # symbol followed by heading serves as the top level i.e. one # = Top Level. Similarly for ## = second level and so on. This is in compliance with normal markdown format.
Following the heading section comes the description of the API. Further, headings are used to break up the description section.

Resource Groups:
—————————–
By using group keyword at the starting of a heading , we create a group of related resources. Just like in below screenshot we have created a Group Users.

# Group Users

For using the API you need(mostly) to register as an user. Registering gives you access to all non admin API endpoints. After registration, you need to create your JWT access token to send requests to the API endpoints.


| Parameter | Description | Type | Required |
|:----------|-------------|------|----------|
| `name`  | Name of the user | string | - |
| `password` | Password of the user | string | **yes** |
| `email` | Email of the user | string | **yes** |

Resources:
——————
In the Group Users we have created a resource Users Collection. The heading specifies the URI used to access the resource inside of the square brackets after the heading. We have used here parameters for the resource URI which takes us into how to add parameters to the URI. Below code shows us how to add parameters to the resource URI.

## Users Collection [/v1/users{?page%5bsize%5d,page%5bnumber%5d,sort,filter}]
+ Parameters
    + page%5bsize%5d (optional, integer, `10`) - Maximum number of resources in a single paginated response.
    + page%5bnumber%5d (optional, integer, `2`) - Page number to fetchedfor the paginated response.
    + sort (optional, string, `email`) - Sort the resources according to the given attribute in ascending order. Append '-' to sort in descending order.
    + filter(optional, string, ``) - Filter according to the flask-rest-jsonapi filtering system. Please refer: http://flask-rest-jsonapi.readthedocs.io/en/latest/filtering.html for more.

Actions:
————–
An action is specified with a sub-heading inside of a resource as the name of Action, followed by HTTP method inside the square brackets.
Before we get on further, let us discuss what a payload is. A payload is an HTTP transaction message including its discussion and any additional assets such as entity-body validation schema.

There are two payloads inside an Action:

Request: It is a payload containing one specific HTTP Request, with Headers and an optional body.
Response: It is a payload containing one HTTP Response.

A payload may have an identifier-a string for a request payload or an HTTP status code for a response payload.

+ Request

    + Headers

            Accept: application/vnd.api+json

            Authorization: JWT <Auth Key>

+ Response 200 (application/vnd.api+json)

Types of HTTP methods for Actions:

GET – In this action, we simply send the header data like Accept and Authorization and no body. Along with it we can send some GET parameters like page[size]. There are two cases for GET: List and Detail. For example, if we consider users, a GET for List helps us retrieve information about all users in the response, while Details, helps us retrieve information about a particular user.

The API Blueprint examples implementation of both GET list and detail request and response are as follows.

### List All Users [GET]
Get a list of Users.

+ Request

    + Headers

            Accept: application/vnd.api+json

            Authorization: JWT <Auth Key>

+ Response 200 (application/vnd.api+json)

        {
            "meta": {
                "count": 2
            },
            "data": [
                {
                    "attributes": {
                        "is-admin": true,
                        "last-name": null,
                        "instagram-url": null,

### Get Details [GET]
Get a single user.

+ Request

    + Headers

            Accept: application/vnd.api+json

            Authorization: JWT <Auth Key>

+ Response 200 (application/vnd.api+json)

        {
            "data": {
                "attributes": {
                    "is-admin": false,
                    "last-name": "Doe",
                    "instagram-url": "http://instagram.com/instagram",

POST – In this action, apart from the header information, we also need to send a data. The data must be correct with jsonapi specifications. A POST body data for an users API would look something like this:

### Create User [POST]
Create a new user using an email, password and an optional name.

+ Request (application/vnd.api+json)

    + Headers

            Authorization: JWT <Auth Key>

    + Body

            {
              "data":
              {
                "attributes":
                {
                  "email": "example@example.com",
                  "password": "password",

A POST request with this data, would create a new entry in the table and then return in jsonapi format the particular entry that was made into the table along with the id assigned to this new entry.

PATCH – In this action, we change or update an already existing entry in the database. So It has a header data like all other requests and a body data which is almost similar to POST except that it also needs to mention the id of the entry that we are trying to modify.

### Update User [PATCH]
+ `id` (integer) - ID of the record to update **(required)**

Update a single user by setting the email, email and/or name.

Authorized user should be same as user in request body or must be admin.

+ Request (application/vnd.api+json)

    + Headers

            Authorization: JWT <Auth Key>

    + Body

            {
              "data": {
                "attributes": {
                  "password": "password1",
                  "avatar_url": "http://example1.com/example1.png",
                  "first-name": "Jane",
                  "last-name": "Dough",
                  "details": "example1",
                  "contact": "example1",
                  "facebook-url": "http://facebook.com/facebook1",
                  "twitter-url": "http://twitter.com/twitter1",
                  "instagram-url": "http://instagram.com/instagram1",
                  "google-plus-url": "http://plus.google.com/plus.google1",
                  "thumbnail-image-url": "http://example1.com/example1.png",
                  "small-image-url": "http://example1.com/example1.png",
                  "icon-image-url": "http://example1.com/example1.png"
                },
                "type": "user",
                "id": "2"
              }
            }

Just like in POST, after we have updated our entry, we get back as response the new updated entry in the database.

DELETE – In this action, we delete an entry from the database. The entry in our case is soft deleted by default. Which means that instead of deleting it from the database, we set the deleted_at column with the time of deletion. For deleting we just need to send header data and send a DELETE request to the proper endpoint. If deleted successfully, we get a response as “Object successfully deleted”.

### Delete User [DELETE]
Delete a single user.

+ Request

    + Headers

            Accept: application/vnd.api+json

            Authorization: JWT <Auth Key>

+ Response 200 (application/vnd.api+json)

        {
          "meta": {
            "message": "Object successfully deleted"
          },
          "jsonapi": {
            "version": "1.0"
          }
        }

How to check after manually entering all these? We can use the apiary website to render it, or simply use different renderer to do it. How? Checkout for my next blog on apiary and aglio.

Learn more about api blueprint here: https://apiblueprint.org/

Open Event Server: Working with Migration Files

Post author:magdalenesuo
Post published:July 7, 2017
Post category:Event Event Management FOSSASIA GSoC Open Event
Post comments:0 Comments

FOSSASIA‘s Open Event Server uses alembic migration files to handle all database operations and updations. From creating tables to updating tables and database, all works with help of the migration files.
However, many a times we tend to miss out that automatically generated migration files mainly drops and adds columns rather than just changing them. One example of this would be:

def upgrade():
    ### commands auto generated by Alembic - please adjust! ###
    op.add_column('session', sa.Column('submission_date', sa.DateTime(), nullable=True))
    op.drop_column('session', 'date_of_submission')

Here, the idea was to change the has_session_speakers(string) to is_session_speakers_enabled (boolean), which resulted in the whole dropping of the column and creation of a new boolean column. We realize that, on doing so we have the whole data under has_session_speakers lost.

How to solve that? Here are two ways we can follow up:

op.alter_column:
———————————-

When update is as simple as changing the column names, then we can use this. As discussed above, usually if we migrate directly after changing a column in our model, then the automatic migration created would drop the old column and create a new column with the changes. But on doing this in the production will cause huge loss of data which we don’t want. Suppose we want to just change the name of the column of start_time to starts_at. We don’t want the entire column to be dropped. So an alternative to this is using op.alter_column. The two main necessary parameters of the op.alter_column is the table name and the column which you are willing to alter. The other parameters include the new changes. Some of the commonly used parameters are:

nullable – Optional: specify True or False to alter the column’s nullability.
new_column_name – Optional; specify a string name here to indicate the new name within a column rename operation.
type_ – Optional: a TypeEngine type object to specify a change to the column’s type. For SQLAlchemy types that also indicate a constraint (i.e. Boolean, Enum), the constraint is also generated.
autoincrement – Optional: set the AUTO_INCREMENT flag of the column; currently understood by the MySQL dialect.
existing_type– Optional: a TypeEngine type object to specify the previous type. This is required for all column alter operations that don’t otherwise specify a new type, as well as for when nullability is being changed on a column.

So, for example, if you want to change a column name from “start_time” to “starts_at” in events table you would write:
op.alter_column(‘events’, ‘start_time’, new_column_name=’starts_at’)

def upgrade():
    ### commands auto generated by Alembic - please adjust! ###
    op.alter_column('sessions_version', 'end_time', new_column_name='ends_at')
    op.alter_column('sessions_version', 'start_time', new_column_name='starts_at')
    op.alter_column('events_version', 'end_time', new_column_name='ends_at')
    op.alter_column('events_version', 'start_time', new_column_name='starts_at')

Here, session_version and events_version are the tables name altering columns start_time to starts_at and end_time to ends_at with the op_alter_column parameter new_column_name.

op.execute:
——————–

Now with alter_column, most of the alteration in the column name or constraints or types is achievable. But there can be a separate scenario for changing the column properties. Suppose I change a table with column “aspect_ratio” which was a string column and had values “on” and “off” and want to convert the type to Boolean – True/False. Just changing the column type using alte_column() function won’t work since we need to also modify the whole data. So, sometimes we need to execute raw SQL commands. To do that, we can use the op.execute() function.
The way it is done:

def upgrade():
    ### commands auto generated by Alembic - please adjust! ###
    op.execute("ALTER TABLE image_sizes ALTER full_aspect TYPE boolean USING CASE 
            full_aspect WHEN 'on' THEN TRUE ELSE FALSE END", execution_options=None)

    op.execute("ALTER TABLE image_sizes ALTER icon_aspect TYPE boolean USING CASE 
            icon_aspect WHEN 'on' THEN TRUE ELSE FALSE END", execution_options=None)

    op.execute("ALTER TABLE image_sizes ALTER thumbnail_aspect TYPE boolean USING CASE 
            thumbnail_aspect WHEN 'on' THEN TRUE ELSE FALSE END"execution_options=None)

For a little more advanced use of op.execute() command will be:

op.alter_column('events', 'type', new_column_name='event_type_id')
    op.alter_column('events_version', 'type', new_column_name='event_type_id')
    op.execute('INSERT INTO event_types(name, slug) SELECT DISTINCT event_type_id, 
                lower(replace(regexp_replace(event_type_id, \'& |,\', \'\', \'g\'),
                \' \', \'-\')) FROM events where not exists (SELECT 1 FROM event_types 
                where event_types.name=events.event_type_id) and event_type_id is not
                null;')
    op.execute('UPDATE events SET event_type_id = (SELECT id FROM event_types WHERE 
                event_types.name=events.event_type_id)')
    op.execute('ALTER TABLE events ALTER COLUMN event_type_id TYPE integer USING 
                event_type_id::integer')

In this example:

op.alter_column() renames the column type to event_type_id of events table

op.execute() does the following:

Inserts into column name of event_types table the value of event_type_idN (which previously contained the name of the event_type) from events table, and
Inserts into slug column of event_types table the value of event_type_id where all letters are changed to lowercase; “& ” and “,” to “”; and spaces to “-”.
1. Checks whether a type with that name already exists so as to disallow any duplicate entries in the event_types table.
2. Checks whether the event_type_id is null because name of event_types table cannot be null.

You can learn more on Alembic migrations here: http://alembic.zzzcomputing.com/en/latest/ops.html

Open Event Server: Dealing with environment variables

Post author:magdalenesuo
Post published:July 6, 2017
Post category:Event Event Management FOSSASIA GSoC Open Event
Post comments:0 Comments

FOSSASIA‘s Open Event Server uses some environment variables to maintain different configuration for different deployment. However one con of it was, for local development, developers have to run many export commands.

It all started with the “It is not convenient, we humans are lazy.” phrase. A recent suggestion was to read this environment variables with files rather than running commands every time.
This few example environment variable commands were:

export DATABASE_URL=postgresql://open_event_user:test@127.0.0.1:5432/test

and

export INTEGRATE_SOCKETIO=false

Open Event Server now uses .env file and envparse to handle this.

The changes were:

We store the variables in .env file, let it get ignored by git, and
Use envparse to parse the file.

What is envpase:
envparse is a simple utility to parse environment variables. It aims to eliminate duplicated, often inconsistent parsing code and instead provide a single, easy-to-use wrapper.

So instead of reading the string from the command line everytime:

export DATABASE_URL=postgresql://open_event_user:password@127.0.0.1:5432/open_event_db

We read it from the file:

To read from command line we had:

SQLALCHEMY_DATABASE_URI = os.environ.get('DATABASE_URL', None)

Which now was parsed using envparse as:

SQLALCHEMY_DATABASE_URI =  env('DATABASE_URL', default=None)

In the above example, DATABASE_URL is a string and we store this string in SQLALCHEMY, setting the default by the default parameter(None in this case).

The type of environment variable is originally string, we can simply use env to parse these strings. For simplicity and understanding we can also use env.str. For different types, we can simply type cast them like env.bool (for boolean), env.int (for integer) etc. One good advantage of this envparse feature is the direct use of these type cast rather than checking for strings (i.e. reading the string from command line).

For example, earlier we had something like this:

socketio_integration = os.environ.get('INTEGRATE_SOCKETIO’)
if socketio_integration == true’:
INTEGRATE_SOCKETIO =True

Using envparse, we can have something like this:

INTEGRATE_SOCKETIO = env.bool('INTEGRATE_SOCKETIO’, default=False)

This helps in parsing as boolean instead of string. So, if we have INTEGRATE_SOCKETIO=true in .env file, we can write INTEGRATE_SOCKETIO = env.bool(‘INTEGRATE_SOCKETIO’, default=False). This automatically converts the true to boolean True in python.

In order to use envparse we install envparse with:

$ pip install envparse

In the case of Open Event Server project, you only have to install the requirements.txt and copy the .env.example already mentioned in installation doc.

This is how we finally remove the headache of writing big exports again and again and fairly helps beginners to start off with Open Event Server project much more easily and quickly.

Sorting language-translation in Open Event Server project using Jinja 2 dictsort.

Post author:magdalenesuo
Post published:June 5, 2017
Post category:Event Event Management FOSSASIA GSoC Open Event
Post comments:0 Comments

Working on the Open Event Server project an issue about arranging language-translation listing in alphabetical order came up. To solve this issue of language listing arrangement i.e. #2817, I found the ‘d0_dictsort’ function in jinja2 to sort dictionaries. It is a defined in jinja2.filters. Python dicts are unsorted and in our web application we at times may want to order them by either their key or value. So this function comes handy.

This is what the function looks like:

do_dictsort(value, case_sensitive=False, by='key')

We can write them in three ways as:

{% for record in my_dictionary|dictsort %}
    case insensitive and sort the dict by key

{% for record in my_dictionary|dicsort(true) %}
    case sensitive and sort the dict by key

{% for record in my_dictionary|dictsort(false, 'value') %}
    sort the dict by value, normally sorted and case insensitive

The first way is easily understood that dict has been sorted by key not taking case into consideration. It is just in the same way written as dictsort(false).
Second way is basically the first being case sensitive. dictsort(true) here tells us that case is sensitive.
Third way is dictsort(false,’value’). The first parameter defines that case insensitive while second parameter defines that it is sorted by ‘value’.

The issues was to sort translation selector for the page in alphabetical order. The languages were stored in a dictionary which to change in order, I found this function very easy and useful.

Basically what we had was:

This is how the function was used in the code for the sort. Like this:

<ul class="dropdown-menu lang-list">
   {% for code in all-languages|dictsort(false,'value') %}
       <li><a  href="#" style="#969191" class="translate" id="{{ code[0] }}">{{  all_languages[code[0]] }}<>a><li>
    {% endfor %}
<ul>

Here: {{ all_languages }} is the list which contained the languages like French, English, etc., which could be accessed with its global language code. code here(index for all_languages) is a tuple of {‘global_language_code’,’language’} (An example would be (‘fr’,’French’), so code[0] gave me the language_code.

Finally, the result:

This is one of the simple ways to sort your dictionaries.

Open Event Server: No (no-wrap) Ellipsis using jquery!

Post author:magdalenesuo
Post published:June 5, 2017
Post category:Event Event Management FOSSASIA GSoC Open Event
Post comments:0 Comments

Yes, the title says it all i.e., Enabling multiple line ellipsis. This was used to solve an issue to keep Session abstract view within 200 characters (#3059) on FOSSASIA‘s Open Event Server project.

There is this one way to ellipsis a paragraph in html-css and that is by using the text-overflow property:

.div_class{
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}’’

But the downside of this is the one line ellipis. Eg: My name is Medozonuo. I am…..

And here you might pretty much want to ellipsis after a few characters in multiple lines, given that your div space is small and you do want to wrap your paragraph. Or maybe not.

So jquery to the rescue.

There are two ways you can easily do this multiple line ellipsis:

1) Height-Ellipsis (Using the do-while loop):

//script:
if ($('.div_class').height() > 100) {
    var words = $('.div_class').html().split(/\s+/);
    words.push('...');

    do {
        words.splice(-2, 1);
        $('.div_class').html( words.join(' ') );
    } while($('.div_class').height() > 100);
}

Here, you check for the div content’s height and split the paragraph after that certain height and add a “…”, do- while making sure that the paragraphs are in multiple lines and not in one single line. But checkout for that infinite loop.

2) Length-Ellipsis (Using substring function):

//script:
$.each($('.div_class'), function() {
        if ($(this).html().length > 100) {
               var cropped_words = $(this).html();
               cropped_words = cropped_words.substring(0, 200) + "...";
               $(this).html(cropped_words);
        }
 });

Here, you check for the length/characters rather than the height, take in the substring of the content starting from 0-th character to the 200-th character and then add in extra “…”.

This is exactly how I used it in the code.

$.each($('.short_abstract',function() {
   if ($(this).html().length > 200) {
       var  words = $(this).html();
       words = words.substring(0,200 + "...";
       $(this).html(words);
    }
});

So ellipsing paragraphs over heights and lengths can be done using jQuery likewise.