Author: schedutron

This author has written 15 articles
Read more about the article Implementing Scheduler Actions on Open Event Frontend

Implementing Scheduler Actions on Open Event Frontend

After the functionality to display scheduled sessions was added to Open Event Frontend, the read-only implementation of the scheduler had been completed. What was remaining now in the scheduler were the write actions, i.e., the sessions’ scheduling which event organizers do by deciding its timings, duration and venue. First of all, these actions required the editable flag to be true for the fullcalendar plugin. This allowed the sessions displayed to be dragged and dropped. Once this was enabled, the next task was to embed data in each of the unscheduled sessions so that when they get dropped on the fullcalendar space, they get recognized by the calendar, which can place it at the appropriate location. For this functionality, they had to be jQuery UI draggables and contain an “event” data within them. This was accomplished by the following code: this.$().draggable({ zIndex : 999, revert : true, // will cause the event to go back to its revertDuration : 0 // original position after the drag }); this.$().data('event', { title : this.$().text().replace(/\s\s+/g, ' '), // use the element's text as the event title id : this.$().attr('id'), serverId : this.get('session.id'), stick : true, // maintain when user navigates (see docs on the renderEvent method) color : this.get('session.track.color') }); Here, “this” refers to each unscheduled session. Note that the session color is fetched via the corresponding session track. Once the unscheduled sessions contain enough relevant data and are of the right type (i.e, jQuery UI draggable type), they’re ready to be dropped on the fullcalendar space. Now, when an unscheduled session is dropped on the fullcalendar space, fullcalendar’s eventReceive callback is triggered after its drop callback. In this callback, the code removes the session data from the unscheduled sessions’ list, so it disappears from there and gets stuck to the fullcalendar space. Then the code in the drop callback makes a PATCH request to Open Event Server with the relevant data, i.e, start and end times as well as microlocation. This updates the corresponding session on the server. Similarly, another callback is generated when an event is resized, which means when its duration is changed. This again sends a corresponding session PATCH request to the server. Furthermore, the functionality to pop a scheduled event out of the calendar and add it back to the unscheduled sessions’ list is also implemented, just like in Eventyay version 1. For this, a cross button is implemented, which is embedded in each scheduled session. Clicking this pops the session out of the calendar and adds it back to the unscheduled sessions list. Again, a corresponding PATCH request is sent to the server. After getting the response of such requests, a notification is displayed on the screen, which informs the users whether the action was successful or not. The main PATCH functionality is in a separate function which is called by different callbacks accordingly, so code reusability is increased: updateSession(start, end, microlocationId, sessionId) { let payload = { data: { attributes: { 'starts-at' : start ? start.toISOString() :…

Continue ReadingImplementing Scheduler Actions on Open Event Frontend
Read more about the article Implementing Scheduled Sessions in Open Event Scheduler

Implementing Scheduled Sessions in Open Event Scheduler

Until recently, the Open Event Frontend version 2 didn’t have the functionality to display the already scheduled sessions of an event on the sessions scheduler. Displaying the already scheduled sessions is important so that the event organizer can always use the sessions scheduler as a draft and not worry about losing progress or data about scheduled sessions’ timings. Therefore, just like a list of unscheduled sessions was implemented for the scheduler, the provision for displaying scheduled sessions also had to be implemented. The first step towards implementing this was to fetch the scheduled sessions’ details from Open Event Server. To perform this fetch, an appropriate filter was required. This filter should ideally ask the server to send only those sessions that are “scheduled”. Thus, scheduled sessions need to be defined as sessions which have a non-null value of its starts-at and ends-at fields. Also, few more details are required to be fetched for a clean display of scheduled sessions. First, the sessions’ speaker details should be included so that the speakers’ names can be displayed alongside the sessions. Also, the microlocations’ details need to be included so that each session is displayed according to its microlocation. For example, if a session is to be delivered in a place named ‘Lecture Hall A’, it should appear under the ‘Lecture Hall A’ microlocation column. Therefore, the filter goes as follows: let scheduledFilterOptions = [ { and: [ { name : 'starts-at', op : 'ne', val : null }, { name : 'ends-at', op : 'ne', val : null } ] } ];   After fetching the scheduled sessions’ details, they need to be delivered to the fulllcalendar code for displaying on the session scheduler. For that, the sessions need to be converted in a format which can be parsed by the fullcalendar add-on of emberJS. For example, fullcalendar calls microlocations as ‘resources’. Here is the format which fullcalendar understands: { title : `${session.title} | ${speakerNames.join(', ')}`, start : session.startsAt.format('YYYY-MM-DDTHH:mm:SS'), end : session.endsAt.format('YYYY-MM-DDTHH:mm:SS'), resourceId : session.microlocation.get('id'), color : session.track.get('color'), serverId : session.get('id') // id of the session on BE }   Once the sessions are in the appropriate format, their data is sent to the fullcalendar template, which renders them on the screen: This completes the implementation of displaying the scheduled sessions of an event on the Open Event Scheduler. Resources Fullcalendar Events and Scheduling Tutorial by Drifting Ruby Blog - How to Implement jQuery Fullcalendar in Laravel - Krissanawat​ Kaewsanmuang

Continue ReadingImplementing Scheduled Sessions in Open Event Scheduler
Read more about the article Implementing Unscheduled Sessions List for Event Scheduler

Implementing Unscheduled Sessions List for Event Scheduler

Until recently, Open Event Server didn’t allow the storage of unscheduled sessions. However, having the provision of unscheduled sessions was necessary so that event organizers can easily schedule the unscheduled sessions and keep track of them. Also, it allows them to remove scheduled sessions from the scheduler and place them in the unscheduled sessions list, so that they can be scheduled later. Also, since the unscheduled sessions list was also present in Eventyay version 1, it was decided to have the same in version 2. The first step was to enable the storage of unscheduled sessions on the server. For this, the starts-at and ends-at fields of the session model were modified to be non-required (earlier they were mandatory). Once this change was done, the next step was to fetch the list of unscheduled sessions on the frontend, from the server. Unscheduled sessions were the ones which had the starts-at and ends-at fields as null. Also, the speakers’ details needed to be fetched so that their names can be mentioned along with sessions’ titles, in accordance with Eventyay version 1. Thus, the following were the filter options for the unscheduled sessions’ fetching: let unscheduledFilterOptions = [ { and: [ { name : 'starts-at', op : 'eq', val : null }, { name : 'ends-at', op : 'eq', val : null } ] } ]; let unscheduledSessions = await eventDetails.query('sessions', { include : 'speakers,track', filter : unscheduledFilterOptions });   This gave us the list of unscheduled sessions on the frontend appropriately. After this, the next step was to display this list to the event organizer. For this, the scheduler’s Handlebars template file was modified appropriately. The colors and sizes were chosen so that the list looks similar to the one in Eventyay version 1. Also, the Ember add-on named ember-drag-drop was used to make these unscheduled session components draggable, so that they can be ultimately scheduled on the scheduler. After installing this package and making the necessary changes to the project’s package.json file, the component file for unscheduled sessions was modified accordingly to adapt for the draggable components’ UI. This was the final step and completed the implementation of listing unscheduled sessions. Resources Ember Drag-Drop Addon Documentation Stackoverflow answer on Dragging external events to FullCalendar from slick slider, by Freeman Lambda

Continue ReadingImplementing Unscheduled Sessions List for Event Scheduler
Read more about the article Implementing User Email Verification in Open Event Frontend

Implementing User Email Verification in Open Event Frontend

Open Event Server provides the functionality of user email verification after a user registers, but it was not implemented on Open Event Frontend until recently. For users, this meant they were still not able to verify themselves, even after receiving confirmation links in their inboxes, which were sent by the server. Thus, implementing it on frontend was crucial for a complete user registration workflow. Since the server had already exposed an endpoint to perform the registration, all that was required on the frontend to be done was to make a call to this endpoint with the necessary data. The entire process can be summarized as follows: The recently registered user clicks on the verification link she receives on her email The above step opens the link, which is of the format http://fossasia.github.io/open-event-frontend/verify?token= As soon as the frontend server receives this request, it extracts the token from the URL query parameter The token is now sent to the backend server as a patch request The response of the above request confirms whether the user verification is successful or not, and an appropriate message is displayed In the frontend code, the above algorithm is spread across 3 files: the router, verify route and verify controller. A new route named /verify was implemented for the user verification, and was registered in the project’s main router.js file. After that, in the verify route, the beforeModel() method is used to trigger the above algorithm before the page is loaded: // in app/routes/verify.js beforeModel(transition) { this.controllerFor('verify').verify(transition.queryParams.token); } The main algorithm above is implemented in the verify controller: // in app/controllers/verify.js ... queryParams : ['token'], token : null, success : false, error : null, verify(tokenVal) { let payload = { data: { token: tokenVal } }; return this.get('loader') .post('auth/verify-email', payload) .then(() => { this.set('success', true); }) .catch(reason => { this.set('error', reason); this.set('success', false); }); } });   A template for displaying the success or failure messages to the user was also created. It uses the value of the success boolean set above to decide the message to be displayed to the user. The user registration workflow is now complete and the user sees the following message after clicking on the verification link she receives: Resources Blog - Setting Up Email Verification in FeatherJS, by Imre Gelens EmberJS PATCH Request Discussion

Continue ReadingImplementing User Email Verification in Open Event Frontend
Read more about the article Upgrading Open Event to Use Sendgrid API v3

Upgrading Open Event to Use Sendgrid API v3

Sendgrid recently upgraded their web API to send emails, and support for previous versions was deprecated. As a result, Open Event Server’s mail sending tasks were rendered unsuccessful, because the requests they were sending to Sendgrid were not being processed. On top of that, it was also found out later that the existing Sendgrid API key on the development server was expired. This had to be fixed at the earliest because emails are a core part of Open Event functionality. The existing way for emails to be sent via Sendgrid used to hit the endpoint “https://api.sendgrid.com/api/mail.send.json” to send emails. Also, the payload structure was as follows: payload = { 'to': to, 'from': email_from, 'subject': subject, 'html': html } Also, a header  "Authorization": "Bearer " accompanied the above payload. However, Sendgrid changed the payload structure to be of the following format: { "personalizations": [ {"to": [ {"email": "example@example.com"} ] } ], "from": { "email": "example@example.com" }, "subject": "Hello, World!", "content": [ { "type": "text/plain", "value": "Heya!" } ] } Furthermore, the endpoint was changed to be “https://api.sendgrid.com/v3/mail/send”. To incorporate all these changes with the minimum number of modified lines in the codebase, it was required for that the structure change itself happens at a fairly low level. This was because there are lots of features in the server that perform a wide variety of email actions. Thus, it was clear that changing all of them will not be the most efficient thing to do. So the perfect place to implement the API changes was the function send_email() in mail.py, because all other higher-level email functions are built on top of this function. But this was not the only change, because this function itself used another function, called send_email_task() in tasks.py, specifically for sending email via Sendgrid. So, in conclusion, the header modifications were made in send_email() and payload structure as well as endpoint modifications were made within send_email_task(). This brought the server codebase back on track to send emails successfully. Finally, the key for development server was also renewed and added to its settings in the Heroku Postgres database. Screenshots: Resources Implement Email in Open Event Server SendGrid API v3 docs

Continue ReadingUpgrading Open Event to Use Sendgrid API v3
Read more about the article Implementing Checkout Times for Attendees on Open Event Server

Implementing Checkout Times for Attendees on Open Event Server

As of this writing, Open Event Server did not have the functionality to add, manipulate and delete checkout times of attendees. Event organizers should have access to log and update attendee checkout times. So it was decided to implement this functionality in the server. This boiled down to having an additional attribute checkout_times in the ticket holder model of the server. So the first step was to add a string column named checkout_times in the ticket holder database model, since this was going to be a place for comma-separated values (CSV) of attendee checkout times. An additional boolean attribute named is_checked_out was also added to convey whether an attendee has checked out or not. After the addition of these attributes in the model, we saved the file and performed the required database migration: To create the migration file for the above changes: $ python manage.py db migrate To upgrade the database instance: $ python manage.py db upgrade Once the migration was done, the API schema file was modified accordingly: class AttendeeSchemaPublic(SoftDeletionSchema): """ Api schema for Ticket Holder Model """ … checkout_times = fields.Str(allow_none=True) # ← is_checked_out = fields.Boolean() # ← … After the schema change, the attendees API file had to have code to incorporate these new fields. The way it works is that when we receive an update request on the server, we add the current time in the checkout times CSV to indicate a checkout time, so the checkout times field is essentially read-only: from datetime import datetime ... class AttendeeDetail(ResourceDetail): def before_update_object(self, obj, data, kwargs): … if 'is_checked_out' in data and data['is_checked_out']: ... else: if obj.checkout_times and data['checkout_times'] not in \ obj.checkout_times.split(","): data['checkout_times'] = '{},{},{}'.format( obj.checkout_times, data['checkout_times'], datetime.utcnow())   This completes the implementation of checkout times, so now organizers can process attendee checkouts on the server with ease. Resources SQLAlchemy Docs: https://docs.sqlalchemy.org/en/latest/ Alembic Docs: http://alembic.zzzcomputing.com/en/latest/

Continue ReadingImplementing Checkout Times for Attendees on Open Event Server
Read more about the article Adding Tickets Relationship with Discount Codes in Open Event Server

Adding Tickets Relationship with Discount Codes in Open Event Server

Recently (as of this writing), it was discovered that the relationship between discount codes and tickets was not implemented yet in Open Event Server. It turns out that the server has two types of discount codes - discount codes for entire events and discount codes for individual tickets of a specific event. More information on how discount code themselves are implemented in the server can be found in this blog post from 2017 - Discount Codes in Open Event Server. So, for implementing the relationship of discount codes with tickets, it was decided to be present only for discount codes that have the DiscountCodeSchemaTicket schema, since those are the discount codes that are used for individual tickets. As a first step, the `tickets` attribute of the discount code model was removed, as it was redundant. The already implemented used_for attribute did the same job, and with better validation. At the same time, discount code was added as an attribute. In the ticket model file: discount_code_id = db.Column(db.Integer, db.ForeignKey('discount_codes.id', ondelete='CASCADE')) discount_code = db.relationship('DiscountCode', backref="tickets") Also, in the __init__ constructor: def __init__(self, ..., discount_code_id=None, ...): ... ... self.discount_code_id = discount_code_id After that, we added a discount_code_id field in the ticket schema file: discount_code_id = fields.Integer(allow_none=True) In this file, we also removed the redundant tickets field. Now, we migrated the Open Event Server database via the following commands: $ python manage.py db migrate then $ python manage.py db upgrade Next, in the discount code schema file, we added the tickets relationship. Note that this is a one-to-many relationship. One discount code (for tickets) can be mapped to many tickets. Here is the code for that relationship, in the discount code schema file, under the DiscountCodeSchemaTicket class: tickets = Relationship(attribute='tickets', self_view='v1.discount_code_tickets', self_view_kwargs={'id': '<id>'}, related_view='v1.ticket_list', related_view_kwargs={'discount_code_id': '<id>'}, schema='TicketSchemaPublic', many=True, type_='ticket') For this, we, of course, imported the TicketSchemaPublic in this file first. After that, we created a DiscountCodeTicketRelationship class in the discount codes API file: class DiscountCodeTicketRelationship(ResourceRelationship): """ DiscountCode Ticket Relationship """ decorators = (jwt_required,) methods = ['GET', 'PATCH'] schema = DiscountCodeSchemaTicket data_layer = {'session': db.session, 'model': DiscountCode} The next step was to add the query code to fetch the tickets related to a particular discount code from the database. For this, we added the following snippet to the query() method of the TicketList class in the tickets API file: if view_kwargs.get('discount_code_id'): discount_code = safe_query(self, DiscountCode, 'id', view_kwargs['discount_code_id'], 'discount_code_id') # discount_code - ticket :: one-to-many relationship query_ = self.session.query(Ticket).filter_by(discount_code_id=discount_code.id) The only thing that remains now is adding the API routes for this relationship. We do that in the project’s __init__.py file: api.route(TicketList, 'ticket_list', '/events/<int:event_id>/tickets', '/events/<event_identifier>/tickets', '/ticket-tags/<int:ticket_tag_id>/tickets', '/access-codes/<int:access_code_id>/tickets', '/orders/<order_identifier>/tickets', '/discount-codes/<int:discount_code_id>/tickets') … api.route(DiscountCodeTicketRelationship, 'discount_code_tickets', '/discount-codes/<int:id>/relationships/tickets')   Many routes already map to TicketList, we added one for that comes from discount codes API. Now we can use Postman to check this relationship, and it indeed works as expected, as seen below! Here’s the end: References: Discount Codes in Open Event Server flask-rest-jsonapi Docs SQLAlchemy Docs

Continue ReadingAdding Tickets Relationship with Discount Codes in Open Event Server
Read more about the article Migrating Event Ratings of Open Event with Stored Procedures

Migrating Event Ratings of Open Event with Stored Procedures

Many developers know about procedural languages and have used them in some form or another, but this is really an unpopular tool, despite its power. There are many advantages (and few disadvantages) of these languages, which we will learn about soon. Having a right amount of database-stored procedure code with the help of these languages can really enhance the speed and responsiveness of an application. This article will teach us how procedural languages can be utilized in database management and how they were used recently for a bug fix in Open Event Server. PostgreSQL, like any other powerful, relational database management system (RDBMS), provides the functionality to create and use stored procedures. Essentially, a stored procedure is database logic code which is saved on the database server. This code can be executed directly in the database, and can (and is!) often used to shift business logic from the application layer of a software to the database layer. This simple shift often has many advantages - including faster execution (as code executes at a lower stack level) and better security. When firing database queries from the application layer (i.e., the code that programmers write for storing programmable objects, performing business logic and so on), it often happens that parameters from the programming language itself are passed in to SQL, which then generates a complete SQL query. For example, here’s how a novice query might look like: import psycopg2 conn = psycopg2.connect(dbname="oevent", user="john", password="start") cur = conn.cursor() name = "Sam" cur.execute("SELECT * FROM users WHERE name='%s'" % name) # DANGEROUS! This is an extremely “exposed” code that can be exploited for malicious access, with a technique called SQL injection. This technique essentially “injects” malicious code via these passed parameters, like the variable name mentioned in the above code. With having stored procedures for business logic, there is no room for SQL injection. They solve this problem by writing the query beforehand, and having the parameterized data as a different entity. The pre-processed query within the corresponding stored procedure now looks like SELECT * FROM users WHERE name=?   The database driver sends the name of this stored procedure (or, in standard parameterised queries, just the query text itself) and a list of parameters, as distinct separate entities in the protocol. More details on how stored procedures enhance security can be found here. After learning so much about the advantages of stored procedures (which are enabled by procedural languages), let’s write one! Postgres supports multiple languages for writing stored procedures; here we will use PL/pgSQL, which is the most popular choice for Postgres. This procedural language, inspired (heavily) by Oracle’s PL/SQL language, looks very similar to SQL. To use this procedural language, we have to first install it. In Postgres, procedural languages are installed per-database, not server-wide. We can use the popular Postgres client psql for this purpose, or simply the createlang command on the command line: $ createlang plpgsql yourdb   Now let’s create a simple procedure that prints the corresponding grades…

Continue ReadingMigrating Event Ratings of Open Event with Stored Procedures
Read more about the article Implementing Event Average Rating with SQLAlchemy

Implementing Event Average Rating with SQLAlchemy

While implementing Open Event Server version 2, we decided to have a better way of ranking events by their quality. To define the “quality” of events, the programmers decided to accumulate the feedbacks of specific events and take the average of the ratings involved. Thus, the average rating of an event proves to be a good (enough) measure of its quality. While there are many ways to implement aggregate relationships in an app, here I demonstrate a rather modern methodology which insists on storing such aggregates once they’re computed. Since there is always a space-time/computation tradeoff in software development, this task was no exception. At first, the straightforward idea that came to my mind was to query the Postgres database every time a request for average rating was made. This sounds simple, but with hundreds of events stored on a server, and potentially thousands of users querying for events, this seemed to be a computationally expensive approach. It was costly because the average rating aggregate would be computed for each request, and there could potentially be thousands of such concurrent requests. Therefore, a better idea is to compute the aggregate once, store it in the database (compromising space in the tradeoff mentioned above, but saving a large amount of computation at the same time), and update only when a change is made. In our specific case, the update should happen only when a new rating is added, a rating is deleted or an existing rating is modified. Since the advantages outnumbered the disadvantages, this was the strategy to be implemented. The first step in implementing average rating was to modify the database model of events accordingly. For this, I performed the necessary imports in the events’ database model file: from sqlalchemy_utils import aggregated from app.models.feedback import Feedback Now comes the tricky part. We want an average_rating column in the events table, that contains the mean rating of events. The values in this column should be updated every time a change is made to the feedbacks table. To perform this sort of functionality, the best, raw tool is a Postgres trigger. A trigger should be created that is fired after every update to the feedbacks table, which should update the average rating values in the events table. Here’s how the raw code of such a trigger looks like: create or replace function UpdateAverageRating() returns trigger AS $$ BEGIN UPDATE events SET average_rating=( SELECT avg(rating) FROM feedbacks WHERE event_id=NEW.event_id GROUP BY event_id ) WHERE id = NEW.event_id END $$ language plpgsql Fortunately, the translation of such a trigger into SQLAlchemy-speak is not only easy, but also very elegant. The imports I showed above already set the context for this translation. The event model class looks like the following: class Event(db.Model): """Event object table""" __tablename__ = 'events' __versioned__ = { 'exclude': ['schedule_published_on', 'created_at'] } id = db.Column(db.Integer, primary_key=True) identifier = db.Column(db.String) name = db.Column(db.String, nullable=False) external_event_url = db.Column(db.String) … … … The list of attributes continues, and to the end of this list,…

Continue ReadingImplementing Event Average Rating with SQLAlchemy
Read more about the article Enforcing Constraints Throughout a Flask Back-End

Enforcing Constraints Throughout a Flask Back-End

Recently it was discovered that Open Event Server does not validate attendees’ tickets. Specifically, it was possible to create an arbitrary number of attendees who’d be attending an event on the same ticket! To fix this, a constraint had to be set up across different layers of Open Event Server, which is based on Flask and Postgres. This post will demonstrate how the constraint was added in the server, and these steps should apply in general to any Flask-based server with a relational back-end. First of all, the immediate idea that comes after investigating such an issue, is to add a UNIQUE constraint to the database. For this specific case, the problem was in ticket_holders table of the Open Event database. There was originally no check imposed on the ticket_id and event_id columns. As can be seen in the ticket_holders schema (using the \d+ ticket_holders command), there is no mention of uniqueness on either column. The initial guess was that the combination of ticket_id and event_id should be unique throughout the table to avoid multiple holders attending on the same ticket. However,imposing uniqueness on just the ticket_id column would’ve also worked. So, to be on the safer side, I moved ahead by adding uniqueness on both the columns. To fix this, we need to make changes to the ticket_holder model. So, in the ticket_holder model file, we add a __table_args__ attribute to the TicketHolder class. This attribute represents the various constraints imposed on the ticket_holders table: class TicketHolder(db.Model): __tablename__ = "ticket_holders" __table_args__ = ( db.UniqueConstraint('ticket_id', 'event_id', name='ticket_event'), ) # this is the constraint we add id = db.Column(db.Integer, primary_key=True) firstname = db.Column(db.String, nullable=False) lastname = db.Column(db.String, nullable=False) … … … The TicketHolder class has attributes named ticket_id and event_id, so to add a unique constraint over them, we pass their names to the UniqueConstraint constructor. Also, any suitable name can be given to the constraint, I chose ‘ticket_event’ to simply emphasize the relationship. Now that we’ve edited the database model file, we have to perform a database migration. Before we command the migration, we have to remove the entries that potentially violate the constraint we just imposed. As a temporary fix, I connected to the database and deleted all non-unique rows via plain SQL. For a more consistent fix, I will implement this simple deletion code in the database migration file, if need be. So, once the non-unique rows are gone, we perform the database migration as follows: $ python manage.py db migrate And then, $ python manage.py db upgrade These commands may be different for different projects, but their purpose is the same - to update the database. The upgrade command generates a migration file which looks as follows: from alembic import op import sqlalchemy as sa import sqlalchemy_utils # revision identifiers, used by Alembic. revision = '9d21de792967' down_revision = '194a5a2a44ef' def upgrade(): op.create_unique_constraint('ticket_event', 'ticket_holders', ['ticket_id', 'event_id']) def downgrade(): op.drop_constraint('ticket_event', 'ticket_holders', type_='unique') We can see that the upgrade() function has the command for adding our constraint. Once the database…

Continue ReadingEnforcing Constraints Throughout a Flask Back-End