Open Event Server – Export Event as an iCalendar File

FOSSASIA‘s Open Event Server is the REST API backend for the event management platform, Open Event. Here, the event organizers can create their events, add tickets for it and manage all aspects from the schedule to the speakers. Also, once he makes his event public, others can view it and buy tickets if interested. To make event promotion easier, we also provide the event organizer to export his event as an iCalendar file. Going by the Wikipedia definition, iCalendar is a computer file format which allows Internet users to send meeting requests and tasks to other Internet users by sharing or sending files in this format through various methods. The files usually have an extension of .ics. With supporting software, such as an email reader or calendar application, recipients of an iCalendar data file can respond to the sender easily or counter propose another meeting date/time. The file format is specified in a proposed internet standard (RFC 5545) for calendar data exchange. Server side - generating the iCal file Here we will be using the icalendar package for Python as the file writer. from icalendar import Calendar, vCalAddress, vText We define a class ICalExporter which has a static method export(event_id). Query the event using the event_id passed and start forming the calendar: event = EventModel.query.get(event_id) cal = Calendar() cal.add('prodid', '-//fossasia//open-event//EN') cal.add('version', '2.0') cal.add('x-wr-calname', event.name) cal.add('x-wr-caldesc', "Schedule for sessions at " + event.name) We query for the accepted sessions of the event and store it in sessions. sessions = Session.query \   .filter_by(event_id=event_id) \   .filter_by(state='accepted') \   .filter(Session.deleted_at.is_(None)) \   .order_by(asc(Session.starts_at)).all() We then iterate through all the sessions in sessions. If it is a valid session, we instantiate an icalendar event and store required details. event_component = icalendar.Event() event_component.add('summary', session.title) event_component.add('uid', str(session.id) + "-" + event.identifier) event_component.add('geo', (event.latitude, event.longitude)) event_component.add('location', session.microlocation.name or '' + " " + event.location_name) event_component.add('dtstart', tz.localize(session.starts_at)) event_component.add('dtend', tz.localize(session.ends_at)) event_component.add('email', event.email) event_component.add('description', session.short_abstract) event_component.add('url', url_for('event_detail.display_event_detail_home',                                  identifier=event.identifier, _external=True)) We then loop through all the speakers in that particular session and add it to the iCal Event object as well. for speaker in session.speakers:   # Ref: http://icalendar.readthedocs.io/en/latest/usage.html#file-structure   # can use speaker.email below but privacy reasons   attendee = vCalAddress('MAILTO:' + event.email if event.email else 'undefined@email.com')   attendee.params['cn'] = vText(speaker.name)   event_component.add('attendee', attendee) This event_component is then added to the cal object that we created in the beginning. cal.add_component(event_component) And finally, the cal.to_ical() is returned. This is the iCalendar file contents. This can be directly written to a file. return cal.to_ical() Obtaining the iCal file: Firstly, we have an API endpoint which starts the task on the server. GET - /v1/events/{event_identifier}/export/ical Here, event_identifier is the unique ID of the event. This endpoint starts a celery task on the server to export the event as an iCal file. It returns the task of the URL to get the status of the export task. A sample response is as follows: {  "task_url": "/v1/tasks/b7ca7088-876e-4c29-a0ee-b8029a64849a" } The user can go to the above returned URL and check the status of his Celery task. If the task completed successfully he will get…

Continue ReadingOpen Event Server – Export Event as an iCalendar File

Open Event Frontend – Implement Access Event API via REST API

FOSSASIA‘s Open Event Frontend uses the Open Event Server as the REST API backend. The user can create an event using the Frontend. He can add sessions, tickets speakers etc. and all this updates the database tables in Open Event Server. The server provides certain endpoints for the user to access and/or update the information. It is important that the user is aware of the expected response from the server for his API request. Let’s see how this is displayed in the frontend. In the event-view page of the frontend, which is accessible to the organizers, there is an Export tab, along with Overview, Tickets, Scheduler, Sessions, Speakers. This tab has an Access Event Information via REST API section which displays the URL to be used by the user and the expected response. It looks as follows : The user can choose between various options which he can include or exclude. The GET URL is modified accordingly and the appropriate response is shown to the user. Example of this - How is this implemented in Code? We maintain two variables baseUrl and displayUrl to display the URL. baseUrl is the URL which is common in all requests, ie, till the include tag. baseUrl: computed('eventId', function() { return `${`${ENV.APP.apiHost}/${ENV.APP.apiNamespace}/events/`}${this.get('eventId')}`; }) displayUrl is the variable which stores the URL being displayed on the webpage. It is initialized to the same as baseUrl. displayUrl: computed('eventId', function() { return `${`${ENV.APP.apiHost}/${ENV.APP.apiNamespace}/events/`}${this.get('eventId')}`; }) To store the value of the toggle switches we use toggleSwitches as follows: toggleSwitches: { sessions       : false, microlocations : false, tracks         : false, speakers       : false, sponsors       : false, tickets        : false } Whenever any of the switches are toggled, an action checkBox is called. This method updates the value of toggleSwitches, calls the method to update the displayUrl and make the corresponding API request to update the displayed response. The code looks like this : makeRequest() { this.set('isLoading', true); this.get('loader')   .load(this.get('displayUrl'), { isExternal: true })   .then(json => {     json = JSON.stringify(json, null, 2);     this.set('json', htmlSafe(syntaxHighlight(json)));   })   .catch(() => {     this.get('notify').error(this.get('l10n').t('Could not fetch from the server'));     this.set('json', 'Could not fetch from the server');   })   .finally(() => {     this.set('isLoading', false);   }); }, buildDisplayUrl() { let newUrl = this.get('baseUrl'); const include = []; for (const key in this.get('toggleSwitches')) {   if (this.get('toggleSwitches').hasOwnProperty(key)) {     this.get('toggleSwitches')[key] && include.push(key);   } } this.set('displayUrl', buildUrl(newUrl, {   include: include.length > 0 ? include : undefined }, true)); }, actions: { checkboxChange(data) {   this.set(`toggleSwitches.${data}`, !this.get(`toggleSwitches.${data}`));   this.buildDisplayUrl();   this.makeRequest(); } } The above code uses some utility methods such as buildUrl and this.get(‘loager’).load(). The complete codebase is available here -> Open Event Frontend Repository. References Open Event Frontend Open Event Server JavaScript Ember

Continue ReadingOpen Event Frontend – Implement Access Event API via REST API

SUSI AI 5 Star Skill Rating System

For making a system more reliable and robust, continuous evaluation is quite important. So is in case of SUSI AI. User feedback is important to improve SUSI skills and create new ones. Previously we had only thumbs up / thumbs down as a feedback method, from the susi chat client. But now a 5 star rating system has been added to the SUSI Skill CMS so that users can rate a skill there. Before the implementation of API  let’s look how data is stored in SUSI AI Susi_server uses DAO in which skill rating is stored as JSONTray. The server side implementation A new java class has been created for the API, FiveStarRateSkillService.java. public class FiveStarRateSkillService extends AbstractAPIHandler implements APIHandler { private static final long serialVersionUID =7947060716231250102L; @Override public BaseUserRole getMinimalBaseUserRole() { return BaseUserRole.USER; } @Override public JSONObject getDefaultPermissions(BaseUserRole baseUserRole) { return null; } @Override public String getAPIPath() { return "/cms/rateSkill.json"; } ... } The getMinimalBaseRole method tells the minimum User role required to access this servlet it can also be ADMIN, USER or ANONYMOUS. In our case it is USER. A user needs to be logged in to rate a skill on a scale of 1-5 stars.  The API runs at “/cms/fiveStarRateSkill.json” endpoint. Next, create serviceImpl method in the above class to handle the request from the client and respond to it. 1. Fetch the required query parameters and store them in variables. They include skill model, group, language, skill name and starts that the user has given in the rating. String skill_name = call.get("skill", null); String skill_stars = call.get("stars", null); 2. Then check if the skill exists. If not them throw an exception. Otherwise, increment the count of the corresponding rating. The rating object has keys as one_star, two_star, three_star, four_star and five_star that has the count of that star rating.        if (skill_stars.equals("1")) { skillName.put("one_star", skillName.getInt("one_star") + 1 + ""); } else if (skill_stars.equals("2")) { skillName.put("two_star", skillName.getInt("two_star") + 1 + ""); } else if (skill_stars.equals("3")) { skillName.put("three_star", skillName.getInt("three_star") + 1 + ""); } else if (skill_stars.equals("4")) { skillName.put("four_star", skillName.getInt("four_star") + 1 + ""); } else if (skill_stars.equals("5")) { skillName.put("five_star", skillName.getInt("five_star") + 1 + ""); } 3. Re-calculate the total rating done on that skill and its average rating and update the object. If the skill has not been already rated then create a new rating object and initialize it with the 0 star counts. public JSONObject createRatingObject(String skill_stars) { JSONObject skillName = new JSONObject(); JSONObject skillStars = new JSONObject(); skillStars.put("one_star", 0); skillStars.put("two_star", 0); skillStars.put("three_star", 0); skillStars.put("four_star", 0); skillStars.put("five_star", 0); skillStars.put("avg_star", 0); skillStars.put("total_star", 0); skillName.put("stars", skillStars); } The complete FiveStarRateSkillService.java is available here : - https://github.com/fossasia/susi_server/blob/development/src/ai/susi/server/api/cms/FiveStarRateSkillService.java Rating a skill Sample endpoint https://api.susi.ai/cms/fiveStarRateSkill.json?model=general&group=Knowledge&language=en&skill=aboutsusi&stars=3&callback=p&_=1526813916145 This gives 3 star rating to the “aboutsusi” skill. Parameters Model Group Language Skill Stars Response { "ratings": { "one_star": 0, "four_star": 0, "five_star": 1, "total_star": 1, "three_star": 0, "avg_star": 5, "two_star": 0 }, "session": {"identity": { "type": "email", "name": "1anuppanwar@gmail.com", "anonymous": false }}, "accepted": true, "message": "Skill ratings updated" } Getting the stats…

Continue ReadingSUSI AI 5 Star Skill Rating System

Adding “All” in Skill Categories

The SUSI SKill CMS has various filters to explore the skills of interest.  For example skill category and skill language. The skills are stored in the susi_skill_data Github repo in the following structure: susi_skill_data/models/<model_name>/<group_name>/<language_name>/<skill_name>.txt NOTE: group and category are same terms and can be used interchangeably So when a category filter is applied the skills from the corresponding directory are returned. susi_skill_data/models/<model_name>/<group_name>/<language_name>/<skill_name>.txt But there’s no directory called “All”,  so how to get skills of all groups? For this, we need to loop through all the directories present in the model. Server side implementation Create a helper function that returns a list of all the folders present in a directory. The function accepts the parent directory name and an empty list. First, fetch all the items (files and folders) present in that directory and store them in an array. Then apply a filter over the array to check if the element is a directory and doesn’t start with a dot(.) i.e., it's not hidden. Add the filtered array to the list. private void listFoldersForFolder(final File folder, ArrayList<String> fileList) { File[] filesInFolder = folder.listFiles(); if (filesInFolder != null) { Arrays.stream(filesInFolder) .filter(fileEntry -> fileEntry.isDirectory() && !fileEntry.getName().startsWith(".")) .forEach(fileEntry -> fileList.add(fileEntry.getName() + "")); } } Fetch the group name form the request and add a check if the CMS is asking for all skill. Otherwise, return the skills of a particular group only. String group_name = call.get("group", "Knowledge"); if (group_name.equals("All")) { // Return the list of all skills } else { // Return the list of a skills in a particular group only } To fetch the list of all skills, call the listFoldersForFolders() function with the model name and an empty list as arguments. The function adds all the directories, present in that model directory, to folderList. File allGroup = new File(String.valueOf(model)); ArrayList<String> folderList = new ArrayList<String>(); listFoldersForFolder(allGroup, folderList); Then loop over all the groups present in the list to get all the skills present in that group. This process is the same as the existing process of getting skills of a particular category. Just keep adding the skill list to a global array. CMS side implementation The list of categories is first fetched from the API and then added to the dropdown menu. Since the API doesn’t return “All” in it, so we need to push it to the list manually. groups.push(<MenuItem value="All" key="All" primaryText="All" />); References StackOverflow : https://stackoverflow.com/questions/5694385/getting-the-filenames-of-all-files-in-a-folder

Continue ReadingAdding “All” in Skill Categories

Adding System Image for Event Categories

The Open Event Server is using the JSON 1.0 Specification and build on top of Flask Rest Json API (for building Rest APIs) and Marshmallow (for Schema). In this blog, we will talk about how to add feature of System Image for Event Categories on Open Event Server. The focus is on Model updation, Schema updation and migrating the Database. Model Updation For adding System Image, we’ll update our Model EventTopic. In this feature, we are providing rights to the Admin to add a system image for each Event Category so that if no image is given by a organizer of event on event creation then it will use the system image of that Event Category as event image by default. Here we are adding a Column named system_image_url which is of type String. This value cannot be nullable and having a default value. Migrating the Database For the migrating the Database we will use simple commands. This command runs migrations. If it cause problems naming Multiple Migration Head, then you need to run This problem is caused when two developers push a migration file without merging two heads to achieve one head. The above command will give us ids of two migration heads. This command is merging two migration heads. This command is upgrading the migrations. Finally, we migrate the Database using above command. Schema Updation For the system image, we’ll update the Schema EventTopicSchema as follows In this feature, to provide system image for each Event Category we’ll add a field named system_image_url in the Schema. Here we are adding a field named system_image_url which is of marshmallow field type URL. This value cannot be none. Validating the Event Image and using System Image by default In this step, we’ll check if a event image is provided by organizer. If that is not provided then we’ll use system image of Event Category as Event Image. Here, we will first take the event topic of event as added by the organizer. Then we will fetch the the database row in Event Topic model which has id == event_topic_id . Then we will return the system image url of that event topic to the event image. So we saw how we could provide a default image for any event. Resources Documentation | Marshmallow : https://marshmallow-jsonapi.readthedocs.io/en/latest/ Documentation | Flask Rest JSONAPI : http://flask-rest-jsonapi.readthedocs.io/en/latest/

Continue ReadingAdding System Image for Event Categories

Adding Event Roles concerning a User on Open Event Server

The Open Event Server enables organizers to manage events from concerts to conferences and meetups. It offers features for events with several tracks and venues. Event managers can create invitation forms for speakers and build schedules in a drag and drop interface. The event information is stored in a database. The system provides API endpoints to fetch the data, and to modify and update it. The Open Event Server is based on JSON 1.0 Specification and hence build on top of Flask Rest Json API (for building Rest APIs) and Marshmallow (for Schema). In this blog, we will talk about how to add different events role concerning a user on Open Event Server. The focus is on its model and Schema updation. Model Updation For the User Table, we’ll update our User Model as follows: Now, let’s try to understand these hybrid properties. In this feature, we are providing Admin the rights to see whether a user is acting as a organizer, co-organizer, track_organizer, moderator, attendee and registrar of any of the event or not. Here, _is_role method is used to check whether an user plays a event role like organizer, co-organizer, track_organizer, moderator, attendee and registrar or not. This is done by querying the record from UserEventsRole model. If the record is present then the returned value is True otherwise False. Schema Updation For the User Model, we’ll update our Schema as follows Now, let’s try to understand this Schema. Since all the properties will return either True or false so these all properties are set to Boolean in Schema. Here dump_only means, we will return this property in the Schema. So, we saw how User Model and Schema is updated to show events role concerning a user on Open Event Server. Resources Documentation | Marshmallow : https://marshmallow-jsonapi.readthedocs.io/en/latest/ Documentation | Flask Rest JSONAPI : http://flask-rest-jsonapi.readthedocs.io/en/latest/

Continue ReadingAdding Event Roles concerning a User on Open Event Server

Adding Defaults Prior to Schema Validation Elegantly

The Open Event Server offers features for events with several tracks and venues. When we were designing the models for the API, we wanted to add default values for some fields in case they aren’t provided by the client. This blog discusses how we have implemented in the project using python decorators that complies to the DRY principle and is easy to test and maintain. Problem Let’s first discuss the problem at hand in detail. We use Marshmallow extensively in our project. Marshmallow is an ORM/ODM/framework-agnostic library for converting complex data types, such as objects, to and from native python data types. We use it for Validating the input data, Deserializing the input data to app-level objects and Serializing app-level objects to primitive Python types. We can define Schema’s very easily using Marshmallow. It also provides an easy way to declare default values to the fields. Below is a sample schema declaration: class SampleSchema(Schema): """ Sample Schema declaration """ class Meta: """ Meta class for the Sample Schema """ type_ = 'sample-schema' id = fields.Str(dump_only=True) field_without_default = fields.Str(required=True) field_with_default = fields.Boolean(required=True, default=False) We have defined an id field for storing the unique ID of the Model. We have also defined two other fields. One of them named as “field_with_default” is a Boolean field and has a default value specified as False. When a client makes a POST/PATCH request to the server, we first validate the input data sent to us by the clients against the defined schema. Marshmallow also supports schema validation but it doesn’t support using default values during deserialization of the input data. It meant that whenever the input data had a missing field, the server would throw a validation error even for a field for which the default values are defined. It was clearly wrong since if the default values are defined, we would want that value to be used for the field. This defeats the entire purpose of declaring default values at the first place. So, we would ideally like the following behaviour from the Server: If the values are defined in the input data, use it during validation. If the value for a required field is not defined but default value has been defined in the Schema, then use that value. If no value has been defined for a required field and it doesn’t have any default value specified, then throw an error. Solution Marshmallow provides decorators like @pre_load and @post_load for adding pre-processing and post-processing methods. We can use them to add a method in each of the Schema classes which takes in the input data and the schema and adds default values to fields before we validate the input. The first approach which we took was to add the following method to each of the schema classes defined in the project. @pre_load def patch_defaults(schema, in_data): data = in_data.get('data') if data is None or data.get('attributes') is None: return in_data attributes = data.get('attributes') for name, field in schema.fields.items(): dasherized_name = dasherize(name) attribute = attributes.get(dasherized_name) if attribute is…

Continue ReadingAdding Defaults Prior to Schema Validation Elegantly

Adding Statistics of Event-Type on Open Event Server

The Open Event Server enables organizers to manage events from concerts to conferences and meet-ups. It offers features for events with several tracks and venues. In this blog, we will talk about how to add statistics of event-type on Open Event Server. The focus is on to get number of events of a specific event type. Number of events of a specific event type Now, let’s try to understand this API. Here, we are using flask Blueprints to add this API to the API index. First of all, we are using the decorator of event_statistics which will append this API route with that of mentioned in the Blueprint event_statistics. We will just allow logged in user to access this API using JWT (JSON Web Token) To return the response having all the event types alongwith number of events of it, requires a lot of queries if tried to fulfilled by SQLALchemy ORM. So instead of using ORM we will query using SQL command so that we query the number of all the events of different event types in just one query, which will eventually reduces the time of server to return the response. In function event_types_count we are using db.engine.execute to run the SQL command of getting the statistics of events respective to event types. The response will include id of event_type, name of event_type and count of events of corresponding event_type. Finally, we jsonify the list having objects of statistics of events respective to event types. Similarly, event topics statistics can be implemented to return the number of events of all the event topics. Resources Documentation | Flask : http://flask.pocoo.org/docs/1.0/ Documentation | Flask-JWT : https://pythonhosted.org/Flask-JWT/

Continue ReadingAdding Statistics of Event-Type on Open Event Server

Ember Data Integration In Badgeyay Frontend

Badgeyay is an open source utility to develop badges for events and tech conferences. Badgeyay project is divided into two components. Frontend part is designed with ember and backend part is designed with Flask and database as PostgreSQL and Firebase as PaaS. After refactoring the backend API for generation of badges, now it is time to consume the API in frontend by ember, and the way to consume the api in ember front--end is with the use of in built ember-data library. Ember data behaves in a way similar to server side ORM’s (Object Relational Mappers). It is a very versatile library and can be equipped with variety of backend services. It can be used with REST as well as sockets and other transfer protocols for communication. For better understanding the working of ember data, let’s see how to use the same to consume the File Upload endpoint in the backend. Procedure Enabling CORS on server, to allow cross-domain requests to the API. from flask_cors import CORS CORS(app, resources={r"*": {"origins": "*"}}) Creating Adapter for the model in frontend. In our case it is csv-file. In the adapter we need to specify the host and the path, because our backend api is not running on the same port. import DS from 'ember-data'; const { RESTAdapter } = DS; export default RESTAdapter.extend({ host : 'http://localhost:5000', pathForType : () => { return 'api/upload/file'; } }); After creating the adapter we need to create the record in the controller of the respective component. The record is like an object of a class, which when pushed to store will make a network request to backend (POST) and fetch the response from the backend. Backend response will provide the id to save in store import Controller from '@ember/controller'; import { inject as service } from '@ember/service'; export default Controller.extend({ routing : service('-routing'), actions : { mutateCSV(csvData) { let csv_ = this.get('store').createRecord('csv-file', { csvFile : csvData, extension : 'csv' }); csv_.save(); }, mutateText(txtData) { console.log(txtData); } } }); Model for the csv-file import DS from 'ember-data'; const { Model, attr } = DS; export default Model.extend({ csvFile : attr('string'), extension : attr('string') }); Next is to create serializers for the model. Serializers gets triggered at two moments, first when the data is sent to the server and second when data is received from the server. Each time an independent function gets executed. As the naming conventions of the functions pretty much explains their role, but for the sake of clarification serialize function gets executed when we send request to the server and normalizeResponse gets executed when we are getting response from the server. import DS from 'ember-data'; const { JSONAPISerializer } = DS; export default JSONAPISerializer.extend({ serialize(snapshot, options) { let json = this._super(...arguments); json.csvFile = { 'csvFile' : json.data.attributes['csv-file'], 'extension' : json.data.attributes.extension }; delete json.data; return json; }, normalizeResponse(store, primaryModelClass, payload, id, requestType) { return payload; } }); After receiving the response a promise is returned by the push method to save the record in the store…

Continue ReadingEmber Data Integration In Badgeyay Frontend