Open Event Server – Export Attendees as CSV File

FOSSASIA‘s Open Event Server is the REST API backend for the event management platform, Open Event. Here, the event organizers can create their events, add tickets for it and manage all aspects from the schedule to the speakers. Also, once he/she makes his event public, others can view it and buy tickets if interested.

The organizer can see all the attendees in a very detailed view in the event management dashboard. He can see the statuses of all the attendees. The possible statuses are completed, placed, pending, expired and canceled, checked in and not checked in. He/she can take actions such as checking in the attendee.

If the organizer wants to download the list of all the attendees as a CSV file, he or she can do it very easily by simply clicking on the Export As and then on CSV.

Let us see how this is done on the server.

Server side – generating the Attendees CSV file

Here we will be using the csv package provided by python for writing the csv file.

import csv
  • We define a method export_attendees_csv which takes the attendees to be exported as a CSV file as the argument.
  • Next, we define the headers of the CSV file. It is the first row of the CSV file.
def export_attendees_csv(attendees):
   headers = ['Order#', 'Order Date', 'Status', 'First Name', 'Last Name', 'Email',
              'Country', 'Payment Type', 'Ticket Name', 'Ticket Price', 'Ticket Type']
  • A list is defined called rows. This contains the rows of the CSV file. As mentioned earlier, headers is the first row.
rows = [headers]
  • We iterate over each attendee in attendees and form a row for that attendee by separating the values of each of the columns by a comma. Here, every row is one attendee.
  • The newly formed row is added to the rows list.
for attendee in attendees:
   column = [str(attendee.order.get_invoice_number()) if attendee.order else '-',
             str(attendee.order.created_at) if attendee.order and attendee.order.created_at else '-',
             str(attendee.order.status) if attendee.order and attendee.order.status else '-',
             str(attendee.firstname) if attendee.firstname else '',
             str(attendee.lastname) if attendee.lastname else '',
             str(attendee.email) if attendee.email else '',
             str(attendee.country) if attendee.country else '',
             str(attendee.order.payment_mode) if attendee.order and attendee.order.payment_mode else '',
             str(attendee.ticket.name) if attendee.ticket and attendee.ticket.name else '',
             str(attendee.ticket.price) if attendee.ticket and attendee.ticket.price else '0',
             str(attendee.ticket.type) if attendee.ticket and attendee.ticket.type else '']

   rows.append(column)
  • rows contains the contents of the CSV file and hence it is returned.
return rows
  • We iterate over each item of rows and write it to the CSV file using the methods provided by the csv package.
writer = csv.writer(temp_file)
from app.api.helpers.csv_jobs_util import export_attendees_csv
content = export_attendees_csv(attendees)
for row in content:
   writer.writerow(row)

Obtaining the Attendees CSV file:

Firstly, we have an API endpoint which starts the task on the server.

GET - /v1/events/{event_identifier}/export/attendees/csv

Here, event_identifier is the unique ID of the event. This endpoint starts a celery task on the server to export the attendees of the event as a CSV file. It returns the URL of the task to get the status of the export task. A sample response is as follows:

{
  "task_url": "/v1/tasks/b7ca7088-876e-4c29-a0ee-b8029a64849a"
}

The user can go to the above-returned URL and check the status of his/her Celery task. If the task completed successfully he/she will get the download URL. The endpoint to check the status of the task is:

and the corresponding response from the server –

{
  "result": {
    "download_url": "/v1/events/1/exports/http://localhost/static/media/exports/1/zip/OGpMM0w2RH/event1.zip"
  },
  "state": "SUCCESS"
}

The file can be downloaded from the above-mentioned URL.

References

Open Event Server – Export Orders as CSV File

FOSSASIA‘s Open Event Server is the REST API backend for the event management platform, Open Event. Here, the event organizers can create their events, add tickets for it and manage all aspects from the schedule to the speakers. Also, once he/she makes his event public, others can view it and buy tickets if interested.

The organizer can see all the orders in a very detailed view in the event management dashboard. He can see the statuses of all the orders. The possible statuses are completed, placed, pending, expired and canceled.

If the organizer wants to download the list of all the orders as a CSV file, he or she can do it very easily by simply clicking on the Export As and then on CSV.

Let us see how this is done on the server.

Server side – generating the Orders CSV file

Here we will be using the csv package provided by python for writing the csv file.

import csv
  • We define a method export_orders_csv which takes the orders to be exported as a CSV file as the argument.
  • Next, we define the headers of the CSV file. It is the first row of the CSV file.
def export_orders_csv(orders):
   headers = ['Order#', 'Order Date', 'Status', 'Payment Type', 'Total Amount', 'Quantity',
              'Discount Code', 'First Name', 'Last Name', 'Email']
  • A list is defined called rows. This contains the rows of the CSV file. As mentioned earlier, headers is the first row.
rows = [headers]
  • We iterate over each order in orders and form a row for that order by separating the values of each of the columns by a comma. Here, every row is one order.
  • The newly formed row is added to the rows list.
for order in orders:
   if order.status != "deleted":
       column = [str(order.get_invoice_number()), str(order.created_at) if order.created_at else '',
                 str(order.status) if order.status else '', str(order.paid_via) if order.paid_via else '',
                 str(order.amount) if order.amount else '', str(order.get_tickets_count()),
                 str(order.discount_code.code) if order.discount_code else '',
                 str(order.user.first_name)
                 if order.user and order.user.first_name else '',
                 str(order.user.last_name)
                 if order.user and order.user.last_name else '',
                 str(order.user.email) if order.user and order.user.email else '']
       rows.append(column)
  • rows contains the contents of the CSV file and hence it is returned.
return rows
  • We iterate over each item of rows and write it to the CSV file using the methods provided by the csv package.
writer = csv.writer(temp_file)
from app.api.helpers.csv_jobs_util import export_orders_csv
content = export_orders_csv(orders)
for row in content:
   writer.writerow(row)

Obtaining the Orders CSV file:

Firstly, we have an API endpoint which starts the task on the server.

GET - /v1/events/{event_identifier}/export/orders/csv

Here, event_identifier is the unique ID of the event. This endpoint starts a celery task on the server to export the orders of the event as a CSV file. It returns the URL of the task to get the status of the export task. A sample response is as follows:

{
  "task_url": "/v1/tasks/b7ca7088-876e-4c29-a0ee-b8029a64849a"
}</span

The user can go to the above-returned URL and check the status of his/her Celery task. If the task completed successfully he/she will get the download URL. The endpoint to check the status of the task is:

and the corresponding response from the server –

{
  "result": {
    "download_url": "/v1/events/1/exports/http://localhost/static/media/exports/1/zip/OGpMM0w2RH/event1.zip"
  },
  "state": "SUCCESS"
}

The file can be downloaded from the aabove-mentionedURL.

References

Open Event Server – Export Event as a Pentabarf XML File

FOSSASIA‘s Open Event Server is the REST API backend for the event management platform, Open Event. Here, the event organizers can create their events, add tickets for it and manage all aspects from the schedule to the speakers. Also, once he makes his event public, others can view it and buy tickets if interested.

To make event promotion easier, we also provide the event organizer to export his event as a Pentabarf XML file. Pentabarf XML is used to store events/conferences in a format which most of the scheduling applications can read and add that particular event/conference to the user’s schedule.

Server side – generating the Pentabarf XML file

Here we will be using the pentabarf package for Python for parsing and creating the file.

from pentabarf.Conference import Conference
from pentabarf.Day import Day
from pentabarf.Event import Event
from pentabarf.Person import Person
from pentabarf.Room import Room
  • We define a class PentabarfExporter which has a static method export(event_id).
  • Query the event using the event_id passed and start forming the event in the required format:
event = EventModel.query.get(event_id)
diff = (event.ends_at - event.starts_at)

conference = Conference(title=event.name, start=event.starts_at, end=event.ends_at,
                       days=diff.days if diff.days > 0 else 1,
                       day_change="00:00", timeslot_duration="00:15",
                       venue=event.location_name)
dates = (db.session.query(cast(Session.starts_at, DATE))
        .filter_by(event_id=event_id)
        .filter_by(state='accepted')
        .filter(Session.deleted_at.is_(None))
        .order_by(asc(Session.starts_at)).distinct().all())
  • We have queried for the dates of the event and saved it in dates.
  • We will now iterate over each date and query the microlocations who have a session on that particular date.
for date in dates:
   date = date[0]
   day = Day(date=date)
   microlocation_ids = list(db.session.query(Session.microlocation_id)
                            .filter(func.date(Session.starts_at) == date)
                            .filter_by(state='accepted')
                            .filter(Session.deleted_at.is_(None))
                            .order_by(asc(Session.microlocation_id)).distinct())
  • For each microlocation thus obtained, we will query for accepted sessions to be held at those microlocations.
  • We will also initialize a Room for each microlocation.
for microlocation_id in microlocation_ids:
   microlocation_id = microlocation_id[0]
   microlocation = Microlocation.query.get(microlocation_id)
   sessions = Session.query.filter_by(microlocation_id=microlocation_id) \
       .filter(func.date(Session.starts_at) == date) \
       .filter_by(state='accepted') \
       .filter(Session.deleted_at.is_(None)) \
       .order_by(asc(Session.starts_at)).all()

   room = Room(name=microlocation.name)
  • We will now iterate over the aabove-obtained sessions and instantiate an Event for each session.
  • Then we will iterate over all the speakers of that session and instantiate a Person for each speaker.
  • Finally, we will add that Event to the Room we created earlier.
for session in sessions:

   session_event = Event(id=session.id,
                         date=session.starts_at,
                         start=session.starts_at,
                         duration=str(session.ends_at - session.starts_at) + "00:00",
                         track=session.track.name,
                         abstract=session.short_abstract,
                         title=session.title,
                         type='Talk',
                         description=session.long_abstract,
                         conf_url=url_for('event_detail.display_event_detail_home',
                                          identifier=event.identifier),
                         full_conf_url=url_for('event_detail.display_event_detail_home',
                                               identifier=event.identifier, _external=True),
                         released="True" if event.schedule_published_on else "False")

   for speaker in session.speakers:
       person = Person(id=speaker.id, name=speaker.name)
       session_event.add_person(person)

   room.add_event(session_event)
  • Then we will add the room to the day and then add each day to the conference.
day.add_room(room)
conference.add_day(day)
  • Finally, we will call the generate method of the conference to generate the XML file. This can be directly written to the file.
return conference.generate("Generated by " + get_settings()['app_name'])

Obtaining the Pentabarf XML file:

Firstly, we have an API endpoint which starts the task on the server.

GET - /v1/events/{event_identifier}/export/pentabarf

Here, event_identifier is the unique ID of the event. This endpoint starts a celery task on the server to export the event as a Pentabarf XML file. It returns the task of the URL to get the status of the export task. A sample response is as follows:

{
  "task_url": "/v1/tasks/b7ca7088-876e-4c29-a0ee-b8029a64849a"
}

The user can go to the above-returned URL and check the status of his Celery task. If the task completed successfully he will get the download URL. The endpoint to check the status of the task is:

and the corresponding response from the server –

{
  "result": {
    "download_url": "/v1/events/1/exports/http://localhost/static/media/exports/1/zip/OGpMM0w2RH/event1.zip"
  },
  "state": "SUCCESS"
}

The file can be downloaded from the above-mentioned URL.

Hence, now the event can be added to any scheduling app which recognizes the Pentabarf XML format.

References

Open Event Server – Export Event as xCalendar File

FOSSASIA‘s Open Event Server is the REST API backend for the event management platform, Open Event. Here, the event organizers can create their events, add tickets for it and manage all aspects from the schedule to the speakers. Also, once he makes his event public, others can view it and buy tickets if interested.

To make event promotion easier, we also provide the event organizer to export his event as an xCalendar file. xCal is an XML representation of the iCalendar standard. xCal is not an alternative nor next generation of iCalendar. xCal represents iCalendar components, properties, and parameters as defined in iCalendar. This format was selected to ease its translation back to the iCalendar format using an XSLT transform.

Server side – generating the xCal file

Here we will be using the xml.etree.ElementTree package for Python for parsing and creating XML data.

from xml.etree.ElementTree import Element, SubElement, tostring
  • We define a class XCalExporter which has a static method export(event_id).
  • Query the event using the event_id passed and start forming the calendar:
event = Event.query.get(event_id)

tz = event.timezone or 'UTC'
tz = pytz.timezone(tz)

i_calendar_node = Element('iCalendar')
i_calendar_node.set('xmlns:xCal', 'urn:ietf:params:xml:ns:xcal')
v_calendar_node = SubElement(i_calendar_node, 'vcalendar')
version_node = SubElement(v_calendar_node, 'version')
version_node.text = '2.0'
prod_id_node = SubElement(v_calendar_node, 'prodid')
prod_id_node.text = '-//fossasia//open-event//EN'
cal_desc_node = SubElement(v_calendar_node, 'x-wr-caldesc')
cal_desc_node.text = "Schedule for sessions at " + event.name
cal_name_node = SubElement(v_calendar_node, 'x-wr-calname')
cal_name_node.text = event.name
  • We query for the accepted sessions of the event and store it in sessions
sessions = Session.query \
   .filter_by(event_id=event_id) \
   .filter_by(state='accepted') \
   .filter(Session.deleted_at.is_(None)) \
   .order_by(asc(Session.starts_at)).all()
  • We then iterate through all the sessions in sessions.
  • If it is a valid session, we instantiate a SubElement and store required details
v_event_node = SubElement(v_calendar_node, 'vevent')

method_node = SubElement(v_event_node, 'method')
method_node.text = 'PUBLISH'

uid_node = SubElement(v_event_node, 'uid')
uid_node.text = str(session.id) + "-" + event.identifier

dtstart_node = SubElement(v_event_node, 'dtstart')
dtstart_node.text = tz.localize(session.starts_at).isoformat()

…. So on
  • We then loop through all the speakers in that particular session and add it to the xCal calendar node object as well.
for speaker in session.speakers:
   attendee_node = SubElement(v_event_node, 'attendee')
   attendee_node.text = speaker.name
  • And finally, the string of the calendar node is returned. This is the xCalendar file contents. This can be directly written to a file.
return tostring(i_calendar_node)

Obtaining the xCal file:

Firstly, we have an API endpoint which starts the task on the server.

GET - /v1/events/{event_identifier}/export/xcal

Here, event_identifier is the unique ID of the event. This endpoint starts a celery task on the server to export the event as an xCal file. It returns the URL of the task to get the status of the export task. A sample response is as follows:

{
  "task_url": "/v1/tasks/b7ca7088-876e-4c29-a0ee-b8029a64849a"
}

The user can go to the above-returned URL and check the status of his Celery task. If the task completed successfully he will get the download URL. The endpoint to check the status of the task is:

and the corresponding response from the server –

{
  "result": {
    "download_url": "/v1/events/1/exports/http://localhost/static/media/exports/1/zip/OGpMM0w2RH/event1.zip"
  },
  "state": "SUCCESS"
}

The file can be downloaded from the above mentioned URL.

Hence, now the event can be added to any scheduling app which recognizes the xcs format.

References

Open Event Server – Export Event as an iCalendar File

FOSSASIA‘s Open Event Server is the REST API backend for the event management platform, Open Event. Here, the event organizers can create their events, add tickets for it and manage all aspects from the schedule to the speakers. Also, once he makes his event public, others can view it and buy tickets if interested.

To make event promotion easier, we also provide the event organizer to export his event as an iCalendar file. Going by the Wikipedia definition, iCalendar is a computer file format which allows Internet users to send meeting requests and tasks to other Internet users by sharing or sending files in this format through various methods. The files usually have an extension of .ics. With supporting software, such as an email reader or calendar application, recipients of an iCalendar data file can respond to the sender easily or counter propose another meeting date/time. The file format is specified in a proposed internet standard (RFC 5545) for calendar data exchange.

Server side – generating the iCal file

Here we will be using the icalendar package for Python as the file writer.

from icalendar import Calendar, vCalAddress, vText
  • We define a class ICalExporter which has a static method export(event_id).
  • Query the event using the event_id passed and start forming the calendar:
event = EventModel.query.get(event_id)

cal = Calendar()
cal.add('prodid', '-//fossasia//open-event//EN')
cal.add('version', '2.0')
cal.add('x-wr-calname', event.name)
cal.add('x-wr-caldesc', "Schedule for sessions at " + event.name)
  • We query for the accepted sessions of the event and store it in sessions.
sessions = Session.query \
   .filter_by(event_id=event_id) \
   .filter_by(state='accepted') \
   .filter(Session.deleted_at.is_(None)) \
   .order_by(asc(Session.starts_at)).all()
  • We then iterate through all the sessions in sessions.
  • If it is a valid session, we instantiate an icalendar event and store required details.
event_component = icalendar.Event()
event_component.add('summary', session.title)
event_component.add('uid', str(session.id) + "-" + event.identifier)
event_component.add('geo', (event.latitude, event.longitude))
event_component.add('location', session.microlocation.name or '' + " " + event.location_name)
event_component.add('dtstart', tz.localize(session.starts_at))
event_component.add('dtend', tz.localize(session.ends_at))
event_component.add('email', event.email)
event_component.add('description', session.short_abstract)
event_component.add('url', url_for('event_detail.display_event_detail_home',
                                  identifier=event.identifier, _external=True))
  • We then loop through all the speakers in that particular session and add it to the iCal Event object as well.
for speaker in session.speakers:
   # Ref: http://icalendar.readthedocs.io/en/latest/usage.html#file-structure
   # can use speaker.email below but privacy reasons
   attendee = vCalAddress('MAILTO:' + event.email if event.email else '[email protected]')
   attendee.params['cn'] = vText(speaker.name)
   event_component.add('attendee', attendee)
  • This event_component is then added to the cal object that we created in the beginning.
cal.add_component(event_component)
  • And finally, the cal.to_ical() is returned. This is the iCalendar file contents. This can be directly written to a file.
return cal.to_ical()

Obtaining the iCal file:

Firstly, we have an API endpoint which starts the task on the server.

GET - /v1/events/{event_identifier}/export/ical

Here, event_identifier is the unique ID of the event. This endpoint starts a celery task on the server to export the event as an iCal file. It returns the task of the URL to get the status of the export task. A sample response is as follows:

{
  "task_url": "/v1/tasks/b7ca7088-876e-4c29-a0ee-b8029a64849a"
}

The user can go to the above returned URL and check the status of his Celery task. If the task completed successfully he will get the download URL. The endpoint to check the status of the task is:

and the corresponding response from the server –

{
  "result": {
    "download_url": "/v1/events/1/exports/http://localhost/static/media/exports/1/zip/OGpMM0w2RH/event1.zip"
  },
  "state": "SUCCESS"
}

The file can be downloaded from the above mentioned URL.

Hence, now the event can be added to any scheduling app which recognizes the ics format.

References

Open Event Frontend – Implement Access Event API via REST API

FOSSASIA‘s Open Event Frontend uses the Open Event Server as the REST API backend. The user can create an event using the Frontend. He can add sessions, tickets speakers etc. and all this updates the database tables in Open Event Server. The server provides certain endpoints for the user to access and/or update the information. It is important that the user is aware of the expected response from the server for his API request. Let’s see how this is displayed in the frontend.

In the event-view page of the frontend, which is accessible to the organizers, there is an Export tab, along with Overview, Tickets, Scheduler, Sessions, Speakers.

This tab has an Access Event Information via REST API section which displays the URL to be used by the user and the expected response. It looks as follows :

The user can choose between various options which he can include or exclude. The GET URL is modified accordingly and the appropriate response is shown to the user.

Example of this –

How is this implemented in Code?

We maintain two variables baseUrl and displayUrl to display the URL. baseUrl is the URL which is common in all requests, ie, till the include tag.

baseUrl: computed('eventId', function() {
 return `${`${ENV.APP.apiHost}/${ENV.APP.apiNamespace}/events/`}${this.get('eventId')}`;
})

displayUrl is the variable which stores the URL being displayed on the webpage. It is initialized to the same as baseUrl.

displayUrl: computed('eventId', function() {
 return `${`${ENV.APP.apiHost}/${ENV.APP.apiNamespace}/events/`}${this.get('eventId')}`;
})

To store the value of the toggle switches we use toggleSwitches as follows:

toggleSwitches: {
 sessions       : false,
 microlocations : false,
 tracks         : false,
 speakers       : false,
 sponsors       : false,
 tickets        : false
}

Whenever any of the switches are toggled, an action checkBox is called. This method updates the value of toggleSwitches, calls the method to update the displayUrl and make the corresponding API request to update the displayed response. The code looks like this :

makeRequest() {
 this.set('isLoading', true);
 this.get('loader')
   .load(this.get('displayUrl'), { isExternal: true })
   .then(json => {
     json = JSON.stringify(json, null, 2);
     this.set('json', htmlSafe(syntaxHighlight(json)));
   })
   .catch(() => {
     this.get('notify').error(this.get('l10n').t('Could not fetch from the server'));
     this.set('json', 'Could not fetch from the server');
   })
   .finally(() => {
     this.set('isLoading', false);
   });
},

buildDisplayUrl() {
 let newUrl = this.get('baseUrl');
 const include = [];

 for (const key in this.get('toggleSwitches')) {
   if (this.get('toggleSwitches').hasOwnProperty(key)) {
     this.get('toggleSwitches')[key] && include.push(key);
   }
 }

 this.set('displayUrl', buildUrl(newUrl, {
   include: include.length > 0 ? include : undefined
 }, true));
},

actions: {
 checkboxChange(data) {
   this.set(`toggleSwitches.${data}`, !this.get(`toggleSwitches.${data}`));
   this.buildDisplayUrl();
   this.makeRequest();
 }
}

The above code uses some utility methods such as buildUrl and this.get(‘loager’).load(). The complete codebase is available here -> Open Event Frontend Repository.

References

Adding JSON-API to Badgeyay Backend

Badgeyay has two main components, the Python-Flask backend server and the EmberJS frontend.

EmberJS frontend uses ember data to save the data from the backend server api into the store of EmberJS frontend. To make the ember data frontend comply with backend api we need the backend server to send responses that comply with the standards of the JSON-API.

What is JSON-API?

As stated by JSONAPI.ORG

"If you've ever argued with your team about the way your JSON responses should be formatted, JSON API can be your anti-bikeshedding tool."

To put it up simply, JSON-API is a way of representing the JSON data that is being generated by the server backend. In this way we represent the JSON data in a particular way that follows the JSON-API convention. An example of data that follows json-api standards is given below:

{
"data": {
"id": "1",
"type": "posts",
"attributes": {
"title": "This is a JSON API data"
},
"relationships": {
"author": {
"links": {
"related": "/example/number"
}
},
"comments": {
"links": {
"related": "/example/number/article/"
}
"data": [
{"id": 5, "type": "example"},
{"id": 12, "type": "example"}
],
}
},
}
}

Adding JSON-API using Marshmallow-JSONAPI

We proceeded on to adding json-api into the Python-Flask backend. Before we proceed to adding json-api, we first need to install marshmallow_jsonapi

To install marshmallow_jsonapi

$ ~ pip install marshmallow-jsonapi

After installing marshmallow_jsonapi, we proceed onto making our first schema.

A schema is a layer of abstraction that is provided over a database model that can be used to dump data from or into an object. This object can therefore be used to either store in database or to dump it to the EmberJS frontend. Let us create a schema for File.

from marshmallow_jsonapi.flask import Schema
from marshmallow_jsonapi import fields


class FileSchema(Schema):
class Meta:
type_ = 'File'
self_view = 'fileUploader.get_file'
kwargs = {'id': '<id>'}

id = fields.Str(required=True, dump_only=True)
filename = fields.Str(required=True)
filetype = fields.Str(required=True)
user_id = fields.Relationship(
self_url='/api/upload/get_file',
self_url_kwargs={'file_id': '<id>'},
related_url='/user/register',
related_url_kwargs={'id': '<id>'},
include_resource_linkage=True,
type_='User'
)

So we have successfully created a Schema for getting files. This schema has an id, filename and filetype. It also has a relationship with the User.

Let us now create a route for this Schema. The below snippet of code is used to find a given file using this schema.

@router.route('/get_file', methods=['GET'])
def get_file():
input_data = request.args
file = File().query.filter_by(filename=input_data.get('filename')).first()
return jsonify(FileSchema().dump(file).data)

 

Now to get details of a file using our newly created route and schema all we need to do is use the following cURL command:

$ ~ curl -X GET "http://localhost:5000/api/upload/get_file?filename={your_file_name}"

You will get something like this as a response:

{
"data": {
"attributes": {
"filename": "13376967-8846-4c66-bcab-4a6b7d58aca7.csv",
"filetype": "csv"
},
"id": "967dc51b-289a-43a1-94c1-5cfce04b0fbf",
"links": {
"self": "/api/upload/get_file"
},
"relationships": {
"user_id": {
"data": {
"id": "J9v2LBIai1MOc8LijeLx7zWsP4I2",
"type": "User"
},
"links": {
"related": "/user/register",
"self": "/api/upload/get_file"
}
}
},
"type": "File"
},
"links": {
"self": "/api/upload/get_file"
}
}

Further Improvements

After adding JSON-API standards to the backend API we can easily integrate it with the EmberJS frontend. Now we can work on adding more schemas as a method of layers of abstraction so that the backend can serve more functionalities and the data can be consumed by the frontend as well.

Resources

Generating Badges from Badgeyay API

Badgeyay is a badge generator and its main functionality is generating badges. Since the beginning of GSoC 2018 period, Badgeyay is under refactoring and remodeling process. We have introduced many APIs to make sure that Badgeyay works. Now, the badge generator has an endpoint to generate badges for your events/meetups

How to create badges?

Creating badges using the newly formed API is simpler than before. All you need to do is pass some basic details of the image you want, the data you want, the size and the color of font etc to the API and woosh! Within a blink of your eye the badges are generated.

Backend requires some data fields to generate badges

{
"csv" : "a731h-jk12n-bbau2-saj2-nxg31.csv",
"image" : "p2ja7-gna398-c23ba-naj31a.png",
"text-color" : "#ffffff"
}

“csv” is the filename of the csv that the user uploads and get back as a result, “image” is the image name that user gets after a successful upload to the respective APIs, “text-color” is the color of the text that the user wants on the badges.

Output of the API

{
"output" :  "path-to-the-pdf-of-the-badge-generated",
.
.
}

What is happening behind the scene?

Once the user sends the data to the API, the required route is triggered and the data is checked,If the data is not present an error response is sent back to the user so as to inform them about the misplacement or improper format of data.

import os
from flask import Blueprint, jsonify, request
from flask import current_app as app
# from api.helpers.verifyToken import loginRequired
from api.utils.response import Response
from api.utils.svg_to_png import SVG2PNG
from api.utils.merge_badges import MergeBadges


router = Blueprint('generateBadges', __name__)


@router.route('/generate_badges', methods=['POST'])
def generateBadges():
try:
data = request.get_json()
except Exception as e:
return jsonify(
Response(401).exceptWithMessage(str(e),'Could not find any JSON'))

if not data.get('csv'):
return jsonify(
Response(401).generateMessage('No CSV filename found'))
if not data.get('image'):
return jsonify(Response(401).generateMessage('No Image filename found'))
csv_name = data.get('csv')
image_name = data.get('image')
text_color = data.get('text-color') or '#ffffff'
svg2png = SVG2PNG()
svg2png.do_text_fill('static/badges/8BadgesOnA3.svg', text_color)
merge_badges = MergeBadges(image_name, csv_name)
merge_badges.merge_pdfs()

output = os.path.join(app.config.get('BASE_DIR'), 'static', 'temporary', image_name)
return jsonify(
Response(200).generateMessage(str(output)))

 

After the data is received, we send it to MergeBadges which internally calls the GenerateBadges class which creates the badges.

Brief explanation of the Badge Generation Process:
- Gather data from the user- Fill the SVG for badges with the text color

- Load the image from uploads directory
- Generate badges for every individual
- Create PDFs for individual Badges
- Merge those PDFs to provide an all-badges pdf to the user

 

And this is how we generated badges for the user using the Badgeyay Backend API.

How is this effective?

We are making sure that the user chooses the image and csv that he/she has uploaded only,

In this way we maintain a proper workflow, we also manage these badges into the database and hence using the filenames helps a lot.It does not involve sending huge files and a lot of data like we had in the previous API.

Earlier, we used to send the image and the csv altogether that caused a serious mismanagement of the project. In this case we are accepting the CSVs and the Images on different API routes and then using the specific image and csv to make badges. We can now more easily relate to the files associated with each and every badge and henceforth we can easily manage them in the database.

Further Improvements

We will work on adding security to the route so that not anyone can create badges. We also need to integrate database into badges generated service so that we can maintain the badges that the user has generated.

Resources

File and Image Upload API in Badgeyay

Badgeyay has seen many changes in the recent past during its refactoring. It started off with backend and we have now transition to remodeling backend as well.

The backend transition is working perfectly. We have established sufficient APIs so far to get it working.

Some of the most important APIs that we created are

  • Image Upload API
  • File Upload API

Why do we need APIs?

We need APIs so that the frontend written in Ember JS can coordinate with the backend written in Python Flask with the database being PostgreSQL.

Creating the APIs

Creating these APIs is easy and straightforward. The following APIs are written in Python Flask with a backend database support of PostgreSQL.

Image Upload API

The image upload API considers that the frontend is sending the Image as a base64 encoded string and the backend is supposed to accept this string and convert this string into an image and save it onto the server.

We proceed by creating a file named fileUploader.py and code the following API.

First of all, we need to declare the imports

from flask import Blueprint, request, jsonify
from api.utils.response import Response
from api.helpers.verifyToken import loginRequired
from api.helpers.uploads import saveToImage, saveToCSV

Now, let’s create a route for image upload.

router = Blueprint('fileUploader', __name__)

@router.route('/image', methods=['POST'])
@loginRequired
def uploadImage():
try:
image = request.json['data']
except Exception as e:
return jsonify(
Response(400).exceptWithMessage(
str(e),
'No Image is specified'))

extension = request.json['extension']
try:
imageName = saveToImage(imageFile=image, extension=extension)
except Exception as e:
return jsonify(
Response(400).exceptWithMessage(
str(e),
'Image could not be uploaded'))

return jsonify(
Response(200).generateMessage({
'message': 'Image Uploaded Successfully',
'unique_id': imageName}))

We are using the saveToImage function to actually save the image to the backend server.

The function definition of saveToImage function is given below.

def generateFileName():
return str(uuid.uuid4())def saveToImage(imageFile=None, extension='.png'):
imageName = generateFileName() + extension
imageDirectory = os.path.join(app.config.get('BASE_DIR'), 'static', 'uploads', 'image')if not os.path.isdir(imageDirectory):
os.makedirs(imageDirectory)imagePath = os.path.join(imageDirectory, imageName)
image = open(imagePath, "wb")
image.write(imageFile.decode('base64'))
image.close()

return imageName

Similarly, we are using file upload route to upload files to backend server.

The route for uploading files along with its helper function saveToCSV is given below.

def saveToCSV(csvFile=None, extension='.csv'):
csvName = generateFileName() + extension
csvDirectory = os.path.join(app.config.get('BASE_DIR'), 'static', 'uploads', 'csv')if not os.path.isdir(csvDirectory):
os.makedirs(csvDirectory)csvPath = os.path.join(csvDirectory, csvName)
csvFile.save(csvPath)return csvName
@router.route('/file', methods=['POST'])
@loginRequired
def fileUpload():
if 'file' not in request.files:
return jsonify(
Response(401).generateMessage(
'No file is specified'))file = request.files['file']
try:
csvName = saveToCSV(csvFile=file, extension='.csv')
except Exception as e:
return jsonify(
Response(400).exceptWithMessage(
str(e),
'CSV File could not be uploaded'))return jsonify(
Response(200).generateMessage({
'message': 'CSV Uploaded successfully',
'unique_id': csvName}))

What happens to the uploaded files?

The uploaded files gets saved into their respective directories, i.e. static/uploads/csv for CSV files and static/uploads/images for Image uploads.

The developer can view them from their respective folders. The static folder has been added to .gitignore  so that it does not gets uploaded to github repository.

Everything has been taken care of with immense accuracy and proper error handling.

Further Improvements

Further improvements in Badgeyay includes adding separate database models, work on adding a beautiful frontend and to add proper routes for completing the backend.

Resources

Unit Tests for REST-API in Python Web Application

Badgeyay backend is now shifted to REST-API and to test functions used in REST-API, we need some testing technology which will test each and every function used in the API. For our purposes, we chose the popular unit tests Python test suite.

In this blog, I’ll be discussing how I have written unit tests to test Badgeyay  REST-API.

First, let’s understand what is unittests and why we have chosen it. Then we will move onto writing API tests for Badgeyay. These tests have a generic structure and thus the code I mention would work in other REST API testing scenarios, often with little to no modifications.

Let’s get started and understand API testing step by step.

What is Unittests?

Unitests is a Python unit testing framework which supports test automation, sharing of setup and shutdown code for tests, aggregation of tests into collections, and independence of the tests from the reporting framework. The unittest module provides classes that make it easy to support these qualities for a set of tests.

Why Unittests?

We get two primary benefits from unit testing, with a majority of the value going to the first:

  • Guides your design to be loosely coupled and well fleshed out. If doing test driven development, it limits the code you write to only what is needed and helps you to evolve that code in small steps.
  • Provides fast automated regression for re-factors and small changes to the code.
  • Unit testing also gives you living documentation about how small pieces of the system work.

We should always strive to write comprehensive tests that cover the working code pretty well.

Now, here is glimpse of how  I wrote unit tests for testing code in the REST-API backend of Badgeyay. Using unittests python package and requests modules, we can test REST API in test automation.

Below is the code snippet for which I have written unit tests in one of my pull requests.

def output(response_type, message, download_link):
    if download_link == '':
        response = [
            {
                'type': response_type,
                'message': message
            }
        ]
    else:
        response = [
            {
                'type': response_type,
                'message': message,
                'download_link': download_link
            }
        ]
    return jsonify({'response': response})

 

To test this function, I basically created a mock object which could simulate the behavior of real objects in a controlled way, so in this case a mock object may simulate the behavior of the output function and return something like an JSON response without hitting the real REST API. Now the next challenge is to parse the JSON response and feed the specific value of the response JSON to the Python automation script. So Python reads the JSON as a dictionary object and it really simplifies the way JSON needs to be parsed and used.

And here’s the content of the backend/tests/test_basic.py file.

 #!/usr/bin/env python3
"""Tests for Basic Functions"""
import sys
import json
import unittest

sys.path.append("../..")
from app.main import *


class TestFunctions(unittest.TestCase):
      """Test case for the client methods."""
    def setup(self):
        app.app.config['TESTING'] = True
        self.app = app.app.test_client()
      # Test of Output function
    def test_output(self):
        with app.test_request_context():
            # mock object
            out = output('error', 'Test Error', 'local_host')
            # Passing the mock object
            response = [
                {
                    'type': 'error',
                    'message': 'Test Error',
                    'download_link': 'local_host'
                }
            ]
            data = json.loads(out.get_data(as_text=True))
            # Assert response
            self.assertEqual(data['response'], response)


if __name__ == '__main__':
    unittest.main()

 

And finally, we can verify that everything works by running nosetests .

This is how I wrote unit tests in BadgeYaY repository. You can find more of work here.

Resources:

  • The Purpose of Unit Testing – Link
  • Unit testing framework – Link