Open Event API Server: Implementing FAQ Types

In the Open Event Server, there was a long standing request of the users to enable the event organisers to create a FAQ section.

The API of the FAQ section was implemented subsequently. The FAQ API allowed the user to specify the following request schema

{
 "data": {
   "type": "faq",
   "relationships": {
     "event": {
       "data": {
         "type": "event",
         "id": "1"
       }
     }
   },
   "attributes": {
     "question": "Sample Question",
     "answer": "Sample Answer"
   }
 }
}

 

But, what if the user wanted to group certain questions under a specific category. There was no solution in the FAQ API for that. So a new API, FAQ-Types was created.

Why make a separate API for it?

Another question that arose while designing the FAQ-Types API was whether it was necessary to add a separate API for it or not. Consider that a type attribute was simply added to the FAQ API itself. It would mean the client would have to specify the type of the FAQ record every time a new record is being created for the same. This would mean trusting that the user will always enter the same spelling for questions falling under the same type. The user cannot be trusted on this front. Thus the separate API made sure that the types remain controlled and multiple entries for the same type are not there.

Helps in handling large number of records:

Another concern was what if there were a large number of FAQ records under the same FAQ-Type. Entering the type for each of those questions would be cumbersome for the user. The FAQ-Type would also overcome this problem

Following is the request schema for the FAQ-Types API

{
 "data": {
   "attributes": {
     "name": "abc"
   },
   "type": "faq-type",
   "relationships": {
     "event": {
       "data": {
         "id": "1",
         "type": "event"
       }
     }
   }
 }
}

 

Additionally:

  • FAQ to FAQ-type is a many to one relation.
  • A single FAQ can only belong to one Type
  • The FAQ-type relationship will be optional, if the user wants different sections, he/she can add it ,if not, it’s the user’s choice.

Related links

Continue ReadingOpen Event API Server: Implementing FAQ Types

Implementing Permissions for Orders API in Open Event API Server

Open Event API Server Orders API is one of the core APIs. The permissions in Orders API are robust and secure enough to ensure no leak on payment and ticketing.The permission manager provides the permissions framework to implement the permissions and proper access controls based on the dev handbook.

The following table is the permissions in the developer handbook.

 

List View Create Update Delete
Superadmin/admin
Event organizer [1] [1] [1] [1][2] [1][3]
Registered user [4]
Everyone else
  1. Only self-owned events
  2. Can only change order status
  3. A refund will also be initiated if paid ticket
  4. Only if order placed by self

Super Admins and admins are allowed to create any order with any amount but any coupon they apply is not consumed on creating order. They can update almost every field of the order and can provide any custom status to the order. Permissions are applied with the help of Permission Manager which takes care the authorization roles. For example, if a permission is set based on admin access then it is automatically set for super admin as well i.e., to the people with higher rank.

Self-owned events

This allows the event admins, Organizer and Co-Organizer to manage the orders of the event they own. This allows then to view all orders and create orders with or without discount coupon with any custom price and update status of orders. Event admins can provide specific status while others cannot

if not has_access('is_coorganizer', event_id=data['event']):
   data['status'] = 'pending'

And Listing requires Co-Organizer access

elif not has_access('is_coorganizer', event_id=kwargs['event_id']):
   raise ForbiddenException({'source': ''}, "Co-Organizer Access Required")

Can only change order status

The organizer cannot change the order fields except the status of the order. Only Server Admin and Super Admins are allowed to update any field of the order.

if not has_access('is_admin'):
   for element in data:
       if element != 'status':
           setattr(data, element, getattr(order, element))

And Delete access is prohibited to event admins thus only Server admins can delete orders by providing a cancelling note which will be provided to the Attendee/Buyer.

def before_delete_object(self, order, view_kwargs):
   if not has_access('is_coorganizer', event_id=order.event.id):
       raise ForbiddenException({'source': ''}, 'Access Forbidden')

Registered User

A registered user can create order with basic details like the attendees’ records and payment method with fields like country and city. They are not allowed to provide any custom status to the order they are creating. All orders will be set by default to “pending”

Also, they are not allowed to update any field in their order. Any status update will be done internally thus maintaining the security of Order System. Although they are allowed to view their place orders. This is done by comparing their logged in user id with the user id of the purchaser.

if not has_access('is_coorganizer_or_user_itself', event_id=order.event_id, user_id=order.user_id):
   return ForbiddenException({'source': ''}, 'Access Forbidden')

Event Admins

The event admins have one more restriction, as an event admin, you cannot provide discount coupon and even if you do it will be ignored.

# Apply discount only if the user is not event admin
if data.get('discount') and not has_access('is_coorganizer', event_id=data['event']):

Also an event admin any amount you will provide on creating order will be final and there will be no further calculation of the amount will take place

if not has_access('is_coorganizer', event_id=data['event']):
   TicketingManager.calculate_update_amount(order)

Creating Attendees Records

Before sending a request to Orders API it is required to create to attendees mapped to some ticket and for this registered users are allowed to create the attendees without adding a relationship of the order. The mapping with the order is done internally by Orders API and its helpers.

Resources

  1. Dev Handbook – Niranjan R
    The Open Event Developer Handbook
  2. Flask-REST-JSONAPI Docs
    Permissions and Data layer | Flask-REST-JSONAPI
  3. A guide to use permission manager in API Server
    https://blog.fossasia.org/a-guide-to-use-permission-manager-in-open-event-api-server/

 

Continue ReadingImplementing Permissions for Orders API in Open Event API Server

Generating Ticket PDFs in Open Event API Server

In the ordering system of Open Event API Server, there is a requirement to send email notifications to the attendees. These attendees receive the URL of the pdf of the generated ticket. On creating the order, first the pdfs are generated and stored in the preferred storage location and then these are sent to the users through the email.

Generating PDF is a simple process, using xhtml2pdf we can generate PDFs from the html. The generated pdf is then passed to storage helpers to store it in the desired location and pdf-url is updated in the attendees record.

Sample PDF

PDF Template

The templates are written in HTML which is then converted using the module xhtml2pdf.
To store the templates a new directory was created at  app/templates where all HTML files are stored. Now, The template directory needs to be updated at flask initializing app so that template engine can pick the templates from there. So in app/__init__.py we updated flask initialization with

template_dir = os.path.dirname(__file__) + "/templates"

app = Flask(__name__, static_folder=static_dir, template_folder=template_dir)

This allows the template engine to pick the templates files from this template directory.

Generating PDFs

Generating PDF is done by rendering the html template first. This html content is then parsed into the pdf

file = open(dest, "wb")

pisa.CreatePDF(cStringIO.StringIO(pdf_data.encode('utf-8')), file)

file.close()

The generated pdf is stored in the temporary location and then passed to storage helper to upload it.

uploaded_file = UploadedFile(dest, filename)

upload_path = UPLOAD_PATHS['pdf']['ticket_attendee'].format(identifier=get_file_name())

new_file = upload(uploaded_file, upload_path)

This generated pdf path is returned here

Rendering HTML and storing PDF

for holder in order.ticket_holders:

  if holder.id != current_user.id:

      pdf = create_save_pdf(render_template('/pdf/ticket_attendee.html', order=order, holder=holder))

  else:

      pdf = create_save_pdf(render_template('/pdf/ticket_purchaser.html', order=order))

  holder.pdf_url = pdf

  save_to_db(holder)

The html is rendered using flask template engine and passed to create_save_pdf and link is updated on the attendee record.

Sending PDF on email

These pdfs are sent as a link to the email after creating the order. Thus a ticket is sent to each attendee and a summarized order details with attendees to the purchased.

send_email(

  to=holder.email,

  action=TICKET_PURCHASED_ATTENDEE,

  subject=MAILS[TICKET_PURCHASED_ATTENDEE]['subject'].format(

      event_name=order.event.name,

      invoice_id=order.invoice_number

  ),

  html= MAILS[TICKET_PURCHASED_ATTENDEE]['message'].format(

      pdf_url=holder.pdf_url,

      event_name=order.event.name

  )

)

References

  1. Readme – xhtml2pdf
    https://github.com/xhtml2pdf/xhtml2pdf/blob/master/README.rst
  2. Using xhtml2pdf and create pdfs
    https://micropyramid.com/blog/generating-pdf-files-in-python-using-xhtml2pdf/

 

Continue ReadingGenerating Ticket PDFs in Open Event API Server

Discount Codes in Open Event Server

The Open Event System allows usage of discount codes with tickets and events. This blogpost describes what types of discount codes are present and what endpoints can be used to fetch and update details.

In Open Event API Server, each event can have two types of discount codes. One is ‘event’ discount code, while the other is ‘ticket’ discount code. As the name suggests, the event discount code is an event level discount code and the ticket discount code is ticket level.

Now each event can have only one ‘event’ discount code and is accessible only to the server admin. The Open Event server admin can create, view and update the ‘event’ discount code for an event. The event discount code followsDiscountCodeEvent Schema. This schema is inherited from the parent class DiscountCodeSchemaPublic. To save the unique discount code associated with an event, the event model’s discount_code_id field is used.

The ‘ticket’ discount is accessible by the event organizer and co-organizer. Each event can have any number of ‘ticket’ discount codes. This follows the DiscountCodeTicket schema, which is also inherited from the same base class ofDiscountCodeSchemaPublic. The use of the schema is decided based on the value of the field ‘used_for’ which can have the value either ‘events’ or ‘tickets’. Both the schemas have different relationships with events and marketer respectively.

We have the following endpoints for Discount Code events and tickets:
‘/events/<int:event_id>/discount-code’
‘/events/<int:event_id>/discount-codes’

The first endpoint is based on the DiscountCodeDetail class. It returns the detail of one discount code which in this case is the event discount code associated with the event.

The second endpoint is based on the DiscountCodeList class which returns a list of discount codes associated with an event. Note that this list also includes the ‘event’ discount code, apart from all the ticket discount codes.

class DiscountCodeFactory(factory.alchemy.SQLAlchemyModelFactory):
   class Meta:
       model = DiscountCode
       sqlalchemy_session = db.session
event_id = None
user = factory.RelatedFactory(UserFactory)
user_id = 1


Since each discount code belongs to an event(either directly or through the ticket), the factory for this has event as related factory, but to check for 
/events/<int:event_id>/discount-code endpoint we first need the event and then pass the discount code id to be 1 for dredd to check this. Hence, event is not included as a related factory, but added as a different object every time a discount code object is to be used.

@hooks.before("Discount Codes > Get Discount Code Detail of an Event > Get Discount Code Detail of an Event")
def event_discount_code_get_detail(transaction):
   """
   GET /events/1/discount-code
   :param transaction:
   :return:
   """
   with stash['app'].app_context():
       discount_code = DiscountCodeFactory()
       db.session.add(discount_code)
       db.session.commit()
       event = EventFactoryBasic(discount_code_id=1)
       db.session.add(event)
       db.session.commit()


The other tests and extended documentation can be found 
here.

References:

Continue ReadingDiscount Codes in Open Event Server

Open Event Server: Getting The Identity From The Expired JWT Token In Flask-JWT

The Open Event Server uses JWT based authentication, where JWT stands for JSON Web Token. JSON Web Tokens are an open industry standard RFC 7519 method for representing claims securely between two parties. [source: https://jwt.io/]

Flask-JWT is being used for the JWT-based authentication in the project. Flask-JWT makes it easy to use JWT based authentication in flask, while on its core it still used PyJWT.

To get the identity when a JWT token is present in the request’s Authentication header , the current_identity proxy of Flask-JWT can be used as follows:

@app.route('/example')
@jwt_required()
def example():
   return '%s' % current_identity

 

Note that it will only be set in the context of function decorated by jwt_required(). The problem with the current_identity proxy when using jwt_required is that the token has to be active, the identity of an expired token cannot be fetched by this function.

So why not write a function on our own to do the same. A JWT token is divided into three segments. JSON Web Tokens consist of three parts separated by dots (.), which are:

  • Header
  • Payload
  • Signature

The first step would be to get the payload, that can be done as follows:

token_second_segment = _default_request_handler().split('.')[1]

 

The payload obtained above would still be in form of JSON, it can be converted into a dict as follows:

payload = json.loads(token_second_segment.decode('base64'))

 

The identity can now be found in the payload as payload[‘identity’]. We can get the actual user from the paylaod as follows:

def jwt_identity(payload):
   """
   Jwt helper function
   :param payload:
   :return:
   """
   return User.query.get(payload['identity'])

 

Our final function will now be something like:

def get_identity():
   """
   To be used only if identity for expired tokens is required, otherwise use current_identity from flask_jwt
   :return:
   """
   token_second_segment = _default_request_handler().split('.')[1]
   missing_padding = len(token_second_segment) % 4
   payload = json.loads(token_second_segment.decode('base64'))
   user = jwt_identity(payload)
   return user

 

But after using this function for sometime, you will notice that for certain tokens, the system will raise an error saying that the JWT token is missing padding. The JWT payload is base64 encoded, and it requires the payload string to be a multiple of four. If the string is not a multiple of four, the remaining spaces can pe padded with extra =(equal to) signs. And since Python 2.7’s .decode doesn’t do that by default, we can accomplish that as follows:

missing_padding = len(token_second_segment) % 4

# ensures the string is correctly padded to be a multiple of 4
if missing_padding != 0:
   token_second_segment += b'=' * (4 - missing_padding)

 

Related links:

Continue ReadingOpen Event Server: Getting The Identity From The Expired JWT Token In Flask-JWT

How to customize LXQT for Meilix

We had a task of customizing the LXQT (LXQt is the Qt port and the upcoming version of LXDE, the Lightweight Desktop Environment) desktop environment in Meilix for events. For example, we can use a distro at events for presentations so during presentations things like system sounds, notifications and panel can be disturbing elements of presentations so we required LXQT to be pre configured for that so the time required in configuration is not wasted or none of the presentations are disturbed.

The default configuration of LXQT are present in ~/.config/lxqt. Configuration files are in This directory is initialized automatically. The default configuration for new users is found in /etc/xdg/lxqt but we are going to use skel for this so that if there are some changes in future in the LXQT code and they do not match default settings we can always be back to default settings by deleting user side changes. Similar to LXDE. LXQt provides a GUI applications to change its settings as well.

While Openbox is the default window manager we have used in Meilix with  LXQt, you can specify a different window manager to use with LXQt via by editing ~/.config/lxqt/session.conf. To a window manager of choice. Change the following line:

window_manager=openbox

 

We have used the Openbox for Meilix.

To create a configuration for panel we can go to ~/.config/lxqt and edit the panel.conf file if it is not present we can create a file named panel.conf and add/edit the code.

[panel1]
hidable=true 

 

GIF representing auto hide of panel.

In order to configure more things like notification we can edit the ~/.config/lxqt/notification.conf.

We can change things like the placement or the size of notification or the timeout of notification in this file.

For eg:

 [General]
__userfile__=true
placement=top-left
server_decides=1
spacing=6
width=295

 

We can now place all the configurations inside the skel folder so that every new user gets the same configurations we have made.

Resources

Continue ReadingHow to customize LXQT for Meilix

Automatic Signing and Publishing of Android Apps from Travis

As I discussed about preparing the apps in Play Store for automatic deployment and Google App Signing in previous blogs, in this blog, I’ll talk about how to use Travis Ci to automatically sign and publish the apps using fastlane, as well as how to upload sensitive information like signing keys and publishing JSON to the Open Source repository. This method will be used to publish the following Android Apps:

Current Project Structure

The example project I have used to set up the process has the following structure:

It’s a normal Android Project with some .travis.yml and some additional bash scripts in scripts folder. The update-apk.sh file is standard app build and repo push file found in FOSSASIA projects. The process used to develop it is documented in previous blogs. First, we’ll see how to upload our keys to the repo after encrypting them.

Encrypting keys using Travis

Travis provides a very nice documentation on encrypting files containing sensitive information, but a crucial information is buried below the page. As you’d normally want to upload two things to the repo – the app signing key, and API JSON file for release manager API of Google Play for Fastlane, you can’t do it separately by using standard file encryption command for travis as it will override the previous encrypted file’s secret. In order to do so, you need to create a tarball of all the files that need to be encrypted and encrypt that tar instead. Along with this, before you need to use the file, you’ll have to decrypt in in the travis build and also uncompress it for use.

So, first install Travis CLI tool and login using travis login (You should have right access to the repo and Travis CI in order to encrypt the files for it)

Then add the signing key and fastlane json in the scripts folder. Let’s assume the names of the files are key.jks and fastlane.json

Then, go to scripts folder and run this command to create a tar of these files:

tar cvf secrets.tar fastlane.json key.jks

 

secrets.tar will be created in the folder. Now, run this command to encrypt the file

travis encrypt-file secrets.tar

 

A new file secrets.tar.enc will be created in the folder. Now delete the original files and secrets tar so they do not get added to the repo by mistake. The output log will show the the command for decryption of the file to be added to the .travis.yml file.

Decrypting keys using Travis

But if we add it there, the keys will be decrypted for each commit on each branch. We want it to happen only for master branch as we only require publishing from that branch. So, we’ll create a bash script prep-key.sh for the task with following content

#!/bin/sh
set -e

export DEPLOY_BRANCH=${DEPLOY_BRANCH:-master}

if [ "$TRAVIS_PULL_REQUEST" != "false" -o "$TRAVIS_REPO_SLUG" != "iamareebjamal/android-test-fastlane" -o "$TRAVIS_BRANCH" != "$DEPLOY_BRANCH" ]; then
    echo "We decrypt key only for pushes to the master branch and not PRs. So, skip."
    exit 0
fi

openssl aes-256-cbc -K $encrypted_4dd7_key -iv $encrypted_4dd7_iv -in ./scripts/secrets.tar.enc -out ./scripts/secrets.tar -d
tar xvf ./scripts/secrets.tar -C scripts/

 

Of course, you’ll have to change the commands and arguments according to your need and repo. Specially, the decryption command keys ID

The script checks if the repo and branch are correct, and the commit is not of a PR, then decrypts the file and extracts them in appropriate directory

Before signing the app, you’ll need to store the keystore password, alias and key password in Travis Environment Variables. Once you have done that, you can proceed to signing the app. I’ll assume the variable names to be $STORE_PASS, $ALIAS and $KEY_PASS respectively

Signing App

Now, come to the part in upload-apk.sh script where you have the unsigned release app built. Let’s assume its name is app-release-unsigned.apk.Then run this command to sign it

cp app-release-unsigned.apk app-release-unaligned.apk
jarsigner -verbose -tsa http://timestamp.comodoca.com/rfc3161 -sigalg SHA1withRSA -digestalg SHA1 -keystore ../scripts/key.jks -storepass $STORE_PASS -keypass $KEY_PASS app-release-unaligned.apk $ALIAS

 

Then run this command to zipalign the app

${ANDROID_HOME}/build-tools/25.0.2/zipalign -v -p 4 app-release-unaligned.apk app-release.apk

 

Remember that the build tools version should be the same as the one specified in .travis.yml

This will create an apk named app-release.apk

Publishing App

This is the easiest step. First install fastlane using this command

gem install fastlane

 

Then run this command to publish the app to alpha channel on Play Store

fastlane supply --apk app-release.apk --track alpha --json_key ../scripts/fastlane.json --package_name com.iamareebjamal.fastlane

 

You can always configure the arguments according to your need. Also notice that you have to provide the package name for Fastlane to know which app to update. This can also be stored as an environment variable.

This is all for this blog, you can read more about travis CLI, fastlane features and signing process in these links below:

Continue ReadingAutomatic Signing and Publishing of Android Apps from Travis

Setting Loklak Server with SSL

Loklak Server is based on embedded Jetty Server which can work both with or without SSL encryption. Lately, there was need to setup Loklak Server with SSL. Though the need was satisfied by CloudFlare. Alternatively, there are 2 ways to set up Loklak Server with SSL. They are:-

1) Default Jetty Implementation

There is pre-existing implementation of Jetty libraries. The http mode can be set in configuration file. There are 4 modes on which Loklak Server can work: http mode, https mode, only https mode and redirect to https mode. Loklak Server listens to port 9000 when in http mode and to port 9443 when in https mode.

There is also a need of SSL certificate which is to be added in configuration file.

2) Getting SSL certificate with Kube-Lego on Kubernetes Deployment

I got to know about Kube-Lego by @niranjan94. It is implemented in Open-Event-Orga-Server. The approach is to use:

a) Nginx as ingress controller

For setting up Nginx ingress controller, a yml file is needed which downloads and configures the server.

The configurations for data requests and response are:

proxy-connect-timeout: "15"
 proxy-read-timeout: "600"
 proxy-send-imeout: "600"
 hsts-include-subdomains: "false"
 body-size: "64m"
 server-name-hash-bucket-size: "256"
 server-tokens: "false"

Nginx is configured to work on both http and https ports in service.yml

ports:
- port: 80
  name: http
- port: 443
  name: https

 

b) Kube-Lego for fetching SSL certificates from Let’s Encrypt

Kube-Lego was set up with default values in yml. It uses the host-name, email address and secretname of the deployment to validate url and fetch SSL certificate from Let’s Encrypt.

c) Setup configurations related to TLS and no-TLS connection

These configuration files mentions the path and service ports for Nginx Server through which requests are forwarded to backend Loklak Server. Here for no-TLS and TLS requests, the requests are directly forwarded to localhost at port 80 of Loklak Server container.

rules:
- host: staging.loklak.org
  http:
  paths:
  - path: /
    backend:
    serviceName: server
    servicePort: 80

For TLS requests, the secret name is also mentioned. Kube-Lego fetches host-name and secret-name from here for the certificate

tls:
- hosts:
- staging.loklak.org
  secretName: loklak-api-tls

d) Loklak Server, ElasticSearch and Mosquitto at backend

These containers work at backend. ElasticSearch and Mosquitto are only accessible to Loklak Server. Loklak Server can be accessed through Nginx server. Loklak Server is configured to work at http mode and is exposed at port 80.

ports:
- port: 80
  protocol: TCP
  targetPort: 80

To deploy the Loklak Server, all these are deployed in separate pods and they interact through service ports. To deploy, we use deploy script:

# For elasticsearch, accessible only to api-server
kubectl create -R -f ${path-to-config-file}/elasticsearch/

# For mqtt, accessible only to api-server
kubectl create -R -f ${path-to-config-file}/mosquitto/

# Start KubeLego deployment for TLS certificates
kubectl create -R -f ${path-to-config-file}/lego/
kubectl create -R -f ${path-to-config-file}/nginx/

# Create web namespace, this acts as bridge to Loklak Server
kubectl create -R -f ${path-to-config-file}/web/

# Create API server deployment and expose the services
kubectl create -R -f ${path-to-config-file}/api-server/

# Get the IP address of the deployment to be used
kubectl get services --namespace=nginx-ingress

References

Continue ReadingSetting Loklak Server with SSL

Showing location response in SUSI.AI bots

SUSI.AI has a capability to tell the basic information about a location, it is asked for. Along with the basic information about that place, it shows a map (i.e. open street map) pointing to that location. The task at hand is to inculcate this “location” feature to the SUSI.AI messenger bots. The SUSI Tweetbot and SUSI Fbbot are used as examples in this blog.

Let’s first check on what response do we get, when a person asks a query like “where is london” to the SUSI API. Along with the basic information about that location, the result as shown below has the type of reply (i.e. map), latitude, longitude and a link to the open street map.

"actions": [
      {
        "type": "answer",
        "language": "en",
        "expression": "Ludhiana is a city and a municipal corporation in Ludhiana district in the Indian state of Punjab, and is the largest city north of Delhi."
      },
      {
        "type": "anchor",
        "link": "https://www.openstreetmap.org/#map=13/30.912040570244187/75.85379021980509",
        "text": "Here is a map",
        "language": "en"
      },
      {
        "type": "map",
        "latitude": "30.912040570244187",
        "longitude": "75.85379021980509",
        "zoom": "13",
        "language": "en"
      }
    ]

The response for a location type query has these 3 main parts:

  1. Clickable static map image.
  2. A basic information of the place asked about.
  3. The link i.e. when the static map image is clicked it should redirect to the corresponding map location.

Let’s try to make up with the 1st part of the response i.e. Static map image.

The map quest API is used to result in a static map image of the location. We need an API key to access the map quest API, which can be requested from their developer site.

Along with that we need the latitude and longitude of the location at hand.

The SUSI API’s response helps us to get the latitude value:

// if body represents the response object 
var lat = body.answers[0].actions[2].latitude;

And the longitude value:

var lon = body.answers[0].actions[2].longitude;

Using the three values that are API key, latitude and longitude, the static image is rendered by this link:

var static_image_url = "https://open.mapquestapi.com/staticmap/v4/getmap?key=API_KEY&size=600,400&zoom=13&center="+lat+","+lon;

The second part is, basic information about the place asked, can be fetched from:

// if body is the JSON response object from SUSI API
var mapMessage = body.answers[0].actions[0].expression;

The link to the map location can be easily fetched from the SUSI API’s response:

var link = body.answers[0].actions[1].link;

As all the three parts are ready, let’s look on how to render them on the SUSI.AI bot’s screen.

Facebook:

Sending a generic template message object:

message = {
        "type":"template",
        "payload":{
                    "template_type":"generic",
                    "elements":[{
                        "title": mapMessage,
                       "image_url": static_image_url,
                       "Item_url": link
                    }]
        }
};

sendTextMessage(sender, message, 1);

Twitter:

The Twitter API does not need a static image of the map to be rendered. It does that work for us. We just need to send an event to the Twitter API with the message data object constituting of the map message, the latitude and longitude values:

"message_data": {
            "text": mapMessage,
            "attachment": {
                "type": "location",
                "location": {
                    "type": "shared_coordinate",
                    "shared_coordinate": {
                        "coordinates": {
                            "type": "Point",
                            "coordinates": [lon, lat]
                        }
                    }
                }
            }
}

Resources:

  1. Speed up customer service with quick replies and welcome messages by Ian Cairns from Twitter blog.
  2. Drive discovery of bots and other customer experiences in direct messages by By Travis Lull from Twitter blog.
  3. By Seth Rosenberg from Facebook developers blogLink Ads to Messenger, Enhanced Mobile Websites, Payments and More.
Continue ReadingShowing location response in SUSI.AI bots

Adding Tweet Streaming Feature in World Mood Tracker loklak App

The World Mood Tracker was added to loklak apps with the feature to display aggregated data from the emotion classifier of loklak server. The next step in the app was adding the feature to display the stream of Tweets from a country as they are discovered by loklak. With the addition of stream servlet in loklak, it was possible to utilise it in this app.

In this blog post, I will be discussing the steps taken while adding to introduce this feature in World Mood Tracker app.

Props for WorldMap component

The WorldMap component holds the view for the map displayed in the app. This is where API calls to classifier endpoint are made and results are displayed on the map. In order to display tweets on clicking a country, we need to define react props so that methods from higher level components can be called.

In order to enable props, we need to change the constructor for the component –

export default class WorldMap extends React.Component {
    constructor(props) {
        super(props);
        ...
    }
    ...
}

[SOURCE]

We can now pass the method from parent component to enable streaming and other components can close the stream by using props in them –

export default class WorldMoodTracker extends React.Component {
    ...
    showStream(countryName, countryCode) {
        /* Do something to enable streaming component */
        ...
    }
 
    render() {
        return (
             ...
                <WorldMap showStream={this.showStream}/>
             ...
        )
    }
}

[SOURCE]

Defining Actions on Clicking Country Map

As mentioned in an earlier blog post, World Mood Tracker uses Datamaps to visualize data on a map. In order to trigger a piece of code on clicking a country, we can use the “done” method of the Datamaps instance. This is where we use the props passed earlier –

done: function(datamap) {
    datamap.svg.selectAll('.datamaps-subunit').on('click', function (geography) {
        props.showStream(geography.properties.name, reverseCountryCode(geography.id));
    })
}

[SOURCE]

The name and ID for the country will be used to display name and make API call to stream endpoint respectively.

The StreamOverlay Component

The StreamOverlay components hold all the utilities to display the stream of Tweets from loklak. This component is used from its parent components whose state holds info about displaying this component –

export default class WorldMoodTracker extends React.Component {
    ...
    getStreamOverlay() {
        if (this.state.enabled) {
            return (<StreamOverlay
                show={true} channel={this.state.channel}
                country={this.state.country} onClose={this.onOverlayClose}/>);
        }
    }

    render() {
        return (
            ...
                {this.getStreamOverlay()}
            ...
        )
    }
}

[SOURCE]

The corresponding props passed are used to render the component and connect to the stream from loklak server.

Creating Overlay Modal

On clicking the map, an overlay is shown. To display this overlay, react-overlays is used. The Modal component offered by the packages provides a very simple interface to define the design and interface of the component, including style, onclose hook, etc.

import {Modal} from 'react-overlays';

<Modal aria-labelledby='modal-label'
    style={modalStyle}
    backdropStyle={backdropStyle}
    show={true}
    onHide={this.close}>
    <div style={dialogStyle()}>
        ...
    </div>
</Modal>

[SOURCE]

It must be noted that modalStyle and backdropStyle are React style objects.

Dialog Style

The dialog style is defined to provide some space at the top, clicking where, the overlay is closed. To do this, vertical height units are used –

const dialogStyle = function () {
    return {
        position: 'absolute',
        width: '100%',
        top: '5vh',
        height: '95vh',
        padding: 20
        ...
    };
};

[SOURCE]

Connecting to loklak Tweet Stream

loklak sends Server Sent Events to clients connected to it. To utilise this stream, we can use the natively supported EventSource object. Event stream is started with the render method of the StreamOverlay component –

render () {
    this.startEventSource(this.props.channel);
    ...
}

[SOURCE]

This channel is used to connect to twitter/country/<country-ID> channel on the stream and then this can be passed to EventStream constructor. On receiving a message, a list of Tweets is appended and later rendered in the view –

startEventSource(country) {
    let channel = 'twitter%2Fcountry%2F' + country;
    if (this.eventSource) {
        return;
    }
    this.eventSource = new EventSource(host + '/api/stream.json?channel=' + channel);
    this.eventSource.onmessage = (event) => {
        let json = JSON.parse(event.data);
        this.state.tweets.push(json);
        if (this.state.tweets.length > 250) {
            this.state.tweets.shift();
        }
        this.setState(this.state);
    };
}

[SOURCE]

The size of the list is restricted to 250 here, so when a newer Tweet comes in, the oldest one is chopped off. And thanks to fast DOM actions in React, the rendering doesn’t take much time.

Rendering Tweets

The Tweets are displayed as simple cards on which user can click to open it on Twitter in a new tab. It contains basic information about the Tweet – screen name and Tweet text. Images are not rendered as it would make no sense to load them when Tweets are coming at a high rate.

function getTweetHtml(json) {
    return (
        <div style={{padding: '5px', borderRadius: '3px', border: '1px solid black', margin: '10px'}}>
            <a href={json.link} target="_blank">
            <div style={{marginBottom: '5px'}}>
                <b>@{json['screen_name']}</b>
            </div>
            <div style={{overflowX: 'hidden'}}>{json['text']}</div>
            </a>
        </div>
    )
}

[SOURCE]

They are rendered using a simple map in the render method of StreamOverlay component –

<div className={styles.container} style={{'height': '100%', 'overflowY': 'auto',
    'overflowX': 'hidden', maxWidth: '100%'}}>
    {this.state.tweets.reverse().map(getTweetHtml)}
</div>

[SOURCE]

Closing Overlay

With the previous setup in place, we can now see Tweets from the loklak backend as they arrive. But the problem is that we will still be connected to the stream when we click-close the modal. Also, we would need to close the overlay from the parent component in order to stop rendering it.

We can use the onclose method for the Modal here –

close() {
    if (this.eventSource) {
        this.eventSource.close();
        this.eventSource = null;
    }
    this.props.onClose();
}

[SOURCE]

Here, props.onClose() disables rendering of StreamOverlay in the parent component.

Conclusion

In this blog post, I explained how the flow of props are used in the World Mood Tracker app to turn on and off the streaming in the overlay defined using react-overlays. This feature shows a basic setup for using the newly introduced stream API in loklak.

The motivation of such application was taken from emojitracker by mroth as mentioned in fossasia/labs.fossasia.org#136. The changes were proposed in fossasia/apps.loklak.org#315 by @singhpratyush (me).

The app can be accessed live at https://singhpratyush.github.io/world-mood-tracker/index.html.

Resources

Continue ReadingAdding Tweet Streaming Feature in World Mood Tracker loklak App