Adding Defaults Prior to Schema Validation Elegantly

The Open Event Server offers features for events with several tracks and venues. When we were designing the models for the API, we wanted to add default values for some fields in case they aren’t provided by the client. This blog discusses how we have implemented in the project using python decorators that complies to the DRY principle and is easy to test and maintain.

Problem

Let’s first discuss the problem at hand in detail. We use Marshmallow extensively in our project. Marshmallow is an ORM/ODM/framework-agnostic library for converting complex data types, such as objects, to and from native python data types. We use it for Validating the input data, Deserializing the input data to app-level objects and Serializing app-level objects to primitive Python types.

We can define Schema’s very easily using Marshmallow. It also provides an easy way to declare default values to the fields. Below is a sample schema declaration:

class SampleSchema(Schema):
    """
    Sample Schema declaration
    """

    class Meta:
        """
        Meta class for the Sample Schema
        """
        type_ = 'sample-schema'

    id = fields.Str(dump_only=True)
    field_without_default = fields.Str(required=True)
    field_with_default = fields.Boolean(required=True, default=False)

We have defined an id field for storing the unique ID of the Model. We have also defined two other fields. One of them named as “field_with_default” is a Boolean field and has a default value specified as False.

When a client makes a POST/PATCH request to the server, we first validate the input data sent to us by the clients against the defined schema. Marshmallow also supports schema validation but it doesn’t support using default values during deserialization of the input data. It meant that whenever the input data had a missing field, the server would throw a validation error even for a field for which the default values are defined. It was clearly wrong since if the default values are defined, we would want that value to be used for the field. This defeats the entire purpose of declaring default values at the first place.

So, we would ideally like the following behaviour from the Server:

  1. If the values are defined in the input data, use it during validation.
  2. If the value for a required field is not defined but default value has been defined in the Schema, then use that value.
  3. If no value has been defined for a required field and it doesn’t have any default value specified, then throw an error.

Solution

Marshmallow provides decorators like @pre_load and @post_load for adding pre-processing and post-processing methods. We can use them to add a method in each of the Schema classes which takes in the input data and the schema and adds default values to fields before we validate the input.

The first approach which we took was to add the following method to each of the schema classes defined in the project.

@pre_load
def patch_defaults(schema, in_data):
        data = in_data.get('data')
        if data is None or data.get('attributes') is None:
            return in_data
        attributes = data.get('attributes')
        for name, field in schema.fields.items():
            dasherized_name = dasherize(name)
            attribute = attributes.get(dasherized_name)
            if attribute is None:
                attributes[dasherized_name] = field.default
        return in_data

The method loops over all the fields defined in the schema class using schema.fields.item(). dasherize is a helper function defined in the utils class which converts underscores(_) in the variable name to dashes(-). After replacing the underscores with dashes we check if the value for the attribute is None. If it is None, then we assign it the specified default value.

Enhancing the solution

The above solution works but there is a problem. We have around 50 schemas defined in the project. Copy pasting this method 50 times would definitely violate the DRY principle. Moreover if we need to change this method in the future, we would have to do it 50 times.

One way to avoid it would be to add the patch_defaults method in a separate file and add a helper method make_object in each of the schema classes which just calls it.

@pre_load
def make_object(self, in_data):
    return patch_defaults(self, in_data)

We would still be repeating the helper method in 50 different files but since it’s sole purpose is to call the patch_defaults method, we won’t have to make changes in 50 files.

It certainly works well but we can go a step further and make it even easier. We can define a class decorator which would add the above make_object method to the class.

def use_defaults():
    """
    Decorator added to model classes which have default values specified for one of it's fields
    Adds the make_object method defined above to the class.
    :return: wrapper
    """
    def wrapper(k, *args, **kwargs):
        setattr(k, "make_object", eval("make_object", *args, **kwargs))
        return k
    return wrapper

Now we can simply add the use_defaults() decorator on the schema class and it would work.

References

Adding Modules API on Open Event Server

The Open Event Server enables organizers to manage events from concerts to conferences and meet-ups. It offers features for events with several tracks and venues. Event managers can create invitation forms for speakers and build schedules in a drag and drop interface. The event information is stored in a database. The system provides API endpoints to fetch the data, and to modify and update it.

The Open Event Server is based on JSON 1.0 Specification and hence build on top of Flask Rest Json API (for building Rest APIs) and Marshmallow (for Schema).

In this blog, we will talk about how to add API for accessing the Modules on Open Event Server. The focus is on Schema creation and it’s API creation.

Schema Creation

For the ModuleSchema, we’ll make our Schema as follows

Now, let’s try to understand this Schema.

In this feature, we are providing Admin the rights to set whether Admin wants to include tickets, payment and donation in the open event application.

  1. First of all, we will provide three fields in this Schema, which are ticket_include, payment_include and donation_include.
  2. The very first attribute ticket_include should be Boolean as we want Admin to update it whether he wants to include ticketing system in the application from default one which is False.
  3. Next attribute payment_include should be Boolean as we want Admin to update it whether he wants to include payment system in the application from default one which is False.
  4. Next attribute donation_include should be Boolean as we want Admin to update it whether he wants to include donation system in the application from default one which is False.

API Creation

For the ModuleDetail, we’ll make our API as follows

Now, let’s try to understand this API.

In this API, we are providing Admin the rights to set whether Admin wants to include tickets, payment and donation in the open event application.

  1. First of all, there is the need to know that this API has two method GET and PATCH.
  2. Decorators shows us that only Admin has permissions to access PATCH method for this API i.e. only Admins can modify the modules .
  3. before_get method shows us that this API will give first record of Modules model irrespective of the id requested by user.
  4. Schema used here is default one of Modules
  5. Hence, GET Request is accessible to all the users.

So, we saw how Module Schema and API is created to allow users to get it’s values and Admin users to modify it’s values.

Resources

Handling Click Events using Custom Binding Adapters

The Open Event Organiser Android App is the Event management app for organizers using the Open Event Platform. It is currently released in the Alpha phase on the Google Play Store here and is being actively developed by the community.

The Data Binding Library is one of the most popular libraries among the android developers. We use it extensively in the application in order to greatly simplify the UI binding logic. While trying to show the details of a speaker in the application, we wanted to list his/her social media links using Image buttons.

Upon clicking one of these buttons, the user was supposed to be directed to the link after opening the default web browser. This blog post discusses how we used custom Binding Adapters to handle click events on an Image Button by defining a custom attribute.

Defining the Binding Adapter

We defined a simple Binding Adapter for an Image button meant to handle social media links. We used “imageOnClick” as the custom attribute name for specifying the URL that will be opened once the button is clicked.

@BindingAdapter("imageOnClick")
public static void bindOnImageButtonClickListener(ImageButton imageButton, String url) {
  imageButton.setOnClickListener(view -> {
    if (url != null) {
      Context context = imageButton.getContext();
      Intent intent = new Intent(Intent.ACTION_VIEW);
      intent.setData(Uri.parse(url));
      if (intent.resolveActivity(context.getPackageManager()) != null) {
        context.startActivity(intent);
      } else {
        Toast.makeText(context, "No Web browser found", Toast.LENGTH_SHORT).show();
      }
    }
  });
}

 

The method can be named anything you want and can be placed anywhere in the project but we would recommend creating a separate class for all the Binding adapters.
The important things to take away from the above method are:

  • The method needs to be public otherwise the Data binding framework won’t be able to find it.
  • We need to pass in the view as the first parameter and the attribute value as the second parameter.

Then we simply set the familiar click listener to handle the click interaction. We use the Context from the view passed in the method as the first parameter. Then we create an Intent and set the passed in URL as the data. We make sure that the user has a browser installed on his/her android phone before we try to open the browser. We show a suitable error message if they don’t.

Using it in Layout

Using the custom attribute in the layout was extremely simple. We specified the url using the attribute “imageOnClick” and the rest was handled by the Binding Adapter and the Data binding framework.  

<ImageButton
     android:id="@+id/action_speakers_linkedin"
     android:layout_width="@dimen/spacing_larger"
     android:layout_height="match_parent"
     android:contentDescription="@string/linkedin_icon"
     app:imageOnClick="@{ speaker.linkedin }"
     android:background="#ededed"
     android:visibility="@{ (speaker.linkedin != null) ? View.VISIBLE : View.GONE }"
     app:srcCompat="@drawable/ic_linkedin_colored"/>

References

Creating Onboarding Screens for SUSI iOS

Onboarding screens are designed to introduce users to how the application works and what main functions it has, to help them understand how to use it. It can also be helpful for developers who intend to extend the current project.

When you enter in the SUSI iOS app for the first time, you see the onboarding screen displaying information about SUSI iOS features. SUSI iOS is using Material design so the UI of Onboarding screens are following the Material design.

There are four onboarding screens:

  1. Login (Showing the login features of SUSI iOS) – Login to the app using SUSI.AI account or else signup to create a new account or just skip login.
  2. Chat Interface (Showing the chat screen of SUSI iOS) – Interact with SUSI.AI asking queries. Use microphone button for voice interaction.
  3. SUSI Skill (Showing SUSI Skills features) – Browse and try your favorite SUSI.AI Skill.
  4. Chat Settings (SUSI iOS Chat Settings) – Personalize your chat settings for the better experience.

Onboarding Screens User Interface

 

There are three important components of every onboarding screen:

  1. Title – Title of the screen (Login, Chat Interface etc).
  2. Image – Showing the visual presentation of SUSI iOS features.
  3. Description – Small descriptions of features.

Onboarding screen user control:

  • Pagination – Give the ability to the user to go next and previous onboarding screen.
  • Swiping – Left and Right swipe are implemented to enable the user to go to next and previous onboarding screen.
  • Skip Button – Enable users to skip the onboarding instructions and go directly to the login screen.

Implementation of Onboarding Screens:

  • Initializing PaperOnboarding:
override func viewDidLoad() {
super.viewDidLoad()

UIApplication.shared.statusBarStyle = .lightContent
view.accessibilityIdentifier = "onboardingView"

setupPaperOnboardingView()
skipButton.isHidden = false
bottomLoginSkipButton.isHidden = true
view.bringSubview(toFront: skipButton)
view.bringSubview(toFront: bottomLoginSkipButton)
}

private func setupPaperOnboardingView() {
let onboarding = PaperOnboarding()
onboarding.delegate = self
onboarding.dataSource = self
onboarding.translatesAutoresizingMaskIntoConstraints = false
view.addSubview(onboarding)

// Add constraints
for attribute: NSLayoutAttribute in [.left, .right, .top, .bottom] {
let constraint = NSLayoutConstraint(item: onboarding,
attribute: attribute,
relatedBy: .equal,
toItem: view,
attribute: attribute,
multiplier: 1,
constant: 0)
view.addConstraint(constraint)
}
}

 

  • Adding content using dataSource methods:

    let items = [
    OnboardingItemInfo(informationImage: Asset.login.image,
    title: ControllerConstants.Onboarding.login,
    description: ControllerConstants.Onboarding.loginDescription,
    pageIcon: Asset.pageIcon.image,
    color: UIColor.skillOnboardingColor(),
    titleColor: UIColor.white, descriptionColor: UIColor.white, titleFont: titleFont, descriptionFont: descriptionFont),OnboardingItemInfo(informationImage: Asset.chat.image,
    title: ControllerConstants.Onboarding.chatInterface,
    description: ControllerConstants.Onboarding.chatInterfaceDescription,
    pageIcon: Asset.pageIcon.image,
    color: UIColor.chatOnboardingColor(),
    titleColor: UIColor.white, descriptionColor: UIColor.white, titleFont: titleFont, descriptionFont: descriptionFont),OnboardingItemInfo(informationImage: Asset.skill.image,
    title: ControllerConstants.Onboarding.skillListing,
    description: ControllerConstants.Onboarding.skillListingDescription,
    pageIcon: Asset.pageIcon.image,
    color: UIColor.loginOnboardingColor(),
    titleColor: UIColor.white, descriptionColor: UIColor.white, titleFont: titleFont, descriptionFont: descriptionFont),OnboardingItemInfo(informationImage: Asset.skillSettings.image,
    title: ControllerConstants.Onboarding.chatSettings,
    description: ControllerConstants.Onboarding.chatSettingsDescription,
    pageIcon: Asset.pageIcon.image,
    color: UIColor.iOSBlue(),
    titleColor: UIColor.white, descriptionColor: UIColor.white, titleFont: titleFont, descriptionFont: descriptionFont)]
    extension OnboardingViewController: PaperOnboardingDelegate, PaperOnboardingDataSource {
    func onboardingItemsCount() -> Int {
    return items.count
    }
    
    func onboardingItem(at index: Int) -> OnboardingItemInfo {
    return items[index]
    }
    }
    

     

  • Hiding/Showing Skip Buttons:
    func onboardingWillTransitonToIndex(_ index: Int) {
    skipButton.isHidden = index == 3 ? true : false
    bottomLoginSkipButton.isHidden = index == 3 ? false : true
    }
    

Resources:

Adding multiple email support for users on Open Event Server

The Open Event Server enables organizers to manage events from concerts to conferences and meet-ups. It offers features for events with several tracks and venues. Event managers can create invitation forms for speakers and build schedules in a drag and drop interface. The event information is stored in a database. The system provides API endpoints to fetch the data, and to modify and update it.

The Open Event Server is based on JSON 1.0 Specification and hence build on top of Flask Rest Json API (for building Rest APIs) and Marshmallow (for Schema).

In this blog, we will talk about how to add support of multiple emails for a user in Open Event Server. The focus is on model and schema creation for this support.

Model Creation

For the UserEmail, we’ll make our model as follows

from app.models import db

class UserEmail(db.Model):
“””user email model class”””
__tablename__ = ‘user_emails’
id = db.Column(db.Integer, primary_key=True)
email = db.Column(db.String(120), unique=True, nullable=False)
verified = db.Column(db.Boolean, default=False)
user_id = db.Column(db.Integer, db.ForeignKey(‘users.id’, ondelete=’CASCADE’))
user = db.relationship(“User”, backref=”emails”, foreign_keys=[user_id])

def __init__(self, email=None, user_id=None):
self.email = email
self.user_id = user_id

def __str__(self):
return ‘User:’ + unicode(self.user_id).encode(‘utf-8’) + ‘ email: ‘ + unicode(self.email).encode(‘utf-8’)

def __unicode__(self):
return unicode(self.id)

Now, let’s try to understand the attributes of this model.

  1. id is most important Column required in every model to set it as primary key and to uniquely identify an UserEmail object.
  2. email is that attribute which is required hence should be unique and non-nullable.
  3. Verified attribute is used to check whether a email is verified or not (thus should be boolean)
  4. User_id is the attribute which specifies id of the user whose email is contained in the UserEmail object.
  5. Finally, a relationship with the user of id user_id and these emails (associated with the User.id == user_id) will be stored in the attribute emails in User Model.

Schema Creation

For the model UserEmail, we’ll make our schema UserEmailSchema as follows

from marshmallow_jsonapi import fields
from marshmallow_jsonapi.flask import Schema, Relationshipfrom app.api.helpers.utilities import dasherizeclass UserEmailSchema(Schema):
“””   API Schema for user email Model   “””class Meta:
“””  Meta class for user email API schema  “””
type_ = ‘user-emails’
self_view = ‘v1.user_emails_detail’
self_view_kwargs = {‘id’: ‘<id>’}
inflect = dasherize

id = fields.Str(dump_only=True)
email = fields.Email(allow_none=False)
user_id = fields.Integer(allow_none=False)
user = Relationship(attribute=’user’,
self_view=’v1.user_email’,
self_view_kwargs={‘id’: ‘<id>’},
related_view=’v1.user_detail’,
related_view_kwargs={‘user_id’: ‘<id>’},
schema=’UserSchema’,
type_=’user’
)

  • Marshmallow-jsonapi provides a simple way to produce JSON API-compliant data in any Python web framework.

Now, let’s try to understand the schema UserEmailSchema

  1. id : Same as in model id is used as uniquely identify an UserEmail object.
  2. email : Same as in model email is required thus allow_none is set to False.
  3. User_id : user_id is the id of user whose email is contained in a UserEmailSchema object.
  4. User : It tells whole attributes of the user to which this email belongs to.

So, we saw how to add multiple email support for users on Open Event Server. We just required to create a model and its schema to add this feature. Similarly, to add support for any database model in the project, we need to create Model and Schema with all the attributes as specified in the model too. This Schema creation is done with guidelines of JSONAPI 1.0 Specification using Marshmallow.

Resources

The Road to Success in Google Summer of Code 2017

It’s the best time when GCI students can get the overview experience of GSoC and all the aspiring participant can get themselves into different projects of FOSSASIA.

I’m a Junior year undergraduate student pursuing B.Tech in Electrical Engineering from Indian Institute of Technology Patna. This summer, I spent coding in Google Summer of Code with FOSSASIA organization. It feels great to be an open-source enthusiast, and Google as a sponsor make it as icing on the cake. People can learn new things here and meet new people.

I came to know about GSoC through my senior colleagues who got selected in GSoC in the year 2016. It was around September 2016 and I was in 2nd year of my college. At that time, last year, result of GSoC was declared.

What is GSoC?

Consider GSoC as a big bowl which has lots of small balls and those small balls are open-source organizations. Google basically acts as a sponsor for the open-source organizations. A timeline is proposed according to the applied organization and then student select their favorite organization and start to contribute to it. Believe me, it’s not only computer science branch specific, anyone can take part in it and there is no minimum CPI requirement. I consider myself to be one of the examples who have an electrical branch with not so good academic performance yet successfully being part of GSoC 2017.

How to select an organization?

This is the most important step and it takes time. I wandered around 100 organizations to find where my interest actually lies. But now, I’ll describe how to sort this and find your organization a little quicker. Take a pen and paper (kindly don’t use notepad of pc) and write down your field of interest in computer science. Number every point in decreasing order of your interest. Then for each respective field write down its basic pre-requisites. Visit GSoC website, go to organization tab and there is a slide for searching working field of the organization. Select only one organization, dig out its website, see the previous project and its application. If nothing fits you, repeat the same with another organization. And if that organization interests you, then look for a project of that organization. First of all, look at that application of the project, and give that application a try and must give a feedback to the organization. Then try to find that what languages, modules, etc that project used to work and how the project works. Don’t worry if nothing goes into your mind. Find out the developers mailing list, their chat channel, their code base area. And ask developers out there for help.

First Love It:

Open-Source, it’s a different world which exists on Earth. All organizations are open-source and all their codes are open and free to view. Find things that interests you the most and start to love the work. If you don’t understand a code, learn things by doing and asking. Most of the times we don’t get favorable responses, in such times we need to carry on and have patience for the best to happen.

My Favourite part:

GSoC has been my dream since the day I came to know about it. It’s only through this that one gets a chance to explore open-source softwares, and organizations get a chance to hire on board developers. This is the great initiative taken by Google which brings hope for the developers to increase the use of open-source. This is one of the ways through which one can look into the codes of the developers and help them out and even also get helped.

GSoC is the platform through which one can implement lots of new things, meet new people, develop new softwares and see the world around in a different way. That’s what happened with me, it’s just at the end of the first phase, my love towards open-source increased exponentially. Now I see every problem in my life as a way to solve it through the open-source. Rather it’s part of arranging an event or designing an invitation, I am encouraged to use open-source tools to help me out. It becomes very easy to distribute data and convey information through open-source, so the people can reach to you much easier.

You always see a thing according to your perspective and it’s always the best but the open-source gives it a view through the perspective of the world and gets the best from them through a compilation of all the sources. One can give ideas, their views, find something that other can’t even see and increase its karma through contribution. And all these things have been made possible through GOOGLE only. I became such that I can donate the rest of my life working for open-source. GSoC is responsible for including the open-source contribution in my daily life. It made me feel really bad if my Github profile page has 0 contributions at the end of the day. Open Source opens door to another world.

Challenging part:

To conclude, I would say that GSoC made me love the challenge. I became such that the things that come easily to me don’t taste good to me at all. Specifically, GSoC’s most challenging part is to get into it that is to get selected. I still can’t believe that I was selected. Now onwards it’s just fun and learning. Each and every day, I encountered several issues, bugs, etc but just before going to bed at night, there were things which collectively made me feel that whether the bug has been solved or not, but I was able to break the upper most covering of that conch shell. And such things increases the motivation and light up the enthusiasm to tackle the problem. Open-Source not only taught me to control different snapshots of software but also of time. I learn to manage different works of day efficiently and it includes the contribution in open-source as part of my daily life.

Advice to students:

The only problem new developers have is to get started. I’ll advise them to close their eyes and dive into it without thinking whether they would be able to complete this task or not. Believe me, you will gradually find that whether the task is completed or not but you are much above the condition than you were at the time of beginning the task.

Just learn by doing the things.

Make mistakes and enlist them as “things that will not work” so one may read it and avoid it.

GSoC Project link: https://summerofcode.withgoogle.com/projects/#5560333780385792
Final Code Submission: https://gist.github.com/meets2tarun/270f151d539298831ce542be5f733c82

Implementing Direct URL in loklak Media Wall

Direct URL is a web address which redirects the user to the preset customized media wall so that the media wall can directly be used to be displayed on the screen. Loklak media wall provides direct URL which has information related to customizations set by the user included in the web address. These customizations, as the query parameters are detected when the page is initialized and actions are dispatched to make changes in the state properties, and hence, the UI properties and the expected behaviour of media wall.

In this blog, I would be explaining how I implemented direct URL in loklak media wall and how customizations are detected to build on initialization of component, a customized media wall.

Flow Chart

Working

Media Wall Direct URL effect

This effect detects when the WALL_GENERATE_DIRECT_URL action is dispatched and creates a direct URL string from all the customization state properties and dispatches a side action WallShortenDirectUrlAction() and stores direct URL string as a state property. For this, we need to get individual wall customization state properties and create an object for it and supply it as a parameter to the generateDirectUrl() function. Direct URL string is returned from the function and now, the action is dispatched to store this string as a state property.

@Effect()
generateDirectUrl$: Observable<Action>
= this.actions$
.ofType(mediaWallDirectUrlAction.ActionTypes.WALL_GENERATE_DIRECT_URL)
.withLatestFrom(this.store$)
.map(([action, state]) => {
return {
query: state.mediaWallQuery.query,
.
.
.
wallBackground: state.mediaWallCustom.wallBackground
};
})
.map(queryObject => {
const configSet = {
queryString: queryObject.query.displayString,
.
.
.
wallBackgroundColor: queryObject.wallBackground.backgroundColor
}
const shortenedUrl = generateDirectUrl(configSet);
return new mediaWallDirectUrlAction.WallShortenDirectUrlAction(shortenedUrl);
});

Generate Direct URL function

This function generates Direct URL string from all the current customization options value. Now,  keys of the object are separated out and for each element of the object, it checks if there is some current value for the elements and it then first parses the value of the element into URI format and then, adds it to the direct URL string. In such a way, we are creating a direct URL string with these customizations provided as the query parameters.

export function generateDirectUrl(customization: any): string {
const shortenedUrl = ;const activeFilterArray: string[] = new Array<string>();
let qs = ;
Object.keys(customization).forEach(config => {
if (customization[config] !== undefined && customization[config] !== null) {
if (config !== ‘blockedUser’ && config !== ‘hiddenFeedId’) {
qs += `${config}=${encodeURIComponent(customization[config])}&`;
}
else {
if (customization[config].length > 0) {
qs += `${config}= ${encodeURIComponent(customization[config].join(‘,’))}&`;
}
}
}
});
qs += `ref=share`;
return qs;
}

Creating a customized media wall

Whenever the user searches for the URL link on the web, a customized media wall must be created on initialization. The media wall component detects and subscribes to the URL query parameters using the queryParams API of the ActivatedRoute. Now, the values are parsed to a required format of payload and the respective actions are dispatched according to the value of the parameters. Now, when all the actions are dispatched, state properties changes accordingly. This creates a unidirectional flow of the state properties from the URL parameters to the template. Now, the state properties that are supplied to the template are detected and a customized media wall is created.

private queryFromURL(): void {
this.__subscriptions__.push(
this.route.queryParams
.subscribe((params: Params) => {
const config = {
queryString: params[‘queryString’] || ,
imageFilter: params[‘imageFilter’] || ,
profanityCheck: params[‘profanityCheck’] || ,
removeDuplicate: params[‘removeDuplicate’] || ,
wallHeaderBackgroundColor: params[‘wallHeaderBackgroundColor’] || ,
wallCardBackgroundColor: params[‘wallCardBackgroundColor’] || ,
wallBackgroundColor: params[‘wallBackgroundColor’] ||
}
this.setConfig(config);
})
);
}public setConfig(configSet: any) {
if (configSet[‘displayHeader’]) {
const isTrueSet = (configSet[‘displayHeader’] === ‘true’);
this.store.dispatch(new mediaWallDesignAction.WallDisplayHeaderAction(isTrueSet));
}
.
.
if (configSet[‘queryString’] || configSet[‘imageFilter’] || configSet[‘location’]) {
if (configSet[‘location’] === ‘null’) {
configSet[‘location’] = null;
}
const isTrueSet = (configSet[‘imageFilter’] === ‘true’);
const query = {
displayString: configSet[‘queryString’],
queryString: ,
routerString: configSet[‘queryString’],
filter: {
video: false,
image: isTrueSet
},
location: configSet[‘location’],
timeBound: {
since: null,
until: null
},
from: false
}
this.store.dispatch(new mediaWallAction.WallQueryChangeAction(query));
}
}

Now, the state properties are rendered accordingly and a customized media wall is created. This saves a lot of effort by the user to change the customization options whenever uses the loklak media wall.

Reference

Open Event API Server: Implementing FAQ Types

In the Open Event Server, there was a long standing request of the users to enable the event organisers to create a FAQ section.

The API of the FAQ section was implemented subsequently. The FAQ API allowed the user to specify the following request schema

{
 "data": {
   "type": "faq",
   "relationships": {
     "event": {
       "data": {
         "type": "event",
         "id": "1"
       }
     }
   },
   "attributes": {
     "question": "Sample Question",
     "answer": "Sample Answer"
   }
 }
}

 

But, what if the user wanted to group certain questions under a specific category. There was no solution in the FAQ API for that. So a new API, FAQ-Types was created.

Why make a separate API for it?

Another question that arose while designing the FAQ-Types API was whether it was necessary to add a separate API for it or not. Consider that a type attribute was simply added to the FAQ API itself. It would mean the client would have to specify the type of the FAQ record every time a new record is being created for the same. This would mean trusting that the user will always enter the same spelling for questions falling under the same type. The user cannot be trusted on this front. Thus the separate API made sure that the types remain controlled and multiple entries for the same type are not there.

Helps in handling large number of records:

Another concern was what if there were a large number of FAQ records under the same FAQ-Type. Entering the type for each of those questions would be cumbersome for the user. The FAQ-Type would also overcome this problem

Following is the request schema for the FAQ-Types API

{
 "data": {
   "attributes": {
     "name": "abc"
   },
   "type": "faq-type",
   "relationships": {
     "event": {
       "data": {
         "id": "1",
         "type": "event"
       }
     }
   }
 }
}

 

Additionally:

  • FAQ to FAQ-type is a many to one relation.
  • A single FAQ can only belong to one Type
  • The FAQ-type relationship will be optional, if the user wants different sections, he/she can add it ,if not, it’s the user’s choice.

Related links

Open Event Server: Getting The Identity From The Expired JWT Token In Flask-JWT

The Open Event Server uses JWT based authentication, where JWT stands for JSON Web Token. JSON Web Tokens are an open industry standard RFC 7519 method for representing claims securely between two parties. [source: https://jwt.io/]

Flask-JWT is being used for the JWT-based authentication in the project. Flask-JWT makes it easy to use JWT based authentication in flask, while on its core it still used PyJWT.

To get the identity when a JWT token is present in the request’s Authentication header , the current_identity proxy of Flask-JWT can be used as follows:

@app.route('/example')
@jwt_required()
def example():
   return '%s' % current_identity

 

Note that it will only be set in the context of function decorated by jwt_required(). The problem with the current_identity proxy when using jwt_required is that the token has to be active, the identity of an expired token cannot be fetched by this function.

So why not write a function on our own to do the same. A JWT token is divided into three segments. JSON Web Tokens consist of three parts separated by dots (.), which are:

  • Header
  • Payload
  • Signature

The first step would be to get the payload, that can be done as follows:

token_second_segment = _default_request_handler().split('.')[1]

 

The payload obtained above would still be in form of JSON, it can be converted into a dict as follows:

payload = json.loads(token_second_segment.decode('base64'))

 

The identity can now be found in the payload as payload[‘identity’]. We can get the actual user from the paylaod as follows:

def jwt_identity(payload):
   """
   Jwt helper function
   :param payload:
   :return:
   """
   return User.query.get(payload['identity'])

 

Our final function will now be something like:

def get_identity():
   """
   To be used only if identity for expired tokens is required, otherwise use current_identity from flask_jwt
   :return:
   """
   token_second_segment = _default_request_handler().split('.')[1]
   missing_padding = len(token_second_segment) % 4
   payload = json.loads(token_second_segment.decode('base64'))
   user = jwt_identity(payload)
   return user

 

But after using this function for sometime, you will notice that for certain tokens, the system will raise an error saying that the JWT token is missing padding. The JWT payload is base64 encoded, and it requires the payload string to be a multiple of four. If the string is not a multiple of four, the remaining spaces can pe padded with extra =(equal to) signs. And since Python 2.7’s .decode doesn’t do that by default, we can accomplish that as follows:

missing_padding = len(token_second_segment) % 4

# ensures the string is correctly padded to be a multiple of 4
if missing_padding != 0:
   token_second_segment += b'=' * (4 - missing_padding)

 

Related links:

Adding Tweet Streaming Feature in World Mood Tracker loklak App

The World Mood Tracker was added to loklak apps with the feature to display aggregated data from the emotion classifier of loklak server. The next step in the app was adding the feature to display the stream of Tweets from a country as they are discovered by loklak. With the addition of stream servlet in loklak, it was possible to utilise it in this app.

In this blog post, I will be discussing the steps taken while adding to introduce this feature in World Mood Tracker app.

Props for WorldMap component

The WorldMap component holds the view for the map displayed in the app. This is where API calls to classifier endpoint are made and results are displayed on the map. In order to display tweets on clicking a country, we need to define react props so that methods from higher level components can be called.

In order to enable props, we need to change the constructor for the component –

export default class WorldMap extends React.Component {
    constructor(props) {
        super(props);
        ...
    }
    ...
}

[SOURCE]

We can now pass the method from parent component to enable streaming and other components can close the stream by using props in them –

export default class WorldMoodTracker extends React.Component {
    ...
    showStream(countryName, countryCode) {
        /* Do something to enable streaming component */
        ...
    }
 
    render() {
        return (
             ...
                <WorldMap showStream={this.showStream}/>
             ...
        )
    }
}

[SOURCE]

Defining Actions on Clicking Country Map

As mentioned in an earlier blog post, World Mood Tracker uses Datamaps to visualize data on a map. In order to trigger a piece of code on clicking a country, we can use the “done” method of the Datamaps instance. This is where we use the props passed earlier –

done: function(datamap) {
    datamap.svg.selectAll('.datamaps-subunit').on('click', function (geography) {
        props.showStream(geography.properties.name, reverseCountryCode(geography.id));
    })
}

[SOURCE]

The name and ID for the country will be used to display name and make API call to stream endpoint respectively.

The StreamOverlay Component

The StreamOverlay components hold all the utilities to display the stream of Tweets from loklak. This component is used from its parent components whose state holds info about displaying this component –

export default class WorldMoodTracker extends React.Component {
    ...
    getStreamOverlay() {
        if (this.state.enabled) {
            return (<StreamOverlay
                show={true} channel={this.state.channel}
                country={this.state.country} onClose={this.onOverlayClose}/>);
        }
    }

    render() {
        return (
            ...
                {this.getStreamOverlay()}
            ...
        )
    }
}

[SOURCE]

The corresponding props passed are used to render the component and connect to the stream from loklak server.

Creating Overlay Modal

On clicking the map, an overlay is shown. To display this overlay, react-overlays is used. The Modal component offered by the packages provides a very simple interface to define the design and interface of the component, including style, onclose hook, etc.

import {Modal} from 'react-overlays';

<Modal aria-labelledby='modal-label'
    style={modalStyle}
    backdropStyle={backdropStyle}
    show={true}
    onHide={this.close}>
    <div style={dialogStyle()}>
        ...
    </div>
</Modal>

[SOURCE]

It must be noted that modalStyle and backdropStyle are React style objects.

Dialog Style

The dialog style is defined to provide some space at the top, clicking where, the overlay is closed. To do this, vertical height units are used –

const dialogStyle = function () {
    return {
        position: 'absolute',
        width: '100%',
        top: '5vh',
        height: '95vh',
        padding: 20
        ...
    };
};

[SOURCE]

Connecting to loklak Tweet Stream

loklak sends Server Sent Events to clients connected to it. To utilise this stream, we can use the natively supported EventSource object. Event stream is started with the render method of the StreamOverlay component –

render () {
    this.startEventSource(this.props.channel);
    ...
}

[SOURCE]

This channel is used to connect to twitter/country/<country-ID> channel on the stream and then this can be passed to EventStream constructor. On receiving a message, a list of Tweets is appended and later rendered in the view –

startEventSource(country) {
    let channel = 'twitter%2Fcountry%2F' + country;
    if (this.eventSource) {
        return;
    }
    this.eventSource = new EventSource(host + '/api/stream.json?channel=' + channel);
    this.eventSource.onmessage = (event) => {
        let json = JSON.parse(event.data);
        this.state.tweets.push(json);
        if (this.state.tweets.length > 250) {
            this.state.tweets.shift();
        }
        this.setState(this.state);
    };
}

[SOURCE]

The size of the list is restricted to 250 here, so when a newer Tweet comes in, the oldest one is chopped off. And thanks to fast DOM actions in React, the rendering doesn’t take much time.

Rendering Tweets

The Tweets are displayed as simple cards on which user can click to open it on Twitter in a new tab. It contains basic information about the Tweet – screen name and Tweet text. Images are not rendered as it would make no sense to load them when Tweets are coming at a high rate.

function getTweetHtml(json) {
    return (
        <div style={{padding: '5px', borderRadius: '3px', border: '1px solid black', margin: '10px'}}>
            <a href={json.link} target="_blank">
            <div style={{marginBottom: '5px'}}>
                <b>@{json['screen_name']}</b>
            </div>
            <div style={{overflowX: 'hidden'}}>{json['text']}</div>
            </a>
        </div>
    )
}

[SOURCE]

They are rendered using a simple map in the render method of StreamOverlay component –

<div className={styles.container} style={{'height': '100%', 'overflowY': 'auto',
    'overflowX': 'hidden', maxWidth: '100%'}}>
    {this.state.tweets.reverse().map(getTweetHtml)}
</div>

[SOURCE]

Closing Overlay

With the previous setup in place, we can now see Tweets from the loklak backend as they arrive. But the problem is that we will still be connected to the stream when we click-close the modal. Also, we would need to close the overlay from the parent component in order to stop rendering it.

We can use the onclose method for the Modal here –

close() {
    if (this.eventSource) {
        this.eventSource.close();
        this.eventSource = null;
    }
    this.props.onClose();
}

[SOURCE]

Here, props.onClose() disables rendering of StreamOverlay in the parent component.

Conclusion

In this blog post, I explained how the flow of props are used in the World Mood Tracker app to turn on and off the streaming in the overlay defined using react-overlays. This feature shows a basic setup for using the newly introduced stream API in loklak.

The motivation of such application was taken from emojitracker by mroth as mentioned in fossasia/labs.fossasia.org#136. The changes were proposed in fossasia/apps.loklak.org#315 by @singhpratyush (me).

The app can be accessed live at https://singhpratyush.github.io/world-mood-tracker/index.html.

Resources