Decorators in Open Event API Server

One of the interesting features of Python is the decorator. Decorators dynamically alter the functionality of a function, method, or class without having to directly use subclasses or change the source code of the function being decorated.

Open Event API Server makes use of decorator in various ways. The ability to wrap a function and run the decorator(s) before executing that function solves various purpose in Python. Earlier before decoupling of Orga Server into API Server and Frontend, decorators were being used for routes, permissions, validations and more.

Now, The API Server mainly uses decorators for:

  • Permissions
  • Filtering on the basis of view_kwargs or injecting something into view_kwargs
  • Validations

We will discuss here first two because validations are simple and we are using them out of the box from marshmall-api

The second one is custom implementation made to ensure no separate generic helpers are called which can add additional database queries and call overheads in some scenarios.

Permissions Using Decorators

Flask-rest-jsonapi provides an easy way to add decorators to Resources. This is as easy as defining this into Resource class

  1. decorators = (some_decorator, )

On working to event role decorators to use here, I need to follow only these 3 rules

  • If the user is admin or super admin, he/she has full access to all event roles
  • Then check the user’s role for the given event
  • Returns the requested resource’s view if authorized unless returns Forbidden Error response.

One of the examples is:

def is_organizer(view, view_args, view_kwargs, *args, **kwargs):
  user = current_identity
 
  if user.is_staff:
      return view(*view_args, **view_kwargs)
 
  if not user.is_organizer(kwargs['event_id']):
      return ForbiddenError({'source': ''}, 'Organizer access is required').respond()
 
  return view(*view_args, **view_kwargs)

From above example, it is clear that it is following those three guidelines

Filtering on the basis of view_kwargs or injecting something into view_kwargs

This is the main point to discuss, starting from a simple scenario where we have to show different events list for different users. Before decoupling API server, we had two different routes, one served the public events listing on the basis of event identifier and other to show events to the event admins and managers, listing only their own events to their panel.

In API server there are no two different routes for this. We manage this with a single route and served both cases using the decorator. This below is the magic decorator function for this purpose

def accessible_role_based_events(view, view_args, view_kwargs, *args, **kwargs):
  if 'POST' in request.method or 'withRole' in request.args:
      _jwt_required(app.config['JWT_DEFAULT_REALM'])
      user = current_identity
      if 'GET' in request.method and user.is_staff:
          return view(*view_args, **view_kwargs)
      view_kwargs['user_id'] = user.id
  return view(*view_args, **view_kwargs)

It works simply by looking for ‘withRole’ in requests and make a decision to include user_idinto kwargs as per these rules

  1. If the request is POST then it has to be associated with some user so add the user_id
  2. If the request is GET and ‘withRole’ GET parameter is present in URL then yes add the user_id. This way user is asking to list the events in which I have some admin or manager role
  3. If the request is GET and ‘withRole’ is defined but the logged in user is admin or super_adminthen there is no need add user_id since staff can see all events in admin panel
  4. The last one is GET and no ‘withRole’ parameter is defined therefore ignores and continues the same request to list all events.

The next work is of query method of EventList Resource

if view_kwargs.get('user_id'):
          if 'GET' in request.method:
              query_ = query_.join(Event.roles).filter_by(user_id=view_kwargs['user_id']) \
                  .join(UsersEventsRoles.role).filter(Role.name != ATTENDEE)

This query joins the UsersEventsRoles model whenever user_id is defined. Thus giving role-based events only.

The next interesting part is the Implementation of permission manager to ensure permission decorators doesn’t break at any point. We will see it in next post.

References:

https://wiki.python.org/moin/PythonDecorators

Continue ReadingDecorators in Open Event API Server

Relationships and its usage in Open Event Orga Server

JSON API is a specification for writing RESTFul APIs (CRUD interfaces). This specification basically sets the standard for a client to request the resources and how a server is supposed to response minimizing the redundancy and number of requests.

If we look at the general implementation of RESTful APIs, we see that we are working on creating every endpoint manually, there are no relations. Sometimes different endpoints are being created for some slightly different business logic than other. We solve this purpose specifically the relationships using JSON API spec.

Features of JSON API

Apart from CRUD interface, JSON-API-Spec provides

  • Fetching Resources
  • Fetching Relationships
  • Inclusion of Related Resources
  • Sparse Fieldsets
  • Sorting
  • Pagination
  • Filtering

For Open Event API Server we need these below

  • Proper relationship definitions
  • Sorting
  • Filtering
  • Pagination

So JSON-API spec is a good choice for us at Orga Server since it solves our every basic need.

Overview of Changes

Firstly the main task was shifting to the library flask-rest-jsonapi because this library stands to our four needs in API. The changes included:

  • ensuring JSON-API spec in our requests and responses (although the most of the work is done by the library)
  • Reusing the current implementation of JWT authorization.
  • To locate the new API to /v1. Since Orga server is going to be API server with Open Event system following the API-centric approach, therefore, there is no need to have /api/v1
  • Now out timestamps in response and request will be timezone aware thus following ISO 8601 with timezone information (Eg. 2017-05-22T09:12:44+00:00)

Media type to use: application/vnd.api+json

A Relationship in JSON API

To begin with APIs, I started working on Sessions API of Orga server and the relation of a session with the event was represented as one of the attribute of Schema of the Session API like this below,

event = Relationship(attribute='event',
                        self_view='v1.session_event',
                        self_view_kwargs={'id': '<id>'},
                        related_view='v1.event_detail',
                        related_view_kwargs={'session_id': '<id>'},
                        schema='EventSchema',
                        type_='event')


  • attribute: name of the attribute with which this will be referenced in response API
  • self_view: A view name which represents the view of this relationship. This is a relationship endpoint of sessions API.
  • self_view_kwargs: view_kwargs for self_view, this is used to provide ID of the specific record to the relationship endpoint.
  • related_view: An endpoints to the related API/Object. Here the related object is ‘event’ so I have provided the endpoint to get the event detail.
  • related_view_kwargs: Here we can provide kwargs to the related object’s endpoint. Here we are sending the value of <Session_id> URL parameter on the related endpoint by mapping it with “id” of the current session object.
  • Schema: this is the schema of the related object. Since we have related object is event, therefore, added EventSchema of it.
  • type_: this is the type of related object which is event here.

After defining them, the magic here is no need to define and inject the relationship endpoints in the responses. We just need to add one route to v1.event_detail and we have relationship ready.

To make this work, I added these on routes file:

  • ‘/sessions/<int:session_id>/event’ to the v1.event_detail
  • api.route(SessionRelationship, ‘session_event’,
             ‘/sessions/<int:id>/relationships/event’)

And we have Relationship ready as Session -> Event in the API ready. We can use these relationships to Get the relationship Object(s), Updated them or Delete them. This helps Orga Server is a very efficient scale since many of our endpoints are related with events directly so instead of separately defining relationships we are able to do this with the help of JSON API and flask-rest-jsonapi

An Example Response

HTTP/1.1 200 OK
Content-Type: application/vnd.api+json

{
  "data": {
    "relationships": {
      "event": {
        "links": {
          "self": "/v1/speakers-calls/3/relationships/event",
          "related": "/v1/speakers-calls/3/event"
        }
      }
    },
    "attributes": {
      "announcement": "Google",
      "ends-at": "2023-05-30T09:30:10+00:00",
      "hash": null,
      "starts-at": "2022-05-30T09:30:10+00:00",
      "privacy": "public"
    },
    "type": "speakers-call",
    "id": "3",
    "links": {
      "self": "/v1/speakers-calls/3"
    }
  },
  "jsonapi": {
    "version": "1.0"
  },
  "links": {
    "self": "/v1/speakers-calls/3"
  }
}

Above example shows the relationships in the response object. We can directly check in the application using these APIs that to which type of objects this object is related with and endpoints to get related data.

Next steps in the implementation are Docs for APIs, permissions implementations to secure the endpoints and setting up unit testing of the endpoints which will be discussed in next posts.

Continue ReadingRelationships and its usage in Open Event Orga Server

Open Event Server: Dealing with environment variables

FOSSASIA‘s Open Event Server uses some environment variables to maintain different configuration for different deployment. However one con of it was, for local development, developers have to run many export commands.

It all started with the “It is not convenient, we humans are lazy.” phrase. A recent suggestion was to read this environment variables with files rather than running commands every time.
This few example environment variable commands were:

export DATABASE_URL=postgresql://open_event_user:test@127.0.0.1:5432/test

and

export INTEGRATE_SOCKETIO=false

 

Open Event Server now uses .env file and envparse to handle this.

The changes were:

  1. We store the variables in .env file, let it get ignored by git, and
  2. Use envparse to parse the file.

What is envpase:
       envparse is a simple utility to parse environment variables. It aims to eliminate duplicated, often inconsistent parsing code and instead provide a single, easy-to-use wrapper.

So instead of reading the string from the command line everytime:

export DATABASE_URL=postgresql://open_event_user:password@127.0.0.1:5432/open_event_db

We read it from the file:

To read from command line we had:

SQLALCHEMY_DATABASE_URI = os.environ.get('DATABASE_URL', None)

Which now was parsed using envparse as:

SQLALCHEMY_DATABASE_URI =  env('DATABASE_URL', default=None)

In the above example, DATABASE_URL is a string and we store this string in SQLALCHEMY, setting the default by the default parameter(None in this case).

The type of environment variable is originally string, we can simply use env to parse these strings. For simplicity and understanding we can also use env.str. For different types, we can simply type cast them like env.bool (for boolean), env.int (for integer) etc. One good advantage of this envparse feature is the direct use of these type cast rather than checking for strings (i.e. reading the string from command line).

For example, earlier we had something like this:

socketio_integration = os.environ.get('INTEGRATE_SOCKETIO’)
if socketio_integration == true’:
INTEGRATE_SOCKETIO =True

Using envparse, we can have something like this:

INTEGRATE_SOCKETIO = env.bool('INTEGRATE_SOCKETIO’, default=False)

This helps in parsing as boolean instead of string. So, if we have INTEGRATE_SOCKETIO=true in .env file, we can write INTEGRATE_SOCKETIO = env.bool(‘INTEGRATE_SOCKETIO’, default=False). This automatically converts the true to boolean True in python.

In order to use envparse we install envparse with:

$ pip install envparse


In the case of
Open Event Server project, you only have to install the requirements.txt and copy the .env.example already mentioned in installation doc.

This is how we finally remove the headache of writing big exports again and again and fairly helps beginners to start off with Open Event Server  project much more easily and quickly.

Continue ReadingOpen Event Server: Dealing with environment variables

Permission Dependent Schema for Admin Settings in Open Event Server

For implementing the next version of the API in Open Event, the schema is a very important thing. It tells you exactly what all information you need to send in the body and how the response will look. In flask-rest-jsonapi, we usually mention a schema for an API which is then used for validating requests and sending response. Using decorators, we restrict who all can create, edit or get responses from a particular API endpoint. However, a scenario may so arise that you need to show data to users at different permissions level, but the amount of data shown significantly varies with the permission.

For example, for the settings API in our case. There are few informations like the app name, app tagline that we want to be available to users at all permission levels. However, informations such as aws secret key, or mailing secret keys or any other secret key, we want that to be available only to the admin and super admin. And the responses should be such that users at different permission level should feel that whatever information shown to them is complete and not missing.

So, what we do is we create different schemas, in our case 2 different schemas. Depending on the permission of the user, we show them a particular schema. In our case, the two schemas are SettingSchemaAdmin and SettingSchemaNonAdmin. In SettingSchemaAdmin, we have all the attributes or fields that are present and is accessible to the Admin and Super Admin. In the SettingSchemaNonAdmin however, we have only those fields and attributes that we want to show to all non admin users.

from flask_jwt import current_identity
 
class SettingDetail(ResourceDetail):
    """
    setting detail by id
    """
 
    def before_get(self, args, kwargs):
        kwargs['id'] = 1
        if current_identity.is_admin or current_identity.is_super_admin:
            self.schema = SettingSchemaAdmin
        else:
            self.schema = SettingSchemaNonAdmin

 

The above code helps us achieve this. If you have read previous blogs about the API server, you would already know that we are using JWT for authenticating our users. In this code, we are importing current_identity from flask_jwt. Current_identity, returns us an object of the User type which has properties such as is_admin, is_super_admin, etc. to help us identify the permission level of that user.
Using this object, we check whether the user who is making the request via jwt authentication is an admin or super admin, or just a normal registered user.

        if current_identity.is_admin or current_identity.is_super_admin:
            self.schema = SettingSchemaAdmin
        else:
            self.schema = SettingSchemaNonAdmin

 

So, if the current user sending the request is an admin, then we set the schema for the Resource manager class of the flask-rest-jsonapi as SettingSchemaAdmin, which we have already declared before containing all the fields, else, we set it as SettingSchemaNonAdmin which has limited number of attributes.

Continue ReadingPermission Dependent Schema for Admin Settings in Open Event Server

Viewing an Event in Giggity

Giggity is an Android app that loads xCal, pentabarf, frab, wafer and iCal files (that contain schedules of conferences, festivals and other events) and lets you browse them in various convenient formats. It is a generic app that can be used for any event that publishes their schedule in an open format. Recently support to see Open Event JSON format has been added in the Giggity. In this blog I describe the simple steps you need to follow to see the event that is created in the Open Event server and can be seen in the Giggity app. Although the process of creating an event is quite detailed but the UI is very user friendly and simple to use so here I describe the process after creating the event and adding it in Giggity app

1. Go to your event dashboard

2. Click on the export button

3. Select sessions and microlocations by clicking the switch buttons. Get the url generated and copy it. Skipping the microlocations will disable the option to navigate to the location in map opening from the app.

4. Click on the “+” button on the toolbar and paste the link.

   

5. Now you can see the sessions with locations. Slide the navigation drawer to see the links and signup url or create the home screen shortcut.

   

So here you can see your event following these five simple steps. Watch this video to see this in action.

https://www.youtube.com/watch?v=RairLTXqqDI

Continue ReadingViewing an Event in Giggity

Differentiating between received file formats for Event Apps

Sometimes we need to operate on files of different formats in the same app itself for example Giggity app parses the file in iCal, xCal, pentabarf and now in Open Event JSON format.

It could be problematic as every format needs different type of manipulation to read and get the relevant data from it, for example the JSON format is based on objects and arrays in it from which we can get the data by referencing titles of the data while pentabarf XML identifies the elements with tags. Also it is not evident from the link itself which format is being received. So to see which type of file is received instead of analyzing the entire file we look for the few initials of the syntax from which it becomes evident.

I recently used this method to differentiate between the JSON and iCal format to display sessions of an event in the Giraffe app in Open Event JSON format which is also being used in Giggity.

Let’s have a look on how to actually implement this. It is simple after you get the data from the url. Instead of iterating for the entire data we read the initial data only to see what kind of file we have received.

BufferedReader r = new BufferedReader(new InputStreamReader(inputStream));
String line = r.readLine();

if(line.contains("BEGIN:VCALENDAR")) {
 SharedPreferences.Editor edit = prefs.edit();
 edit.putString("type", "ical");
 edit.commit();
 Log.e("Response type","ical");
} 
else if(line.contains("{")) {
 SharedPreferences.Editor edit = prefs.edit();
 edit.putString("type", "json");
 edit.commit();
 Log.e("Response type","json");
}

We saved the type of response received in the shared preferences so we retrieve it later from anywhere in the app to see what kind of data we have received instead of passing it in between functions or activities.

See this to check as you download the data from the URL asynchronously, so you don’t need to do it later or manipulations needs to be done asynchronously too.

So when you are checking in asynchronously using AsyncTask class get shared preferences in onPreExecute() and update it in doInBackground() as we download the data and check.

The other advantage of this method is that we can close the thread or return to the main activity if the file format found is not readable or not of desired format.

Continue ReadingDifferentiating between received file formats for Event Apps

Creating inter-component actions in Open Event Front-end

Open Event Front-end project is built using Ember JS, which lets us create modular components. While implementing the sessions route on the project, we faced a challenge of sending inter-component actions. To solve this problem we used the ember action helpers which bubbles up the action to the controller and sends it to the desired component. How did we solve it?

Handling actions in ember

In ember we can handle actions using the {{action ‘function’}} where function executes every time an action on element is triggered. This can be used to handle actions for the component. You can define actions for elements as:

<a href="#" class="item {{if (eq selectedTrackId track.id) 'active'}}" {{action 'filter' track.id}}>
  {{track.name}}
</a>

All the actions defined using the {{action}}  helper are defined inside the actions section of the component. Here the action filter is getting binded to onClick event of the anchor tag. The above helper will pass the name of the track(track) as a parameter to the filter function defined in the component.

Whenever the element is clicked the filter function defined in the component will get triggered. This method works for handling actions within a component, however when we need to trigger inter-component actions this approach does not help.

Sending actions from component to controller

We will send an action from the component to the controller of the route in which the component is rendered using a computed property defined inside the controller, which watched the selectedTrackId.

{{public/session-filter selectedTrackId=selectedTrackId tracks=tracks}}

Whenever the anchor is clicked it passes the id of the selected track to the filter function of the component. Inside the filter function we set the selectedTrackId variable passed to the component inside the route template as specified above.

actions: {
  filter(trackId = null) {
    this.set('selectedTrackId', trackId);
  }
}

The selectedTrackId is a observed by a computed property defined in the controller which modified the track list based on the id passed by the sessions-filter component.

Handling action in the controller

Inside the controller we have a computed property which observes the sessions array and the selectedTrackId which is passed by the session-filter component to the controller called sessionsByTracks.

sessionsByTracks: computed('model.sessions.[]', 'selectedTrackId', function() {
  if (this.get('selectedTrackId')) {
    return chain(this.get('model.sessions'))
      .filter(['track.id', this.get('selectedTrackId')])
      .orderBy(['track.name', 'startAt'])
      .value();
  } else {
    return chain(this.get('model.sessions'))
      .orderBy(['track.name', 'startAt'])
      .groupBy('track.name')
      .value();
  }
}),
tracks: computed('model.sessions.[]', function() {
  return chain(this.get('model.sessions'))
    .map('track')
    .uniqBy('id')
    .orderBy('name')
    .value();
})

sessionsByTracks is the property gets filtered using the lodash functions based upon the id of session passed to it. On the first render all the sessions are displayed as the selectedTrackId is set to null.

{{public/session-list sessions=sessionsByTracks}}

This property is passed to the session-list component which renders the filtered list of session based on the selected session in session-filter component. Check the source code for the example here.

Resources

Continue ReadingCreating inter-component actions in Open Event Front-end

Testing Asynchronous Code in Open Event Orga App using RxJava

In the last blog post, we saw how to test complex interactions through our apps using stubbed behaviors by Mockito. In this post, I’ll be talking about how to test RxJava components such as Observables. This one will focus on testing complex situations using RxJava as the library itself provides methods to unit test your reactive streams, so that you don’t have to go out of your way to set contraptions like callback captors, and implement your own interfaces as stubs of the original one. The test suite (kind of) provided by RxJava also allows you to test the fate of your stream, like confirming that they got subscribed or an error was thrown; or test an individual emitted item, like its value or with a predicate logic of your own, etc. We have used this heavily in Open Event Orga App (Github Repo) to detect if the app is correctly loading and refreshing resources from the correct source. We also capture certain triggers happening to events like saving of data on reloading so that the database remains in a consistent state. Here, we’ll look at some basic examples and move to some complex ones later. So, let’s start.

public class AttendeeRepositoryTest {

    private AttendeeRepository attendeeRepository;

    @Before
    public void setUp() {
        testDemo = new TestDemo();
    }

    @Test
    public void shouldReturnAttendeeByName() {
        // TODO: Implement test
    }

}

 

This is our basic test class setup with general JUnit settings. We’ll start by writing our tests, the first of which will be to test that we can get an attendee by name. The attendee class is a model class with firstName and lastName. And we will be checking if we get a valid attendee by passing a full name. Note that although we will be talking about the implementation of the code which we are writing tests for, but only in an abstract manner, meaning we won’t be dealing with production code, just the test.

So, as we know that Observables provide a stream of data. But here, we are only concerned with one attendee. Technically, we should be using Single, but for generality, we’ll stick with Observables.

So, a person from the background of JUnit would be tempted to write this code below.

Attendee attendee = attendeeRepository.getByAttendeeName("John Wick")
    .blockingFirst();

assertEquals("John Wick", attendee.getFirstName() + attendee.getLastName());

 

So, what this code is doing is blocking the thread till the first attendee is provided in the stream and then checking that the attendee is actually John Wick.

While this code works, this is not reactive. With the reactive way of testing, not only you can test more complex logic than this with less verbosity, but it naturally provides ways to test other behaviors of Reactive streams such as subscriptions, errors, completions, etc. We’ll only be covering a few. So, let’s see the reactive version of the above test.

attendeeRepository.getByAttendeeName("John Wick")
    .firstElement()
    .test()
    .assertNoErrors()
    .assertValue(attendee -> "John Wick".equals(
        attendee.getFirstName() + attendee.getLastName()
    ));

 

So clean and complete. Just by calling test() on the returned observable, we got this whole suite of testing methods with which, not only did we test the name but also that there are no errors while getting the attendee.

Testing for Network Error on loading of Attendees

OK, so let’s move towards a more realistic test. Suppose that you call this method on AttendeeRepository, and that you can fetch attendees from the network. So first, you want to handle the simplest case, that there should be an error if there is no connection. So, if you have (hopefully) set up your project using abstractions for the model using MVP, then it’ll be a piece of cake to test this. Let’s suppose we have a networkUtil object with a method isConnected.

The NetworkUtil class is a dependency of AttendeeRepository and we have set it up as a mock in our test using Mockito. If this is sounding somewhat unfamiliar, please read my previous article “The Joy of Testing with MVP”.

So, our test will look like this

@Test
public void shouldStreamErrorOnNetworkDown() {
    when(networkUtils.isConnected()).thenReturn(false);
    
    attendeeRepository.getAttendees()
        .test()
        .assertErrorMessage("No Network");
}

 

Note that, if you don’t define the mock object’s behavior like I have here, attendeeRepository will likely throw an NPE as it will be calling isConnected() on an undefined object.

With RxJava, you get a whole lot of methods for each use case. Even for checking errors, you get to assert a particular Throwable, or a predicate defining an operation on the Throwable, or error message as I have shown in this case.

Now, if you run this code, it’ll probably fail. That’s because if you are testing this by offloading the networking task to a different thread by using subscribeOn observeOn methods, the test body may be detached from Main Thread while the requests complete. Furthermore, if testing in an application made for Android, you would have use AndroidSchedulers.mainThread(), but as it is an Android dependency, the test will fail. Well actually, crash. There were some workarounds by creating abstractions for even RxJava schedulers, but RxJava2 provides a very convenient method to override the default schedulers in the form of RxJavaPlugins. Similarly, RxAndroidPlugins is present in the rx-android package. Let’s suppose you have the plan to use Schedulers.io() for asynchronous work and want to get the stream on Android’s Main Thread, meaning you use AndroidSchedulers.mainThread() in the observeOn method. To override these schedulers to Schedulers.trampoline() which queues your tasks and performs them one by one, like the main thread, your setUp will include this:

RxJavaPlugins.setIoSchedulerHandler(scheduler ->  Schedulers.trampoline());
RxAndroidPlugins.setInitMainThreadSchedulerHandler(scheduler -> Schedulers.trampoline());

 

And if you are not using isolated tests and need to resume the default scheduler behavior after each test, then you’ll need to add this in your tearDown method

RxJavaPlugins.reset();
RxAndroidPlugins.reset();

Testing for Correct loading of Attendees

Now that we have tested that our Repository is correctly throwing an error when the network is down, let’s test that it correctly loads attendees when the network is connected. For this, we’ll need to mock our EventService to return attendees when queried, since we don’t want our unit tests to actually hit the servers.

So, we’ll need to keep these things in mind:

  • Mock the network until it shows that it is connected to the Internet
  • Mock the EventService to return attendees when queried
  • Call the getter on the attendeeRepository and test that it indeed returned a list of attendees

For these conditions, our test will look like this:

@Test
public void shouldLoadAttendeesSuccessfully() {
    List<Attendee> attendees = Arrays.asList(
        new Attendee(),
        new Attendee(),
        new Attendee()
    );

    when(networkUtils.isConnected()).thenReturn(true);
    when(eventService.getAttendees()).thenReturn(Observable.just(attendees));

    attendeeRepository.getAttendees()
        .test()
        .assertValues(attendees.toArray(new Attendee[attendees.size()]));
}

 

The assertValues function asserts that these values were emitted by the observable. And if you want to be terser, you can even verify that in fact EventService’s getAttendees function was called by

verify(eventService).getAttendees();

 

But the problem in this way is that the getAttendees function returns an observable and just calling it does not necessarily means that it was subscribed, emitting the results, hence we need to test to ensure that it was indeed subscribed. If we call the normal test() function on the observable, it is already subscribed, making the result of testSubscribed always true. In order to test that correctly, let’s look at our final use case.

Testing for saving of Attendees

In the Open Event Orga App, we have strived to create self-sufficient and intelligent classes, thus, our repository is also built this way. It detects that new attendees are loaded from the server and saves them in the database. Now we’d want to test this functionality.

In this test, there is an added dependency of DatabaseRepository for saving the attendees, which we will mock. The conditions for this test will be:

  • Network is connected
  • EventService returns attendees
  • DatabaseRepository mocks the saving of attendees

For DatabaseRepository’s save method, we’ll be returning a Completable, which will notify when the saving of data is completed. The primary purpose of this test will be to assert that this completable is indeed subscribed when the attendee loading is triggered. This will not only ensure that the correct function to save the attendees is called, but also that it is indeed triggered and not just left hanging after the call. So, our test will look like this.

@Test
public void shouldSaveAttendeesInDatabase() {
    List<Attendee> attendees = Arrays.asList(
        new Attendee(),
        new Attendee(),
        new Attendee()
    );

    TestObserver testObserver = TestObserver.create();
    Completable completable = Completable.complete()
        .doOnSubscribe(testObserver::onSubscribe);

    when(networkUtils.isConnected()).thenReturn(true);
    when(databaseRepository.save(attendees)).thenReturn(completable);
    when(eventService.getAttendees()).thenReturn(Observable.just(attendees));

    attendeeRepository.getAttendees()
        .test()
        .assertNoErrors();

    testObserver.assertSubscribed();
}

 

Here, we have created a separate test observable and set it to be subscribed when the Completable is subscribed and we have returned that Completable when the save method is called. In the last, we have asserted that the test observer is indeed subscribed.

You can create more complex use cases and assert subscriptions, errors, the emptiness of a stream and much more, by using the built-in test functionalities of RxJava2. So, that’s all for this blog, you can visit these links for more details on unit testing RxJava

http://fedepaol.github.io/blog/2015/09/13/testing-rxjava-observables-subscriptions/

https://www.infoq.com/articles/Testing-RxJava

Continue ReadingTesting Asynchronous Code in Open Event Orga App using RxJava

Deploying documentations generated by Yaydoc to Heroku

There are many web applications available online that generates static websites. Among these projects are two unique projects developed here at FOSSASIA. These are the Open Event WebApp Generator and Yaydoc (an automatic documentation generation and deployment project.). Since Yaydoc already supports the deployment of the generated documentations to Github pages, it was just a matter of time that the deployment to Heroku is also supported.

Heroku is an excellent cloud-based platform used as a web application deployment service. Heroku provides most of its services at free of cost to the users and is excellent to host static websites provided that a little bit of tweaking is done.

For this implementation, we use the `Platform API` provided by Heroku. Stating it’s description mentioned in the documentation,

The platform API empowers developers to automate, extend and combine Heroku with other services. You can use the platform API to programmatically create apps, provision add-ons and perform other tasks that could previously only be accomplished with Heroku toolbelt or dashboard.

In order to deploy the static websites to Heroku, we need to first prepare a bundle of source code that has been compiled and is ready for execution on the Heroku runtime. This bundle is known as a Slug.

cd temp/$EMAIL/${UNIQUE_ID}_preview
mkdir -p app
cd app

curl https://nodejs.org/dist/v6.11.0/node-v6.11.0-linux-x64.tar.gz | tar xzv > /dev/null

cp $BASE/web.js
rsync -av --progress ../ . --exclude app

cd ..
tar czfv slug.tgz ./app > /dev/null

We are using the files generated for preview to bundle them in a slug. Also, we download the NodeJS runtime files since we are deploying a static website to Heroku. Along with the static files, we require bundling a NodeJS server file (web.js) that will be used to reference the static files in the application.

After preparing the Slug, we publish the static web application to Heroku. For this, we start by creating a Heroku app using the command `heroku create <app-name>`. The app name is decided by the user when he or she fills the form in the Yaydoc Web App. Following that, we request Heroku to allocate a new slug for your app. After that, we upload the slug tar file to the platform.

# Create Heroku 
heroku create $APP_NAME

# Allocating new Slug
Arr=($(curl -u “:$API_KEY” -X \
-H ‘Content-Type:application/json’ \
-H ‘Accept: application/vnd.heroku+json;version=3’ \
-d ‘{“process_types”:{“web”:”node-v6.11.0-linux-x64/bin/node web.js”}}’ \
-n https://api.heroku.com/apps/${APP_NAME}/slugs | \
python -c “import sys,json; obj=json.load(sys.stdin);
print(obj[‘blob’][‘url’] + ‘\n’ + obj[‘id’])”))

# Upload the slug tar file
curl -X PUT \
-H “Content-Type:”\
--data-binary @slug.tgz \
“${Arr[0]}”

After uploading the slug to Heroku, we need to release the app. This is done using the following command.

curl -u “:$API_KEY” -X POST \
-H “Accept: application/vnd.heroku+json; version=3” \
-H “Content-Type: application/json” \
-d ‘{“slug”:”’${Arr[1]}’”}’ \
-n https://api.heroku.com/apps/$APP_NAME/releases

Releasing the application completes the process of deployment, making the documentation generated by Yaydoc up and running at the following URL: https://<app-name>.herokuapp.com/

 

Continue ReadingDeploying documentations generated by Yaydoc to Heroku

Using HTTMock to mock Third Party APIs for Development of Open Event API server

In the process of implementing the connected social media in Open Event API server, there was a situation where we need to mock third party API services like Google OAuth, Facebook Graph API. In mocking, we try to run our tests on APIs Simulation instead of Original API such that it responds with dummy data similar to original APIs.

To implement this first we need the library support in our Orga Server to mock APIs, so the library that was used for this purpose is the httmock library for Python. One of my mentors @hongquan helped me to understand the approach that we will follow to get this implemented. So according to implementation, when we make a HTTP request to any API through tests then our implementation with httmock will be such that it

  • stands in the middle of the request,
  • Stops the request from going to the original API,
  • and returns a dummy response as if the response is from original API.

The content of this response is written by us in the test case. We have to make sure that it is same type of object as we receive from original API.

Steps to follow ( on mocking Google OAuth API )

  1. Look for response object on two requests (OAuth and profile details).
  2. Create the dummy response using the sample response object.
  3. Creating endpoints using the httpmock library.
  4. During test run, calling the specific method with HTTMock

Sample object of OAuth Response from Google is:

{
"access_token":"2YotnFZFEjr1zCsicMWpAA",
"token_type":"Bearer",
"expires_in":3600,
"refresh_token":"tGzv3JOkF0XG5Qx2TlKWIA",
"example_parameter":"example_value"
}

and from the sample object of Google Profile API we needed the link of profile for our API-server:

{'link':'http://google.com/some_id'}

 

Creating the dummy response

Creating dummy response was easy. All I had to do is provide proper header and content in response and use @urlmatch decorator

# response for getting userinfo from google

@urlmatch(netloc='https://www.googleapis.com/userinfo/v2/me')
def google_profile_mock(url, request):
   headers = {'content-type': 'application/json'}
   content = {'link':'http://google.com/some_id'}
   return response(200, content, headers, None, 5, request)

@urlmatch(netloc=r'(.*\.)?google\.com$')
def google_auth_mock(url, request):
   headers = {'content-type': 'application/json'}
   content = {
       "access_token":"2YotnFZFEjr1zCsicMWpAA",
       "token_type":"Bearer",
       "expires_in":3600,
       "refresh_token":"tGzv3JOkF0XG5Qx2TlKWIA",
       "example_parameter":"example_value"
   }
   return response(200, content, headers, None, 5, request)

 

So now we have the end points to mock the response. All we need to do is to use HTTMock inside the test case.

To use this setup all we need to do is:

with HTTMock(google_auth_mock, google_profile_mock):
                self.assertTrue('Open Event' in self.app.get('/gCallback/?state=dummy_state&code=dummy_code',
                                                          follow_redirects=True).data) 
            self.assertEqual(self.app.get('/gCallback/?state=dummy_state&code=dummy_code').status_code, 302)
            self.assertEqual(self.app.get('/gCallback/?state=dummy_state&code=dummy_code').status_code, 302)

And we were able to mock the Google APIs in our test case. Complete implementation in FOSSASIA API-Server can be seen here

 

Continue ReadingUsing HTTMock to mock Third Party APIs for Development of Open Event API server