Implemeting Permissions for Speakers API in Open Event API Server

In my previous blogpost I talked about what the permissions enlisted in developer handbook means and which part of the codebase defines what part of the permissions clauses. The permission manager provides the permissions framework to implement the permissions and proper access controls based on the dev handbook.

In this blogpost, the actual implementation of the permissions is described. (Speakers API is under consideration here). The following table is the permissions in the developer handbook.

List

View

Create

Update

Delete

Superadmin/admin

Event organizer

✓ [1]

✓ [1]

✓ [1]

✓ [1]

✓ [1]

Registered User

✓ [3]

✓ [3]

✓ [4]

✓ [3]

✓ [3]

Everyone else

✓ [2][4]

✓ [2][4]

  1. Only self-owned events
  2. Only of sessions with state approved or accepted
  3. Only of self-submitted sessions
  4. Only to events with state published.

Super admin and admin should be able to access all the methods – list, view, create, update and delete. All the permissions are implemented through functions derived from permissions manager.Since all the functions have first check for super admin and admin, these are automatically taken care of.

Only of self-submitted sessions
This means that a registered user can list, view, edit or delete speakers of a session which he himself submitted. This requires adding a ‘creator’ attribute to session object which will help us determine if the session was created by the user. So before making a post for sessions, the current user identity is included as part of the payload.

def before_post(self, args, kwargs, data):
   data['creator_id'] = current_identity.id


Now that we have added creator id to a session, a method is used to check if session was created by the same user.

def is_session_self_submitted(view, view_args, view_kwargs, *args, **kwargs):
    user = current_identity


Firstly the current identity is set as user which will later be used to check id. Sequentially, admin, superadmin, organizer and co-organizers are checked. After this a session is fetched using 
kwargs[session_id]. Then if the current user id is same as the creator id of the session fetched, access is granted, else Forbidden Error is returned.

if session.creator_id == user.id:
   return view(*view_args, **view_kwargs)


In the before_post method of speakers class, the session ids received in the data are passed to this function in 
kwargs as session_id. The permissions are then checked there using current user. If the session id are not those of self submitted sessions, ‘Session Not Found’ is returned.

 if not has_access('is_session_self_submitted', session_id=session_id):
                    raise ObjectNotFound({'parameter': 'session_id'},
                                         "Session: {} not found".format(session_id))


Only of sessions with state approved or accepted
This check is required for user who has not submitted the session himself, so he can only see speaker profiles of accepted sessions. First, if the user is not authenticated, permissions are not checked. If co-organizer access is available, then the user can see all the speakers, so for this case filtering is not done. If not, then ‘is_session_self_submitted’ is checked. If yes, then then again no filtering, but if not then the following query filters accepted sessions.

if not has_access('is_session_self_submitted', session_id=session.id):
    query_ = query_.filter(Session.state == "approved" or Session.state == "accepted")

Similarly all the permissions first generate a list of all objects and then filtering is done based on the access level, instead of getting the list based on permissions.

Only to events with state published
It is necessary that users except the organizers and co-organizers can not see the events which are in draft state. The same thing follows for speaker profiles – a user cannot submit or view a speaker profile to an unpublished event. Hence, this constraint. So before POST of speakers, if event is not published, an event not found error is returned.

if event.state == "draft":
    raise ObjectNotFound({'parameter': 'event_id'},
                        "Event: {} not found".format(data['event_id'])


For GET, the  implementation of this is similar to the previous permission. A basic query is generated as such:

query_ = query_.join(Event).filter(Event.id == event.id)


Now if the user does not have at least 
co-organizer access, draft events must be filtered out.

if not has_access('is_coorganizer', event_id=event.id):
    query_ = query_.filter(Event.state == "published")


Some of the finer details have been skipped here, which can be found in the 
code.

Resources

Understanding Permissions for Various APIs in Open Event API Server

Since the Open Event Server has various elements, a proper permissions system is essential. This huge list of permissions is well compiled in the developer handbook which can be found here. In this blogpost, permissions listed in the developer handbook are discussed. Let’s start with what we wish to achieve, that is, how to make sense of these permissions and where does each clause fit in the API Server’s codebase.

For example, Sponsors API has the following permissions.

List

View

Create

Update

Delete

Superadmin/admin

Event organizer

✓ [1]

✓ [1]

✓ [1]

✓ [1]

✓ [1]

Registered User

✓ [3]

✓ [3]

✓ [4]

✓ [3]

✓ [3]

Everyone else

✓ [2][4]

✓ [2][4]

  1. Only self-owned events
  2. Only sessions with state approved or accepted
  3. Only self-submitted sessions
  4. Only to events with state published.

Based on flask-rest-jsonapi resource manager, we get list create under ResourceList through ResourceList’s GET and POST methods, whereas View, Update, Delete work on single objects and hence are provided by ResourceDetail’s GET, PATCH and DELETE respectively. Each function of the permission manager has a jwt_required decorator.

@jwt_required
def is_super_admin(view, view_args, view_kwargs, *args, **kwargs):

@jwt_required
def is_session_self_submitted(view, view_args, view_kwargs, *args, **kwargs):


This
 ensures that whenever a check for access control is made to the permission manager, the user is signed in to Open Event. Additionally, the permissions are written in a hierarchical way such that for every permission, first the useris checked for admin or super admin, then for other accesses. Similar hierarchy is kept for organizer accesses like track organizer, registrar, staff or organizer and coorganizer.

Some APIs resources require no authentication for List. To do this we need to add a check for Authentication token in the headers. Since each of the functions of permission manager have jwt_required as decorator, it is important to checkfor the presence of JWT token in request headers, because we can proceed to check for specific permissions in that case only.

if 'Authorizationin request.headers:
 _jwt_required(current_app.config['JWT_DEFAULT_REALM'])


Since the resources are created by endpoints of the form : 
‘/v1/<resource>/` , this is derived from the separate ResourceListPost class. This class is POST only and has a before_create object method where the required relationships and permissions are checked before inserting the data in the tables. In the before_create method, let’s say that event is a required relationship, which will be defined by the ResourceRelationRequired , then we use our custom method

def require_relationship(resource_list, data):
    for resource in resource_list:
        if resource not in data:
            raise UnprocessableEntity({'pointer': '/data/relationships/{}'.format(resource)},
                                      "A valid relationship with {} resource is required".format(resource))


to check if the required relationships are present in the data. The event_id here can also be used to check for organizer or co-organizer access in the permissions manager for a particular event.

Here’s another permissions structure for a different API – Settings.

List

View

Create

Update

Delete

Superadmin/admin

Everyone else

✓ [1]

  1. Only app_nametaglineanalytics_keystripe_publishable_keygoogle_urlgithub_urltwitter_urlsupport_urlfacebook_urlyoutube_urlandroid_app_urlweb_app_url fields .

This API does not allow access to the complete object, but to only some fields which are listed above. The complete details can be checked here.

Resources

Using Custom Forms In Open Event API Server

One feature of the  Open Event management system is the ability to add a custom form for an event. The nextgen API Server exposes endpoints to view, edit and delete forms and form-fields. This blogpost describes how to use a custom-form in Open Event API Server.

Custom forms allow the event organizer to make a personalized forms for his/her event. The form object includes an identifier set by the user, and the form itself in the form of a string. The user can also set the type for the form which can be either of text or checkbox depending on the user needs. There are other fields as well, which are abstracted. These fields include:

  • id : auto generated unique identifier for the form
  • event_id : id of the event with which the form is associated
  • is_required : If the form is required
  • is_included : if the form is to be included
  • is_fixed : if the form is fixedThe last three of these fields are boolean fields and provide the user with better control over forms use-cases in the event management.

Only the event organizer has permissions to edit or delete these forms, while any user who is logged in to eventyay.com can see the fields available for a custom form for an event.

To create a custom-form for event with id=1, the following request is to be made:
POST  https://api.eventyay.com/v1/events/1/custom-forms?sort=type&filter=[]

with all the above described fields to be included in the request body.  For example:

{
 "data": {
   "type": "custom_form",
   "attributes": {
     "form": "form",
     "type": "text",
     "field-identifier": "abc123",
     "is-required": "true",
     "is-included": "false",
     "is-fixed": "false"
   }
 }
}

The API returns the custom form object along with the event relationships and other self and related links. To see what the response looks like exactly, please check the sample here.

Now that we have created a form, any user can get the fields for the same. But let’s say that the event organiser wants to update some field or some other attribute for the form, he can make the following request along with the custom-form id.

PATCH https://api.eventyay.com/v1/custom-forms/1

(Note: custom-form id must be included in both the URL as well as request body)

Similarly, to delete the form,
DELETE https://api.eventyay.com/v1/custom-forms/1     can be used.

Resources

Writing Dredd Test for Event Topic-Event Endpoint in Open Event API Server

The API Server exposes a large set of endpoints which are well documented using apiary’s API Blueprint. Ton ensure that these documentations describe exactly what the API does, as in the response made to a request, testing them is crucial. This testing is done through Dredd Documentation testing with the help of FactoryBoy for faking objects.

In this blogpost I describe how to use FactoryBoy to write Dredd tests for the Event Topic- Event endpoint of Open Event API Server.

The endpoint for which tests are described here is this: For testing this endpoint, we need to simulate the API GET request by making a call to our database and then compare the response received to the expected response written in the api_blueprint.apib file. For GET to return some data we need to insert an event with some event topic in the database.

The documentation for this endpoint is the following:

To add the event topic and event objects for GET events-topics/1/events, we use a hook. This hook is written in hook_main.py file and is run before the request is made.

We add this decorator on the function which will add objects to the database. This decorator basically traverses the APIB docs following level with number of ‘#’ in the documentation to ‘>’ in the decorator. So for
 we have,

Now let’s write the method itself. In the method here, we first add the event topic object using EventTopic Factory defined in the factories/event-topic.py file, the code for which can be found here.

Since the endpoint also requires some event to be created in order to fetch events related to an event topic, we add an event object too based on the EventFactoryBasic class in factories/event.py  file. [Code]

To fetch the event related to a topic, the event must be referenced in that particular event topic. This is achieved by passing event_topic_id=1 when creating the event object, so that for the event that is created by the constructor, event topic is set as id = 1.
event = EventFactoryBasic(event_topic_id=1)
In the EventFactoryBasic class, the event_topic_id is set as ‘None’, so that we don’t have to create event topic for creating events in other endpoints testing also. This also lets us to not add event-topic as a related factory. To add event_topic_id=1 as the event’s attribute, an event topic with id = 1 must be already present, hence event_topic object is added first.
After adding the event object also, we commit both of these into the database. Now that we have an event topic object with id = 1, an event object with id = 1 , and the event is related to that event topic, we can make a call to GET event-topics/1/events and get the correct response.

Related:

Working with Activity API in Open Event API Server


Recently, I added the Activities API with documentation and dredd tests for the same in
Open Event API Server. The Activity Model in the Open Event Server is basically a log of all the things that happen while the server is running, like – event updates, speaker additions, invoice generations and other similar things. This blogpost explains how to implement Activity API in the Open Event API Server’s nextgen branch. In the Open Event Server, we first add the endpoints, then document these so that the consumers ( Open Event Orga App, Open Event Frontend) find it easy to work with all the endpoints.

We also test the documentation against backend implementation to ensure that a end-developer who is working with the APIs is not misled to believe what each endpoint actually does in the server.

We also test the documentation against backend implementation to ensure that a end-developer who is working with the APIs is not misled to believe what each endpoint actually does in the server.
The Activities API endpoints are based on the Activity database model. The Activity table has four columns – 
id, actor, time, action, the names are self-explanatory. Now for the API schema, we need to make fields corresponding these columns.
Since id is auto generated, we do not need to add it as a field for API. Also the activity model’s __init__ method stamps time with the current system time. So this field is also not required in the API fields. We are left with two fields- actor and action.

Defining API Schema

Next, we define the API Schema class for Activities model. This will involve a Meta class and fields for the class.The Meta class contains the metadata of the class. This includes details about type_,

self_view, self_view_kwargs and an inflect parameter to dasherize the input fields from request body.

We define the four fields – id, actor, time and action according to marshmallow fields based on the data type and parameters from the activities model. Since id, actor and action are string columns and time is a DateTime column, the fields are written as following:

The id field is marked as dump only because it is a read-only type field. The other fields are marked with allow_none as they are all non-required field.

ActivityList Class:
The activity list class will provide us with the endpoint: “/v1/activities”

This endpoint will list all the activities. Since we wanted only GET requests to be working for this, so defined method = [‘GET’, ] for ResourceList. The activities are to be internally created based on different actions like creating an event, updating an event, adding speakers to sessions and likewise. Since the activities are to be shown only to the server admin, 
is_admin permission is used from the permission manager.

ActivityDetail Class:

The activity detail gives methods to work with an activity based on the activity id.
The endpoint provided is :  ‘/v1/activity/<int:activity_id>’

Since this is also an admin-only accessible GET only endpoint the following was written:

Writing Documentation:

The documentation is written using API Blueprint. Since we have two endpoints to document : /v1/activities and /v1/activities/<int:activity_id> both GET only.

So we begin by defining the ‘Group Activities’ , under which we first list  ‘Activity Collection’ which essentially is the Activity List class.

For this class, we have the endpoint:  /v1/activities. This is added for GET request. The parameters – actor, time and action are described along with description, type and whether they are required or not.

The request headers are written as part of the docs followed by the expected response.

Next we write the ‘Activity Details’ which represents the ActivityDetail class. Here the endpoint /v1/activities/<int:activity_id> is documented for GET. The parameter here is activity_id, which is the id of the activity to get the details of.

Writing DREDD Test for Documentation

To imitate the request responses, we need a faker script which creates an object of the the class we are testing the docs for then makes the request. For this we use FactoryBoy and dredd hooks to insert data into the database.

Defining Factory Model for Activity db model

The above is the factory model for activity model. It is derived from

factory.alchemy.SQLAlchemyModelFactory. The meta class defines the db model and sqlalchemy session to be used. The actor and action have dummy strings as part of the request body.

Writing Hooks
Now to test these endpoints we need to add objects to the database so that the GET requests have an object to fetch. This is done by dredd hooks. Before each request, an object of the corresponding factory class is initialised and committed into the database. Thus a dummy object is available for dredd to test on. The request is made and the real output is compared with the expected output written in the API Blueprint documentation.

This is what the hooks look like for  this endpoint: GET /activities

Now if the expected responses and actual responses match, the dredd test successfully passes. This dredd test in run on each build of the project on Travis to ensure that documented code does exactly what is says!

This concludes the process to write an API right from Schema to Resources and Documentation and Dredd tests.

Additional Resources:

Managing Related Endpoints in Permission Manager of Open Event API Server

Open Event API Server has its permission manager to manage all permission to different endpoints and some of the left gaps were filled by new helper method has_access. The next challenge for permission manager was to incorporate a feature many related endpoints points to the same resource.
Example:

  • /users-events-roles/<int:users_events_role_id>/user or
  • /event-invoices/<int:event_invoice_id>/user

Both endpoints point to Users API where they are fetching the record of a single user and for this, we apply the permission “is_user_itself”. This permission ensures that the logged in user is the same user whose record is asked through the API and for this we need the “user_id” as the “id” in the permission function, “is_user_itself”
Thus there is need to add the ability in permission manager to fetch this user_id from different models for different endpoints. For example, if we consider above endpoints then we need the ability to get user_id from UsersEventsRole and EventInvoice models and pass it to permission function so that it can use it for the check.

Adding support

To add support for multiple keys, we have to look for two things.

  • fetch_key_url
  • model

These two are key attributes to add this feature, fetch_key_url will take the comma separated list which will be matched with view_kwargs and model receives the array of the Model Classes which will be used to fetch the related records from the model
This snippet provides the main logic for this:

for index, mod in enumerate(model):
   if is_multiple(fetch_key_url):
       f_url = fetch_key_url[index]
   else:
       f_url = fetch_key_url
   try:
       data = mod.query.filter(getattr(mod, fetch_key_model) == view_kwargs[f_url]).one()
   except NoResultFound, e:
       pass
   else:
       found = True

if not found:
   return NotFoundError({'source': ''}, 'Object not found.').respond()

From the above snippet we are:

  • We iterate through the models list
  • Check if fetch_key_url has multiple keys or not
  • Get the key from fetch_key_url on the basis of multiple keys or single key in it.
  • We try to attempt to get object from model for the respective iteration
  • If there is any record/object in the database then it’s our data. Skipping further process
  • Else continue iteration till we get the object or to the end.

To use multiple mode

Instead of providing the single model to the model option of permission manager, provide an array of models. Also, it is optional to provide comma separated values to fetch_key_url
Now there can be scenario where you want to fetch resource from database model using different keys present on your view_kwargs
for example, consider these endpoints

  1. `/notifications/<notification_id>/event`
  2. `/orders/<order_id>/event`

Since they point to same resource and if you want to ensure that logged in user is organizer then you can use these two things as:

  1. fetch_key_url=”notification_id, order_id”
  2. model=[Notification, Order]

Permission manager will always match indexes in both options, the first key of fetch_key_url will be only used for the first key of the model and so on.
Also, fetch_key_url is an optional parameter and even in multiple mode you can provide a single value as well.  But if you provide multiple commas separated values make sure you provide all values such that no of values in fetch_key_url and model must be equal.

Resources

Custom Data Layer in Open Event API Server

Open Event API Server uses flask-rest-jsonapi module to implement JSON API. This module provides a good logical abstraction in the data layer.
The data layer is a CRUD interface between resource manager and data. It is a very flexible system to use any ORM or data storage. The default layer you get in flask-rest-jsonapi is the SQLAlchemy ORM Layer and API Server makes use of default alchemy layer almost everywhere except the case where I worked on email verification part.

To add support for adding user’s email verification in API Server, there was need to create an endpoint for POST /v1/users/<int:user_id>/verify
Clearly here we are working on a single resource i.e, specific user record. This requires us to use ResourceDetail and the only issue was there is no any POST method or view in ResourceDetail class. To solve this I created a custom data layer which enables me to redefine all methods and views by inheriting abstract class. A custom data layer must inherit from flask_rest_jsonapi.data_layers.base.Base.

Creating Custom Layer

To solve email verification process, a custom layer was created at app/api/data_layers/VerifyUserLayer.py

def create_object(self, data, view_kwargs):
   user = safe_query(self, User, 'id', view_kwargs['user_id'], 'user_id')
   s = get_serializer()
   try:
       data = s.loads(data['token'])
   except Exception:
       raise UnprocessableEntity({'source': 'token'}, "Invalid Token")

   if user.email == data[0]:
       user.is_verified = True
       save_to_db(user)
       return user
   else:
       raise UnprocessableEntity({'source': 'token'}, "Invalid Token")

Using custom layer in API

We can easily provide custom layer in API Resource using one of the properties of the Resource Class

data_layer = {
   'class': VerifyUserLayer,
   'session': db.session
}

This is all we have to provide in the custom layer, now all CRUD method will be directed to our custom data layer.

Solution to our issue
Setting up custom layer provides us the ability to create our custom resource methods, i.e, modifying the view for POST request and allowing us to verify the registered users in API Server.
On Setting up the data layer all I need to do is create a ResourceList with using this layer and with permissions

class VerifyUser(ResourceList):

   methods = ['POST', ]
   decorators = (jwt_required,)
   schema = VerifyUserSchema
   data_layer = {
       'class': VerifyUserLayer,
       'session': db.session
   }

This enables me to use the custom layer, VerifyUserLayer for ResourceList resource.

Resources

A guide to use Permission Manager in Open Event API Server

This article provides a simple guide to use permission manager in Open Event API Server. Permission manager is constantly being improved and new features are being added into it. To ensure that all co-developers get to know about it and make use of them, this blog posts describes every part of permission manager.

Bootstrapping

Permission manager as a part of flask-rest-jsonapi works as a decorator for different resources of the API. There are two ways to provide the permission decorator to any view

  • First one is to provide it in the list of decorators
decorators = (api.has_permission('is_coorganizer', fetch="event_id",
                                fetch_as="event_id", model=StripeAuthorization),)
    • Second way is to explicitly provide it as a decorator to any view
@api.has_permission('custom_arg', custom_kwargs='custom_kwargs')
    def get(*args, **kwargs):
        return 'Hello world !'

In the process of booting up, we first need to understand the flow of Resources in API. All resources even before doing any schema check, call the decorators. So this way you will not get any request data in the permission methods. All you will receive is a dict of the URL parameters but again it will not include the filter parameters.
Permission Manager receives five parameters as: 

def permission_manager(view, view_args, view_kwargs, *args, **kwargs):

First three are provided into it implicitly by flask-rest-jsonapi module

  • view: This is the resource’s view method which is called through the API. For example, if I go to /events then the get method of ResourceList will be called.
  • view_args: These are args associated with that view.
  • view_kwargs: These are kwargs associated with that resource view. It includes all your URL parameters as well.
  • args: These are the custom args which are provided when calling the permission manager. Here at permission manager is it expected that the first index of args will be the name of permission to check for.
  • kwargs: This is the custom dict which is provided on calling the permission manager. The main pillar of the permission manager. Described below in usage.

Using Permission Manager

Using permission manager is basically understanding the different options you can send through the kwargs so here is the list of the things you can send to permission manager
These are all described in the order of priority in permission manager

  • method (string): You can provide a string containing the methods where permission needs to be checked as comma separated values of different methods in a string.
    For example: method=”GET,POST”
  • leave_if (lambda): This receives a lambda function which should return boolean values. Based on returned value if is true then it will skip the permission check. The provided lambda function receives only parameter, “view_kwargs”
    Example use case can be the situation where you can leave the permission for any specifically related endpoint to some resource and would like to do a manual check in the method itself.
  • check (lambda): Opposite to leave_if. It receives a lambda function that will return boolean values. Based on returned value, If it is true then only it will go further and check the request for permissions else will throw forbidden error.
  • fetch (string): This is the string containing the name of the key which has to be fetched for the fetch_as key (described below). Permission manager will first look for this value in view_kwargs dict object. If it is not there then it will make the query to get one(described below at model )
  • fetch_as (string): This is the string containing the name of a key. The value of fetch key will be sent to the permission functions by this name.
  • model (string): This is one most interesting concept here. To get the value of the fetch key. Permission manager first looks into view_kwargs and if there no such value then you can still get one through the model. The model attribute here receives the class of the database model which will be used to get the value of the fetch key.
    It makes the query to get the single resource from this model and look for the value of the fetch key and then pass it to the permission functions/methods.
    The interesting part is that by default it uses <id> from view_kwargs to get the resource from the model but in any case if there is no specific ID with name <id> on the view_kwargs. You can use these two options as:
  • fetch_key_url (string): This is the name of the key whose value will be fetched from view_kwargs and will be used to match the records in database model to get the resource.
  • fetch_key_model (string): This is the name of the match column in the database model for the fetch_key_url, The value of it will be matched with the column named as the value of fetch_key_model.
    In case there is no record found in the model then permission manager will throw NotFound 404 Error.

A helper for permissions

The next big thing in permission manager is the addition of new helper function “has_access”

def has_access(access_level, **kwargs):
   if access_level in permissions:
       auth = permissions[access_level](lambda *a, **b: True, (), {}, (), **kwargs)
       if type(auth) is bool and auth is True:
           return True
   return False

This method allows you to check the permission at the mid of any method of any view and of any resource. Just provide the name of permission in the first parameter and then the additional options needed by the permission function as the kwargs values.
This does not throw any exception. Just returns the boolean value so take care of throwing any exception by yourselves.

Anything to improve on?

I will not say this exactly as the improvement but I would really like to make it more meaningful and interesting to add permission. May be something like this below:

permission = "Must be co_organizer OR track_organizer, fetch event_id as event_id, use model Event"

This clearly needs time to make it. But I see this as an interesting way to add permission. Just provide meaningful text and rest leave it to the permission manager.

Image Uploading in Open Event API Server

Open Event API Server manages image uploading in a very simple way. There are many APIs such as “Event API” in API Server provides you data pointer in request body to send the image URL. Since you can send only URLs here if you want to upload any image you can use our Image Uploading API. Now, this uploading API provides you a temporary URL of your uploaded file. This is not the permanent storage but the good thing is that developers do not have to do anything else. Just send this temporary URL to the different APIs like the event one and rest of the work is done by APIs.
API Endpoints which receives the image URLs have their simple mechanism.

  • Create a copy of an uploaded image
  • Create different sizes of the uploaded image
  • Save all images to preferred storage. The Super Admin can set this storage in admin preferences

To better understand this, consider this sample request object to create an event

{
  "data": {
    "attributes": {
      "name": "New Event",
      "starts-at": "2002-05-30T09:30:10+05:30",
      "ends-at": "2022-05-30T09:30:10+05:30",
      "email": "[email protected]",
      "timezone": "Asia/Kolkata",
      "original-image-url": "https://cdn.pixabay.com/photo/2013/11/23/16/25/birds-216412_1280.jpg"
    },
    "type": "event"
  }
}

I have provided one attribute as “original-image-url”, server will open the image and create different images of different sizes as

      "is-map-shown": false,
      "original-image-url": "http://example.com/media/events/3/original/eUpxSmdCMj/43c6d4d2-db2b-460b-b891-1ceeba792cab.jpg",
      "onsite-details": null,
      "organizer-name": null,
      "can-pay-by-stripe": false,
      "large-image-url": "http://example.com/media/events/3/large/WEV4YUJCeF/f819f1d2-29bf-4acc-9af5-8052b6ab65b3.jpg",
      "timezone": "Asia/Kolkata",
      "can-pay-onsite": false,
      "deleted-at": null,
      "ticket-url": null,
      "can-pay-by-paypal": false,
      "location-name": null,
      "is-sponsors-enabled": false,
      "is-sessions-speakers-enabled": false,
      "privacy": "public",
      "has-organizer-info": false,
      "state": "Draft",
      "latitude": null,
      "starts-at": "2002-05-30T04:00:10+00:00",
      "searchable-location-name": null,
      "is-ticketing-enabled": true,
      "can-pay-by-cheque": false,
      "description": "",
      "pentabarf-url": null,
      "xcal-url": null,
      "logo-url": null,
      "can-pay-by-bank": false,
      "is-tax-enabled": false,
      "ical-url": null,
      "name": "New Event",
      "icon-image-url": "http://example.com/media/events/3/icon/N01BcTRUN2/65f25497-a079-4515-8359-ce5212e9669f.jpg",
      "thumbnail-image-url": "http://example.com/media/events/3/thumbnail/U2ZpSU1IK2/4fa07a9a-ef72-45f8-993b-037b0ad6dd6e.jpg",

We can clearly see that server is generating three other images on permanent storage as well as creating the copy of original-image-url into permanent storage.
Since we already have our Storage class, all we need to do is to make the little bit changes in it due to the decoupling of the Open Event. Also, I had to work on these points below

  • Fix upload module, provide support to generate url of locally uploaded file based on static_domain defined in settings
  • Using PIL create a method to generate new image by converting first it to jpeg(lower size than png) and resize it according to the aspect ratio
  • Create a helper method to create different sizes
  • Store all images in preferred storage.
  • Update APIs to incorporate this feature, drop any URLs in image pointers except original_image_url

Support for generating locally uploaded file’s URL
Here I worked on adding support to check if any static_domain is set by a user and used the request.url as the fallback.

if get_settings()['static_domain']:
        return get_settings()['static_domain'] + \
            file_relative_path.replace('/static', '')
    url = urlparse(request.url)
    return url.scheme + '://' + url.host + file_relative_path

Using PIL create a method to create image

This method is created to create the image based on any size passed it to as a parameter. The important role of this is to convert the image into jpg and then resize it on the basis of size and aspect ratio provided.
Earlier, in Orga Server, we were directly using the “open” method to open Image files but since they are no longer needed to be on the local server, a user can provide the link to any direct image. To add this support, all we needed is to use StringIO to turn the read string into a file-like object

image_file = cStringIO.StringIO(urllib.urlopen(image_file).read())

Next, I have to work on clearing the temporary images from the cloud which was created using temporary APIs. I believe that will be a cakewalk for locally stored images since I already had this support in this method.

if remove_after_upload:
        os.remove(image_file)

Update APIs to incorporate this feature
Below is an example how this works in an API.

if data.get('original_image_url') and data['original_image_url'] != event.original_image_url:
            uploaded_images = create_save_image_sizes(data['original_image_url'], 'event', event.id)
            data['original_image_url'] = uploaded_images['original_image_url']
            data['large_image_url'] = uploaded_images['large_image_url']
            data['thumbnail_image_url'] = uploaded_images['thumbnail_image_url']
            data['icon_image_url'] = uploaded_images['icon_image_url']
        else:
            if data.get('large_image_url'):
                del data['large_image_url']
            if data.get('thumbnail_image_url'):
                del data['thumbnail_image_url']
            if data.get('icon_image_url'):
                del data['icon_image_url']

Here the method “create_save_image_sizes” provides the different URL of different images of different sizes and we clearly dropping any other images of different sizes is provided by the user.

General Suggestion
Sometimes when we work on such issues there are some of the things to take care of for example, if you checked the first snippet, I tried to ensure that you will get the URL although it is sure that static_domain will not be blank, because even if the user (admin) doesn’t fill that field then it will be filled by server hostname
A similar situation is the one where there is no record in Image Sizes table, may be server admin didn’t add one. In that case, it will use the standard sizes stored in the codebase to create different images of different sizes.

Resources:

Permission Manager in Open Event API Server

Open Event API Server uses different decorators to control permissions for different access levels as discussed here. Next challenging thing for permissions was reducing redundancy and ensuring permission decorators are independent of different API views. They should not look to the view for which they are checking the permission or some different logic for different views.

In API Server, we have different endpoints that leads to same Resource this way we maintain relationships between different entities but this leads to a problem where permission decorators has to work on different API endpoints that points to different or same resource and but to check a permission some attributes are required and one or more endpoints may not provide all attributes required to check a permission.

For instance, PATCH /session/id` request requires permissions of a Co-Organizer and permission decorator for this requires two things, user detail and event details. It is easy to fetch user_id from logged in user while it was challenging to get “event_id”. Therefore to solve this purpose I worked on a module named “permission_manager.py” situated at “app/api/helpers/permission_manager.py” in the codebase

Basic Idea of Permission Manager

Permission manager basically works to serve the required attributes/view_kwargs to permission decorators so that these decorators do not break

Its logic can be described as:

    1. It first sits in the middle of a request and permission decorator
    2. Evaluates the arguments passed to it and ensure the current method of the request (POST, GET, etc ) is the part of permission check or not.
    3. Uses two important things, fetch and fetch_as
      fetch => value of this argument is the URL parameter key which will be fetched from URL or the database ( if not present in URL )
      fetch_as => the value received from fetch will be sent to permission decorator by the name as the value of this option.
    4. If the fetch key is not there in URL, It uses third parameter model which is Model if the table from where this key can be fetched and then passes it to permission decorator
    5. Returns the requested view on passing access level and Forbidden error if fails

This way it ensures that if looks for the only specific type of requests allowing us to set different rules for different methods.

if 'methods' in kwargs:
        methods = kwargs['methods']

    if request.method not in methods:
        return view(*view_args, **view_kwargs)

Implementing Permission Manager

Implementing it was a simple thing,

  1. Firstly, registration of JSON API app is shifted from app/api/__init__.py to app/api/bootstrap.py so that this module can be imported anywhere
  2. Added permission manager to the app_v1 module
  3. Created permission_manager.py in app/api/helpers
  4. Added it’s usage in different APIs

An example Usage:

decorators = (api.has_permission('is_coorganizer', fetch='event_id', fetch_as="event_id", methods="POST",
                                     check=lambda a: a.get('event_id') or a.get('event_identifier')),)

Here we are checking if the request has the permission of a Co-Organizer and for this, we need to fetch event_id  from request URI. Since no model is provided here so it is required for event_id in URL this also ensures no other endpoints can leak the resource. Also here we are checking for only POST requests thus it will pass the GET requests as it is no checking.

What’s next in permission manager?

Permission has various scopes for improving, I’m still working on a module as part of permission manager which can be used directly in the middle of views and resources so that we can check for permission for specific requests in the middle of any process.

The ability to add logic so that we can leave the check on the basis of some logic may be adding some lambda attributes will work.

Resources