API

Read more about the article Password Recovery Link Generation in SUSI with JSON

Password Recovery Link Generation in SUSI with JSON

In this blog, I will discuss how the SUSI server processes requests using JSON when a request for password recovery is made.. The blog post will also cover some parts of the client side implementation as well for better insight. All the clients function in the same way. When you click on forget password button, client asks you that whether you want to recover password for your own custom server or the standard one. This choice of user defines the base URL where to make the forget password request. If you select custom server radio button, then you will be asked to enter the URL to your server, Otherwise standard base URL will be used. The API endpoint used will be /aaa/recoverpassword.json The client will make a POST request with “forgotemail” parameter containing the email id of the user making the request. Following the email id provided, server generates a client identity if and only if the email id is registered. If email id is not found in the database, server throws an exception with error code 422. String usermail = call.get("forgotemail", null); ClientCredential credential = new ClientCredential(ClientCredential.Type.passwd_login, usermail); ClientIdentity identity = new ClientIdentity(ClientIdentity.Type.email, credential.getName()); if (!DAO.hasAuthentication(credential)) { throw new APIException(422, "email does not exist"); } If the email id is found to be registered against a valid user in the database, call to a method is made which returns a random string of length passed in as a parameter. This returned random string acts as a token. Below is the implementation of the createRandomString(int length) method. public static String createRandomString(Integer length){ char[] chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789".toCharArray(); StringBuilder sb = new StringBuilder(); Random random = new Random(); for (int i = 0; i < length; i++) { char c = chars[random.nextInt(chars.length)]; sb.append(c); } return sb.toString(); } This method is defined in AbsractAPIHandler class. The function createRandomString(int length) initialises an array with alphabet (both upper and lower cases) and integers 1 to 10. An object of StringBuilder is declared and initialised in the fol loop. Next, the token generated is hashed against the user’s email id. Since we have used a token of length 30, there will be 30! (30 factorial) combinations and hence chances of two tokens to be same is ver very low (approximately Zero). Validity for token is set for 7 days (i.e. one week). After that the token will expire and to reset the password a new token will be needed. String token = createRandomString(30); ClientCredential tokenkey = new ClientCredential(ClientCredential.Type.resetpass_token, token); Authentication resetauth = new Authentication(tokenkey, DAO.passwordreset); resetauth.setIdentity(identity); resetauth.setExpireTime(7 * 24 * 60 * 60); resetauth.put("one_time", true); Everything is set by now. Only thing left is send a mail to the user. For that we call a method sendEmail() of EmailHandler class. This requires 3 parameters. User email id, subject for the email, and the body of the email. The body contains a verification link. To get this verification link, getVerificationMailContent(String token) is called and token generated in the previous step is sent to it as a parameter.…

Continue ReadingPassword Recovery Link Generation in SUSI with JSON
Read more about the article Controlling Camera Actions Using Voice Interaction in Phimpme Android

Controlling Camera Actions Using Voice Interaction in Phimpme Android

In this blog, I will explain how I implemented Google voice actions to control camera features on the Phimpme Android project. I will cover the following features I have implemented on the Phimpme project: Opening the application using Google Voice command. Switching between the cameras. Clicking a Picture and saving it through voice command. Opening application when the user gives a command to Google Now.                       When the user gives command “Take a selfie” or “Click a picture” to Google Now it directly opens Phimpme camera activity.  First                                                                                                                                        We need to add an intent filter to the manifest file so that Google Now can  detect Phimpme camera activity <activity   android:name=".opencamera.Camera.CameraActivity"   android:screenOrientation="portrait"   android:theme="@style/Theme.AppCompat.NoActionBar">   <intent-filter>       <action android:name="android.media.action.IMAGE_CAPTURE"/>       <category android:name="android.intent.category.DEFAULT"/>       <category android:name="android.intent.category.VOICE"/>   </intent-filter> </activity> category android:name=”android.intent.category.VOICE” is added to the IMAGE_CAPTURE intent filter for the Google Now to detect the camera activity. For the Google Now assistance to accept the command in the camera activity we need to add the following in the STILL_IMAGE_CAMERA intent filter in the camera activity. <intent-filter>   <action android:name="android.media.action.STILL_IMAGE_CAMERA"/>   <category android:name="android.intent.category.DEFAULT"/>   <category android:name="android.intent.category.VOICE"/> </intent-filter> So, now when the user says “OK Google” and “Take a Picture” the camera activity on Phimpme opens. Integrating Google Voice assistance in Camera Activity Second,                                                                                                                               After opening the camera application the Google Assistance should ask a question. The cameraActivity in Phimpme can be opened in two ways, such as: When opened from a different application. When given the command to Goole Now assistance. We need to check whether the camera activity is prompted from Google assistance or not to activate voice command. We will check it in onResume function. @Override public void onResume() { if (CameraActivity.this.isVoiceInteraction()) {     startVoiceTrigger();  } } If is.VoiceInteraction gives “true” then voice Assistance prompts.             Assistance to ask which camera to use Third,                                                                                                                              …

Continue ReadingControlling Camera Actions Using Voice Interaction in Phimpme Android

How to Implement Memory like Servlets in SUSI.AI

In this blog, I'll be discussing about how a server gets previous messages from the Log files in SUSI server. SUSI AI clients, Android, iOS and web chat, follow a very simple rule. Whenever a user logs in to the app, the app makes a http GET call to the server in the background and in response, server returns the chat history. Link to the API endpoint -> http://api.susi.ai/susi/memory.json But parsing a lot of data might depend on the connection speed. If the connection is poor or lacking speed, the history would cost user’s time. To prevent this, server by default returns last 10 pair of messages. It is up to the client that how many messages they want to render. So for example, if the client requests last 5 messages, then the client has to make a GET request and pass the cognitions parameter. Hence the modified end point will be : http://api.susi.ai/susi/memory.json?cognitions=2 But how does the server process it? Let us see. Browse to susi_server/src/ai/susi/server/api/susi/UserService.java file. This is the main working servlet. If you are new and wondering how servlets for susi are implemented, Please go through this first how-to-add-a-new-servletapi-to-susi-server This is how serviceImpl() method looks like : @Override public ServiceResponse serviceImpl(Query post, HttpServletResponse response, Authorization user, final JsonObjectWithDefault permissions) throws APIException { int cognitionsCount = Math.min(10, post.get("cognitions", 10)); String client = user.getIdentity().getClient(); List<SusiCognition> cognitions = DAO.susi.getMemories().getCognitions(client); JSONArray coga = new JSONArray(); for (SusiCognition cognition: cognitions) { coga.put(cognition.getJSON()); if (--cognitionsCount <= 0) break; } JSONObject json = new JSONObject(true); json.put("cognitions", coga); return new ServiceResponse(json); } In the first step, we find the minimum of default value (i.e. 10) and the value of cognitions received as GET parameter. Messages equivalent to minimum variable are encoded in JSONArray and sent to the client. Whenever the server receives a valid signup request, It makes a directory with the name “email_emailid”. In this directory, a log.txt file is maintained which stores all the queries along with the other details associated with it. For example if user has signed up with the email id example@example.com, Then the path of this directory will be /data/susi/email_example@example.com. If a user queries “http://api.susi.ai/susi/chat.json?timezoneOffset=-330&q=flip+a+coin”,  then { "query": "flip a coin", "count": 1, "client_id": "", "query_date": "2017-06-30T12:22:05.918Z", "answers": [{ "data": [{ "0": "flip a coin", "token_original": "coin", "token_canonical": "coin", "token_categorized": "coin", "timezoneOffset": "-330", "answer": "tails", "skill": "/susi_skill_data/models/general/entertainment/en/flip_coin.txt", "_etherpad_dream": "cricket" }, "metadata": { "count": 1 }, "actions": [{ "type": "answer", "expression": "tails" }], "skills": ["/susi_skill_data/models/general/entertainment/en/flip_coin.txt"] }], "answer_date": "2017-06-30T12:22:05.928Z", "answer_time": 10, "language": "en" } The server has user’s identity. It will use this identity and store (will be appended) in the respective log file. The next steps in retrieving the message are pretty easy and includes getting the identity of the current user session. Use this identity to populate the JSONArray named coga. This is finally encoded in a JSONObject along with other basic details and returned to the clients where they render the data received, and show the messages in an appropriate way. Resources Start your own Jetty Server How…

Continue ReadingHow to Implement Memory like Servlets in SUSI.AI
Read more about the article Share Images on Pinterest from Phimpme Android Application

Share Images on Pinterest from Phimpme Android Application

After successfully establishing Pinterest authentication in Phimpme our next goal was to share the image on the Pinterest website directly from Phimpme, without using any native Android application. Adding Pinterest Sharing option in Sharing Activity in Phimpme To add various sharing options in Sharing Activity in the Phimpme project, I have applied a ScrollView for the list of the different sharing options which include: Facebook, Twitter, Pinterest, Imgur, Flickr and Instagram. All the App icons with the name are arranged in a TableLayout in the activity_share.xml file. Table rows consist of two columns. In this way, it is easier to add more app icons for future development. <ScrollView   android:layout_width="wrap_content"   android:layout_height="@dimen/scroll_view_height"   android:layout_above="@+id/share_done"   android:id="@+id/bottom_view">   <LinearLayout       android:layout_width="wrap_content"       android:layout_height="wrap_content"       android:orientation="vertical">       <TableLayout Adding Pinterest app icon on the icons_drawable array. This array is then used to inflate the icon on the list view. private int[] icons_drawables = {R.drawable.ic_facebook_black, R.drawable.ic_twitter_black,R.drawable.ic_instagram_black, R.drawable.ic_wordpress_black, R.drawable.ic_pinterest_black); Adding Pinterest text on the titles_text array. This array is then used to inflate the names of the various sharing activity. private int[] titles_text = {R.string.facebook, R.string.twitter, R.string.instagram,       R.string.wordpress, R.string.pinterest); Prerequisites to share Image on Pinterest To share an Image on Pinterest a user has to add a caption and Board ID. Our first milestone was to get the input of the Board ID  by the user. I have achieved this by taking the input in a Dialog Box. When the user clicks on the Pinterest option, a dialog box pops and then the user can add their Board ID. private void openPinterestDialogBox() {   AlertDialog.Builder captionDialogBuilder = new AlertDialog.Builder(SharingActivity.this, getDialogStyle());   final EditText captionEditText = getCaptionDialog(this, captionDialogBuilder);   captionEditText.setHint(R.string.hint_boardID);   captionDialogBuilder.setNegativeButton(getString(R.string.cancel).toUpperCase(), null);   captionDialogBuilder.setPositiveButton(getString(R.string.post_action).toUpperCase(), new DialogInterface.OnClickListener() {       @Override       public void onClick(DialogInterface dialog, int which) {           //This should be empty it will be overwrite later           //to avoid dismiss of the dialog on the wrong password       }   });   final AlertDialog passwordDialog = captionDialogBuilder.create();   passwordDialog.show();   passwordDialog.getButton(AlertDialog.BUTTON_POSITIVE).setOnClickListener(new View.OnClickListener() {       @Override       public void onClick(View v) {           String captionText = captionEditText.getText().toString();           boardID =captionText;           shareToPinterest(boardID);           passwordDialog.dismiss();       }   }); } A user can fetch the Board ID by following the steps: Copy the Board URL from the Pinterest main page. Paste the URL in the dialog box of the website: https://www.nutt.net/how-do-i-get-pinterest-board-id/ Copy the Board ID from the website and then past it in the AlertDialog Box. Board ID is necessary because it specifies where the image needs to be posted. Creating custom post function for Phimpme The image is posted using a function in PDKClient class. PDKClient is found in the PDK module which we get after importing Pinterest SDK. Every image is posted on Pinterest is called a Pin. So we will call createPin function. I have made my custom createPin function so that it also accepts Bitmaps as a parameter. In the Pinterest SDK it only accepts image URL to share, The image should already be on the internet to be shared. For this reason, we to add a custom create Pin function to accept Bitmaps as an option. public void createPin(String note, String boardId, Bitmap image, String link, PDKCallback callback) {   if (Utils.isEmpty(note)…

Continue ReadingShare Images on Pinterest from Phimpme Android Application
Read more about the article Using Marshmallow Fields in Open Event API Server

Using Marshmallow Fields in Open Event API Server

The nextgen Open Event API Server  provides API endpoints to fetch the data, and to modify and update it. These endpoints have been written using flask-rest-jsonapi, which is a flask extension to build APIs around the specifications provided by JSONAPI 1.0. This extension helps you, quoting from their website: flask-rest-jsonAPI’s data abstraction layer lets us expose the resources in a flexible way. This is achieved by using Marshmallow fields by marshmallow-jsonapi. This blog post explains how we use the marshmallow fields for building API endpoints in Open Event API Server. The marshmallow library is used to serialize, deserialize and validate input data. Marshmallow uses classes to define output schemas. This makes it easier to reuse and configure the and also extend the schemas. When we write the API Schema for any database model from the Open Event models, all the columns have to be added as schema fields in the API class. This is the API Server’s event schema using marshmallow fields: These are the Marshmallow Field classes for various types of data. You can pass the following parameters when creating a field object. The ones which are used in the API Server as described below. For the rest, you can read more on marshmallow docs. Let’s take a look at each of these fields. Each of the following snippets sample writing fields for API Schema. identifier = fields.Str(dump_only=True) This is a field of data-type String. dump_only :  This field will be skipped during deserialization as it is set to True here. Setting this true essentially means marking `identifier` as read-only( for HTTP API)  name = fields.Str(required=True) This again is a field of data-type String. This is a required field and a ValidationError is raised if found missing during deserialization. Taking a look at the database backend: Since this field is set to non-nullable in the database model, it is made required in API Schema.  external_event_url = fields.Url(allow_none=True) This is a field of datatype URL. Since this is not a required field, NULL values are allowed for this field in the database model. To reflect the same in the API, we have to add allow_none=True. If missing=None is unset, it defaults to false. ends_at = fields.DateTime(required=True, timezone=True) Field of datatype DateTime It is a required field for an event and the time used here is timezone aware. latitude = fields.Float(validate=lambda n: -90 <= n <= 90, allow_none=True) Field of datatype Float. In marshmallow fields, we can use validator clauses on the input value of a field using the validate: parameter. It returns a boolean, which when false raises a Validation Error. These validators are called during deserialization. is_map_shown = fields.Bool(default=False) Field of datatype boolean. Default value for the marshmallow fields can be set by defining using default: Here, is_map_shown attribute is set to false as default for an event. privacy = fields.Str(default="public") privacy is set to default “public”. When the input value for a field is missing during serialization, the default value will be used. This parameter can either be a value or a callable. As described in the examples above, you can…

Continue ReadingUsing Marshmallow Fields in Open Event API Server
Read more about the article Adding Pinterest Integration in Phimpme Android

Adding Pinterest Integration in Phimpme Android

After establishing Facebook and Twitter share in Phimpme, our next goal was to integrate more social networking sites. Pinterest is an ideal website for us as it is widely used among various photography enthusiasts. Our goal was to share the image on the Pinterest website without the use of any other native android application. Note: First, we added Phimpme app in the developer section https://developers.pinterest.com/apps/. This step is crucial as it generates APP ID and it is necessary for authentication. Adding Pinterest option in Accounts Activity in Phimpme In accounts activity, we have added a list of accounts that we have integrated in a Recyclerview. Currently, we have integrated Facebook and Twitter. We need to add Pinterest App icon and Pinterest option in the Recyclerview. To add Pinterest app icon I downloaded the icon from iconfinder in SVG format. Using SVG format allows icon to resize itself according to the screen sizes of the users. We saved the icon file name as ic_pinterest_black.xml Pinterest icon SVG: <vector android:height="24dp" android:viewportHeight="32.0"   android:viewportWidth="32.0" android:width="24dp" xmlns:android="http://schemas.android.com/apk/res/android">   <path android:fillColor="#231F20" android:pathData="M16,0.27C7.16,0.27 0,7.43 0,16.27c0,6.55 3.94,12.18 9.58,14.65c-0.05,-1.12 -0.01,-2.46 0.28,-3.67c0.31,-1.3 2.06,-8.72 2.06,-8.72s-0.51,-1.02 -0.51,-2.53c0,-2.37 1.37,-4.14 3.08,-4.14c1.46,0 2.16,1.09 2.16,2.4c0,1.46 -0.93,3.65 -1.41,5.68c-0.4,1.7 0.85,3.08 2.53,3.08c3.03,0 5.07,-3.89 5.07,-8.51c0,-3.51 -2.36,-6.13 -6.66,-6.13c-4.85,0 -7.88,3.62 -7.88,7.66c0,1.39 0.41,2.38 1.05,3.14c0.29,0.35 0.34,0.49 0.23,0.89C9.51,20.37 9.33,21.08 9.26,21.36c-0.11,0.4 -0.44,0.55 -0.8,0.4c-2.23,-0.91 -3.28,-3.36 -3.28,-6.11c0,-4.55 3.83,-9.99 11.44,-9.99c6.11,0 10.13,4.42 10.13,9.16c0,6.28 -3.49,10.97 -8.63,10.97c-1.73,0 -3.35,-0.93 -3.91,-1.99c0,0 -0.93,3.69 -1.13,4.4c-0.34,1.23 -1,2.46 -1.61,3.43C12.9,32.04 14.43,32.27 16,32.27c8.84,0 16,-7.16 16,-16S24.84,0.27 16,0.27z"/> </vector> We made an array of all the accounts list in the Accounts Activity. I added Pinterest in the array list. This array list is inflated in the Accounts Activity. public static String[] accountName = {"Facebook", "Twitter", "Drupal", "NextCloud", "Wordpress", "Pinterest"}; To add the icon the recyclerview, we have to get the resource id from the drawable folder and then set is in the recyclerview. We have done it dynamically so that we don’t have to fetch the id of every accounts icons. Integer id = getContext().getResources().getIdentifier(context.getString(R.string.ic_) +               (accountName[position].toLowerCase()) + "_black"       , context.getString(R.string.drawable)       , getContext().getPackageName()); holder.accountAvatar.setImageResource(id); Importing Pinterest SDK in Phimpme project To import the Pinterest SDK in the project, please download the Pinterest SDK from the link: http://assets.pinterest.com/sdk/android-pdk.tar. Go to File->import new module->Import Gradle Project. Build the project, if there is any error resolve the error and then build Gradle again. compile project(‘:pdk’) Add APP ID in the manifest file of the project We need to add the APP ID that was generated while making the app the Pinterest in the manifest folder of the Phimpme Android. In the case of Phimpme, we have added the APP ID in the intent filter in the Accounts Activity. This is necessary for the interaction of the Phimpme with the Pinterest website. <activity   android:name=".accounts.AccountActivity"   android:screenOrientation="portrait"   android:theme="@style/Theme.AppCompat.NoActionBar">   <intent-filter>       <action android:name="android.intent.action.VIEW" />       <category android:name="android.intent.category.DEFAULT" />       <category android:name="android.intent.category.BROWSABLE" />       <data android:scheme="pdk4910600717739247160" />   </intent-filter> </activity> Establishing Pinterest authentication in Accounts Activity First, we need to import all the necessary class from the Pinterest SDK. These class includes callback functions, PDKClient which manages the interface between the Phimpme app and the Pinterest website. We need to pass the APP ID in the PDKClient…

Continue ReadingAdding Pinterest Integration in Phimpme Android
Read more about the article Working With Inter-related Resource Endpoints In Open Event API Server

Working With Inter-related Resource Endpoints In Open Event API Server

The Open Event API Server provides endpoints to access and modify data. The nextgen is built using flask-rest-jsonapi. In this library we create resource classes for each database model. The resource class has three classes for list, detail and relationship. For each resource object we have the endpoints related to it - GET, POST for List - GET, PATCH, DELETE for Detail - GET, POST, PATCH, DELETE for Relationship In this blogpost I will discuss how the resource inter-related endpoints work. These are the endpoints which involve two resource objects which are also related to another same resource object. The discussion in this post is of the endpoints related to Sessions Model. In the API server, there exists a relationship between event and sessions. Apart from these, session also has relationships with microlocations, tracks, speakers and session-types. Let’s take a look at the endpoints related with the above. `/v1/tracks/<int:track_id>/sessions` is a list endpoint which can be used to list and create the sessions related to a particular track of an event. To get the list we define the query() method in ResourceList class as such: The query method is executed for GET requests, so this if clause looks for track_id in view_kwargs dict. When the request is made to `/v1/tracks/<int:track_id>/sessions`, track id will be present as ‘track_id in the view_kwargs. The tracks are filtered based on the id passed here and then joined on the query with all sessions object from database. For the POST method, we need to add the track_id from view_kwargs to pass into the track_id field of database model. This is achieved by using flask-rest-jsonapi’s before_create_object() method. The implementation for track_id is the following: When a POST request is made to `/v1/tracks/<int:track_id>/sessions`, the view_kwargs dict will have ‘track_id’ in it. So if track_id is present in the url params, we first ensure that a track with the passed id exists, then only proceed to create a sessions object under the given track. Now the safe_query() method is a generic custom method written to check for such things. The model is passed along with the filter attribute, and a field to include in the error message. This method throws an ObjectNotFound exception if no such object exists for a given id. We also need to take care of the permissions for these endpoints. As the decorators are called even before schema validation, it was difficult to get the event_id for permissions unless adding highly endpoint-specific code in the permission manager core, leading to loss of generality.  So the leave_if parameter of permission check was used to overcome this issue. Since the permissions manager isn’t fully developed yet, this is to be changed in the improved implementation of the permissions manager. Similar implementations for micro locations and session types was done. All the same is not explained in this blogpost. For extended code, take a look at the source code of this schema. For speaker relation, a few things were different. This is because speakers-sessions is a many-to-many relationship. Let’s take a look at this: As it…

Continue ReadingWorking With Inter-related Resource Endpoints In Open Event API Server
Read more about the article Create Scraper in Javascript for Loklak Scraper JS

Create Scraper in Javascript for Loklak Scraper JS

Loklak Scraper JS is the latest repository in Loklak project. It is one of the interesting projects because of expected benefits of Javascript in web scraping. It has a Node Javascript engine and is used in Loklak Wok project as bundled package. It has potential to be used in different repositories and enhance them. Scraping in Python is easy (at least for Pythonistas) as one needs to just import Request library and BeautifulSoup library (lxml as better option), write some lines of code using Request library to get webpage and some lines of bs4 to walk through html and scrape data. This sums up to about less than a hundred lines of coding, where as Javascript coding isn't easily readable (at least to me) as compared to Python. But it has an advantage, it can easily deal with Javascript in the pages we are scraping. This is one of the motive, Loklak Scraper JS repository was created and we contributed and worked on it. I recently coded a Javascript scraper in loklak_scraper_js repository. While coding, I found it’s libraries similar to the libraries, I use to code in Python. Therefore, this blog is for Pythonistas how they can start scraping in Javascript as they finish reading and also contribute to Loklak Scraper JS. First, replace Python interpreter, Request and Beautifulsoup library with Node JS interpreter, Request and Cheerio JS library. 1) Node JS Interpreter: Node JS Interpreter is used to interpret Javascript files. This is different from Python as it deals with the project instead of a module in case of Python. The most compatible Node for most of the libraries is 6.0.0 , where as latest version available(as I checked) is 8.0.0 TIP: use `--save` with npm like here while installing a library. 2) Request Library :- This is used to load webpage to be processed. Similar to one in Python. Request-promise library, a wrapper around Request with implementation of Bluebird library, improves readability and makes code cleaner (how?).   3) Cheerio Library:- A Pythonista (a rookie one) can call it twin of BeautifulSoup Library. But this is faster and is Javascript. It's selector implementation is nearly identical to jQuery's. Let us code a basic Javascript scraper. I will take TimeAndDate scraper from loklak_scraper_js as example here. It inputs place and outputs its local time. Step#1: fetching HTML from webpage with the help of Request library. We input url to Request function to fetch the webpage and is saved to `html` variable. This scrapeTimeAndDate() function scrapes data from html url = "http://www.timeanddate.com/worldclock/results.html?query=London"; request(url, function(error, response, body) { if(error) { console.log("Error: " + error); process.exit(-1); } html = body; scrapeTimeAndDate() });   Step#2: To scrape important data from html using Cheerio JS list of date and time of locations is embedded in table tag, So we will iterate through <td> and extract text. a) Load html to Cheerio as we do in beautifulsoup In Python soup = BeautifulSoup(html,'html5lib')   In Cheerio JS $ = cheerio.load(html);   b) This line finds…

Continue ReadingCreate Scraper in Javascript for Loklak Scraper JS
Read more about the article Isolating each request in Dredd testing for Open Event Server

Isolating each request in Dredd testing for Open Event Server

In the Open Event Server, we are using Api-blueprint along with aglio for API documentation in the project. The foremost concern with any API documentation is making sure it remains updated to the actual implementation of the API backend. It was only a matter of days that we realized we needed some sort of testing mechanism for the API documentation. Dredd came to our rescue in that moment. Quoting the official documentation: Dredd is a language-agnostic command-line tool for validating API description document against backend implementation of the API. Dredd reads your API description and step by step validates whether your API implementation replies with responses as they are described in the documentation.  For loading DB fixtures, handling authentication, sessions etc. Dredd Hooks are used. Quoting their official documentation: Similar to any other testing framework, Dredd supports executing code around each test step. Hooks are code blocks executed in defined stage of execution lifecycle. In the hooks code you have an access to compiled HTTP transaction object which you can modify. Now after setting up dredd testing successfully, the challenge was to make every request remains isolated with respect to the database i.e making sure that database changes by one request do not affect the database used by another request, similar to what is done in case of unit tests ;). In unit tests, you can just access the session of the tests after it is completed and gracefully rollback the changes made by the test. But as dredd is completely separated from the API backend being tested, accessing that session is not possible. So the only choice that remains is purging the database after each request and recreating it with the required data before each request. To do the above, we initialize a vanilla flask app and attach a database to it: @hooks.before_all def before_all(transaction):    app = Flask(__name__)    app.config.from_object('config.TestingConfig')    db.init_app(app)    Migrate(app, db)    stash['app'] = app    stash['db'] = db Before each request, the pre-existing database is purged to remove the changes from the previous request. After that a new db is created with necessary repopulation. @hooks.before_each def before_each(transaction):    with stash['app'].app_context():        db.engine.execute("drop schema if exists public cascade")        db.engine.execute("create schema public")        db.create_all()        stamp()        populate_db_with_required_static_values() Database fixtures specific to the request can be loaded in the before hook in the following way: @hooks.before("Users > Users Collection > List All Users") def user_get_list(transaction):    """    GET /users    :param transaction:    :return:    """    with stash['app'].app_context():        user = UserFactory()        db.session.add(user)        db.session.commit() Relevant links: API-Blueprint: https://apiblueprint.org/ Dredd: https://github.com/apiaryio/dredd Dredd-hooks-python: https://github.com/apiaryio/dredd-hooks-python

Continue ReadingIsolating each request in Dredd testing for Open Event Server
Read more about the article Uploading Images via APIs in the Open Event Server

Uploading Images via APIs in the Open Event Server

APIs help us to send and receive data in some particular data format that can then be used individually or integrated with a frontend UI. In our case, the entire API server is used to manage all the requests from the frontend and send back the necessary response. Usually, the application is to send simple form data which is then stored into the backend database and a valid jsonapi response is shown. However other than normal text, url, datetime data one very important data is media files, in our case event images, user images, slides, etc. In this blog, we will particularly deal with how we can upload images in the server using API. Sending Data Firstly, we need to decide how do we send the data in the post request of the API. So we are sending a base64 encoded string representing the image along with the image extension appended to it, for example, data:image/png;base64,iVBORw0KGgoAAAANS. This is a widely used format for showing images over the web. So when we send a POST request we send a json encoded body like: { "data": "data:image/png;base64,iVBORw0KGgoAAAANS" } Converting Base64 Data to Image There are 2 parts of the data in the string that we receive. The first part basically tells us the format of the image(.gif in this case) and string encoding(base64 in this case), the second part gives us the encoded string. So, from the first part, we extract the file extension for the image file to be created. We use uuid.uuid4() for a random filename. filename = '{}.{}'.format(str(uuid.uuid4()).data.split(";")[0].split("/")[1]) Now to write the base64 encoded string as an image file, we first need to get only the encoded string part from the data and then decode it. We use string decode function of python for the decoding purpose. Then we write the data to the image file and save it. file.write(data.split(",")[1].decode("base64") API Response Finally using whatever logic you are using for serving your static files, you generate the static file path for the image saved. And then create another json encoded response which returns you the url for the saved image in the server. { "url": "https://xyz.storage.com/asd/fgh/hjk/1233456.png" } And there you have your image uploaded and served.

Continue ReadingUploading Images via APIs in the Open Event Server