FOSSASIA

Read more about the article Uploading Files via APIs in the Open Event Server

Uploading Files via APIs in the Open Event Server

There are two file upload endpoints. One is endpoint for image upload and the other is for all other files being uploaded. The latter endpoint is to be used for uploading files such as slides, videos and other presentation materials for a session. So, in FOSSASIA’s Orga Server project, when we need to upload a file, we make an API request to this endpoint which is turn uploads the file to the server and returns back the url for the uploaded file. We then store this url for the uploaded file to the database with the corresponding row entry.

Sending Data

The endpoint /upload/file  accepts a POST request, containing a multipart/form-data payload. If there is a single file that is uploaded, then it is uploaded under the key “file” else an array of file is sent under the key “files”.

A typical single file upload cURL request would look like this:

curl -H “Authorization: JWT <key>” -F file=@file.pdf -x POST http://localhost:5000/v1/upload/file

A typical multi-file upload cURL request would look something like this:

curl -H “Authorization: JWT <key>” -F files=@file1.pdf -F files=@file2.pdf -x POST http://localhost:5000/v1/upload/file

Thus, unlike other endpoints in open event orga server project, we don’t send a json encoded request. Instead it is a form data request.

Saving Files

We use different services such as S3, google cloud storage and so on for storing the files depending on the admin settings as decided by the admin of the project. One can even ask to save the files locally by passing a GET parameter force_local=true. So, in the backend we have 2 cases to tackle- Single File Upload and Multiple Files Upload.

Single File Upload

if 'file' in request.files:
        files = request.files['file']
        file_uploaded = uploaded_file(files=files)
        if force_local == 'true':
            files_url = upload_local(
                file_uploaded,
                UPLOAD_PATHS['temp']['event'].format(uuid=uuid.uuid4())
            )
        else:
            files_url = upload(
                file_uploaded,
                UPLOAD_PATHS['temp']['event'].format(uuid=uuid.uuid4())
            )


We get the file, that is to be uploaded using request.files[‘file’] with the key as ‘file’ which was used in the payload. Then we use the uploaded_file() helper function to convert the file data received as payload into a proper file and store it in a temporary storage. After this, if force_local is set as true, we use the upload_local helper function to upload it to the local storage, i.e. the server where the application is hosted, else we use whatever service is set by the admin in the admin settings.

In uploaded_file() function of helpers module, we extract the filename and the extension of the file from the form-data payload. Then we check if the suitable directory already exists. If it doesn’t exist, we create a new directory and then save the file in the directory

extension = files.filename.split('.')[1]
        filename = get_file_name() + '.' + extension
        filedir = current_app.config.get('BASE_DIR') + '/static/uploads/'
        if not os.path.isdir(filedir):
            os.makedirs(filedir)
        file_path = filedir + filename
        files.save(file_path)


After that the upload function gets the settings key for either s3 or google storage and then uses the corresponding functions to upload this temporary file to the storage.

Multiple File Upload

 elif 'files[]' in request.files:
        files = request.files.getlist('files[]')
        files_uploaded = uploaded_file(files=files, multiple=True)
        files_url = []
        for file_uploaded in files_uploaded:
            if force_local == 'true':
                files_url.append(upload_local(
                    file_uploaded,
                    UPLOAD_PATHS['temp']['event'].format(uuid=uuid.uuid4())
                ))
            else:
                files_url.append(upload(
                    file_uploaded,
                    UPLOAD_PATHS['temp']['event'].format(uuid=uuid.uuid4())
                ))


In case of multiple files upload, we get a list of files instead of a single file. Hence we get the list of files sent as form data using request.files.getlist(‘files[]’). Here ‘files’ is the key that is used and since it is an array of file content, hence it is written as files[]. We again use the uploaded_file() function to get back a list of temporary files from the content that has been uploaded as form-data. After that we loop over all the temporary files that are stored in the variable files_uploaded in the above code. Next, for every file in the list of temporary files, we use the upload() helper function to save these files in the storage system of the application.

In the uploaded_file() function of the helpers module, since this time there are multiple files and their content sent, so things work differently. We loop over all the files that are received and for each of these files we find their filename and extension. Then we create directories to save these files in and then save the content of the file with the corresponding filename and extension. After the file has been saved, we append it to a list and finally return the entire list so that we can get a list of all files.

if multiple:
        files_uploaded = []
        for file in files:
            extension = file.filename.split('.')[1]
            filename = get_file_name() + '.' + extension
            filedir = current_app.config.get('BASE_DIR') + '/static/uploads/'
            if not os.path.isdir(filedir):
                os.makedirs(filedir)
            file_path = filedir + filename
            file.save(file_path)
            files_uploaded.append(UploadedFile(file_path, filename))


The upload() function then finally returns us the urls for the files after saving them.

API Response

The file upload endpoint either returns a single url or a list of urls depending on whether a single file was uploaded or multiple files were uploaded. The url for the file depends on the storage system that has been used. After the url or list of urls is received, we jsonify the entire response so that we can send a proper JSON response that can be parsed properly in the frontend and used for saving corresponding information to the database using the other API services.

A typical single file upload response looks like this:

{
     "url": "https://xyz.storage.com/asd/fgh/hjk/12332233.docx"
 }

Multiple file upload response looks like this:

{
     "url": [
         "https://xyz.storage.com/asd/fgh/hjk/12332233.docx",
         "https://xyz.storage.com/asd/fgh/hjk/66777777.ppt"
     ]
 }

You can find the related documentations and example payloads on how to use this endpoint to upload files here: http://open-event-api.herokuapp.com/#upload-file-upload.

 

Reference:

Continue ReadingUploading Files via APIs in the Open Event Server
Read more about the article How User Event Roles relationship is handled in Open Event Server

How User Event Roles relationship is handled in Open Event Server

Users and Events are the most important part of FOSSASIA‘s Open Event Server. Through the advent and upgradation of the project, the way of implementing user event roles has gone through a lot many changes. When the open event organizer server was first decoupled to serve as an API server, the user event roles like all other models was decided to be served as a separate API to provide a data layer above the database for making changes in the entries. Whenever a new role invite was accepted, a POST request was made to the User Events Roles table to insert the new entry. Whenever there was a change in the role of an user for a particular event, a PATCH request was made. Permissions were made such that a user could insert only his/her user id and not someone else’s entry.

def before_create_object(self, data, view_kwargs):
        """
        method to create object before post
        :param data:
        :param view_kwargs:
        :return:
        """
        if view_kwargs.get('event_id'):
            event = safe_query(self, Event, 'id', view_kwargs['event_id'], 'event_id')
            data['event_id'] = event.id

        elif view_kwargs.get('event_identifier'):
            event = safe_query(self, Event, 'identifier', view_kwargs['event_identifier'], 'event_identifier')
            data['event_id'] = event.id
        email = safe_query(self, User, 'id', data['user'], 'user_id').email
        invite = self.session.query(RoleInvite).filter_by(email=email).filter_by(role_id=data['role'])\
                .filter_by(event_id=data['event_id']).one_or_none()
        if not invite:
            raise ObjectNotFound({'parameter': 'invite'}, "Object: not found")

    def after_create_object(self, obj, data, view_kwargs):
        """
        method to create object after post
        :param data:
        :param view_kwargs:
        :return:
        """
        email = safe_query(self, User, 'id', data['user'], 'user_id').email
        invite = self.session.query(RoleInvite).filter_by(email=email).filter_by(role_id=data['role'])\
                .filter_by(event_id=data['event_id']).one_or_none()
        if invite:
            invite.status = "accepted"
            save_to_db(invite)
        else:
            raise ObjectNotFound({'parameter': 'invite'}, "Object: not found")


Initially what we did was when a POST request was sent to the User Event Roles API endpoint, we would first check whether a role invite from the organizer exists for that particular combination of user, event and role. If it existed, only then we would make an entry to the database. Else we would raise an “Object: not found” error. After the entry was made in the database, we would update the role_invites table to change the status for the role_invite.

Later it was decided that we need not make a separate API endpoint. Since API endpoints are all user accessible and may cause some problem with permissions, it was decided that the user event roles would be handled entirely through the model instead of a separate API. Also, the workflow wasn’t very clear for an user. So we decided on a workflow where the role_invites table is first updated with the particular status and after the update has been made, we make an entry to the user_event_roles table with the data that we get from the role_invites table.

When a role invite is accepted, sqlalchemy add() and commit() is used to insert a new entry into the table. When a role is changed for a particular user, we make a query, update the values and save it back into the table. So the entire process is handled in the data layer level rather than the API level.

The code implementation is as follows:

def before_update_object(self, role_invite, data, view_kwargs):
        """
        Method to edit object
        :param role_invite:
        :param data:
        :param view_kwargs:
        :return:
        """
        user = User.query.filter_by(email=role_invite.email).first()
        if user:
            if not has_access('is_user_itself', id=user.id):
                raise UnprocessableEntity({'source': ''}, "Only users can edit their own status")
        if not user and not has_access('is_organizer', event_id=role_invite.event_id):
            raise UnprocessableEntity({'source': ''}, "User not registered")
        if not has_access('is_organizer', event_id=role_invite.event_id) and (len(data.keys())>1 or 'status' not in data):
            raise UnprocessableEntity({'source': ''}, "You can only change your status")

    def after_update_object(self, role_invite, data, view_kwargs):
        user = User.query.filter_by(email=role_invite.email).first()
        if 'status' in data and data['status'] == 'accepted':
            role = Role.query.filter_by(name=role_invite.role_name).first()
            event = Event.query.filter_by(id=role_invite.event_id).first()
            uer = UsersEventsRoles.query.filter_by(user=user).filter_by(event=event).filter_by(role=role).first()
            if not uer:
                uer = UsersEventsRoles(user, event, role)
                save_to_db(uer, 'Role Invite accepted')


In the above code, there are two main functions – before_update_object which gets executed before the entry in the role_invites table is updated, and after_update_object which gets executed after.

In the before_update_object, we verify that the user is accepting or rejecting his own role invite and not someone else’s role invite. Also, we ensure that the user is allowed to only update the status of the role invite and not any other sensitive data like the role_name or email. If the user tried to edit any other field except status, then an error is shown to him/her. However if the user has organizer access, then he/she can edit the other fields of the role_invites table as well. The has_access() helper permission function helps us ensure the permission checks.

In the after_update_object we make the entry to the user event roles table. In the after_update_object from the role_invite parameter we can get the exact values of the newly updated row in the table. We use the data of this role invite to find the user, event and role associated with this role. Then we create a UsersEventsRoles object with user, event and role as parameters for the constructor. Then we use save_to_db helper function to save the new entry to the database. The save_to_db function uses the session.add() and session.commit() functions of flask-sqlalchemy to add the new entry directly to the database.

Thus, we maintain the flow of the user event roles relationship. All the database entries and operation related to users-events-roles table remains encapsulated from the client user so that they can use the various API features without thinking about the complications of the implementations.

 

Reference:

Continue ReadingHow User Event Roles relationship is handled in Open Event Server
Read more about the article Adding Github buttons to Generated Documentation with Yaydoc

Adding Github buttons to Generated Documentation with Yaydoc

Many times repository owners would want to link to their github source code, issue tracker etc. from the documentation. This would also help to direct some users to become a potential contributor to the repository. As a step towards this feature, we added the ability to add automatically generated GitHub buttons to the top of the docs with Yaydoc.

To do so we created a custom sphinx extension which makes use of http://buttons.github.io/ which is an excellent service to embed GitHub buttons to any website. The extension takes multiple config values and using them generates the `html` which it adds to the top of the internal docutils tree using a raw node.

GITHUB_BUTTON_SPEC = {
    'watch': ('eye', 'https://github.com/{user}/{repo}/subscription'),
    'star': ('star', 'https://github.com/{user}/{repo}'),
    'fork': ('repo-forked', 'https://github.com/{user}/{repo}/fork'),
    'follow': ('', 'https://github.com/{user}'),
    'issues': ('issue-opened', 'https://github.com/{user}/{repo}/issues'),
}

def get_button_tag(user, repo, btn_type, show_count, size):
    spec = GITHUB_BUTTON_SPEC[btn_type]
    icon, href = spec[0], spec[1].format(user=user, repo=repo)
    tag_fmt = '<a class="github-button" href="{href}" data-size="{size}"'
    if icon:
        tag_fmt += ' data-icon="octicon-{icon}"'
    tag_fmt += ' data-show-count="{show_count}">{text}</a>'
    return tag_fmt.format(href=href,
                          icon=icon,
                          size=size,
                          show_count=show_count,
                          text=btn_type.title())

The above snippet shows how it takes various parameters such as the user name, name of the repository, the button type which can be one of fork, issues, watch, follow and star, whether to display counts beside the buttons and whether a large button should be used. Another method named get_button_tags is used to read the various configs and call the above method with appropriate parameters to generate each button.

The extension makes use of the doctree-resolved event emitted by sphinx to hook into the internal doctree. The following snippet shows how it is done.

def on_doctree_resolved(app, doctree, docname):
    if not app.config.github_user_name or not app.config.github_repo:
        return
    buttons = nodes.raw('', get_button_tags(app.config), format='html')
    doctree.insert(0, buttons)

Finally we add the custom javascript using the add_javascript method.

app.add_javascript('https://buttons.github.io/buttons.js')

To use this with yaydoc, users would just need to add the following to their .yaydoc.yml file.

build:
  github_button:
    buttons:
      watch: true
      star: true
      issues: true
      fork: true
      follow: true
    show_count: true
    large: true

Resources

  1.  Homepage of Github:buttons – http://buttons.github.io/
  2. Sphinx extension Tutorial – http://www.sphinx-doc.org/en/stable/extdev/tutorial.html
Continue ReadingAdding Github buttons to Generated Documentation with Yaydoc
Read more about the article UI Espresso Test Cases for Phimpme Android

UI Espresso Test Cases for Phimpme Android

Now we are heading toward a release of Phimpme soon, So we are increasing the code coverage by writing test cases for our app. What is a Test Case? Test cases are the script against which we run our code to test the features implementation. It is basically contains the output, flow and features steps of the app. To release app on multiple platform, it is highly recommended to test the app on test cases.

For example, Let’s consider if we are developing an app which has one button. So first we write a UI test case which checks whether a button displayed on the screen or not? And in response to that it show the pass and fail of a test case.

Steps to add a UI test case using Espresso

Espresso testing framework provides APIs to simulate user interactions. It has a concise API. Even, now in new Version of Android Studio, there is a feature to record Espresso Test cases. I’ll show you how to use Recorder to write test cases in below steps.

  • Setup Project Directory

Android Instrumentation tests must be placed in androidTest directory. If it is not there create a directory in app/src/androidTest/java…

  • Write Test Case

So firstly, I am writing a very simple test case, which checks whether the three Bottom navigation view items are displayed or not?

Espresso Testing framework has basically three components:

ViewMatchers

Which helps to find the correct view on which some actions can be performed E.g. onView(withId(R.id.navigation_accounts). Here I am taking the view of accounts item in Bottom Navigation View.

ViewActions

It allows to perform actions on the view we get earlier. E.g. Very basic operation used is click()

ViewAssertions

It allows to assert the current state of the view E.g. isDisplayed() is an assertion on the view we get. So a basic architecture of an Espresso Test case is

onView(ViewMatcher)       
 .perform(ViewAction)     
   .check(ViewAssertion);

We can also Use Hamcrest framework which provide extra features of checking conditions in the code.

Setup Espresso in Code

Add this in your application level build.gradle

// Android Testing Support Library's runner and rules
androidTestCompile "com.android.support.test:runner:$rootProject.ext.runnerVersion"
androidTestCompile "com.android.support.test:rules:$rootProject.ext.rulesVersion"

// Espresso UI Testing dependencies.
androidTestCompile "com.android.support.test.espresso:espresso-core:$rootProject.ext.espressoVersion"
androidTestCompile "com.android.support.test.espresso:espresso-contrib:$rootProject.ext.espressoVersion"

  • Use recorder to write the Test Case

New recorder feature is great, if you want to set up everything quickly. Go to the Run → Record Espresso Test in Android Studio.

It dumps the current User Interface hierarchy and provide the feature to assert that.

You can edit the assertions by selecting the element and apply the assertion on it.

Save and Run the test cases by right click on Name of the class. Run ‘Test Case name’

Console will show the progress of Test case. Like here it is showing passed, it means it get all the view hierarchy which is required by the Test Case.

Resources

Continue ReadingUI Espresso Test Cases for Phimpme Android
Read more about the article Display a List of Registered Repositories in User’s Dashboard

Display a List of Registered Repositories in User’s Dashboard

Yaydoc, our automatic documentation generation and deployment project was recently introduced with the dashboard for its registered users. Introducing a dashboard to Yaydoc enabled us to include a lot of new functionalities and processes associated with users and their registered repositories.

One of the main reasons of creating a dashboard for registered users was to enable them to access and edit configurations of the repositories they registered to Yaydoc. Hence, there was a need to display all of the registered repositories to users. These repositories include the public repositories owned by the users and organizations’ repositories the user has admin access to.

The organizations and users are retrieved separately in parallel using async.js. Async is a utility module which provides straight-forward, powerful functions for working with asynchronous JavaScript. The required functionality is achieved using async.parallel() method which runs the tasks of a collection of functions in parallel, without waiting until the previous function has completed.

async.parallel({
  organizations: function (callback) {
    github.retrieveOrganizations(user.token, function (error, organizations)) {
      callback(null, organizations);
    }
  },
  ownedRepositories: function (callback) {
    Repository.getRepositoriesByOwner(user.username, function (err, repositories)) {
      callback(null, repositories);
    }
  }, function (err, results) {
    res.render(‘dashboard’, {
      title: ‘Dashboard’,
      organizations: results.organizations,
      ownedRepositories: results.ownedRepositories
    });
  }
});

In order to show the organizations’ registered repositories  the github.retrieveOrganizations() function is updated, retrieving registered repositories from Yaydoc’s database. To perform the operation, we use async’s waterfall method. This method runs the tasks in series, passing the result of each function to the next function in that array. This function is needed since we are retrieving the organizations first and then for each organization, we retrieve all the repositories for each organization. To access each organization asynchronously, we use the async.forEachOf() method.

/**
 * Retrieve a list of organization the user has access to
 * Along with the organizations, also retrieve the registered repositories
 * @param accessToken: Access token of the user
 * @param callback
 */
exports.retrieveOrganizations = function (accessToken, callback) {
  async.waterfall([
    function (callback) {
      // Retrieve organizations
      ....
      organizations.push({
        id: organization.id,
        name: organization.login,
        avatar: organization.avatar_url
      });
      return callback(null, organizations);
    },
    function (organizations, callback) {
      async.forEachOf(organizations, function (value, key, callback)) {
        Repository.getRepositoriesByOwner(value.name, function (error, repositories) {
          value.repositories = repositories;
          update.push(value);
          callback()
        });
      }, function (err) {
        callback(null, update);
      }
    }
  ], function (error, result) {
    callback(error, result);
  });
};

Showing the list of repositories, we now plan to make these repositories configurable. Some of these configurations may include deleting the repository and enabling or disabling it from the build process.

Resources

  1. Async utilities for node and browser – https://caolan.github.io/async/
  2. Github’s organization API https://developer.github.com/v3/orgs/
Continue ReadingDisplay a List of Registered Repositories in User’s Dashboard
Read more about the article Electronics Experiments with PSLab

Electronics Experiments with PSLab

Numerous college level electronics experiments can be performed using Pocket Science Lab (PSLab). The Android app and the Desktop app have all the essential features needed to perform these experiments and both these apps have quite a large number of experiments built-in. Some of the common experiments involve the use of BJT (Bipolar Junction Transistor), Zener Diode, FET (Field Effect Transistor), Op-Amp ( Operational Amplifier) etc. This blog walks through the details of performing some experiments using the above commonly used elements.  

The materials required for all the experiments are minimal and includes a few things like PSLab hardware device, components like Diodes, Transistors, Op-Amps etc., connecting wires/jumpers and secondary components like resistors, capacitors etc. Most of these elements would be a part of the PSLab Accessory Kit.

It is recommended to read this blog here, go through the resources mentioned at the end and also get acquainted with construction of circuits before advancing with the experiments mentioned in this blog.

Half Wave and Full Wave Rectifiers

The Bipolar Junction Transistor (BJT) can be used as a rectifier. Rectifiers are needed in circuits to obtain a nearly constant and stable output voltage and prevent any ripples in the circuit. The rectifier can be half wave or full wave depending on whether it rectifies one or both cycles of Alternating Voltage.

The circuit for the Half and Full Wave rectifier is given as follows:

  • Construct the above circuits on a breadboard.
  • For the half wave rectifier, connect the terminals of CH1 and GND of PSLab on the input side and the terminals of CH2 and GND on the output side.
  • The terminals of W1 and GND are also connected on the input side and they are used to generate a sine wave.
  • Use the PSLab Desktop App and open the Waveform Generator in Control. Set the wave type of W1 to Sine and set the frequency at 100 Hz and magnitude to 10mV. Then go ahead and open the Oscilloscope.
  • CH1 would display the input waveform and CH2 will display the output waveform and the plots can be observed.
  • The plot obtained will have rectification in only half of the cycle. In order to obtain rectification in the complete cycle, the full wave rectifier is needed.
  • For the full wave rectifier, the procedure is the same but an additional diode is used. Use an additional channel CH3 to plot the extra input.

  • The plot obtained from the above steps would still have ripples and so a capacitor is placed in parallel to cancel this effect.
  • Place a 100uF/330uF capacitor in parallel to the resistor RL and an additional 1 ohm resistor in the circuit.

BJT Inverter

  • Transistor has a lot of functions. The most common of them is its use as an amplifier. However, transistor can be used as a switch in a circuit i.e. as an inverter.
  • The circuit for this experiment is shown below. For this experiment, it is recommended to use an external 5V DC supply like a battery. Connect the transistor and the diode initially and then connect the resistors accordingly. (Connect the terminals of diode and transistor carefully else they will be damaged).
  • When the circuit is constructed completely, connect CH1 to Vi and CH2 to Vo. Vi and Vo are input and outputs respectively and are marked in the figure.
  • The terminals of W1 and GND are also connected on the input side and they are used to generate a sine wave.
  • Use the PSLab Desktop App and open the Waveform Generator in Control. Set the wave type of W1 to Sine and set the frequency at 200 Hz and magnitude to 10mV.
  • Then go ahead and open the Oscilloscope. Use the X-Y mode of the oscilloscope to obtain the plot between Vi and Vo which should look like the graph shown below.

Common Mode Gain and Differential Mode Gain in Op-Amps

Gain of any amplifier can be calculated by calculating the ratio of the output and input voltage. On plotting the graph in X-Y mode, a Vo vs Vi graph is obtained. The slope of that graph gives us the gain at any particular input voltage.

  • For finding the Differential Mode gain of an Op-Amp, construct the circuit as shown below.
  • When the circuit is constructed completely, connect CH1 to Vi and CH2 to Vo. Vi and Vo are input and outputs respectively and are marked in the figure. The terminals of W1 and GND are also connected on the input side and they are used to generate a sine wave.
  • The power supply provided to the Op-Amp are set to + 12V. (If faced with any confusion, please refer to the resources mentioned at the end of the blog to learn more about Op-Amps before proceeding ahead.)
  • Use the PSLab Desktop App and open the Waveform Generator in Control. Set the wave type of W1 to Sine and set the frequency at 1000 Hz and magnitude to 0.5V. Then go ahead and open the Oscilloscope. Use the X-Y mode of the oscilloscope to obtain the plot between Vi and Vo.
  • For finding the Common Mode gain of the Op-Amp, remove the waveform generator input i.e W1 from R3 and attach it to R2. The rest of the steps remain the same.

Schmitt Trigger

In electronics, a Schmitt trigger is a comparator circuit with hysteresis implemented by applying positive feedback to the noninverting input of a comparator or differential amplifier. It is an active circuit which converts an analog input signal to a digital output signal.

  • Construct the circuit as shown below. Although the diagram shows a variable resistor be used, a constant value resistor would also work fine.
  • When the circuit is constructed completely, connect CH1 to Vi and CH2 to Vo. Vi and Vo are input and outputs respectively and are marked in the figure. The terminals of W1 and GND are also connected on the input side and they are used to generate a sine wave.
  • The power supply provided to the Op-Amp are set to + 12V. (If faced with any confusion, please refer to the resources mentioned at the end of the blog to learn more about Op-Amps before proceeding ahead. If done incorrectly, Op-Amps will be damaged)
  • Use the PSLab Desktop App and open the Waveform Generator in Control. Set the wave type of W1 to Sine and set the frequency at 1000 Hz and magnitude to 0.5V. Then go ahead and open the Oscilloscope. Use the X-Y mode of the oscilloscope to obtain the plot between Vi and Vo which should look like the graph shown below.

Resources:

  1. Read more about Half wave and Full wave rectifier and their applications – https://en.wikipedia.org/wiki/Rectifier
  2. Read more about the Bipolar Junction Transistor and its use as a switch – http://www.electronicshub.org/transistor-as-switch/
  3. Understand the common mode and differential mode of Op-Amp – https://www.allaboutcircuits.com/video-lectures/op-amps-common-differential/
  4. Find more about Schmitt Trigger and its uses – https://en.wikipedia.org/wiki/Schmitt_trigger
Continue ReadingElectronics Experiments with PSLab
Read more about the article Introduction To Kotlin in SUSI Android App

Introduction To Kotlin in SUSI Android App

Lately, we wrote some of the code of SUSI Android App in Kotlin. Kotlin is a very similar language to Java but with much more advantages than Java. It is easy to adapt and learn. There is no doubt that Kotlin is better than Java but with the announcement of Kotlin Support in Google IO’17 for Android development, Kotlin seems a decent way to write code for an Android App.

Advantages of Kotlin over Java

    1. Reduce Boilerplate Code: It helps making development of app faster as it reduces more than 20 percent of boilerplate code. Writing long statements again and again is a headache for developers. Kotlin comes to rescue in that situation.
    2. Removes Null Pointer Exception: Once a large company faced millions of dollars of loss due to null pointer exception. It causes crashes of apps more often than anything else. Thus Kotlin helps in Null checks and makes app free from Null pointer Exceptions.
    3. Interoperable with Java: Kotlin code and Java code are interoperable. Which means you can write half your code in kotlin and half in Java and it will work like a charm. You can call java methods from Kotlin code and vice versa. So, you can simply move your existing Java based app to Kotlin slowly making your app always running.
    4. Lambda and Inline functions: Yes, Kotlin also has functionalities from functional programming languages. Mainly and most widely used feature of those languages is Lambda functions.
    5. Direct Reference of Views by Id: You do not need to write findViewById(R.id.view_name) or use any other library like Butterknife for view binding. You can simply use the view by its id.
    6. No semicolon:  Last but not the least, you do not need to add a semicolon after each statement. In fact, you do not need to add semicolon at all.

Setting up Android Studio to work with Kotlin

If you have latest Android Studio Canary Version, there is already a build support for Kotlin in it. You need not do anything in that case. But if you don’t have the Canary version, you can add Kotlin Plugin in your Android Studio. Follow the below steps to do that.

  1. Install the Kotlin Plugin:

Android Studio → Preferences… →Plugins → Browse Repository → type “Kotlin” in search box → install

  1. Restart your Android Studio and Rebuild the project. Everything else is already set up in SUSI Android App but if you want to do it for your other apps, follow this link.

Implementation in SUSI Android App

So, I am not going to give unnecessary code but will point out specific things where Kotlin helped a lot to reduce unnecessary code and made the code compact.

1. Listeners:

Earlier with Java

Button signup = (Button) findViewById(R.id.sign_up);

signup.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
               startActivity(new Intent(LoginActivity.this, SignUpActivity.class));
            }
        });

Now, with Kotlin

fun signUp() {
   sign_up.setOnClickListener { startActivity(Intent(this@LoginActivity, SignUpActivity::class.java)) }
}

2. Models

With Java

public class MapData {

    private double latitude;
    private double longitude;
    private double zoom;

    public MapData(double latitude, double longitude, double zoom) {
        this.latitude = latitude;
        this.longitude = longitude;
        this.zoom = zoom;
    }

    public double getLatitude() {
        return latitude;
    }

    public void setLatitude(double latitude) {
        this.latitude = latitude;
    }

    public double getLongitude() {
        return longitude;
    }

    public void setLongitude(double longitude) {
        this.longitude = longitude;
    }

    public double getZoom() {
        return zoom;
    }

    public void setZoom(double zoom) {
        this.zoom = zoom;
    }
}

With Kotlin

class MapData (var latitude: Double, var longitude: Double, var zoom: Double)

3. Constructor

With Java

public class LoginPresenter {
    private LoginActivity loginActivity;
    public LoginPresenter(loginActivity: LoginActivity){
        this.loginActivity = loginActivity;
    }
}

With Kotlin

class LoginPresenter(loginActivity: LoginActivity) {
}

Summary

So, this blog was to give you an idea about Kotlin programming language, it’s advantages over java and information how you can set it up on your Android Studio so as to help you a little in understanding the codebase of SUSI Android App a little more.

Resources

  1. Official Kotlin Guide for Syntax Reference and further learning  https://kotlinlang.org/docs/reference/
  2. Blog by Elye on Setting up Kotlin on Android Studio https://android.jlelse.eu/setup-kotlin-for-android-studio-1bffdf1362e8
  3. Youtube Video tutorial by Derek Banas on Kotlin https://www.youtube.com/watch?v=H_oGi8uuDpA
Continue ReadingIntroduction To Kotlin in SUSI Android App
Read more about the article Parsing SUSI.AI Blog Feed

Parsing SUSI.AI Blog Feed

Our SUSI.AI web chat is improving every day.Recently we decided to add a blog page to our SUSI.AI web chat to show all the latest blogs related to SUSI.AI. These blogs are written by our developer team. We have the following feed from which we need to parse the information of blogs and display:

http://blog.fossasia.org/tag/susi-ai/feed/

This feed is in RSS (Really Simple Syndication) format. We decided to use Google feed API service to parse this RSS content, but that service is now deprecated. Then we decided to convert this RSS into JSON format and then we will parse the information.
We have used the rss2json API for converting our RSS content into JSON. We make an ajax call to this API for fetching the JSON content:

$.ajax({
        url: 'https://api.rss2json.com/v1/api.json',
        method: 'GET',
        dataType: 'json',
        data: {
            'rss_url': 'http://blog.fossasia.org/tag/susi-ai/feed/',
            'api_key': api_key: '0000000000000000000000000000000000000000', // put your api key here,
            'count': 50
        }
        }).done(function (response) {
            if(response.status !== 'ok'){ throw response.message; }
            this.setState({ posts: response.items, postRendered: true});
        }.bind(this));

Explanation:

  • URL: This is base URL of API to which we are making calls in order to fetch JSON data.
  • rss_url: This is the URL of RSS feed which needs to be converted into JSON format.
  • api_key: This is the key, which can be generated after making an account on the website
  • count: Count of feed items to return, the default is 20.

The converted JSON response looks like:

This can be checked here.

Now we have used cards of material-ui to show the content of this JSON response. On the success of our ajax call, we update the array(named as posts) present in the initial state of our component with response.items( an array containing information of blogs).

We map through each element in an array named posts and return the corresponding card, containing relevant information in it. We parsed the following information from JSON:

  • Name of the author: posts.author
  • Title of the blog: posts.title
  • Link to blog on WordPress: posts.link

Publish date of the blog:
The publish date is in this format: “2017-08-05 09:05:27” (example), we need to format this date. We used following code to do that:

let date = posts.pubDate.split(' ');
let d = new Date(date[0]);
dateFormat(d, 'dddd, mmmm dS, yyyy') // dateFormat is an npm package

This converts “2017-08-05 09:05:27” to “Saturday, August 5th, 2017”.

Description and Featured Images of the blog:
We needed to show a short description of the blog and a featured image. There is an element in our posts array with name description that contains HTML, starting from featured image and followed by a short description. We needed to convert this HTML into simple text for using this. I used htmlToText npm package for this purpose:
let description = htmlToText.fromString(posts.description).split(‘…’);
description variable now contains simple text. A simple example :

“[https://blog.fossasia.org/wp-content/uploads/2017/08/image2-768×442.png] Our SUSI.AI Web Chat has many static pages like Overview, Devices, Team and Support.We have separate CSS files for each component. Recently, we faced a problem regarding design pattern where CSS files of one component were affecting another component. This blog is all about solving this issue and we take an example of distortion ”

Now, this text contains both link for featured image and our short description text. For getting the link for featured image, I have used regex(Regular Expression) in the following code and saved the link in the variable image and for short description I have used the split function :

let text = description[0].split(']');
let image = susi // temporary image for initialisation
let regExp = /\[(.*?)\]/;
let imageUrl = regExp.exec(description[0]);
if(imageUrl) {
   image = imageUrl[1]
}

After successfully parsing this information from JSON, we can have cards with details and card looks like this:

As we need all this to be rendered when the component mounts, so we put our ajax call inside componentDidMount() function of our react component.

Resources:

Continue ReadingParsing SUSI.AI Blog Feed
Read more about the article Using Mosquitto as a Message Broker for MQTT in loklak Server

Using Mosquitto as a Message Broker for MQTT in loklak Server

In loklak server, messages are collected from various sources and indexed using Elasticsearch. To know when a message of interest arrives, users can poll the search endpoint. But this method would require a lot of HTTP requests, most of them being redundant. Also, if a user would like to collect messages for a particular topic, he would need to make a lot of requests over a period of time to get enough data.

For GSoC 2017, my proposal was to introduce stream API in the loklak server so that we could save ourselves from making too many requests and also add many use cases.

Mosquitto is Eclipse’s project which acts as a message broker for the popular MQTT protocol. MQTT, based on the pub-sub model, is a lightweight and IOT friendly protocol. In this blog post, I will discuss the basic setup of Mosquitto in the loklak server.

Installation and Dependency for Mosquitto

The installation process of Mosquitto is very simple. For Ubuntu, it is available from the pre installed PPAs –

sudo apt-get install mosquitto

Once the message broker is up and running, we can use the clients to connect to it and publish/subscribe to channels. To add MQTT client as a project dependency, we can introduce following line in Gradle dependencies file – 

compile group: 'net.sf.xenqtt', name: 'xenqtt', version: '0.9.5'

[SOURCE]

After this, we can use the client libraries in the server code base.

The MQTTPublisher Class

The MQTTPublisher class in loklak would provide an interface to perform basic operations in MQTT. The implementation uses AsyncClientListener to connect to Mosquitto broker – 

AsyncClientListener listener = new AsyncClientListener() {
    // Override methods according to needs
};

[SOURCE]

The publish method for the class can be used by other components of the project to publish messages on the desired channel – 

public void publish(String channel, String message) {
    this.mqttClient.publish(new PublishMessage(channel, QoS.AT_LEAST_ONCE, message));
}

[SOURCE]

We also have methods which allow publishing of multiple messages to multiple channels in order to increase the functionality of the class.

Starting Publisher with Server

The flags which signal using of streaming service in loklak are located in conf/config.properties. These configurations are referred while initializing the Data Access Object and an MQTTPublisher is created if needed – 

String mqttAddress = getConfig("stream.mqtt.address", "tcp://127.0.0.1:1883");
streamEnabled = getConfig("stream.enabled", false);
if (streamEnabled) {
    mqttPublisher = new MQTTPublisher(mqttAddress);
}

[SOURCE]

The mqttPublisher can now be used by other components of loklak to publish messages to the channel they want.

Adding Mosquitto to Kubernetes

Since loklak has also a nice Kubernetes setup, it was very simple to introduce a new deployment for Mosquitto to it.

Changes in Dockerfile

The Dockerfile for master deployment has to be modified to discover Mosquitto broker in the Kubernetes cluster. For this purpose, corresponding flags in config.properties have to be changed to ensure that things work fine – 

sed -i.bak 's/^\(stream.enabled\).*/\1=true/' conf/config.properties && \
sed -i.bak 's/^\(stream.mqtt.address\).*/\1=mosquitto.mqtt:1883/' conf/config.properties && \

[SOURCE]

The Mosquitto broker would be available at mosquitto.mqtt:1883 because of the service that is created for it (explained in later section).

Mosquitto Deployment

The Docker image used in Kubernetes deployment of Mosquitto is taken from toke/docker-kubernetes. Two ports are exposed for the cluster but no volumes are needed – 

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: mosquitto
  namespace: mqtt
spec:
  ...
  template:
    ...
    spec:
      containers:
      - name: mosquitto
        image: toke/mosquitto
        ports:
        - containerPort: 9001
        - containerPort: 8883

[SOURCE]

Exposing Mosquitto to the Cluster

Now that we have the deployment running, we need to expose the required ports to the cluster so that other components may use it. The port 9001 appears as port 80 for the service and 1883 is also exposed – 

apiVersion: v1
kind: Service
metadata:
  name: mosquitto
  namespace: mqtt
  ...
spec:
  ...
  ports:
  - name: mosquitto
    port: 1883
  - name: mosquitto-web
    port: 80
    targetPort: 9001

[SOURCE]

After creating the service using this configuration, we will be able to connect our clients to Mosquitto at address mosquitto.mqtt:1883.

Conclusion

In this blog post, I discussed the process of adding Mosquitto to the loklak server project. This is the first step towards introducing the stream API for messages collected in loklak.

These changes were introduced in pull requests loklak/loklak_server#1393 and loklak/loklak_server#1398 by @singhpratyush (me).

Resources

Continue ReadingUsing Mosquitto as a Message Broker for MQTT in loklak Server
Read more about the article Updating Settings Activity With MVP Architecture in SUSI Android

Updating Settings Activity With MVP Architecture in SUSI Android

SUSI Android app includes settings that allow users to modify app features and behaviours. For example, SUSI Android allows users to specify whether they want voice output i.e text to speech irrespective of input type.

Currently different settings available in SUSI Android are:

Settings                                          Use
Enter As Send It allows users to specify whether they want to use action button of soft keyboard as carriage return or send message
Mic Input It allows users to specify whether they want to use speech for input type or not. User can also use keyboard to provide input.
Speech Always It allows users to specify whether they want speech output i.e text to speech irrespective of input type
Speech Output It allows users to specify whether they want speech output in case of speech input or not.
Select Server It allows user to specify whether they want to use default susi_server or their own server.
Language It allows users to select text to speech engine language.

Android’s standard architecture isn’t always sufficient, especially in the case of complex applications that need to be tested regularly. So we need an architecture that can improve the app’s testability and MVP(Model View Presenter) architecture is one of the best options for that.

Working with MVP architecture to show settings

MVP architecture creates three layers: Model, View and Presenter. Use of each layer is described below

  • Model: The Model holds the business logic of the application. It controls how data can be created, stored, and modified.
  • View: The View is a UI that displays data and routes user actions to the Presenter.
  • Presenter: Presenter is a mediator between View and Model. It retrieves data from the Model and shows it in the View. It also processes user actions forwarded to it by the View.

Steps followed to implement MVP architecture are:

1. I first created three interfaces: ISettingsView, ISettingModel and  ISettingsPresenter. These classes contain all the important functions we needed in model, presenter and view. Presenter’s interaction with model is handled by  ISettingModel and presenter’s interaction with view is handled by ISettingsPresenter and ISettingView.

  1. I created model, view and presenter class i.e SettingModel, SettingsPresenter and SettingsActivity and implemented ISettingModel, ISettingsPresenter and ISettingsView in SettingModel, SettingPresenter and SettingsActivity respectively so that presenter can communicate with both model and view.
class SettingsPresenter(fragmentActivity: FragmentActivity): ISettingsPresenter, ISettingModel.onSettingFinishListener
class SettingsActivity : AppCompatActivity(), ISettingsView
class SettingModel: ISettingModel
  1. After that, I created ChatSettingsFragment class. It is a PreferenceFragment class which resides in SettingsActivity and contains all settings.
class ChatSettingsFragment : PreferenceFragmentCompat() 

How Presenter communicates with Model and View

As already mentioned presenter’s interaction with view is handled by ISettingsView and ISettingsPresenter. So first I created object of ISettingsPresenter in ChatSettingsFragment, which is part of view, as shown below

lateinit var settingsPresenter: ISettingsPresenter
settingsPresenter = SettingsPresenter(activity)

And used it to call method of presenter, SettingsPresenter

settingsPresenter.sendSetting(Constant.ENTER_SEND, newValue.toString())

After that, I created an object of ISettingsView in SettingsPresenter, so that it can communicate with view, SettingsActivity.

Presenter’s interaction with model is handled by ISettingModel. So first I created an object of SettingModel class which implemented ISettingModel and then used it to call methods of SettingModel.

var settingModel: SettingModel = SettingModel()
settingModel.sendSetting(key, value, this)

Here the third parameter in the sendSetting method is ISettingModel.onSettingFinishListener which is part of ISettingModel. SettingModel used it to communicate with SettingsPresenter as shown below

override fun sendSetting(key: String, value: String, listener: ISettingModel.onSettingFinishListener) {

settingResponseCall.enqueue(object : Callback<ChangeSettingResponse> {

 override fun onResponse(call: Call<ChangeSettingResponse>?, response:

Response<ChangeSettingResponse>) {

          listener.onSuccess(response)

      }

  })

}

onSuccess method present in SettingsPresenter.

Reference

Continue ReadingUpdating Settings Activity With MVP Architecture in SUSI Android