PSLab Desktop Architecture

In this blog post we will talk about the software architecture that we have followed in PSLab Desktop. We will discuss in detail about how the three components, ie – Electron, React and Python gel together in our architecture and give the user a smooth, high performance, cross platform experience.

The picture below is all there is to the PSLab Desktop architecture. I have numbered all the main components and will break them down in later section of the article. Once you get this architecture going on in the back of your head while writing the app, you would be able to think clearly and code more efficiently.


An overview of the whole architecture

I am sure there are many other approaches available to build an electron app, but for the stack I used, this architecture turned out to be the only working model that went all the way from development to production and proved its stability after a lot of experimentation.

The PSLab Desktop happens to be a software interface that needs to interact and fetch data from an actual hardware device at a very fast rate and display the results in real time. The stability of this architecture has been tested under various use cases of the PSLab Desktop.


Let’s break it down

There are 4 major parts in the whole architecture, let us talk about them one by one.

1. Main Process

If you read the documentation of electron, it states that the purpose of the main process is to create and manage browser windows. Each browser window can then run its own Web page/GUI. As this process is kind of central to managing the whole app, there can be just one and only one main process throughout an electron app life cycle.

The main process has a secondary yet important task of communicating with the OS to make use of native GUI features like notifications, creating shortcuts, responding to auto-updates and so on.

It also plays a key role in communication, acting as the middle man for all the browser windows it has created. This is in fact the backbone of the whole architecture we are relying on.

2. Renderer Process (Visible)

Every browser window that the main process creates gives rise to a renderer process. For the sake of simplicity, let us call each browser window to be equivalent to a renderer process. The only purpose of a renderer process is to show a webpage. In other words, its task is to render the GUI.

It is also important to note that each browser window that the main process generates runs independently much like tabs on our browser. Even if a tab crashes, the rest of the tabs keep running. This very fundamental feature gives us limitless powers to create stuff.

Q1. Can I make use of native OS features directly from renderer process?

No, you cannot. This is due to security reasons and thus to make use of native OS features, every render process has to communicate with the main process and as it to get the job done on its behalf.

Note: Actually you can access the main process features directly from renderer process by something called the remote module but that is not considered a healthy practice as a careless code may end up leaking resources.

Everything that you need to know about main and renderer process has been mentioned in the electron documentation.

You may have noticed that I have written the word visible separately for the renderer process. Why is that? Because, this is the only renderer that will be visible. As we have mentioned before, we are using React as one of our tech stack elements which is a great tool for writing Single Page Applications. And as you may have guessed, unlike old web apps, Single page Application do not need to open several tabs, rather, it is like a mobile app that gives the user a smooth experience on a single screen/tab. Thus, we will only need one tab to actually render the whole UI for the app. One tab equals one renderer process.

3. Renderer Process (Hidden)

But you just said that we only need one renderer process!!

Well I said only one visible process. Now if you were to make a simple GUI focused app, chances are you’ll never need point number 3 but we are here to push the electron app to new limits.

What happens when you want to perform a thread blocking ( heavy computation ) task? Where would you like to perform it? Main process? Nope, your app will crash if you block the main process. Visible renderer process? Nope, your GUI will start to lag and choke. So where do you do it? Yup, you guessed it, this is where the hidden renderer process comes in.

The hidden process is just another tab/window that is inherently hidden from the user. This is because our goal is not to render a UI in it but to perform a thread blocking task. The idea is to create and destroy hidden windows as and when needed in order to do heavy lifting in the background. Thus to do a thread blocking task, you would simply create a hidden window, finish the task in that hidden window ( running in the background ) without causing any issues with the visible components of your app and when you are done, just destroy the hidden window to claim back your resources. It is that simple.

The best part is you can create as many as you want.

This is the article I read to decide upon the approach I would adopt to tackle CPU intensive background tasks. It would be an interesting read if you want to compare this approach to the other options I had.

4. Python Script

By now you already know that Python is one of the key elements of our teach stack so we have to find a way to make it work effortlessly with the electron code. So the simplest way to do this was making use of an npm module called python-shell. This allows us to execute external python scripts from nodeJS application. As you are allowed to use nodeJS features in an electron app, thus the npm modules will work well in your electron code too.

Q1. From where do you call it?

ans. Obviously if you are using python for performing some task, you are planning to do some heavy lifting. But heavy lifting or not, the ideal spot to run the python script is from hidden renderer processes.

It is important to note that the script is not running inside your electron app, rather running in parallel. This diagram should help you visualize it better.


Script initialization using render process

Now obviously your python script may need some input and will most certainly generate some output when it runs. So as the script runs outside your app, you cannot directly grab or send data to the script. But python-shell has a nice feature which can be used to deal with this situation. We will talk about it in the next section.

The way you would design the script itself is again a topic for discussion which will be covered as we go into the details of coding.

Communications

An important point of discussion is how will each of these four components talk to each other. And to be frank, we just need to take care of three types of communication in our app.

1. Main — Renderer :

The actual means of communication is called the electron IPC (Inter Process Communication). The renderer processes use the ipcRenderer while the main process uses the ipcMain.

Now if you were to ask how it actually looks like then the best analogy is event listeners. You just set up a bunch of listeners on both ends depending on what you plan to listen for. Later on in the app life cycle, you can trigger these listeners manually to get the desired output.


Placing listeners on Main and render threads for bouncing messages

This idea can be extended to any number of render and main process combination. Under the hood, this is all there is.

2. Visible Renderer— Hidden Renderer :

There is no direct means of communication between two renderer processes. So the hidden renderer process cannot directly talk to the visible renderer process. The only way to actually establish a communication is by bouncing messages off the main process much like a relay system.


Hidden render – Visible render process communication

3. Hidden Renderer — Python Script :

The python-shell library provides us a way to make use of stdin and stdout channels for sending and receiving data from the python script.

So the best way to write your python code would be to assume that you would be using it from the terminal where you directly read data from user input and print data out to the terminal using simple print statement.

The message format that we will be using is JSON (JavaScript Object Notation) although many other options are available. JSON actually makes it easier to structure data which in turn makes it really easy to extract data on both ends with very minimal overhead.


Script-Hidden render communication

Note: Keep in mind that I have mentioned that this is the message format I use; don’t end up relating this to HTTP request-response because the term JSON is used quite heavily in that domain.

Resources:

Continue Reading PSLab Desktop Architecture

Building PSLab Desktop Apps using JavaScript, Python  and Electron

So before we get started let me quickly show you what we accomplished over a period of 2 and a half months building the Pocket Science Lab desktop app using the very technology I am going to talk about.

The PSLab desktop app was originally written using pyQt. The UI of the python stack looked really outdated and maintaining it was becoming a problem due to poor developer support. So, after a lot of discussion and experimentation, we finally decided to re-implement the whole app in electronJS. There are three major benefits of writing an app with Electron:

  • You can re-use your HTML, CSS knowledge to write the UI part in no time.
  • You can re-use your JavaScript knowledge to write the business logic of the app.
  • It works…

Why?

Before we even dive down to why use X and why use Y, let us try to answer a very basic question, The reason I had to invest a massive amount of time trying to experiment with different combinations and config parameters is because most of the blogs on the internet just wrote about “How to get started”, nobody talked about how to complete the build. And that is what I am going to cover. Anyone can figure out how to start out but  the real grind only begins when you get to the details of the app. But I hope to hit the escape velocity with my take on the subject.

The Tech Stack — Why we chose Electron for PSLab Desktop app

Now let’s talk about why we need React or Python in an Electron App.

1. React

It is perfectly fine to use everything minimal, plain old HTML, CSS and JS will do just fine for your UI, but for how long? React opens up so many possibilities and makes the code logic flow naturally through a very well designed UI language. The state management system is also brilliant and the most crucial thing that will definitely make your life easier is the life cycle methods offered by React class components. Making use of react also lets you take advantage of well known React oriented libraries like Material UI React, and Styled-Components. These things are game changers and can transform your UI to an absolute gem that your users will enjoy.

Oh, by the way, React UI is blazing fast…

But that is not the best part, we will not configure React on our own, but rather let the crowd favorite CRA do it for us. CRA (Create-React-App) will not only abstract out all the webpack config files, but will also make sure your react dependencies stay updated. This will help in keeping your code up to date in the long run.

2. Python

The task that is being performed by Python in our case was inevitable and could not be performed by JavaScript because PSLab only had Python libraries. We did not have the time to write everything again for JS so we figured a way to make use of Python in the electron app. If you can do everything with JS and have the necessary libraries to do so, then you don’t need this. For example, if you need to predict some value using a model you trained via TensorFlow, then this would be one such use case.

Bear in mind that this could be extended to any language as long as you get your hands on a library that can deal with the communication.

3. Electron

In short, it gives you the platform to make use of the above two. It becomes more interesting when we go into the details of how electron works. But for now, let us just say, it will provide you the space on which react will render the UI, it will provide the means for your UI to talk to the Python code, it will also provide APIs to talk to native system features like filesystem, notifications etc.

Resources:How to build an Electron app using create-react-app. No webpack configuration or “ejecting” necessary. Christian Sepulveda Christian Sepulveda: https://medium.com/free-code-camp/building-an-electron-application-with-create-react-app-97945861647c

Continue Reading Building PSLab Desktop Apps using JavaScript, Python  and Electron