Developing Oscilloscope UI in PSLab Android App

User Interface (UI) is one of the most important part of any software development. In PSLab while developing the UI of the Oscilloscope the following points are very critical.

  1. The UI should expose all the functionalities of Oscilloscope that can be performed using PSLab.
  2. The UI should be very convenient to use. It should allow the user to access functionalities of the PSLab with ease.
  3. The UI should be attractive.

Since Android smartphones come with relatively small size, it was a challenge to develop a UI for Oscilloscope. In this blog, I am going to mention the steps involved in developing the Oscilloscope UI.

Step 1: Creating a mockup

Initially, a mock-up for Oscilloscope UI is created using moqups tool. Later, the mock-up was discussed in public channel where fellow developers and mentors approved it. Let’s discuss some benefits of adopting this layout for Oscilloscope Activity.

  • The graph is ensured maximum screen space as it is the most important component/section of the Oscilloscope. This is also the reason why we kept the screen orientation to landscape.
  • The widgets don’t populate the screen, make the UI look clean.
  • The UI is comparable to basic app’s people use in their daily lives hence very convenient to use.

Mockup of Oscilloscope UI developed using moqups tool

Step 2: Deciding the API to be used

In Oscilloscope Activity, the main component is the graph. The captured data from the PSLab device is plotted on the graph. We decided to use MPAndroidCharts for the same.

Step 3: Deciding the space given to different sections of the UI

The next step was to decide how much screen space each section of Oscilloscope should acquire. There are 3 sections of the Oscilloscope UI.

  1. Graph
  2. Side panel consisting of buttons, each button loads a different set Oscilloscope controls and features in 3.
  3. A lower panel which is basically a fragment displaying controls and features corresponding to the button selected in 2.

By trying different dimensions and arrangements the following configuration fits the best.

To achieve this, the dimensions of different sections is set programmatically. This makes the UI compatible with different screen sizes.

public void onWindowFocusChanged() {

       RelativeLayout.LayoutParams lineChartParams = (RelativeLayout.LayoutParams) mChartLayout.getLayoutParams();
       lineChartParams.height = height * 2 / 3;
       lineChartParams.width = width * 5 / 6;
       mChartLayout.setLayoutParams(lineChartParams);
       RelativeLayout.LayoutParams frameLayoutParams = (RelativeLayout.LayoutParams) frameLayout.getLayoutParams();
       frameLayoutParams.height = height / 3;
       frameLayoutParams.width = width * 5 / 6;
       frameLayout.setLayoutParams(frameLayoutParams);
   
}

onWindowFocusChanged method is called in onCreate method. Here we are first receiving current layout parameters and then setting new layout parameters.

Step 4: Developing each section

  1. Graph

The graph needs to be customized concerning following requirements

  • Dual y axis, one dedicated to CH2 and another to analog input selected.
  • Black background
  • Grid lines
  • Scaling
  • Initial scale for x and y axis.

To achieve this a chartInit method is created which initializes the graph as per required. It is called in onCreate method.

2. Side Panel

It is a simple layout consisting of image buttons and text views. It is used to replace fragments in Lower Panel. To achieve this, image buttons and textviews were added to the layout and image buttons weight is set to 2. Later onClick listeners were added to both image buttons and textviews.

3. Lower Panel

The lower panel is frame layout which accommodates different fragments (one at a time). To achieve these different fragments are created that are ChannelsParametersFragment, TimebaseTriggerFragment, DataAnalysisFragment and XYPlotFragment. In ChannelsParametersFragment, TimebaseTriggerFragment and XYPlotFragment fragments, constraint views are used whereas in TimebaseTriggerFragment table layout is used. Each fragment allows the user to access different controls and features.

The Final Layout

The above is the GIF of the Oscilloscope UI.

This covers various steps for developing Oscilloscope UI in PSLab Android App.

Resources

Continue ReadingDeveloping Oscilloscope UI in PSLab Android App

How to Read PIC Data-Sheet and Add a New Functionality to PSLab Firmware

Reading data-sheets is not a fun task. Going through tens of hundreds of pages with numerical, mathematical and scientific data is not fun reading. This blog post attempts to simplify reading the available data-sheets related to PIC24 micro-controller used in the PSLab device to help reader with implementing a new feature in PSLab firmware.

There are many features available in the PSLab device, such as; UART, SPI, I2C, ADC and Basic I/O reading. The “basic” implementation techniques do not vary much from one feature to other. That being stated this blog will carry out the basic implementation techniques one should follow and basic knowledge on PIC micro-controller programming to save himself from the trouble going through the 500+ pages in PIC data-sheets.

PIC Basics:

Before go into implementation there are few facts one should know about PIC programming.

– In the micro-controller values are saved in a memory block known as Registers. The values saved in these registers are volatile as they are all set to 0 regardless the value they were assigned when the power is off.

– Micro-controller configurations are made by setting values to these registers. Even turning on and off a whole feature like UART in PSLab device can be done using setting 0 to UARTEN register bit.

– When it comes to I/O ports, there are two different types of registers called TRIS and LAT/PORT. By setting 1 to TRIS ports will make the relevant pin an input pin. Setting it to 0 will make it an output pin. Easy way to remember this is think 1 as I in input and 0 as O in output. In UART implementation of PSLab, pin RP40 is set as an input pin to receive the data stream and pin RP39 is set as an output pin to send the data stream out. These settings are made using TRIS port settings. PORT registers save the value received by the relevant input pin attached to it.


The above figure extracted from mikroe learning materials, illustrates different stages an I/O pin can handle. As an extra point, ANSEL register makes the pin support digital signals or analog signals as per user requirements.

– In PIC, some registers such as PORT, TRIS and registers with similar functionalities are combined together. To access the value of each individual register can be done using dot notation. Assume the program requires to access the 8th register in TRISB register set. Note that the registers are indexed from zero. This implies that the 8th register will have the index 8 in the register sub-set. The following syntax is used to access the register;

TRISBbits.TRISB7

 

The above points cover the basic knowledge one should have when developing firmware to PSLab device.

How to implement a feature like UART in PSLab firmware?

The first thing to know when implementing a new feature is that the developer needs to be familiar with the relevant hardware protocols. As an example, to implement UART; relevant protocol is RS232. If the feature is I2C; one should know about the I2C protocol.

Once the feature is familiar, next step is to refer the PIC data-sheet and resources on how to implement it in firmware. As for demonstrative purposes, this blog will continue with UART implementation.

Download the latest data-sheet from MicroChip official website and browse to the table of content. It consists of a set of features supported by the micro-controller implemented in the PSLab device. Find the entry related to the feature being implemented. In this case it will be Universal Asynchronous Receiver Transmitter (UART).

Each feature will contain a description following this format explaining what are the options it support and its constraints.

One must be aware of the fact that not every pin in the micro-controller can be used for any feature as he desires. The “PINOUT I/O DESCRIPTIONS” section in the data-sheet explains which pins are capable of the feature being implemented. According to these details, the pins should be initiated as Input/Output pins as well as Digital/Analog pins.

The next step is to refer to the control registers related to the feature. They are all mentioned in the data-sheet under the specific feature. There are some notations available in this section which resembles something like the following;

PDSEL<1:0>: Parity and Data Selection bits
11 = 9-bit data, no parity
10 = 8-bit data, odd parity
01 = 8-bit data, even parity
00 = 8-bit data, no parity

 

This represents a register with two bits. By setting either 11 or 10 or 01 or 00; different implementations can be achieved.

In PSLab firmware this is implemented as;

U1MODEbits.PDSEL = 0;

 

which implements UART feature with 8-bits stream having no parity bits for error correction.

In UART feature implemented in PSLab device, receiving bit stream is fetched by reading the register values in U1STAbits.URXDA and data is transmitted using U1TXREG. All these registers are mentioned in the control registers section in the feature.

Resources:

Continue ReadingHow to Read PIC Data-Sheet and Add a New Functionality to PSLab Firmware

Detecting password strength in SUSI.AI Web Chat SignUp

Every SignUp page contains a field to enter a password, but it should not be just a dumb component that takes input and saves it on server. A password field on a SignUp page should tell how weak or strong the password is. We decided to implement this in our SUSI.AI Web Chat SignUp Page.

Our SignUp page look like this:

After entering a valid email, user needs to enter the password. It shows that minimum 6 character are required:

We have the following div for our Password Field :

<PasswordField
  name="password"
  style={fieldStyle}
  value={this.state.passwordValue}
  onChange={this.handleChange}
  errorText={this.passwordErrorMessage}
  floatingLabelText="Password" />

In our OnChange Method we need to check the strength of password once the six character limit is crossed and display the strength visually.

We have used Dropbox’s zxcvbn library for the purpose of getting the strength of password.

Major reasons of choosing this library are :

  • Flexibility : It allows different passwords as long as certain level of complexity is matched.
  • Usability : It is very easy use and gives instant feedback. This helps user towards less guessable passwords.

For installing this library :

 npm -S install zxcvbn

For importing this:

import zxcvbn from 'zxcvbn';

Now in our OnChange Method:

handleChange = (event) => {
        let email;
        let password;
        let confirmPassword;
        let serverUrl;
        let state = this.state
      // Checking if event is password
        if (event.target.name === 'password') {
            password = event.target.value;
            let validPassword = password.length >= 6;
            state.passwordValue=password;
            state.passwordError = !(password && validPassword);
            if(validPassword) {
              //getting strength of password from zxcvbn
              let result = zxcvbn(password);
              state.passwordScore=result.score;
              let strength = [
                'Worst',
                'Bad',
                'Weak',
                'Good',
                'Strong'
              ];
              state.passwordStrength=strength[result.score];
            }
            else {
              state.passwordStrength='';
              state.passwordScore=-1;
            }
        }

Explanation:

In the above method result variable gets the strength of password and result.score gives us the score of password in terms of an integer and according to which we have made an array to get result in remarks corresponding to score. We have remarks like Good, Strong, etc.

Initially we have set the score to -1 to know that user has not entered the password yet. Once user has entered password the score changes.
Then we made a wrapper class to help css format the color of remark and display a meter (determining password strength) with corresponding length and color. We have used template strings to make our wrapper class. This helps us in having different names of wrapper class according to the score of the password.

// using Template Strings(look at resources for more info)
const PasswordClass=[`is-strength-${this.state.passwordScore}`];

Then we wrapped our Password field in div with className = “PasswordClass”.

<div className={PasswordClass.join(' ')}>
        <PasswordField
            name="password"
            style={fieldStyle}
            value={this.state.passwordValue}
            onChange={this.handleChange}
            errorText={this.passwordErrorMessage}
            floatingLabelText="Password" />
            <div className="ReactPasswordStrength-strength-bar" />
<div>

All that was left to was add css code corresponding to every score. For example for score=3, the following css was made:

.is-strength-3 { color: #57B8FF; }

.ReactPasswordStrength-strength-bar {
  box-sizing: border-box;
  height: 2px;
  position: relative; top: 1px; right: 1px;
  transition: width 300ms ease-out;
}

.is-strength--1 .ReactPasswordStrength-strength-bar {
  background: #D1462F;
  display: none;
}
// if strength = -1 then setting display of block equals to none
.is-strength--1 .ReactPasswordStrength-strength-bar {
  background: #D1462F;
  display: none;
}

.is-strength-3 .ReactPasswordStrength-strength-bar {
  background: #57B8FF; //Changing color according to password’s strength
  width: 192px; //Changing width according to password’s strength
  display: block;
}

After successfully implementing all these features, We had following SignUP page:

Resources:

1)Dropbox’s library(ZXVBN): https://github.com/dropbox/zxcvbn

2)Template Strings(Used here for making wrapping class of Password Field): https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Template_literals

Test Link:

This can be tested here.

Continue ReadingDetecting password strength in SUSI.AI Web Chat SignUp

Adding Emoji Support in SUSI Web Chat

SUSI.AI web chat sometimes, renders responses which contains emojis. We cannot rely on browser’s capability to render these emojis. The Problem is, that the default support for emojis of browsers does not offer a great variety of emojis to be rendered. The solution we implemented in the SUSI.AI web chat is to make use of a npm package to support our need for displaying emojis.

There were many options to choose from. For example :

Comparison between emoji packages :

Property Twemoji React-easy-emoji React-twemoji React-emojione
Built by Twitter Appfigures ZxMYS Pladaria
Usage Can be used as an object with function parse: twemoji.parse() Can be used as function: emoji() It is a simple wrapper for Twemoji.Can be used as component: <Twemoji> Can be used as function: emojify() or component: <Emojify>
Conversion compatibility Provides standard Unicode emoji support across all platforms Parse only basic emojis.Doesn’t parse emoji names like 🙂 and emoticons like 🙂 Convert emoji characters to Twemoji images Converts shortnames, unicode and ASCII smileys into renderable emojis
Dependencies None loot-web-kit lodash, prop-types, twemoji None

After detailed analysis of the above mentioned packages, we decided to go with React-emojione.

The major reasons are :

  • It is very easy to use.
  • It has no dependencies.
  • It can convert shortnames, unicode and ASCII symbols properly.
  • It can be used as both function and component, which diversifies its usage.

Installation:

npm install -S react-emojione

Basic usage (as function)

import {emojify} from 'react-emojione';
 
ReactDOM.render(
    <div>
        {emojify(':p')}
    </div>,
    document.body
);

Basic usage (as component)

import Emojify from 'react-emojione';
 
ReactDOM.render(
    <Emojify>
        <span>:p</span>
    </Emojify>,
    document.body
);

Some notes about the <Emojify> component:

  • If it has a single child, it won’t be wrapped
  • Otherwise it will be wrapped with a <span>

Difference between component and function?

Functional Stateless Components are just a ‘dumb’ function that takes props as an input. They do not have any state or methods. Just (props) => { return <span>content</span>; }

Class components can have state, variables, methods etc.

Now we needed our react app to render emojis. Our component named MessageListItem.react renders all the text and images of response.

There is a function called imageParse in this component. We use this function to parse our emojis.

Screenshot of SUSI Web Chat

Emoji’s like (:p) are now rendered properly

The implementation is as follows :

function imageParse(stringWithLinks){
  let replacePattern = new RegExp([
                      '((?:https?:\\/\\/)(?:[a-zA-Z]{1}',
                      '(?:[\\w-]+\\.)+(?:[\\w]{2,5}))',
                      '(?::[\\d]{1,5})?\\/(?:[^\\s/]+\\/)',
                      '*(?:[^\\s]+\\.(?:jpe?g|gif|png))',
                      '(?:\\?\\w+=\\w+(?:&\\w+=\\w+)*)?)'
                      ].join(''),'gim');
  let splits = stringWithLinks.split(replacePattern);
  let result = [];
  splits.forEach((item,key)=>{
    let checkmatch = item.match(replacePattern);
    if(checkmatch){
      result.push(
        <img key={key} src={checkmatch}
            style={{width:'95%',height:'auto'}} alt=''/>)
    }
    else{
      result.push(<Emojify  key={key}>{item}</Emojify>);
    }
  });
  return result;
}

Here we put {item} inside <Emojify> tag to render all the emoji’s present inside {item}.

This parses all emojis regardless of browser support. This package fulfills all our needs in this case.

Resources:

react-emojione package: https://www.npmjs.com/package/react-emojione

Testing link: SUSI.AI (Web Chat): http://chat.susi.ai/

Continue ReadingAdding Emoji Support in SUSI Web Chat

Reset Password for SUSI Accounts Using Verification Link

In this blog, I will discuss how the SUSI server interprets the verification link sent to your email id to reset SUSI account password. The email one receives to reset password looks like this :  

http://api.susi.ai/apps/resetpass/index.html?token={30 character long token}

*Original link contains a token of length of 30 characters. The link has been tampered for security purpose.

Taking a close look at the reset link, one would find it easy to decode. It simply contains path to an application on current SUSI Accounts hosting. Name of the application is “resetpass” for Reset Password. But what about the token in the link?

As soon as a user clicks on the link, the app is called and token is passed as a GET parameter. The app in background makes a call to the server where the token is evaluated for whether the token is hashed against some user’s email id and has not expired yet. Below is code of first step the client does which is to make a simple ajax call on Ready state.

$(document).ready(function()
{
	var passerr = false, confirmerr = false, tokenerr = false;

	// get password parameters
	var regex;
	var urltoken = getParameter('token');

	$.ajax(	"/aaa/recoverpassword.json", {
		data: { getParameters: true, token: urltoken },
		dataType: 'json',
		success: function (response) {
			regex = response.regex;
			var regexTooltip = response.regexTooltip;
			$('#pass').tooltip({'trigger':'focus', 'placement': 'left', 'title': regexTooltip});
			$('#status-box').text(response.message);
			tokenerr = false;
		},
		error: function (xhr, ajaxOptions, thrownError) {
			$('#status-box').text(thrownError);
			$('#status-box').addClass("error");
			$('#pass').prop( "disabled", true );
			$('#confirmpass').prop( "disabled", true );
			$('#resetbut').prop( "disabled", true );
			tokenerr = true;
		},
	});

As you can see the call is made to /aaa/recoverypassword.json end point with token as the request parameter. Now, the client has to just evaluate the JSON response and render the message for user accordingly. If this request returns an error then the error message is shown and the entries are disabled to enter the password. Otherwise, user email id is shown with green text and user can now enter new password and confirm it.

In next step, client simply evaluates the password and sends a query to server to reset it. Let us now look at how server functions and processes such requests.

//check if a token is already present
if (call.get("getParameters", false)) {
			if (call.get("token", null) != null && !call.get("token", null).isEmpty()) {
				ClientCredential credentialcheck = new ClientCredential(ClientCredential.Type.resetpass_token,
						call.get("token", null));
				if (DAO.passwordreset.has(credentialcheck.toString())) {
					Authentication authentication = new Authentication(credentialcheck, DAO.passwordreset);
					if (authentication.checkExpireTime()) {
						String passwordPattern = DAO.getConfig("users.password.regex", "^(?=.*\\d).{6,64}$");
						String passwordPatternTooltip = DAO.getConfig("users.password.regex.tooltip",
								"Enter a combination of atleast six characters");
						result.put("message", "Email ID: " + authentication.getIdentity().getName());
						result.put("regex", passwordPattern);
						result.put("regexTooltip", passwordPatternTooltip);
						result.put("accepted", true);
						return new ServiceResponse(result);
					}
					authentication.delete();
					throw new APIException(422, "Expired token");
				}
				throw new APIException(422, "Invalid token");
			} else {
				throw new APIException(422, "No token specified");
			}
		}

In the above code snippet, server evaluates the received token on the basis of three parameters. First it checks whether the token is provided or not. If not, it throws APIException with error code 422 and a message “No token specified”. If it passes the check, next it checks if the token passed is valid or not. If the token is invalid, it throws different APIException with same error code but different message “Invalid token”. Finally it checks whether the token is expired or not {life of each token is 7 days. After that, server marks it as expired}.

If all checks pass, the server finds the valid email id against which the token was hashed and sends it to user in JSON format. Now let us see how the final reset  password call is handled at the server.

If the token is valid  and still has life, user will be asked to enter new password and confirm it. Client locally checks whether new password and confirm password are same or not. It will now make a call to the below given servlet.

/aaa/resetpassword.json

Till now, server has already made the client identity using the token. Next it will check if the password matches regular expression or not. If not, it sends  an error message “Invalid Password” with error code 400.

if (DAO.hasAuthentication(emailcred)) {
			Authentication emailauth = DAO.getAuthentication(emailcred);
			String salt = createRandomString(20);
			emailauth.remove("salt");
			emailauth.remove("passwordHash");
			emailauth.put("salt", salt);
			emailauth.put("passwordHash", getHash(newpass, salt));
		}

Above code snippet shows what happens when new password matches the conditions of regular expression. The server will generate a random string of 20 characters and use it as the new salt to hash the password. It updates the salt and password hash for the particular user. Next time whenever user makes a login request, server will use the new salt-hash pair to authorise the user. Below given is a flowchart for better understanding.

Resources

 

Continue ReadingReset Password for SUSI Accounts Using Verification Link

Performing Fourier Transforms in the PSLab Android App

Oscilloscope is one of the key features of PSLab. When periodic signals such as sine waves are read by the Oscilloscope, curve fitting functions are used to construct a curve that has the best fit to a series of data points. In PSLab, the sine curve fitting involves the Fourier Transforms. FFT (short for “Fast Fourier Transform”) is nothing more than a curve-fit of sines and cosines to some given data. In order to understand the implementation of Fourier Transforms in PSLab Android App let’s first have a look at the Fourier transform equations.

The first equation here is the Forward Fourier transform. It converts the function of time (t) into the function of frequency (ω).
The second equation is Inverse Fourier transform. It does the opposite to first equation ie. it converts the function of frequency (ω) into the function of time (t).

So, first I will answer what is transform?
It is the mapping between two different sets of domains. In this case, the information is changed from the time domain to frequency domain. The data in these domains look different but represent the same information. A transform will get you from one representation to another.

Fourier transforms converts between the time domain f(t) and the frequency domain F(ω).
Performing Fourier Transforms in Android
Let’s perform Forward Fourier transform. This means it converts the function of time (t) into the function of frequency (ω). We will use Apache Maths Commons to perform Fourier transforms. Since we have finite input data set we will calculate Discrete Fourier Transform (DFT).
The algorithm which is being used here is Fast Fourier Transform (FFT) which is the best algorithm to calculate Fourier transforms.

FastFourierTransformer fastFourierTransformer = 
      new FastFourierTransformer(DftNormalization.STANDARD);

Here we are creating an instance of FastFourierTransformer which passed STANDARD normalization convention to its constructor. Normalization other than STANDARD is UNITARY.
Complex complex[] = fastFourierTransformer.transform(input, TransformType.FORWARD);

Here, input array and TransformType. FORWARD is also passed to transform method. Input is an array of data representing time whereas TransformType. FORWARD defines the type of Fourier transform that should be performed ie. forward or inverse.

Complex complex[] = fastFourierTransformer.transform(input, TransformType.FORWARD);

The output will be an array of complex number. Each data point will be represented like the following graph in the complex plane.

Suppose the amplitude of the data point given in above graph is 1 and phase shift is 45°. So, solving this we will get 2/√2 as both real and imaginary components. Therefore, F (ω) would be 1/√2 + (1/√2) i.
Dealing with Complex numbers in Java
A complex number has both real and imaginary part. Using Apache Maths Commons we can use Complex to represent a complex number.

Complex number = new Complex(1,2);
System.out.println(number);
Output
1.0 + 2.0i

We can also get real and imaginary parts separately of Complex numbers.

System.out.println(number.getReal());
System.out.println(number.getImaginary());
Output
1.0
2.0

The array of the Complex numbers can be implemented like the following

Complex[] number;
Complex c1 = new Complex(1, 2);
Complex c2 = new Complex(3, 4);
number = new Complex[]{c1, c2};
System.out.println(Arrays.toString(number));
System.out.println(number[1]);
Output

[(1.0, 2.0), (3.0, 4.0)]
(3.0, 4.0)

Resources

Continue ReadingPerforming Fourier Transforms in the PSLab Android App

Implementing the Message Response Status Indicators In SUSI WebChat

SUSI Web Chat now has indicators reflecting the message response status. When a user sends a message, he must be notified that the message has been received and has been delivered to server. SUSI Web Chat implements this by tagging messages with ticks or waiting clock icons and loading gifs to indicate delivery and response status of messages ensuring good UX.

This is implemented as:

  • When the user sends a message, the message is tagged with a `clock` icon indicating that the message has been received and delivered to server and is awaiting response from the server
  • When the user is waiting for a response from the server, we display a loading gif
  • Once the response from the server is received, the loading gif is replaced by the server response bubble and the clock icon tagged to the user message is replaced by a tick icon.

Lets visit SUSI WebChat and try it out.

Query : Hey

When the message is sent by the user, we see that the displayed message is tagged with a clock icon and the left side response bubble has a loading gif indicating that the message has been delivered to server and are awaiting response.

When the response from server is delivered, the loading gif disappears and the user message tagged with a tick icon.

 

How was this implemented?

The first step is to have a boolean flag indicating the message delivery and response status.

let _showLoading = false;

getLoadStatus(){
  return _showLoading;
},

The `showLoading` boolean flag is set to true when the user just sends a message and is waiting for server response.  When the user sends a message, the CREATE_MESSAGE action is triggered. Message Store listens to this action and along with creating the user message, also sets the showLoading flag as true.

case ActionTypes.CREATE_MESSAGE: {

  let message = action.message;
  _messages[message.id] = message;
  _showLoading = true;
  MessageStore.emitChange();
  
  break;
}

The showLoading flag is used in MessageSection to display the loading gif. We are using a saved gif to display the loading symbol. The loading gif is displayed at the end after displaying all the messages in the message store. Since this loading component must be displayed for every user message, we don’t save this component in MessageStore as a loading message as that would lead to repeated looping thorugh the messages in message store to add and delete loading component.

import loadingGIF from '../../images/loading.gif';

function getLoadingGIF() {

  let messageContainerClasses = 'message-container SUSI';

  const LoadingComponent = (
    <li className='message-list-item'>
      <section className={messageContainerClasses}>
        <img src={loadingGIF}
          style={{ height: '10px', width: 'auto' }}
          alt='please wait..' />
      </section>
    </li>
  );
  return LoadingComponent;
}

We then use this flag in MessageListItem class to tag the user messages with the clock icons. We used Material UI SVG Icons to display the clock and tick messages. We display these beside the time in the messages.

import ClockIcon from 'material-ui/svg-icons/action/schedule';

statusIndicator = (
  <li className='message-time' style={footerStyle}>
    <ClockIcon style={indicatorStyle}
      color={UserPreferencesStore.getTheme()==='light' ? '#90a4ae' : '#7eaaaf'}/>
  </li>
);

When the response from server is received, the CREATE_SUSI_MESSAGE action is triggered to render the server response. This action is again collected in MessageStore where the `showLoading` boolean flag is reset to false. This event also triggers the state of MessageSection where we are listening to showLoading value from MessageStore, hence triggering changes in MessageSection and accordingly in MessageListItem where showLoading is passed as props, removing the loading gif component and displaying the server response and replacing the clock icon with tick icon on the user message.

case ActionTypes.CREATE_SUSI_MESSAGE: {
  
  let message = action.message;
  MessageStore.resetVoiceForThread(message.threadID);
  _messages[message.id] = message;
  _showLoading = false;
  MessageStore.emitChange();
  
  break;
}

This is how the status indicators were implemented for messages. The complete code can be found at SUSI WebChat Repo.

Resources

Continue ReadingImplementing the Message Response Status Indicators In SUSI WebChat

Introducing Priority Kaizen Harvester for loklak server

In the previous blog post, I discussed the changes made in loklak’s Kaizen harvester so it could be extended and other harvesting strategies could be introduced. Those changes made it possible to introduce a new harvesting strategy as PriorityKaizen harvester which uses a priority queue to store the queries that are to be processed. In this blog post, I will be discussing the process through which this new harvesting strategy was introduced in loklak.

Background, motivation and approach

Before jumping into the changes, we first need to understand that why do we need this new harvesting strategy. Let us start by discussing the issue with the Kaizen harvester.

The produce consumer imbalance in Kaizen harvester

Kaizen uses a simple hash queue to store queries. When the queue is full, new queries are dropped. But numbers of queries produced after searching for one query is much higher than the consumption rate, i.e. the queries are bound to overflow and new queries that arrive would get dropped. (See loklak/loklak_server#1156)

Learnings from attempt to add blocking queue for queries

As a solution to this problem, I first tried to use a blocking queue to store the queries. In this implementation, the producers would get blocked before putting the queries in the queue if it is full and would wait until there is space for more. This way, we would have a good balance between consumers and producers as the consumers would be waiting until producers can free up space for them –

public class BlockingKaizenHarvester extends KaizenHarvester {
   ...
   public BlockingKaizenHarvester() {
       super(new KaizenQueries() {
           ...
           private BlockingQueue<String> queries = new ArrayBlockingQueue<>(maxSize);

           @Override
           public boolean addQuery(String query) {
               if (this.queries.contains(query)) {
                   return false;
               }
               try {
                   this.queries.offer(query, this.blockingTimeout, TimeUnit.SECONDS);
                   return true;
               } catch (InterruptedException e) {
                   DAO.severe("BlockingKaizen Couldn't add query: " + query, e);
                   return false;
               }
           }
           @Override
           public String getQuery() {
               try {
                   return this.queries.take();
               } catch (InterruptedException e) {
                   DAO.severe("BlockingKaizen Couldn't get any query", e);
                   return null;
               }
           }
           ...
       });
   }
}

[SOURCE, loklak/loklak_server#1210]

But there is an issue here. The consumers themselves are producers of even higher rate. When a search is performed, queries are requested to be appended to the KaizenQueries instance for the object (which here, would implement a blocking queue). Now let us consider the case where queue is full and a thread requests a query from the queue and scrapes data. Now when the scraping is finished, many new queries are requested to be inserted to most of them get blocked (because the queue would be full again after one query getting inserted).

Therefore, using a blocking queue in KaizenQueries is not a good thing to do.

Other considerations

After the failure of introducing the Blocking Kaizen harvester, we looked for other alternatives for storing queries. We came across multilevel queues, persistent disk queues and priority queues.

Multilevel queues sounded like a good idea at first where we would have multiple queues for storing queries. But eventually, this would just boil down to how much queue size are we allowing and the queries would eventually get dropped.

Persistent disk queues would allow us to store greater number of queries but the major disadvantage was lookup time. It would terribly slow to check if a query already exists in the disk queue when the queue is large. Also, since the queries would always increase practically, the disk queue would also go out of hand at some point in time.

So by now, we were clear that not dropping queries is not an alternative. So what we had to use the limited size queue smartly so that we do not drop queries that are important.

Solution: Priority Queue

So a good solution to our problem was a priority queue. We could assign a higher score to queries that come from more popular Tweets and they would go higher in the queue and do not drop off until we have even higher priority queried in the queue.

Assigning score to a Tweet

Score for a tweet was decided using the following formula –

α= 5* (retweet count)+(favourite count)

score=α/(α+10*exp(-0.01*α))

This equation generates a score between zero and one from the retweet and favourite count of a Tweet. This normalisation of score would ensure we do not assign an insanely large score to Tweets with a high retweet and favourite count. You can see the behaviour for the second mentioned equation here.

Graph?

Changes required in existing Kaizen harvester

To take a score into account, it became necessary to add an interface to also provide a score as a parameter to the addQuery() method in KaizenQueries. Also, not all queries can have a score associated with it, for example, if we add a query that would search for Tweets older than the oldest in the current timeline, giving it a score wouldn’t be possible as it would not be associated with a single Tweet. To tackle this, a default score of 0.5 was given to these queries –

public abstract class KaizenQueries {

   public boolean addQuery(String query) {
       return this.addQuery(query, 0.5);
   }

   public abstract boolean addQuery(String query, double score);
   ...
}

[SOURCE]

Defining appropriate KaizenQueries object

The KaizenQueries object for a priority queue had to define a wrapper class that would hold the query and its score together so that they could be inserted in a queue as a single object.

ScoreWrapper and comparator

The ScoreWrapper is a simple class that stores score and query object together –

private class ScoreWrapper {

   private double score;
   private String query;

   ScoreWrapper(String m, double score) {
       this.query = m;
       this.score = score;
   }

}

[SOURCE]

In order to define a way to sort the ScoreWrapper objects in the priority queue, we need to define a Comparator for it –

private Comparator<ScoreWrapper> scoreComparator = (scoreWrapper, t1) -> (int) (scoreWrapper.score - t1.score);

[SOURCE]

Putting things together

Now that we have all the ingredients to declare our priority queue, we can also declare the strategy to getQuery and putQuery in the corresponding KaizenQueries object –

public class PriorityKaizenHarvester extends KaizenHarvester {

   private static class PriorityKaizenQueries extends KaizenQueries {
       ...
       private Queue<ScoreWrapper> queue;
       private int maxSize;

       public PriorityKaizenQueries(int size) {
           this.maxSize = size;
           queue = new PriorityQueue<>(size, scoreComparator);
       }

       @Override
       public boolean addQuery(String query, double score) {
           ScoreWrapper sw = new ScoreWrapper(query, score);
           if (this.queue.contains(sw)) {
               return false;
           }
           try {
               this.queue.add(sw);
               return true;
           } catch (IllegalStateException e) {
               return false;
           }
       }

       @Override
       public String getQuery() {
           return this.queue.poll().query;
       }
       ...
}

[SOURCE]

Conclusion

In this blog post, I discussed the process in which PriorityKaizen harvester was introduced to loklak. This strategy is a flavour of Kaizen harvester which uses a priority queue to store queries that are to be processed. These changes were possible because of a previous patch which allowed extending of Kaizen harvester.

The changes were introduced in pull request loklak/loklak#1240 by @singhpratyush (me).

Resources

Continue ReadingIntroducing Priority Kaizen Harvester for loklak server

Fetching URL for Embedded Twitter Videos in loklak server

The primary web service that loklak scrapes is Twitter. Being a news and social networking service, Twitter allows its users to post videos directly to Twitter and they convey more thoughts than what text can. But for an automated scraper, getting the links is not a simple task.

Let us see that what were the problems we faced with videos and how we solved them in the loklak server project.

Previous setup and embedded videos

In the previous version of loklak server, the TwitterScraper searched for videos in 2 ways –

  1. Youtube links
  2. HTML5 video links

To fetch the video URL from HTML5 video, following snippet was used –

if ((p = input.indexOf("<source video-src")) >= 0 && input.indexOf("type=\"video/") > p) {
   String video_url = new prop(input, p, "video-src").value;
   videos.add
   continue;
}

Here, input is the current line from raw HTML that is being processed and prop is a class defined in loklak that is useful in parsing HTML attributes. So in this way, the HTML5 videos were extracted.

The Problem – Embedded videos

Though the previous setup had no issues, it was useless as Twitter embeds the videos in an iFrame and therefore, can’t be fetched using simple HTML5 tag extraction.

If we take the following Tweet for example,

the requested HTML from the search page contains video in following format –

<src="https://twitter.com/i/videos/tweet/881946694413422593?embed_source=clientlib&player_id=0&rpc_init=1" allowfullscreen="" id="player_tweet_881946694413422593" style="width: 100%; height: 100%; position: absolute; top: 0; left: 0;">

So we needed to come up with a better technique to get those videos.

Parsing video URL from iFrame

The <div> which contains video is marked with AdaptiveMedia-videoContainer class. So if a Tweet has an iFrame containing video, it will also have the mentioned class.

Also, the source of iFrame is of the form https://twitter.com/i/videos/tweet/{Tweet-ID}. So now we can programmatically go to any Tweet’s video and parse it to get results.

Extracting video URL from iFrame source

Now that we have the source of iFrame, we can easily get the video source using the following flow –

public final static Pattern videoURL = Pattern.compile("video_url\\\":\\\"(.*?)\\\"");

private static String[] fetchTwitterIframeVideos(String iframeURL) {
   // Read fron iframeURL line by line into BufferReader br
   while ((line = br.readLine()) != null ) {
       int index;
       if ((index = line.indexOf("data-config=")) >= 0) {
           String jsonEscHTML = (new prop(line, index, "data-config")).value;
           String jsonUnescHTML = HtmlEscape.unescapeHtml(jsonEscHTML);
           Matcher m = videoURL.matcher(jsonUnescHTML);
           if (!m.find()) {
               return new String[]{};
           }
           String url = m.group(1);
           url = url.replace("\\/", "/");  // Clean URL
           /*
            * Play with url and return results
            */
       }
   }
}

MP4 and M3U8 URLs

If we encounter mp4 URLs, we’re fine as it is the direct link to video. But if we encounter m3u8 URL, we need to process it further before we can actually get to the videos.

For Twitter, the hosted m3u8 videos contain link to further m3u8 videos which are of different resolution. These m3u8 videos again contain link to various .ts files that contain actual video in parts of 3 seconds length each to support better streaming experience on the web.

To resolve videos in such a setup, we need to recursively parse m3u8 files and collect all the .ts videos.

private static String[] extractM3u8(String url) {
   return extractM3u8(url, "https://video.twimg.com/");
}

private static String[] extractM3u8(String url, String baseURL) {
   // Read from baseURL + url line by line
   while ((line = br.readLine()) != null) {
       if (line.startsWith("#")) {  // Skip comments in m3u8
           continue;
       }
       String currentURL = (new URL(new URL(baseURL), line)).toString();
       if (currentURL.endsWith(".m3u8")) {
           String[] more = extractM3u8(currentURL, baseURL);  // Recursively add all
           Collections.addAll(links, more);
       } else {
           links.add(currentURL);
       }
   }
   return links.toArray(new String[links.size()]);
}

And then in fetchTwitterIframeVideos, we can return the all .ts URLs for the video –

if (url.endsWith(".mp4")) {
   return new String[]{url};
} else if (url.endsWith(".m3u8")) {
   return extractM3u8(url);
}

Putting things together

Finally, the TwitterScraper can discover the video links by tweaking a little –

if (input.indexOf("AdaptiveMedia-videoContainer") > 0) {
   // Fetch Tweet ID
   String tweetURL = props.get("tweetstatusurl").value;
   int slashIndex = tweetURL.lastIndexOf('/');
   if (slashIndex < 0) {
       continue;
   }
   String tweetID = tweetURL.substring(slashIndex + 1);
   String iframeURL = "https://twitter.com/i/videos/tweet/" + tweetID;
   String[] videoURLs = fetchTwitterIframeVideos(iframeURL);
   Collections.addAll(videos, videoURLs);
}

Conclusion

This blog post explained the process of extracting video URL from Twitter and the problem faced. The discussed change enabled loklak to extract and serve URLs to video for tweets. It was introduced in PR loklak/loklak_server#1193 by me (@singhpratyush).

The service was further enhanced to collect single mp4 link for videos (see PR loklak/loklak_server#1206), which is discussed in another blog post.

Resources

Continue ReadingFetching URL for Embedded Twitter Videos in loklak server

Creating Sensor Libraries in PSLab

The I2C bus of the PSLab enables access to a whole range of sensors capable of measuring parameters ranging from light intensity, humidity, and temperature, to acceleration, passive infrared, and magnetism.

Support for each sensor in the communication library is implemented as a small Python library that depends in the I2C communication module for PSLab.

However, most sensors have capabilities that are not just limited to data readouts, but also enable various configuration options.

This blog post explains how a common format followed across the sensor libraries enables the graphical utilities such as data loggers and control panels to dynamically create widgets pertaining to the various configuration options.

The following variables and methods must be present in each sensor file in order to enable the graphical utilities to correctly function:

Name: A generic name for the sensor to be shown in menus and such. e.g. ‘Altimeter BMP180’

GetRaw(): A function that returns a list of values read from the sensor. The values may have been directly read from the sensor, or derived based on some parameters/equations.

For example, the BMP180 altitude sensor is actually a pressure and temperature sensor. The altitude value is derived from these two using an equation. Therefore, the getRaw function for the BMP180 returns a list of three values, viz, [temperature, pressure, altitude]

NUMPLOTS: A constant that stores the number of available dataPoints in the list returned by the getRaw function. This enables the graphical utilities to create required number of traces . In case of the BMP180, it is 3

PLOTNAMES: A list of text strings to be displayed in the plot legend . For the BMP180, the following list is defined : [‘Temperature’, ‘Pressure’, ‘Altitude’]

params: A dictionary that stores the function names for configuring various features of the sensor, and options that can be passed to the function. For example, for the BMP180 altimeter, and oversampling parameter is available, and can take values 0,1,2 and 3 . Therefore, params = {‘setOversampling’: [0, 1, 2, 3]}

The Sensor data Logger application uses this dictionary to auto-generate menus with the ‘key’ as the name , and corresponding ‘values’ as a submenu . When the user opens a menu and clicks on a ‘value’ , the ‘value’ is passed to a function whose name is the corresponding key , and which must be defined in the sensor’s class.

When the above are defined, menus and plots are automatically generated, and saves considerable time and effort for graphical utility development since no sensor specific code needs to be added on that front.

The following Params dictionary defined in the class of MPU6050 creates a menu as shown in the second image:

self.params = { 'powerUp':['Go'],
'setGyroRange':[250,500,1000,2000],
'setAccelRange':[2,4,8,16],
'KalmanFilter':[.01,.1,1,10,100,1000,10000,'OFF']
}

As shown in the image , when the user clicks on ‘8’ , MPU6050.setAccelRange(8) is executed.

Improving the flexibility of the auto-generated menus

The above approach is a little limited, since only a fixed set of values can be used for configuration options, and there may be cases where a flexible input is required.

This is the case with the Kalman filter option, where the user may want to provide the intensity of the filter as a decimal value. Therefore we shall implement a more flexible route for the params dictionary, and allow the value to certains keys to be objects other than lists.

Functions with user defined variable inputs are defined as Spinbox/QDoubleSpinBox.

KalmanFilter is defined in the following entry in Params:

‘KalmanFilter’:{‘dataType’:’double’,’min’:0,’max’:1000,’prefix’:’value: ‘}

Screenshot of the improved UI with MPU6050.

In this instance, the user can input a custom value, and the KalmanFilter function is called with the same.

Additional Reading:

[1]: Using sensors with PSLab Android

[2]: Analyzing sensor data with PSLab android
[3]: YouTube video to understand analysis of data from MPU6050 with Arduino – https://www.youtube.com/watch?v=taZHl4Mr-Pk

Continue ReadingCreating Sensor Libraries in PSLab