Implementing Map View in Devices Tab in Settings

The Table View implemented in the Devices tab in settings on SUSI.AI Web Client has a column “geolocation” which displays the latitudinal and longitudinal coordinates of the device. These coordinates needed to be displayed on a map. Hence, we needed a Map View apart from the Table View, dedicated to displaying the devices pinpointed on a map. This blog post explains how this feature has been implemented on the SUSI.AI Web Client.

Modifying the fetched data of devices to suitable format

We already have the fetched data of devices which is being used for the Table View. We need to extract the geolocation data and store it in a different suitable format to be able to use it for the Map View. The required format is as follows:

[
   {
      "location":{
         "lat": latitude1,
         "lng": longitude2
      }
   },
   {
      "location":{
         "lat": latitude1,
         "lng": longitude2
      }
   }
]

 

To modify the fetched data of devices to this format, we modify the apiCall() function to facilitate extraction of the geolocation info of each device and store them in an object, namely ‘mapObj’. Also, we needed variables to store the latitude and longitude to use as the center for the map. ‘centerLat’ and ‘centerLng’ variables store the average of all the latitudes and longitudes respectively. The following code was added to the apiCall() function to facilitate all the above requirements:

let mapObj = [];
let locationData = {
  lat: parseFloat(response.devices[i].geolocation.latitude),
  lng: parseFloat(response.devices[i].geolocation.longitude),
};
centerLat += parseFloat(response.devices[i].geolocation.latitude);
centerLng += parseFloat(response.devices[i].geolocation.longitude);
let location = {
  location: locationData,
};
mapObj.push(location);
centerLat = centerLat / mapObj.length;
centerLng = centerLng / mapObj.length;
if (mapObj.length) {
  this.setState({
    mapObj: mapObj,
    centerLat: centerLat,
    centerLng: centerLng,
  });
}

 

The following code was added in the return function of Settings.react.js file to use the Map component. All the modified data is passed as props to this component.

<MapContainer
  google={this.props.google}
  mapData={this.state.mapObj}
  centerLat={this.state.centerLat}
  centerLng={this.state.centerLng}
  devicenames={this.state.devicenames}
  rooms={this.state.rooms}
  macids={this.state.macids}
/>

 

The implementation of the MapContainer component is as follows:

 componentDidUpdate() {
    this.loadMap();
  }

  loadMap() {
    if (this.props && this.props.google) {
      const {google} = this.props;
      const maps = google.maps;
      const mapRef = this.refs.map;
      const node = ReactDOM.findDOMNode(mapRef);

      const mapConfig = Object.assign({}, 
        
          center: { lat: this.props.centerLat, lng: this.props.centerLng },
          zoom: 2,
        }
      )
      this.map = new maps.Map(node, mapConfig);
    }
  }

 

Let us go over the code of MapContainer component step by step.

  1. Firstly, the componentDidUpdate() function calls the loadMap function to load the google map.

  componentDidUpdate() {
    this.loadMap();
  }

 

  1. In the loadMap() function, we first check whether props have been passed to the MapContainer component. This is done by enclosing all contents of loadMap function inside an if statement as follows:

  if (this.props && this.props.google) {
    // All code of loadMap() function
  } 

 

  1. Then we set the prop value to google, and maps to google maps props. This is done as follows:

  const {google} = this.props;
  const maps = google.maps;

 

  1. Then we look for HTML div ref ‘map’ in the React DOM and name it ‘node’. This is done as follows:

  const mapRef = this.refs.map;
  const node = ReactDOM.findDOMNode(mapRef);

 

  1. Then we set the center and the default zoom level of the map using the props we provided to the MapContainer component.

  {
    center: { lat: this.props.centerLat, lng: this.props.centerLng },
    zoom: 2,
  }

 

  1. Then we create a new Google map on the specified node (ref=’map’) with the specified configuration set above. This is done as follows:

this.map = new maps.Map(node, mapConfig);

 

  1. In the render function of the MapContainer component, we return a div with a ref ‘map’ as follows:

 render() {
    return (
      <div ref="map" style={style}>
        loading map...
      </div>
    );
  }

 

This is how the Map View has been implemented in the Devices tab in Settings on SUSI.AI Web Client.

Resources

Continue ReadingImplementing Map View in Devices Tab in Settings

Implementing Table View in Devices Tab in Settings

We can connect to the SUSI.AI Smart Speaker using our mobile apps (Android or iOS). But there needs to be something implemented which can tell you what all devices are linked to your account. This is in consistency with the way how Google Home devices and Amazon Alexa devices have this feature implemented in their respective apps, which allow you to see the list of devices connected to your account. This blog post explains how this feature has been implemented on the SUSI.AI Web Client.

Fetching data of the connected devices from the server

The information of the devices connected to an account is stored in the Accounting object of that user. This is a part of a sample Accounting object of a user who has 2 devices linked to his/her account. This is the data that we wish to fetch. This data is accessible at the /aaa/listUserSettings.json endpoint.

{
  "devices": {
    "37-AE-5F-7B-CA-3F": {
      "name": "Device 1",
      "room": "Room 1",
      "geolocation": {
        "latitude": "50.34567",
        "longitude": "60.34567"
      }
    },
    "9D-39-02-01-EB-95": {
      "name": "Device 2",
      "room": "Room 2",
      "geolocation": {
        "latitude": "52.34567",
        "longitude": "62.34567"
      }
    }
  }
} 

 

In the Settings.react.js file, we make an AJAX call immediately after the component is mounted on the DOM. This AJAX call is made to the /aaa/listUserSettings.json endpoint. The received response of the AJAX call is then used and traversed to store the information of each connected device in a format that would be more suitable to use as a prop for the table.

apiCall = () => {
    $.ajax({
      url: BASE_URL + '/aaa/listUserSettings.json?' + 'access_token=' +
  cookies.get('loggedIn');,
      type: 'GET',
      dataType: 'jsonp',
      crossDomain: true,
      timeout: 3000,
      async: false,
      success: function(response) {
        let obj = [];
         // Extract information from the response and store them in obj object
        obj.push(myObj);
        this.setState({
          dataFetched: true,
          obj: obj,
        });
      }
      }.bind(this),
  };

 

This is how the extraction of keys takes place inside the apiCall() function. We first extract the keys of the ‘devices’ JSONObject inside the response. The keys of this JSONObject are the Mac Addresses of the individual devices. Then we traverse inside the JSONObject corresponding to each Mac Address and store the name of the device, room and also the geolocation information of the device in separate variables, and then finally push all this information inside an object, namely ‘myObj’. This JSONObject is then pushed to a JSONArray, namely ‘obj’. Then a setState() function is called which sets the value of ‘obj’ to the updated ‘obj’ variable.

let keys = Object.keys(response.devices);
keys.forEach(i => {
    let myObj = {
        macid: i,
        devicename: response.devices[i].name,
        room: response.devices[i].room,
        latitude: response.devices[i].geolocation.latitude,
        longitude: response.devices[i].geolocation.longitude,
    };
}

 

This way we fetch the information of devices and store them in a variable named ‘obj’. This variable will now serve as the data for the table which we want to create.

Creating table from this data

The data is then passed on to the Table component as a prop in the Settings.react.js file.

<TableComplex
    // Other props
    tableData={this.state.obj}
/>

 

The other props passed to the Table component are the functions for handling the editing and deleting of rows, and also for handling the changes in the textfield when the edit mode is on. These props are as follows:

<TableComplex
    startEditing={this.startEditing}
    stopEditing={this.stopEditing}
    handleChange={this.handleChange}
    handleRemove={this.handleRemove}
/> 

 

The 4 important functions which are passed as props to the Table component are:

  1. startEditing() function:

When the edit icon is clicked for any row, then the edit mode should be enabled for that row. This also changes the edit icon to a check icon. The columns corresponding to the room and the name of the device should turn into text fields to enable the user to edit them. The implementation of this function is as follows:

startEditing = i => {
    this.setState({ editIdx: i });
}; 

 

‘editIdx’ is a variable which contains the row index for which the edit mode is currently on. This information is passed to the Table component which then handles the editing of the row.

  1. stopEditing() function:

When the check icon is clicked for any row, then the edit mode should be disabled for that row and the updated data should be stored on the server. The columns corresponding to the room and the name of the device should turn back into label texts. The implementation of this function is as follows:

stopEditing = i => {
  let data = this.state.obj;
  this.setState({ editIdx: -1 });
  // AJAX call to server to store the updated data
  $.ajax({
    url:
      BASE_URL + '/aaa/addNewDevice.json?macid=' + macid + '&name=' + devicename + '&room=' + room + '&latitude=' + latitude + '&longitude=' + longitude + '&access_token=' + cookies.get('loggedIn'),
    dataType: 'jsonp',
    crossDomain: true,
    timeout: 3000,
    async: false,
    success: function(response) {
      console.log(response);
    },
  });
};

 

The value of ‘editIdx’ is also set to -1 as the editing mode is now off for all rows.

  1. handleChange() function:

When the edit mode is on for any row, if we change the value of any text field, then that updated value is stored so that we can edit it without the value getting reset to the initial value of the field.

handleChange = (e, name, i) => {
  const value = e.target.value;
  let data = this.state.obj;
  this.setState({
    obj: data.map((row, j) => (j === i ? { ...row, [name]: value } : row)),
  });
};

 

  1. handleRemove() function:

When the delete icon is clicked for any row, then that row should get deleted. This change should be reflected to us on the site, as well as on the server. Hence, The implementation of this function is as follows:

handleRemove = i => {
  let data = this.state.obj;
  let macid = data[i].macid;

  this.setState({
    obj: data.filter((row, j) => j !== i),
  });
  // AJAX call to server to delete the info of device with specified Mac Address
  $.ajax({
    url:
      BASE_URL + '/aaa/removeUserDevices.json?macid=' + macid + '&access_token=' + cookies.get('loggedIn'),
    dataType: 'jsonp',
    crossDomain: true,
    timeout: 3000,
    async: false,
    success: function(response) {
      console.log(response);
    },
  });
};  

 

This is how the Table View has been implemented in the Devices tab in Settings on SUSI.AI Web Client.

Resources

 

Continue ReadingImplementing Table View in Devices Tab in Settings

How does SUSI AI web bot plugin work

  

In this blog, we’ll learn how SUSI AI web plugin works. Normally, for any bot like Kik bot, we don’t have to worry about the chat boxes or the way chatbot is rendered on screen because all of that is handled by the service on which we are building our bot. So, for these bots, we simply take the text input from user, send a GET request to SUSI server, receive the response and display it.

This is not the case with SUSI AI Web Bot plugin. In this case, there is not Bot platform.

Hence, there are a lot of other things that we need to take care of here. Let’s explore the main ones among them.

Adding bot to website:

The final product that we’re going to provide to the client is going to be just a JavaScript code. Hence the HTML code of bot widget needs to be added to the <body> of the website. This is done usingappend() method of jQuery. The html() method sets or returns the content (innerHTML) of selected elements.

Syntax:

$(selector).append(content)

So we store the HTML code of bot widget in a variable and then returns the content of that variable to the body tag using:

$("body").append(mybot);

Here “mybot” is the variable containing HTML code of bot widget.
The text boxes of user texts and bot texts are added to a HTML element in the same way.

Event handling:

JavaScript’s interaction with HTML is handled through events that occur when the user or the browser manipulates a page. Loading of the page, clicking, pressing a key, closing a window, resizing the window etc are all examples of events.

In the SUSI AI web bot, there are two major events.

  1. Clicking a button

This is required for allowing clicks on send button. Clicking of a button can be done in many ways. There are two major ways that are used in SUSI AI web bot.

Some elements already exist on the webpage. For example – the HTML code of web bot. It is added to the body tag as soon as webpage loads. For these elements we use click(). The click() binding is called “direct” binding which can be attached to the elements that already exists.

Syntax:

$(selector).click(function);

Selector – Class, ID or name of HTML element.
Function – Specifies the function to run when click event occurs.

Some elements are dynamically generated. This means that they are added at a later point of time. For example – the text messages are added when user types in. This happens after loading of page. For this, we have to create “delegated” binding by using on() method.

Syntax:

$(selector).on(event,childSelector,data,function);

Selector – Class, ID or name of HTML element
Event – (Required) Specifies one or more event(s) or namespaces to attach to the selected elements.
childSelector – (Optional) Specifies that the event handler should only be attached to the specified child elements
Data – (Optional) Specifies additional data to pass along to the function
Function – Specifies the function to run when click event occurs.

  1. Pressing the enter key

For identifying which key is pressed, we use key codes. The code for enter key is 13.
The on() method is used for handling this event. It’ll be clear from the following code snippet:

Keyup – Keyboard key is released
Keypress – Keyboard key is pressed

$('#txMessage').on('keyup keypress', function(e) {
        var keyCode = e.keyCode || e.which;
        var text = $("#txMessage").val();
        if (keyCode === 13) {
                if(text == "" ||  $.trim(text) == '') {
                        e.preventDefault();
                        return false;
                } else {
                        $("#chat-input").blur();
                        setUserResponse(text);
                        send(text);
                        e.preventDefault();
                        return false;
                }
        }
});

This is the code used to handle “enter” key event in SUSI AI web bot. Here “txMessage” is the id of text box where user entered text.

Making GET HTTP request to SUSI Server

We use ajax() method of jQuery to perform an asynchronous HTTP request to SUSI Server. Looking at the code will make things more clear:

$.ajax({
        type: "GET",
        url: baseUrl+text,
        contentType: "application/json",
        dataType: "json",
        success: function(data) {
                main(data);
        },
        error: function(e) {
                console.log(e);
        }
});

This is a GET type Ajax request. The endpoint of SUSI API is in url . We can see from dataType that the response received will be a JSON file. If the file is successfully received then main function is called else the error is logged into console.

References :

Continue ReadingHow does SUSI AI web bot plugin work

Fetching Images for RSS Responses in SUSI Web Chat

Initially, SUSI Web Chat rendered RSS action type responses like this:

The response from the server initially only contained

  • Title
  • Description
  • Link

We needed to improvise the web search & RSS results display and also add images for the results.

The web search & RSS results are now rendered as :

How was this implemented?

SUSI AI uses Yacy to fetchRSSs feeds. Firstly the server using the console process to return the RSS feeds from Yacy needs to be configured to return images too.

"yacy":{
  "example":"http://127.0.0.1:4000/susi/console.json?q=%22SELECT%20title,%20link%20FROM%20yacy%20WHERE%20query=%27java%27;%22",
  "url":"http://yacy.searchlab.eu/solr/select?wt=yjson&q=",
  "test":"java",
  "parser":"json",
  "path":"$.channels[0].items",
  "license":""
}

In a console process, we provide the URL needed to fetch data from, the query parameter needed to be passed to the URL and the path to look for the answer in the API response.

  • url = <url>   – the URL to the remote JSON service which will be used to retrieve information. It must contain a $query$ string.
  • test = <parameter> – the parameter that will replace the $query$ string inside the given URL. It is required to test the service.

Here the URL used is :

http://yacy.searchlab.eu/solr/select?wt=yjson&q=QUERY

To include images in RSS action responses, we need to parse the images also from the Yacy response. For this, we need to add `image` in the selection rule while calling the console process

"process":[
  {
    "type":"console",
    "expression":"SELECT title,description,link FROM yacy WHERE query='$1$';"
  }
]

Now the response from the server for RSS action type will also include `image` along with title, description, and link. An example response for the query `Google` :

{
  "title": "Terms of Service | Google Analytics \u2013 Google",
  "description": "Read Google Analytics terms of service.",
  "link": "http://www.google.com/analytics/terms/",
  "image":   "https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_116x41dp.png",
}

However, the results at times, do not contain images because there are none stored in the index. This may happen if the result comes from p2p transmission within Yacy where no images are transmitted. So in cases where images are not returned by the server, we use the link preview service to preview the link and fetch the image.

The endpoint for previewing the link is :

BASE_URL+'/susi/linkPreview.json?url=URL'

On the client side, we first search the response for data objects with images in API actions. And the amongst the remaining data objects in answers[0].data, we preview the link to fetch image keeping a check on the count. This needs to be performed for processing the history cognitions too.To preview the remaining links in a loop, we cannot make ajax calls directly in a loop. To handle this, nested ajax calls are made using the function previewURLForImage() where we loop through the remaining links and on the success we decrement the count and call previewURLForImage() on the next link and on error we try previewURLForImage() on the next link without decrementing the count.

success: function (rssResponse) {
  if(rssResponse.accepted){
    respData.image = rssResponse.image;
    respData.descriptionShort = rssResponse.descriptionShort;
    receivedMessage.rssResults.push(respData);
  }
  if(receivedMessage.rssResults.length === count ||
    j === remainingDataIndices.length - 1){
    let message = ChatMessageUtils.getSUSIMessageData(receivedMessage, currentThreadID);
    ChatAppDispatcher.dispatch({
      type: ActionTypes.CREATE_SUSI_MESSAGE,
      message
    });
  }
  else{
    j+=1;
    previewURLForImage(receivedMessage,currentThreadID,
BASE_URL,data,count,remainingDataIndices,j);
  }
},

And we store the results as rssResults which are used in MessageListItems to fetch the data and render. The nested calling of previewURLForImage() ends when we have the required count of results or we have finished trying all links for previewing images. We then dispatch the message to the message store. We now improvise the UI. I used Material UI Cards to display the results and for the carousel like display, react-slick.

<Card className={cardClass} key={i} onClick={() => {
  window.open(tile.link,'_blank')
}}>
  {tile.image &&
    (
      <CardMedia>
        <img src={tile.image} alt="" className='card-img'/>
      </CardMedia>
    )
  }
  <CardTitle title={tile.title} titleStyle={titleStyle}/>
  <CardText>
    <div className='card-text'>{cardText}</div>
    <div className='card-url'>{urlDomain(tile.link)}</div>
  </CardText>
</Card>

We used the full width of the message section to display the results by not wrapping the result in message-list-item class. The entire card is hyperlinked to the link. Along with title and description, the URL info is also shown at the bottom right. To get the domain name from the link, urlDomain() function is used which makes use of the HTML anchor tag to get the domain info.

function urlDomain(data) {
  var a = document.createElement('a');
  a.href = data;
  return a.hostname;
}

To prevent stretching of images we use `object-fit: contain;` to make the images fit the image container and align it to the middle.

We finally have our RSS results with images and an improvised UI. The complete code can be found at SUSI WebChat Repo. Feel free to contribute

Resources
Continue ReadingFetching Images for RSS Responses in SUSI Web Chat

Modifying SUSI Skills using SUSI Skill CMS

SUSI Skill CMS is a complete solution right from creating a skill to modifying the skill. The skills in SUSI are well synced with the remote repository and can be completely modified using the Edit Skill feature of SUSI Skill CMS. Here’s how to Modify a Skill.

  1. Sign Up/Login to the website using your credentials in skills.susi.ai
  2. Choose the SKill which you want to edit and click on the pencil icon.
  3. The following screen allows editing the skill. One can change the Group, Language, Skill Name, Image and the content as well.
  4. After making the changes the commit message can be added to Save the changes.

To achieve the above steps we require the following API Endpoints of the SUSI Server.

  1. http://api.susi.ai/cms/getSkillMetadata.json – This gives us the meta data which populates the various Skill Content, Image, Author etc.
  2. http://api.susi.ai/cms/getAllLanguages.json – This gives us all the languages of a Skill Group.
  3. http://api.susi.ai/cms/getGroups.json – This gives us all the list of Skill Groups whether Knowledge, Entertainment, Smalltalk etc.

Now since we have all the APIs in place we make the following AJAX calls to update the Skill Process.

  1. Since we are detecting changes in all the fields (Group Value, Skill Name, Language Value, Image Value, Commit Message, Content changes and the format of the content), the AJAX call can only be sent when there is a change in the PR and there is no null or undefined value in them. For that, we make various form validations. They are as follows.
    1. We first detect whether the User is in a logged in state.
if (!cookies.get('loggedIn')) {
            notification.open({
                message: 'Not logged In',
                description: 'Please login and then try to create/edit a skill',
                icon: <Icon type="close-circle" style={{ color: '#f44336' }} />,
            });
        }
  1. We check whether the image uploaded matches the format of the Skill image to be stored which is ::image images/imageName.png
if (!new RegExp(/images\/\w+\.\w+/g).test(this.state.imageUrl)) {
            notification.open({
                message: 'Error Processing your Request',
                description: 'image must be in format of images/imageName.jpg',
                icon: <Icon type="close-circle" style={{ color: '#f44336' }} />,
            });
        }
  1. We check if the commit message is not null and notify the user if he forgot to add a message.
if (this.state.commitMessage === null) {
            notification.open({
                message: 'Please make some changes to save the Skill',
                icon: <Icon type="close-circle" style={{ color: '#f44336' }} />,
            });
        }
  1. We also check whether the old values of the skill are completely similar to the new ones, in this case, we do not send the request.
if (toldValues===newValues {
            notification.open({
                message: 'Please make some changes to save the Skill',
                icon: <Icon type="close-circle" style={{ color: '#f44336' }} />,
            });
        }

To check out the complete code, go to this link.

  1. Next, if the above validations are successful, we send a POST request to the server and show the notification to the user accordingly, whether the changes to the Skill Data have been updated or not. Here’s the AJAX call snippet.
// create a form object
let form = new FormData();       
/* Append the following fields from the Skill Component:- OldModel, OldLanguage, OldSkill, NewModel, NewGroup, NewLanguage, NewSkill, changelog, content, imageChanged, old_image_name, new_image_name, image_name_changed, access_token */  
if (image_name_changed) {
            file = this.state.file;
            // append file to image
        }

        let settings = {
            "async": true,
            "crossDomain": true,
            "url": "http://api.susi.ai/cms/modifySkill.json",
            "method": "POST",
            "processData": false,
            "contentType": false,
            "mimeType": "multipart/form-data",
            "data": form
        };
        $.ajax(settings)..done(function (response) {
         //show success
        }.
        .fail(function(response){
         // show failure
        }
  1. To verify all this we head to the commits section of the SUSI Skill Data repo and see the changes we made. The changes can be seen here https://github.com/fossasia/susi_skill_data/commits/master 

Resources

  1. AJAX POST Request – https://api.jquery.com/jquery.post/ 
  2. Material UI – http://material-ui.com 
  3. Notifications – http://www.material-ui.com/#/components/notification 
Continue ReadingModifying SUSI Skills using SUSI Skill CMS

Change Password for SUSI Accounts Using Access Token and Email-ID

In this blog, I discuss how the SUSI server synchronizes with SUSI Accounts and SUSI webchat for users to Change Password. When a user logs in, the clients store the email id of the user along with the access token in cookies. These are stored once the client gets a positive login response from the server. Both of these are required at the time of making the final call. Web clients store the email id and access token in the following way.

cookies.set('loggedIn', loggedIn, { path: '/', maxAge: time });
cookies.set('emailId', email, { path: '/', maxAge: time });

First, the client has to ask the user to enter their current password. A javascript test is used to validate that at least 6 characters must be entered by the user. A similar test is run on the new password. But while confirming the password, client checks whether the user has entered the same password as new password or not. These are just the basics. In next stage (which is achieved only when all the above conditions are met), client encodes the email id (which it gets from cookies), current password, new password and the access token (which it again extracts from cookies).

Now, Client just has to make an ajax request to the server. The response is processed and sent back to the client. Let us now look at PasswordChange Servlet.

The base user role is defined as USER. Initial steps of the servlet are to extract the values form the request it receives. The values extracted from the request are in turn used to make a client’s identity. Before that, server checks if current and new password have same values or not. If not, then server returns a JSON response to user stating, “Your current password and new password matches”. Otherwise, it will continue its control flow as it is. Look at the code snippet below:

if(password.equals(newpassword)){
            result.put("message", "Your current password and new password matches");
            result.put("accepted", false);
            return new ServiceResponse(result);
        }

The reader here may think that they have discovered a hack. But they have not. Why? Because this is just the first step. In later stages, the hash of passwords are used to match to see whether the passwords match or not. To obtain a proper client identity, first a Client credentials object is made with support from the email id which is received in ‘changepassword’ attribute. Using the ClientCredentials object made above, an object of Authentication class is made. This object uses a method defined in its class to return a valid client identity. Using the client identity, value of password hash is extracted from the database along with the salt used to hash the password. If any error is encountered while extracting the client’s password hash value and/or salt value, an error is thrown towards the client, with a message stating “invalid credentials”.

ClientCredential pwcredential = new ClientCredential(ClientCredential.Type.passwd_login, useremail);
            Authentication authentication = DAO.getAuthentication(pwcredential);
            ClientCredential emailcred = new ClientCredential(ClientCredential.Type.passwd_login,
                authentication.getIdentity().getName());
            ClientIdentity identity = authentication.getIdentity();
            String passwordHash;
            String salt;

            try {
                passwordHash = authentication.getString("passwordHash");
                salt = authentication.getString("salt");
            } catch (Throwable e) {
                Log.getLog().info("Invalid password try for user: " + identity.getName() + " from host: " + post.getClientHost() + " : password or salt missing in database");
                result.put("message", "invalid credentials");
                throw new APIException(422, "Invalid credentials");
            }

Using the same salt value that was used earlier, a hash for password entered by the user will be generated which now matches  the previous value. This is the point where the hack you were thinking you found, failed. Again the server throws an error message if user’s credential did not match. Passwords are hard to handle and easy to guess. So here we have used quite many tests before changing them. Users are not allowed to use their email id as a password as well.

If the server is clear on all the above facts and tests, It finally generates a new hashed value of the password received in the parameter ‘newpassword’ and replaces the old hash value with the new one. To notify the clients that password change exited with a success response, it sends a JSON object with message “Your password has been changed!” and accepted flag set to true.

if (DAO.hasAuthentication(emailcred)) {
                    Authentication emailauth = DAO.getAuthentication(emailcred);
                    String newsalt = createRandomString(20);
                    emailauth.remove("passwordHash");
                    emailauth.put("passwordHash", getHash(newpassword, salt));
                    Log.getLog().info("password change for user: " + identity.getName() + " via newpassword from host: " + post.getClientHost());
                    result.put("message", "Your password has been changed!");
                    result.put("accepted", true);
                }

 

Additional Resources:

Wikipedia article: What is DAO?

Continue ReadingChange Password for SUSI Accounts Using Access Token and Email-ID

Including a Graph Component in the Remote Access Framework for PSLab

The remote-lab software of the pocket science lab enables users to access their devices remotely via the Internet. It includes an API server designed with Python Flask, and a web-app designed with EmberJS that allows users to access the API and carry out various tasks such as writing and executing Python scripts. In a series of blog posts, various aspects of this framework such as  remote execution of function strings, automatic deployment on various domains, creating and submitting python scripts which will be run on the remote server etc have already been explored.  This blog post deals with the inclusion of a graph component in the webapp that will be invoked when the user utilises the `plot` command in their scripts.

The JQPLOT library is being used for this purpose, and has been found to be quite lightweight and has a vast set of example code .

Task list for enabling the plotting feature
  • Add a plot method to the codeEvaluator module in the API server and allow access to it by adding it to the evalGlobals dictionary
  • Create an EmberJS component for handling plots
    • Create a named div in the template
    • Invoke the Jqplot initializer from the JS file and pass necessary arguments and data to the jqplot instance
  • Add a conditional statement to include the jqplot component whenever a plot subsection is present in the JSON object returned by the API server after executing a script
Adding a plot method to the API server

Thus far, in addition to the functions supported by the sciencelab.py instance of PSLab, users had access to print, print_, and button functions. We shall now add a plot function.

def plot(self,x,y,**kwargs):
self.generatedApp.append({"type":"plot","name":kwargs.get('name','myPlot'),"data":[np.array([x,y]).T.tolist()]})

 

The X,Y datasets provided by the user are stacked in pairs because jqplot requires [x,y] pairs . not separate datasets.

We also need to add this to evalGlobals, so we shall modify the __init__ routine slightly:

self.evalGlobals['plot']=self.plot
Building an Ember component for handling plots

First, well need to install jqplot:   bower install –save jqplot

And this must be followed by including the following files using app.import statements in ember-cli-build.js

  • bower_components/jqplot/jquery.jqplot.min.js
  • bower_components/jqplot/plugins/jqplot.cursor.js
  • bower_components/jqplot/plugins/jqplot.highlighter.js
  • bower_components/jqplot/plugins/jqplot.pointLabels.js
  • bower_components/jqplot/jquery.jqplot.min.css

In addition to the jqplot js and css files, we have also included a couple of plugins we shall use later.

Now we need to set up a new component : ember g component jqplot-graph

Our component will accept an object as an input argument. This object will contain the various configuration options for the plot

Add the following line in templates/components/jqplot-graph.hbs:

style="solid gray 1px;" id="{{data.name}}">

The JS file for this template must invoke the jqplot function in order to insert a complete plot into the previously defined <div> after it has been created. Therefore, the initialization routine must override the didInsertElement routine of the component.

components/jqplot-graph.js

import Ember from 'ember';

export default Ember.Component.extend({
  didInsertElement () {
    Ember.$.jqplot(this.data.name,this.data.data,{
        title: this.title,

        axes: {
          xaxis: {
            tickInterval: 1,
            rendererOptions: {
            minorTicks: 4
            }
          },
        },
        highlighter: {
          show: true, 
          showLabel: true, 

          tooltipAxes: 'xy',
          sizeAdjust: 9.5 , tooltipLocation : 'ne'
        },				  
        legend: {
          show: true,
          location: 'e',
          rendererOptions: {
            numberColumns: 1,
          }
        },
        cursor:{ 
          show: true,
          zoom:true, 
          showTooltip:false
          } 

    });
  }
});

Our component is now ready to be used , and we must make the necessary changes to user-home.hbs in order to include the plot component if the output JSON of a script executed on the server contains it.

The following excerpt from the results modal shows how the plot component can be inserted

{{#each codeResults as |element|}}
	{{#if (eq element.type 'text')}}
		{{element.value}}<br>
	{{/if}}
	{{#if (eq element.type 'plot')}}
		{{jqplot-graph data=element}}
	{{/if}}
{{/each}}            

Most of the other components such as buttons and spans have been removed for clarity. Note that the element object is passed to the jqplot-graph component as an argument so that the component may configure itself accordingly.

In conclusion, the following screencast shows what we have created. A simple plot command creates a fancy plot in the output which includes data point highlighting, and can be easily configured to do a lot more. In the next blog post we shall explore how to use this plot to create a persistent application such as an oscilloscope.

Resources:

 

Continue ReadingIncluding a Graph Component in the Remote Access Framework for PSLab

Enhancing the Functionality of User Submitted Scripts in the PSLab-remote framework

The remote-lab framework of the pocket science lab enables users to access their devices remotely via the internet. Its design involves an API server built with Python-Flask and a webapp that uses EmberJS. This post is the latest in a series of blog posts which have explored and elaborated various aspect of the remote-lab such as designing the API server and testing with Postman, remote execution of function strings, automatic deployment on various domains etc. It also supports creating and submitting python scripts which will be run on the remote server, and the console output relayed to the webapp.

In this post, we shall take a look at how we can extend the functionality by providing support for object oriented code in user submitted scripts.

Let’s take an example of a Python script where the user wishes to create a button which when clicked will read a voltage via the API server, and display the value to the remote user. Clearly, an interpreter that only provides the console output is not enough for this task. We need the interpreter to generate an app structure that also includes callbacks for widgets such as buttons, and JSON objects are an obvious choice for relaying such a structure to the webapp.

In a nutshell, we had earlier created an API method that could execute a python script and return a string output, and now we will modify this method to return a JSON encoded structure which will be parsed by the webapp in order to display an output.

Let’s elaborate this with an example : Example.py

print ('testing')
print ('testing some changes..... ')
print_('highlighted print statement')

 

JSON returned by the API [localhost:8000/runScriptById] , for the above script:

{"Date": "Tue, 01 Aug 2017 21:39:12 GMT", "Filename": "example.py", "Id": 4,
 "result": [
  {"name": "print", "type": "span", "value": "('testing',)"},
  {"name": "print", "type": "span", "value": "('testing some changes..... ',)"},
  {"class": "row well", "name": "print", "type": "span", "value": "highlighted print statement"}
  ],
"status": true}
Screenshot of the EmberJS webapp showing the output rendered with the above JSON

Adding Support for Widgets

In the previous section, we laid the groundwork for a flexible platform. Instead of returning a string, the webapp accepts a JSON object and parses it. We shall now add support for a clickable button which can be associated with a valid PSLab function.

An elementary JS twiddle has been made by Niranjan Rajendran which will help newbies to understand how to render dynamic templates via JSON objects retrieved from APIs. The twiddle uses two API endpoints; one to retrieve the compiled JSON output, and another to act as a voltmeter method which returns a voltage value.

To understand how this works in pslab-remote, consider a one line script called button.py:

button('get voltage',"get_voltage('CH1')")

The objective is to create a button with the text ‘get voltage’ on it , and which when clicked will run the command ‘get_voltage(‘CH1’)’ on the API server, and display the result.

When this script is run on the API server, it returns a JSON object with the following structure:

{"Date": "Tue, 01 Aug 2017 21:39:12 GMT", "Filename": "example.py", "Id": 4,
 "result": [  {"type":"button","name":"button-id0","label":"get_voltage","fetched_value":"","action":{"type":"POST","endpoint":"get_voltage('CH1')","success":{"datapoint":'result',"type":"display_number", "target":"button-id0-label"}}},
  {"name": "button-id0label", "type": "label", "value": ""},
  ],
"status": true}

The above JSON object is parsed by the webapp’s user-home template, and a corresponding button and label are generated. The following section of code from user-home.hbs renders the JSON object

{{#each codeResults as |element|}}
  {{#if (eq element.type 'label')}}
    <label  id="{{element.name}}" class="{{element.class}}">{{element.value}}</label>
  {{/if}}
  {{#if (eq element.type 'button')}}
    <button id="{{element.name}}" {{action 'runButtonAction' element.action}}>{{element.label}}</button>
  {{/if}}
{{/each}}    

An action was also associated with the the created button, and this is the “get_voltage(‘CH1’)” string which we had specified in our one line script.

For the concluding section, we shall see how this action is invoked when the button is clicked, and how the returned value is used to update the contents of the label that was generated as part of this button.

Action defined in controllers/user-home.js :

runButtonAction(actionDefinition) {
  if(actionDefinition.type === 'POST') {
    Ember.$.post('/evalFunctionString',{'function':actionDefinition.endpoint},this,"json")
      .then(response => {
        const resultValue = Ember.get(response, actionDefinition.success.datapoint);
        if (actionDefinition.success.type === 'display_number') {
           Ember.$('#' + actionDefinition.success.target).text(resultValue.toFixed(3));
        }
      });
  }
}

The action string is passed to the evalFunctionString endpoint of the API, and the contents are mapped to the display label.

Screencast of the above process
Resources:
Continue ReadingEnhancing the Functionality of User Submitted Scripts in the PSLab-remote framework

Implementing the Feedback Functionality in SUSI Web Chat

SUSI AI now has a feedback feature where it collects user’s feedback for every response to learn and improve itself. The first step towards guided learning is building a dataset through a feedback mechanism which can be used to learn from and improvise the skill selection mechanism responsible for answering the user queries.

The flow behind the feedback mechanism is :

  1. For every SUSI response show thumbs up and thumbs down buttons.
  2. For the older messages, the feedback thumbs are disabled and only display the feedback already given. The user cannot change the feedback already given.
  3. For the latest SUSI response the user can change his feedback by clicking on thumbs up if he likes the response, else on thumbs down, until he gives a new query.
  4. When the new query is given by the user, the feedback recorded for the previous response is sent to the server.

Let’s visit SUSI Web Chat and try this out.

We can find the feedback thumbs for the response messages. The user cannot change the feedback he has already given for previous messages. For the latest message the user can toggle feedback until he sends the next query.

How is this implemented?

We first design the UI for feedback thumbs using Material UI SVG Icons. We need a separate component for the feedback UI because we need to store the state of feedback as positive or negative because we are allowing the user to change his feedback for the latest response until a new query is sent. And whenever the user clicks on a thumb, we update the state of the component as positive or negative accordingly.

import ThumbUp from 'material-ui/svg-icons/action/thumb-up';
import ThumbDown from 'material-ui/svg-icons/action/thumb-down';

feedbackButtons = (
  <span className='feedback' style={feedbackStyle}>
    <ThumbUp
      onClick={this.rateSkill.bind(this,'positive')}
      style={feedbackIndicator}
      color={positiveFeedbackColor}/>
    <ThumbDown
      onClick={this.rateSkill.bind(this,'negative')}
      style={feedbackIndicator}
      color={negativeFeedbackColor}/>
  </span>
);

The next step is to store the response in Message Store using saveFeedback Action. This will help us later to send the feedback to the server by querying it from the Message Store. The Action calls the Dispatcher with FEEDBACK_RECEIVED ActionType which is collected in the MessageStore and the feedback is updated in the Message Store.

let feedback = this.state.skill;

if(!(Object.keys(feedback).length === 0 &&    
feedback.constructor === Object)){
  feedback.rating = rating;
  this.props.message.feedback.rating = rating;
  Actions.saveFeedback(feedback);
}

case ActionTypes.FEEDBACK_RECEIVED: {
  _feedback = action.feedback;
  MessageStore.emitChange();
  break;
}

The final step is to send the feedback to the server. The server endpoint to store feedback for a skill requires other parameters apart from feedback to identify the skill. The server response contains an attribute `skills` which gives the path of the skill used to answer that query. From that path we need to parse :

  • Model : Highest level of abstraction for categorising skills
  • Group : Different groups under a model
  • Language : Language of the skill
  • Skill : Name of the skill

For example, for the query `what is the capital of germany` , the skills object is

"skills": ["/susi_skill_data/models/general/smalltalk/en/English-Standalone-aiml2susi.txt"]

So, for this skill,

    • Model : general
    • Group : smalltalk
    • Language : en
    • Skill : English-Standalone-aiml2susi

The server endpoint to store feedback for a particular skill is :

BASE_URL+'/cms/rateSkill.json?model=MODEL&group=GROUP&language=LANGUAGE&skill=SKILL&rating=RATING'

Where Model, Group, Language and Skill are parsed from the skill attribute of server response as discussed above and the Rating is either positive or negative and is collected from the user when he clicks on feedback thumbs.

When a new query is sent, the sendFeedback Action is triggered with the required attributes to make the server call to store feedback on server. The client then makes an Ajax call to the rateSkill endpoint to send the feedback to the server.

let url = BASE_URL+'/cms/rateSkill.json?'+
          'model='+feedback.model+
          '&group='+feedback.group+
          '&language='+feedback.language+
          '&skill='+feedback.skill+
          '&rating='+feedback.rating;

$.ajax({
  url: url,
  dataType: 'jsonp',
  crossDomain: true,
  timeout: 3000,
  async: false,
  success: function (response) {
    console.log(response);
  },
  error: function(errorThrown){
    console.log(errorThrown);
  }
});

This is how the feedback feedback mechanism works in SUSI Web Chat. The entire code can be found at SUSI Web Chat Repository.

Resources

 

Continue ReadingImplementing the Feedback Functionality in SUSI Web Chat