susi-api – blog.fossasia.org

Adding Support for Displaying Images in SUSI iOS

Post author:Jogendra Kumar
Post published:July 29, 2018
Post category:FOSSASIA GSoC SUSI.AI
Post comments:0 Comments

SUSI provided exciting features in the chat screen. The user can ask a different type of queries and SUSI respond according to the user query. If the user wants to get news, the user can just ask news and SUSI respond with the news. Like news, SUSI can respond to so many queries. Even user can ask SUSI for the image of anything. In response, SUSI displays the image that the user asked. Exciting? Let’s see how displaying images in the chat screen of SUSI iOS is implemented.

Getting image URL from server side –

When we ask susi for the image of anything, it returns the URL of image source in response with answer action type. We get the URL of the image in the expression key of actions object as below:

actions:
[
{
language: "en",
type: "answer",
expression:
"https://pixabay.com/get/e835b60f2cf6033ed1584d05fb1d4790e076e7d610ac104496f1c279a0e8b0ba_640.jpg"
}
]

Implementation in App –

In the app, we check if the response coming from server is image URL or not by following two methods.

One – Check if the expression is a valid URL:

func isValidURL() -> Bool {
if let url = URL(string: self) {
return UIApplication.shared.canOpenURL(url)
}
return false
}

The method above return boolean value if the expression is valid or not.

Two – Check if the expression contains image source in URL:

func isImage() -> Bool {
let imageFormats = ["jpg", "jpeg", "png", "gif"]

if let ext = self.getExtension() {
return imageFormats.contains(ext)
}

return false
}

The method above returns the boolean value if the URL string is image source of not by checking its extension.

If both the methods return true, the expression is a valid URL and contain image source, then we consider it as an Image and proceed further.

In collectionView of the chat screen, we register ImageCell and present the cell if the response is the image as below.

Registering the Cell –

register(_:forCellWithReuseIdentifier:) method registers a class for use in creating new collection view cells.

collectionView?.register(ImageCell.self, forCellWithReuseIdentifier: ControllerConstants.imageCell)

Presenting the Cell using cellForItemAt method of UICollectionView –

The implementation of cellForItemAt method is responsible for creating, configuring, and returning the appropriate cell for the given item. We do this by calling the dequeueReusableCell(withReuseIdentifier:for:) method of the collection view and passing the reuse identifier that corresponds to the cell type we want. That method always returns a valid cell object. Upon receiving the cell, we set any properties that correspond to the data of the corresponding item, perform any additional needed configuration, and return the cell.

if let expression = message.answerData?.expression, expression.isValidURL(), expression.isImage() {
if let cell = collectionView.dequeueReusableCell(withReuseIdentifier: ControllerConstants.imageCell, for: indexPath) as? ImageCell {
cell.message = message
return cell
}
}

In ImageCell, we present a UIImageView that display the image. When cell the message var, it call downloadImage method to catch and display image from server URL using Kingfisher method.

In method below –

Get image URL string and check if it is image URL
Convert image string to image URL
Set image to the imageView

func downloadImage() {
if let imageString = message?.answerData?.expression, imageString.isImage() {
if let url = URL(string: imageString) {
imageView.kf.setImage(with: url, placeholder: ControllerConstants.Images.placeholder, options: nil, progressBlock: nil, completionHandler: nil)
}
}
}

We have added a tap gesture to the imageView so that when the user taps the image, it opens the full image in the browser by using the method below:

@objc func openImage() {
if let imageURL = message?.answerData?.expression {
if let url = URL(string: imageURL) {
UIApplication.shared.open(url, options: [:], completionHandler: nil)
}
}
}

Final Output –

Resources –

Adding Servlet for SUSI Service

Post author:dynamitechetan
Post published:August 15, 2017
Post category:FOSSASIA
Post comments:0 Comments

Whenever we ask anything to SUSI, we get a intelligent reply. The API endpoints which clients uses to give the reply to users is http://api.susi.ai/susi/chat.json, and this API endpoint is defined in SUSIService.java. In this blog post I will explain how any query is processed by SUSI Server and the output is sent to clients.

public class SusiService extends AbstractAPIHandler implements APIHandler

This is a public class and just like every other servlet in SUSI Server, this extends AbstractAPIHandler class, which provides us many options including AAA related features.

    public String getAPIPath() {
           return "/susi/chat.json";
    }

The function above defines the endpoint of the servlet. In this case it is “/susi/chat.json”. The final API Endpoint will be API http://api.susi.ai/susi/chat.json.

This endpoint accepts 6 parameters in GET request . “q” is the query “parameter”, “timezoneOffset” and “language” parameters are for giving user a reply according to his local time and local language, “latitude” and “longitude” are used for getting user’s location.

      String q = post.get("q", "").trim();
        int count = post.get("count", 1);
        int timezoneOffset = post.get("timezoneOffset", 0); // minutes, i.e. -60
        double latitude = post.get("latitude", Double.NaN); // i.e. 8.68
        double longitude = post.get("longitude", Double.NaN); // i.e. 50.11
        String language = post.get("language", "en"); // ISO 639-1

After getting all the parameters we do a database update of the skill data. This is done using DAO.susi.observe() function.

Then SUSI checks that whether the reply was to be given from a etherpad dream, so we check if we are dreaming something.

if (etherpad_dream != null && etherpad_dream.length() != 0) {
    String padurl = etherpadUrlstub + "/api/1/getText?apikey=" + etherpadApikey + "&padID=$query$";

If SUSI is dreaming something then we call the etherpad API with the API key and padID.

After we get the response from the etherpad API we parse the JSON and store the text from the skill in a local variable.

JSONObject json = new JSONObject(serviceResponse);
String text = json.getJSONObject("data").getString("text");

After that, we fill an empty SUSI mind with the dream. SUSI susi_memory_dir is a folder named as “susi” in the data folder, present at the server itself.

SusiMind dream = new SusiMind(DAO.susi_memory_dir);

We need the memory directory here to get a share on the memory of previous dialogues, otherwise, we cannot test call-back questions.

JSONObject rules = SusiSkill.readEzDSkill(new BufferedReader(new InputStreamReader(new ByteArrayInputStream(text.getBytes(StandardCharsets.UTF_8)), StandardCharsets.UTF_8)));
dream.learn(rules, new File("file://" + etherpad_dream));

When we call dream.learn function, Susi starts dreaming. Now we will try to find an answer out of the dream.

The SusiCognition takes all the parameters including the dream name, query and user’s identity and then try to find the answer to that query based on the skill it just learnt.

    SusiCognition cognition = new SusiCognition(dream, q, timezoneOffset, latitude, longitude, language, count, user.getIdentity());

If SUSI finds an answer then it replies back with that answers else it tries to answer with built-in intents. These intents are both existing SUSI Skills and internal console services.

if (cognition.getAnswers().size() > 0) {          DAO.susi.getMemories().addCognition(user.getIdentity().getClient(), cognition);
    return new ServiceResponse(cognition.getJSON());
}

This is how any query is processed by SUSI Server and the output is sent to clients.

Currently people use Etherpad (http://dream.susi.ai/) to make their own skills, but we are also working on SUSI CMS where we can edit and test the skills on the same web application.

References

Java Servlet Docs: http://docs.oracle.com/javaee/6/tutorial/doc/bnafd.html

Embedding Jetty: http://www.eclipse.org/jetty/documentation/9.4.x/embedding-jetty.html

Server-Side Java: http://www.javaworld.com/article/2077634/java-web-development/how-to-get-started-with-server-side-java.html

Pie Chart Responses from SUSI Server

Post author:dynamitechetan
Post published:July 24, 2017
Post category:FOSSASIA
Post comments:0 Comments

Giving out responses in charts and graphs is a very common reply of various assistants. We also have it in SUSI. We can show users the output of stocks, market covers and various percentages output in Pie Charts.

A pie chart is a circular chart/graph which is divided in some segments like a pie. The segments in a pie chart shows the share of each object or category.

The PieChartServlet in SUSI Server is a servlet that takes the JSON data of a the pie chart components as input parameters and returns an Image of the rendered Pie Chart..

public class PieChartServlet extends HttpServlet {

This is a simple HttpServlet. It does not require any authentication or base user role. So we extend the HttpServlet here.

    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws  IOException {

doGet is a method which is triggered whenever the PieChartServlet receives a GET Query. This will contain all the code that we will need to render the final output.

{"ford":"17.272992","toyota":"27.272992","renault":"47.272992"}

This is the sample JSON that we send to the PieChartServlet. This contains the names of the pie chart components and their respective percentages. After we receive these parameters we parse them and store them in our local variables.
These variables are then further used to plot the pie chart.

To plot these values in pie chart we have used a library JFreeChart.

This is a free and well documented Java chart library. This library supports PNGs and JPEGs as well as vector graphics file formats

JFreeChart chart = getChart(json, legendBit, tooltipBit);

From here we call a function getChart This function accept 3 parameters. The json which we sent as the GET parameter, legendBit and tooltilBit. These are also sent as GET parameters. In this example I will use legendBit as true and tooltipBit as false.

        JFreeChart chart = ChartFactory.createPieChart("SUSI Visualizes -   PieChart", dataset, legend, tooltips, urls);

        chart.setBorderPaint(Color.BLACK);
        chart.setBorderStroke(new BasicStroke(5.0f));
        chart.setBorderVisible(true);

        return chart;

This is the function getChart. It creates a chart using the ChartFactory method and set sets the SUSI branding on it as “SUSI Visualizes – PieChart”. It accepts the datasets, legends and tool tips. The variable, dataset is nothing but the json keys and their values.

After the ChartFactory returns the chart we set the border of the chart and returns a pie chart back the function where it was called.

ChartUtilities.writeChartAsPNG(outputStream, chart, width, height);

Finally we write the chart as a PNG image and send it to the user.

Output

This can be tested at

https://api.susi.ai/vis/piechart.png?data={%22ford%22:%2217.272992%22,%22toyota%22:%2227.272992%22,%22renault%22:%2247.272992%22}&width=1000&height=1000&legend=true&tooltip=false

References

JFreeChart Homepage: http://www.jfree.org/jfreechart/
JFreeChart Documentation: http://www.jfree.org/jfreechart/api/javadoc/index.html
SUSI Server: https://github.com/fossasia/susi_server/

Using react-slick for Populating RSS Feeds in SUSI Chat

Post author:rishiraj824
Post published:July 16, 2017
Post category:FOSSASIA GSoC SUSI.AI
Post comments:0 Comments

To populate SUSI RSS Feed generated, while chatting on SUSI Web Chat, I needed a Horizontal Swipeable Tile Slider. For this purpose, I made use of the package react-slick. The information which was supposed to be handled as obtained from the SUSI Server to populate the RSS feed was

Title
Description
Link

Hence to show all of this information like a horizontal scrollable feed, tiles by react-slick solves the purpose. To achieve the same, let’s see follow the steps below.

First step is to install the react-slick package into our project folder, for that we use

npm install react-slick --save

Next we import the Slider component from react-slick package into the file where we want the slider, here MessageListItem.react.js

import Slider from 'react-slick'

Add Slider with settings as given in the docs. This is totally customisable. For more customisable options go to https://github.com/akiran/react-slick

var settings = {
         speed: 500,
         slidesToShow: 3,
         slidesToScroll: 1,
        swipeToSlide:true,
         swipe:true,
         arrows:false
     };

speed – The Slider will scroll horizontally with this speed.

slidesToShow – The number of slides to populate in one visible screen

swipeToSlide, swipe – Enable swiping on touch screen devices.

arrows – Put false, to disable arrows

The next step is to initialize the Slider component inside the render function and populate it with the tiles. The full code snippet is available at MessageListItem.react.js

<Slider {..settings}>//Append the settings which you created
    {yourListToProps} // Add the list tiles you want to see
</Slider>

Adding a little bit of styling, full code available in ChatApp.css

 .slick-slide{
 margin: 0 10px;
}
.slick-list{
  max-height: 100px;
}

This is the output you would get in your screen.

Note – To prevent errors like the following on testing with jest, you will have to add the following lines into the code.

Error log, which one may encounter while using react-slick –

 matchMedia not present, legacy browsers require a polyfill

  at Object.MediaQueryDispatch (node_modules/enquire.js/dist/enquire.js:226:19)
  at node_modules/enquire.js/dist/enquire.js:291:9
  at i (node_modules/enquire.js/dist/enquire.js:11:20)
  at Object.<anonymous> (node_modules/enquire.js/dist/enquire.js:21:2)
  at Object.<anonymous> (node_modules/react-responsive-mixin/index.js:2:28)
  at Object.<anonymous> (node_modules/react-slick/lib/slider.js:19:29)
  at Object.<anonymous> (node_modules/react-slick/lib/index.js:3:18)
  at Object.<anonymous> (src/components/Testimonials.jsx:3:45)
  at Object.<anonymous> (src/pages/Index.jsx:7:47)
  at Object.<anonymous> (src/App.jsx:8:40)
  at Object.<anonymous> (src/App.test.jsx:3:38)
  at process._tickCallback (internal/process/next_tick.js:103:7)

In package.json, add the following lines-

"peerDependencies": {
      "react": "^0.14.0 || ^15.0.1",
      "react-dom": "^0.14.0 || ^15.0.1"
    },
   "jest": {
      "setupFiles": ["./src/setupTests.js", "./src/node_modules/react-scripts/config/polyfills.js"]
   },

In src/setupTests.js, add the following lines.

window.matchMedia = window.matchMedia || (() => { return { matches: false, addListener: () => {}, removeListener: () => {}, }; });

These lines will help resolve any occurring errors while testing with Jest or ESLint.

To have a look at the full project, visit https://github.com/fossasia/chat.susi.ai and feel free to contribute. To test the project visit http://chat.susi.ai

Resources

NPM Package – React-slick
Official Examples – http://neostack.com/opensource/react-slick