Correct the API for Downloading GitHub Content in SUSI.AI android app

The content from github in the SUSI.AI android app is downloaded through simple links and the data is parsed through them and is used in the app depending upon what needs to be done with that data at that time.

A simple example for this that was used in the app was  :

private val imageLink = “https://raw.githubusercontent.com/fossasia/susi_skill_data/master/models/general/”

Above is the link that is used to download and display the images of the skills in the app. All the api calls that generally take place in SUSI are through the SUSI server, and making the call to display the images for the skills which takes place through the github links should be replaced by making the calls to the SUSI server instead, as this is a terrible programming style, with this style the project cannot be cloned from other developers and it cannot be moved to other repositories.

We see that there was a huge programming style issue present in the android app and hence, it was fixed by adding the API that calls the SUSI server for external source images and removing the existing implementation that downloads the image from github directly.

Below is an example of the link to the API call made in the app that was needed for the request to be made to the SUSI server :

${BaseUrl.SUSI_DEFAULT_BASE_URL}/cms/getImage.png?model=${skillData.model}&language=${skillData.language}&group=${skillData.group}&image=${skillData.image}

The link is displayed in the kotlin string interpolation manner, here is what the actual URL would look like :

https://api.susi.ai/cms/getImage.pngmodel=${skillData.model}&language=${skillData.language}&group=${skillData.group}&image=${skillData.image}

Here the values with ‘$’ symbol are the parameters for the API taken from the SkillData.kt file and are put inside the link so that the image needed can be extracted.

Now, since we use this link to set the images, to avoid the duplicate code, an Object class was made for this purpose. The object class contained two functions, one for setting the image and one for parsing the skilldata object and forming a URL out of it. Here is the code for the object class :

object Utils {

  fun setSkillsImage(skillData: SkillData, imageView: ImageView) {
      Picasso.with(imageView.context)
              .load(getImageLink(skillData))
              .error(R.drawable.ic_susi)
              .fit()
              .centerCrop()
              .into(imageView)
  }

  fun getImageLink(skillData: SkillData): String {
      val link = “${BaseUrl.SUSI_DEFAULT_BASE_URL}/cms/getImage.png?model=${skillData.model}&language=${skillData.language}&group=${skillData.group}&image=${skillData.image}”
              .replace(” “,“%20”)
      Timber.d(“SUSI URI” + link)
      return link
  }
}

setSkillsImage() method sets the image in the ImageView and the getImageLink() method returns the image formed out of the SkillData object.

References

 

Continue ReadingCorrect the API for Downloading GitHub Content in SUSI.AI android app

Building SUSI.AI Android App with FDroid

Fdroid is an app store for Free and Open Source Software (FOSS). Building and hosting an app on Fdroid is not an easy process compared to when we host one on Google Play. A certain set of build checks are required to be done prior to making a merge request (which is similar to a pull request in GitHub) in the fdroid-data GitLab repository. SUSI.AI Android app has undergone through all these checks and tests and is now ready for the merge request to be made.

Setting up the fdroid-server and fdroid-data repositories is a separate thing and is fairly easy. Building the app using the tools provided by fdroid is another thing and is the one that causes the most problems. It will involve quite a few steps to get started. Fdroid requires all the apps need to be built using:

$ fdroid build -v -l ai.susi

This will output a set of logs which tell us what went wrong in the builds. The usual one in a first time app is obviously the build is not taking place at all. The reason is our metadata file needs to be changed to initiate a build.

The metadata file is used for the build process and contains all the information about the app. The metadata file for a.susi package was a .yaml file.

Builds:

 – versionName: 1.0.10

   versionCode: 11

   commit: 1ad2fd0e858b1256617e652c6c8ce1b8372473e6

   subdir: app

   gradle:

     – fdroid

This is the metadata reference file’s build section that is used for the build process using the command that was mentioned above.The versionName a nd versionCode is found in the build.gradle file in the app and commit denotes the commit-id of the latest commit that will be checked out and built, subdir shows the subdirectory of the app, here the subdirectory is the app file.

Next is the interesting stuff, since we are using flavors in the app, we have to mention in the gradle the flavor which we are using, in our case we are using the flavor by the name of “fdroid” and by mentioning this we can build only the “fdroid” flavor in the app.

Also when building the app there were many blockers that were faced, the reason for the usual build fails were :

1 actionable task: 1 executed
INFO: Scanning source for common problems…
ERROR: Found usual suspect ‘youtube.*android.*player.*api’ at app/libs/YouTubeAndroidPlayerApi.jar
WARNING: Found JAR file at app/libs/YouTubeAndroidPlayerApi.jar
WARNING: Found possible binary at app/src/main/assets/snowboy/alexa_02092017.umdl
WARNING: Found possible binary at app/src/main/assets/snowboy/common.res
ERROR: Found shared library at app/src/main/jniLibs/arm64-v8a/libsnowboy-detect-android.so
ERROR: Found shared library at app/src/main/jniLibs/armeabi-v7a/libsnowboy-detect-android.so
INFO: Removing gradle-wrapper.jar at gradle/wrapper/gradle-wrapper.jar
ERROR: Could not build app ai.susi: Can‘t build due to 3 errors while scanning
INFO: Finished
INFO: 1 build failed

The reason for these build fails were that fdroid does not allow us to use prebuilt files and any proprietary software if present, the above log indicates the two prebuilt files which should be removed and also the YouTubeAndroidPlayerApi.jar which is proprietary software and hence needs to removed. So, to remove the files that are not used in the fdroid flavor and exclude them in the build process, we have to include the following statements in the build section of the metadata reference file :

   rm:
     – app/src/main/jniLibs/arm64-v8a/libsnowboy-detect-android.so
     – app/src/main/jniLibs/armeabi-v7a/libsnowboy-detect-android.so
     – app/libs/YouTubeAndroidPlayerApi.jar

Once the metadata file is complete we are ready to run the build command once again. If you have properly set the environment in your local PC, build will end successfully assuming there were no Java or any other language syntax errors.

It is worth to mention few other facts which are common to Android software projects. Usually the source code is packed in a folder named “app” inside the repository and this is the common scenario if Android Studio builds up the project from scratch. If this “app” folder is one level below the root, that is “android/app”, the build instructions shown above will throw an error as it cannot find the project files.

The reason for this is as it is mentioned “subdir=app” in the metadata file. Change this to “subdir=android/app” and run the build again. The idea is to direct the build to find where the project files are.

Reference:

  1. Metadata : https://f-droid.org/docs/Build_Metadata_Reference/#Build
  2. Publish an app on fdroid: https://blog.fossasia.org/publish-an-open-source-app-on-fdroid/
Continue ReadingBuilding SUSI.AI Android App with FDroid

Handle response for API calls to the smart speaker

The SUSI.AI android app has the feature to connect to the smart speaker through the WiFi access point of the smart speaker. This connection takes place without internet and is completely offline and takes place locally between the device and the app.

When the APIs are hit, the parameters are sent to the server and are extracted by the speaker and then stored to perform further actions, but like any other API there is some response sent by the server once the parameters are fetched and only when this response is correctly parsed we show the confirmation that the data was required to be sent to the speaker is successfully done.

The SUSI.AI android app was unable to parse this response and so , every time there was a request made to the speaker the error message was displayed on the screen, just like this :

This is a cropped screenshot of the error message shown.

To handle the response from the local server there was a need to add the response parsing classes in the app, also since the app is written in kotlin the specialised classes for this purpose are already present in kotlin which are specifically for the purpose of handling responses. The data classes are added for the three APIs for the local server that are made, the response class added for the WiFi credentials response is :

data class SpeakerWifiResponse(
      val wifi: String,
      val wifi_ssid: String,
      val wifi_password: String
)

Here the first string by the name of ‘wifi’ is used to display a success message and other two variables are for the ssid and password of the wifi network respectively.

The API that is used to set the configuration of the speaker has the response class as follows  :

data class SpeakerConfigResponse(
      val configuration: String,
      val hotword: String,
      val stt: String,
      val tts: String,
      val wake: String
)

The third API which is used to send the auth credentials to the speaker has the following response class :

data class SpeakerAuthResponse(
      val auth: String,
      val authentication: String,
      val email: String,
      val password: String? = null
)

Now these response classes have the variable name similar to the key values in the JSON response generated by the local server. Due to the similar names to the keys an extra step and extra lines of code are eliminated which had to be added otherwise denoting the “Serializable” values to match the JSON response values.

After these data classes were added the Call<> functions of the retrofit service DeviceApi.java were made of the type of the types of the above data classes , which then were able to successfully parse the response from the local server of the speaker.

So, after this the app to speaker connection was successful and a success message was shown as :

References

 

Continue ReadingHandle response for API calls to the smart speaker

Show Option to choose WiFi for Smart Speaker Connection in SUSI.AI Android APP

SUSI.AI android app has the functionality to detect the available WiFi networks and among them check for the hotspot named “SUSI.AI”. Now, this process is required so that the app can connect to the smart speaker and send the WiFi credentials, the authentication credentials and the configuration data to the smart speaker.

After on clicking the “SET UP” button on the available SUSI.AI hotspot as shown in the image below,

The app needs to make API requests where the app will send the data to the speaker, the first API that needs to be hit is for the WiFi credentials. Once the “SET UP” button is clicked the app shows a message “Connecting to your device” with a loader as in the image below :

Now during this step the code to detect the available WiFi networks is run again and the list of the available networks is sent from the DeviceConnectPresenter.kt to the DeviceConnectFragment.kt from the function defined in the presenter as follows :

override fun availableWifi(list: List<ScanResult>) {
  connections = ArrayList<String>()
  for (i in list.indices) {
      connections.add(list[i].SSID)
  }
  if (!list.isEmpty()) {
      deviceConnectView?.setupWiFiAdapter(connections)
  } else {
      deviceConnectView?.onDeviceConnectionError(utilModel.getString(R.string.no_device_found), utilModel.getString(R.string.setup_tut))
  }
}

Now to show the list of available WiFi networks a new ViewHolder had to made which contained a textview and an imageview. The viewholder file named WifiViewHolder.java is responsible for this and is used along with the DeviceAdapter only.

The interesting thing to see is that the DeviceAdapter already inflates the ViewItems of the type DeviceViewHolder. Instead of making a new adapter for the WifiViewHolder view item, I wrote some smart code that handles both the viewholders with the same adapter i.e DeviceAdapter. Let’s now see how this was handled,

In the DeviceAdapter a private variable named viewCode of type integer was made and it would be responsible to segregate the two viewholders using an if command. Here is code below that shows how using viewcode variable allows us to choose and inflate a viewholder among the two we have in onCreateViewHolder() method :

if (viewCode == 1) {
  View v = LayoutInflater.from(parent.getContext()).inflate(R.layout.device_layout, parent, false);
  return new DeviceViewHolder(v, (DeviceConnectPresenter) devicePresenter);
} else {
  View v = LayoutInflater.from(parent.getContext()).inflate(R.layout.layout_wifi_item, parent, false);
  return new WifiViewHolder(v, (DeviceConnectPresenter) devicePresenter);
}

Now, when the viewholder of type WifiViewHolder is inflated the app displays a list of available wifi networks and on clicking any of the Wifi items the app asks the user to enter the credentials of the WiFi network. Below is how the app looks when the available wifi networks are loaded and displayed and also when we click on any of the item in the list.

References :

  1. Android manipulating Wifi networks using WiFiManager : https://medium.com/@josiassena/android-manipulating-wifi-using-the-wifimanager-9af77cb04c6a
  2. Kotlin broadcast intents and receivers : https://www.techotopia.com/index.php/Kotlin_Android_Broadcast_Intents_and_Broadcast_Receivers
  3. Android Viewholder pattern : https://www.javacodegeeks.com/2013/09/android-viewholder-pattern-example.html
Continue ReadingShow Option to choose WiFi for Smart Speaker Connection in SUSI.AI Android APP

Scan and detect available networks in SUSI.AI Android app

We all have experienced Apple HomPad, Google Home, Alexa etc or read about smart speakers which offer interactive actions over voice commands. Smart speaker uses hotword for activation. They utilize WiFi, bluetooth and other wireless protocols.

SUSI.AI is also coming with an Open Source smart speaker using  as simple as a RaspberryPi for the speaker which can perform various actions like playing music etc over voice commands. To use SUSI Smart Speaker, you have to connect it to the SUSI iOS or Android App. You can manage your connected devices in SUSI Android,SUSI iOS and Web clients. Here we will see initial setups for connecting SUSI Smart Speaker with an Android device.

Unlike iOS, Android allows any app to change the WiFi settings of the phone if the specific permissions are added in the app and on the run time while using the app. To connect to a particular WiFi in android given that permissions are enabled a few lines of code is sufficient enough to connect to the new WiFi network, but in our case before connecting to the WiFi we must know that the SUSI.AI hotspot is available around or not.

To know if the SUSI hotspot is available or not we have to scan for the available WiFi networks present and then choose the SUSI WiFi hotspot.

The app simply detects the available WiFi networks and displays a set of them that have the name SUSI.AI. To detect the available WiFi networks we have used a BroadCastReceiver that listens for the action :

WifiManager.SCAN_RESULTS_AVAILABLE_ACTION

The broadcast receiver listens to this action and fetches the list of networks available. The broadcast receiver is registered after the objects of WifiReceiver class and WifiManager class are declared. Once the broadcast receiver is registered the object of WifiManager is used to start the scanning process of the available Wifi networks.

Here is the object of the WifiManager and WifiReceiver declared in the onCreate() method of the DeviceActivity.kt file :

mainWifi = application.getSystemService(Context.WIFI_SERVICE) as WifiManager
receiverWifi = WifiReceiver()

The broadcast receiver class that scans the available networks is as follows :

inner class WifiReceiver : BroadcastReceiver() {
  override fun onReceive(p0: Context?, p1: Intent?) {
      Timber.d(“Inside the app”)
      if (p1 != null) {
          if (p1.action.equals(WifiManager.SCAN_RESULTS_AVAILABLE_ACTION)) {
              var wifiList: List<ScanResult> = ArrayList<ScanResult>()
              wifiList = mainWifi.getScanResults()
              devicePresenter.inflateList(wifiList)
          }

In the above code all the available networks on scanning are sent to the presenter for further processing.

In the presenter class in the method inflateList() the available WiFi networks list is scanned on the basis of their SSID and all the networks with name are “SUSI.AI” are selected and then displayed as a list to the user.

The inflateList() function is as follows :

override fun inflateList(list: List<ScanResult>) {
  Timber.d(“size “ + list.size)
  connections = ArrayList<String>()
  for (i in list.indices) {
      if (list[i].SSID.equals(utilModel.getString(R.string.device_name)))
          connections.add(list[i].SSID)
  }

  if (connections.size > 0) {
      deviceView?.setupAdapter(connections)
  } else {
      deviceView?.onDeviceConnectionError(utilModel.getString(R.string.no_device_found), utilModel.getString(R.string.setup_tut))
  }
}

Now after the wifi list is scanned the sorted and selected list is sent to the Recycler adapter that will display the list to available SUSI networks to the user.

CONCLUSION :

REFERENCES :

  1. Android Manipulation of wifi using WifiManager : https://medium.com/@josiassena/android-manipulating-wifi-using-the-wifimanager-9af77cb04c6a
  2. Scan and list all Wifi in Android : https://www.nplix.com/scan-list-wifi-network-android/
  3. Kotlin android broadcast receivers : https://www.techotopia.com/index.php/Kotlin_Android_Broadcast_Intents_and_Broadcast_Receivers

 

Continue ReadingScan and detect available networks in SUSI.AI Android app

Display image responses in SUSI.AI Android app

SUSI.AI Android app has many response functionalities ranging from giving simple ANSWER type responses to complex TABLE and MAP type responses. Although, even after all these useful response types there were some missing action types all related to media. SUSI.AI app was not capable of playing any kind of image responses.So, to do this the links received in the response were used to fetch the corresponding image stored in the link.

Since, the app now has two build flavors corresponding to the F-Droid version and PlayStore version respectively it had to be considered that while adding the feature to display images any proprietary software was not included with the F-Droid version.

The JSON response from the server whenever a query for was asked gave the link to the image.  For eg : on querying : “Image of cat “ the server gave the response as :

Actions”: [

       {

         “language”: “en”,

         “type”: “answer”,

         “expression”: “https://pixabay.com/get/e830b1072ef5023ed1584d05fb1d4790e076e7d610ac104496f0c77ea0e9bcbf_640.jpg”

       }

So in the android app just like the usual answer type response a message was displayed with the link to the image present in the message.

Catching IMAGE response :

The image responses are of the type “answer” as seen in the server response above. So we a method had to be devised so that the responses which display the images are caught and displayed. In the file ChatFeedRecyclerAdapter.java a separate code to detect the IMAGE response was added as :

private static final int IMAGE = 17;

Next we must specify that what type of view that is to be used whenever an IMAGE response is encountered by the app. Since the action type is “answer” a specification was required to choose the type of the viewholder. Since the images are only displayed through the pixabay the URL of the images end with either “.jpg” or “.png”. So in the expression of the response if we check that it is a link and also it ends with either “.jpg” or “.png” it will be certain that the response given from the server is an image.

The code to identify the view type :

@Override
public int getItemViewType(int position) {
  ChatMessage item = getItem(position);

  if (item.getId() == -404) return DOTS;
  else if (item.getId() == -405) return NULL_HOLDER;
  else if (item.isDate()) return DATE_VIEW;
  else if (item.getContent().endsWith(“.jpg”) || item.getContent().endsWith(“.png”))
      return IMAGE;

Inflating the layout of type IMAGE

Now after determining that the response will be an image we have to inflate the layout of the viewholder to support images in the onCreateViewHolder() method . The layout  of the image response was inflated as follows :

case IMAGE:
  view = inflater.inflate(R.layout.image_holder, viewGroup, false);
  return new ImageViewHolder(view, clickListener);

Here ImageViewHolder is the view holder that is used for displaying the images , we will discuss it later in the post. Also now in the onBindViewHolder() method of the ChatFeedRecyclerAdapter.java file we have to specify the instance of the view holder if it was to support the image response. It was done as follows :

else if (holder instanceof ImageViewHolder) {
  ImageViewHolder imageViewHolder = (ImageViewHolder) holder;
  imageViewHolder.setView(getData().get(position));
}

ViewHolder for Images :

Like all other viewholders for different variety responses from the server a viewholder to display the images was also needed to be included in the app. So, ImageViewHolder.java was created which handles the images to de displayed in the app. In the setView() method ChatMessage object is passed and this object is  used to get the image url so that it is used to display the image using Picasso.

public void setView(ChatMessage model) {
  this.model = model;

  if (model != null) {
      imageURL = model.getContent();
      try {
          Picasso.with(itemView.getContext())
                  .load(imageURL)
                  .placeholder(R.drawable.ic_susi)
                  .into(imageView);
      } catch (Exception e) {
          Timber.e(e);
      }
  }

And on clicking the image we can view the image on full screen using a custom chrome tab as follows  :

imageView.setOnClickListener(new View.OnClickListener() {
  @Override
  public void onClick(View view) {
      CustomTabsIntent.Builder builder = new CustomTabsIntent.Builder();
      CustomTabsIntent customTabsIntent = builder.build();
      customTabsIntent.launchUrl(itemView.getContext(), Uri.parse(imageURL));
  }
});

Conclusion :

References :

  1. Susi server response in case of images : https://api.susi.ai/susi/chat.json?timezoneOffset=-330&q=image+of+cat
  2. Pixabay is the source used for fetching images : https://pixabay.com
  3. Display images using Picasso : https://guides.codepath.com/android/Displaying-Images-with-the-Picasso-Library

 

Continue ReadingDisplay image responses in SUSI.AI Android app

Play Youtube Videos in SUSI.AI Android app

SUSI.AI Android app has many response functionalities ranging from giving simple ANSWER type responses to complex TABLE and MAP type responses. Although, even after all these useful response types there were some missing action types all related to media. SUSI.AI app was not capable of playing any kind of music or video.So, to do this  the largest source of videos in the world was thought and the functionality to play youtube videos in the app was added.

Since, the app now has two build flavors corresponding to the FDroid version and PlayStore version respectively it had to be considered that while adding the feature to play youtube videos any proprietary software was not included with the FDroid version. Google provides a youtube API that can be used to play videos inside the app only instead of passing an intent and playing the videos in the youtube app.

Steps to integrate youtube api in SUSI.AI

The first step was to create an environment variable that stores the API key, so that each developer that wants to test this functionality can just create an API key from the google API console and put it in the build.gradle file in the line below

def YOUTUBE_API_KEY = System.getenv(‘YOUTUBE_API_KEY’) ?: ‘”YOUR_API_KEY”‘    

In the build.gradle file the buildConfigfield parameter names API_KEY was created so that it can used whenever we need the API_KEY in the code. The buildConfigfield was declared for both the release and debug build types as :

release {
  minifyEnabled false
 proguardFiles getDefaultProguardFile(‘proguard-android.txt’), ‘proguard-rules.pro’
 buildConfigField “String”, ‘YOUTUBE_API_KEY’, YOUTUBE_API_KEY
}

debug {
  buildConfigField “String”, ‘YOUTUBE_API_KEY’, YOUTUBE_API_KEY
}

The second step is to catch the audio and video action type in the ParseSusiResponseHelper.kt file which was done by adding two constants “video_play” and “audio_play” in the Constant class. These actions were easily caught in the app as :

Constant.VIDEOPLAY -> try {
  identifier = susiResponse.answers[0].actions[i].identifier
} catch (e: Exception) {
  Timber.e(e)
}

Constant.AUDIOPLAY -> try {
  identifier = susiResponse.answers[0].actions[i].identifier
} catch (e: Exception) {
  Timber.e(e)
}

The third step involves making a ViewHolder class for the audio and video play actions. A simple layout was made for this viewholder which can display the thumbnail of the video and has a play button on the thumbnail, on clicking which plays the youtube video. Also, in the ChatFeedRecyclerAdapter we need to specify the action as Audio play and video play and then load the specific viewholder for the youtube videos whenever the response from the server for this particular action is fetched. The YoutubeViewHolder.java file describes the class that displays the thumbnail of the youtube video whenever a response arrives.

public void setPlayerView(ChatMessage model) {
  this.model = model;

  if (model != null) {
      try {
          videoId = model.getIdentifier();
          String img_url = “http://img.youtube.com/vi/” + videoId + “/0.jpg”;

          Picasso.with(itemView.getContext()).load(img_url).
                  placeholder(R.drawable.ic_play_circle_filled_white_24dp)
                  .into(playerView);
      } catch (Exception e) {
          Timber.e(e);
      }

  }

The above method shows how the thumbnail is set for a particular youtube video.

The fourth step is to pass the response from the server in the ChatPresenter to play the video asked by the user. This is  achieved by calling a function declared in the IChatView so that the youtube video will be played after fetching the response from the server :

if (psh.actionType == Constant.VIDEOPLAY || psh.actionType == Constant.AUDIOPLAY) {
  // Play youtube video
  identifier = psh.identifier
  chatView?.playVideo(identifier)
}

The fifth step is a bit complicated, as we know the app contains two flavors, one for the FDroid version that contains all the Open Source libraries and software and the second is the PlayStore version which may keep the proprietary software. Since, the youtube api is nothing but a piece of proprietary software we cannot include its implementation in the FDroid version and so different methods were devised to play the youtube videos in both versions. Since,  we use flavors and only want that Youtube API is compiled and included in the playstore version the youtube dependency was updated as :

playStoreImplementation files(‘libs/YouTubeAndroidPlayerApi.jar’)

This ensures that the library is only included in the playStore version. But on this step another problem is encountered, as now the code in both the versions will be different an encapsulation method was devised which enabled the app to keep separate code in both flavors.

To keep different code for both variants, in the src folder directories named fdroid and playStore were created and package name was added and then finally a java folder was created in each directory in which the separate code was kept for each flavor. An interface in the main directory was created which was used as a means to provide encapsulation so that it was implemented differently in each of the flavors to provide different functionality. The interface was created as :

interface IYoutubeVid {
  fun playYoutubeVid(videoId: String)
}

An object of the interface was made initialised in the ChatActivity and separate classes

named YoutubeVid.kt was made in each flavor which implemented the interface IYoutubeVid. So, now what happened was that depending on the build variant the YoutubeVid class of the particular flavor was called and the video would play according the sources that are available in that flavor.

In ChatActivity the following implementation was followed :

Declaration :

lateinit var youtubeVid: IYoutubeVid

Initialisation :

youtubeVid = YoutubeVid(this)

Call to the overridden function :

override fun playVideo(videoId: String) {
  Timber.d(videoId)
  youtubeVid.playYoutubeVid(videoId)
}

Final Output

References

  1. Youtube Android player API : https://developers.google.com/youtube/android/player/
  2. Building apps with product flavors – Tomoaki Imai https://medium.com/@tomoima525/building-multiple-regions-apps-with-product-flavors-good-and-bad-fe33e3c31060
  3. Kotlin style guide : https://android.github.io/kotlin-guides/style.html

 

Continue ReadingPlay Youtube Videos in SUSI.AI Android app

Setting up the Circle.CI config for SUSI Android

SUSI.AI Android app uses CircleCI to check for tests whenever a new PR is made. This is done to ensure that the app is consistent with the new changes and there is no problem adding that particular code change. CircleCI has now launched a v2 version of the .yml file and therefore to continue using CircleCI it is time to upgrade to v2 version of the script.

Circle.CI config version 1

Config file tells the CI on what commands to run to test the build, which will determine if the build is successful or failed like tests, lints etc and hooks commands to run if the tests pass, which environment to run the code in – python, java, android, etc.

Circle.CI published that 31, August 2018 is the last date upto when they will support the config version 1.0, therefore it Circle.CI was updated to version 2.0.

The version 2.0 has an updated syntax of the script and previous script was modified so that it could provide the configuration for the CI builds.

The updated script is shown below :

version: 2
jobs:
build:
  working_directory: ~/code
  docker:
    – image: circleci/android:api-27-alpha
  environment:
    JVM_OPTS: -Xmx3200m
  steps:
    – checkout
    – restore_cache:
        key: jars-{{ checksum “build.gradle” }}-{{ checksum  “app/build.gradle” }}
    – run:
        name: Download Dependencies
        command: ./gradlew androidDependencies
    – save_cache:
        paths:
          – ~/.gradle
        key: jars-{{ checksum “build.gradle” }}-{{ checksum  “app/build.gradle” }}
    – run:
        name: lint check
        command: ./gradlew lint
    – run:
        name: Run Tests
        command: ./gradlew build
    – run:
        command:
          bash exec/prep-key.sh
          bash exec/apk-gen.sh
    – store_artifacts:
        path: app/build/reports
        destination: reports
    – store_artifacts:
        path: app/build/outputs
        destination: outputs
    – store_test_results:
        path: app/build/test-results

Few checks such as the connectedCheck, which were required for the UI testing, are not included in this script and instead an approach towards the increasing number of unit tests is followed. The reason being, implementing UI tests is a hard task for apps which have no constant designs and have constantly changing specifications. Since the architecture used is MVP, so on moving all logic to presenter components there won’t be a need of most UI tests at all. This problem with UI tests will further increase in future. Therefore, moving in the direction of increasing unit tests is better because they are both easy, small and quick to code and run and the degree of dependence on flaky UI tests also decreases.

If the view is implementing small sections of display logic, just verifying that the presenter calls those is enough to guarantee that the UI is going to work.

The command in the previous config such as

./gradlew connectedCheck

was removed and instead ./gradlew build was added to start the unit tests instead, also due to the updated gradle dependencies, changes were made to the apk uploading commands and also changes were performed in the apk-gen.sh file.

bash exec/prep-key.sh
chmod +x exec/apk-gen.sh
./exec/apk-gen.sh

In the above code, lines concerned with apk-gen.sh can be combined and the resultant command was written as :

bash exec/apk-gen.sh

The apk-gen.sh script was configured and the latest build tools were added to make the script work.

REFERENCES

Continuous Integration – Martin Fowler:

https://martinfowler.com/articles/continuousIntegration.html

Circle CI version 2.0 for android :

https://circleci.com/docs/2.0/configuration-reference/#run

Circle CI configuration reference :

https://circleci.com/docs/2.0/configuration-reference/

 

Continue ReadingSetting up the Circle.CI config for SUSI Android

Adding product flavors to SUSI Android app

Product flavors are required by production level apps to provide different versions of their app available to people. Each of the flavor has separate functionality depending on the type of flavors chosen. Most common flavors are full and demo flavors, from their name only we can understand that these flavors contain full version and demo version of their apps. Since SUSI.AI android app has to be released on FDroid several libraries that were present in the current app were not  acceptable by them as it only allows apps that completely use only open source software to develop and therefore there was a need to make the app compatible with both the app stores (Play Store and FDroid) product flavors were added in the app.

Proprietary software in the app

There are several libraries which are not acceptable on FDroid but are present in the app, a couple of them being :

  1. Crashlytics using fabric
  2. LinkPreview library

These two libraries were found which are not acceptable in FDroid. Although the link preview library was not proprietary software, but fdroid has guidelines which only allows that recognised and trusted libraries from maven repository is accepted.

Further, youtube api recently integrated in the app is also not acceptable by FDroid as it is under google and categorised as a proprietary software.

Adding product flavors

The flavor names chosen to be made were “fdroid” and “playStore”, fdroid here specifying the build variant for the app that would contain the libraries and code acceptable by fdroid and playStore flavor to contain the sources and libraries for the regular version of the app which can contain proprietary software.

The flavors were added as shown below in the module level build.gradle file :

flavorDimensions “default”

productFlavors {
  fdroid {
      dimension “default”
      applicationIdSuffix “.fdroid”
      versionNameSuffix “-FDroid”
  }

  playStore {
      dimension “default”
      versionNameSuffix “-PlayStore”
  }
}

the flavors are added under the block of code by the name productFlavors, and each flavor needs to have some properties to be set. The applicationId of the app can be changed or can be kept same, in our case for the fdroid flavor the applicationIdSuffix was added which adds “.fdroid” at the end of the applicationId for the fdroid version.

versionNameSuffix property was set in both the flavors which adds “-FDroid” to end of the versionname for the fdroid variant and “-PlayStore” to the end of the versionname for the playstore variant.

Using the product flavors we can generate multiple apks , as in our case after the build flavors were added there were four apks built. These can be seen as  :

Choosing any of these apks we can select the variant that is to be worked upon. The debug apks here are those used for developing purposes.

Conclusion

Adding flavors allows SUSI.AI android app to keep separate code and resources for different variants. Any code that is different to both the variants is added separately after creating the directories named “fdroid” and “playStore” in the src folder.

This was added after switching to the Project view in Android Studio and then moving to the app folder and then inside the src folder make a directories named fdroid and playStore.

References

  1. Configure Build Variants: https://developer.android.com/studio/build/
  2. Android Product Flavors: https://medium.com/@iammert/android-product-flavors-1ef276b2bbc1

Handling multiple java sources and flavors using flavors on Gradle: https://medium.com/@thiagolopessilva/the-handling-multiple-java-source-and-resources-using-flavors-on-gradle-18a4b581285b

Continue ReadingAdding product flavors to SUSI Android app

Added “table” type action support in SUSI android app

SUSI.AI has many actions supported by it, for eg: answer, anchor, map, piechart, websearch and rss.These actions are a few of those that can be supported in the SUSI.AI android app, but there are many actions implemented on the server side and the web client even has the implementation of how to handle the “table” type response.

The table response is generally a JSON array response with different json objects, where each json object have similar keys, and the actions key in the JSON response has the columns of the table response which are nothing but the keys in the data object of the response.

To implement the table type response in the susi android app a separate file needed to made to parse the table type response, since the keys and values both are required to the display the response. The file ParseTableSusiResponseHelper.kt was made which parsed the JSON object using the Gson converter factory to get the key value of the actions :

“actions”: [

       {

         “columns”: {

           “ingredients”: “Ingredients”,

           “href”: “Instructions Link”,

           “title”: “Recipe”

         },

         “count”: -1,

         “type”: “table”

       }

     ]

 

The inside the columns the keys and the values, both were extracted, values were to displayed in the title of the column and keys used were to extract the values from the “data” object of the response.

The files TableColumn.java, TableData.java are POJO classes that were used for storing the table columns and the data respectively. The TableDatas.java class was used to store the column list and the data list for the table response.

To fetch the table type response from the server a TableSusiResponse.kt file was added that contained serializable entities which were used to map the response values fetched from the server. A variable that contained the data stored in the “answers” key of the response was made of type of an ArrayList of TableAnswers.

@SerializedName(“answers”)
@Expose
val answers: List<TableAnswer> = ArrayList()

The TableAnswer.kt is another file added that contains serializable variables to store values inside the keys of the “answers” object. The actions object shown above is inside the answers object and it was stored in the form of an ArrayList of TableAction.

@SerializedName(“actions”)
@Expose
val actions: List<TableAction> = ArrayList()

Similar to TableAnswer.kt file TableAction.kt file also contains serializable variables that map the values stored in the “actions” object.

In the retrofit service interface SusiService.java a new call was added to fetch the data from the server as follows :

@GET(“/susi/chat.json”)
Call<TableSusiResponse> getTableSusiResponse(@Query(“timezoneOffset”) int timezoneOffset,
                                           @Query(“longitude”) double longitude,
                                           @Query(“latitude”) double latitude,
                                           @Query(“geosource”) String geosource,
                                           @Query(“language”) String language,
                                           @Query(“q”) String query);

Now, after the data was fetched, the table response can be parsed using the Gson converter factory in the ParseTableSusiResponseHelper.kt file. Below is the implementation :

fun parseSusiResponse(response: Response<TableSusiResponse>) {
  try {
      var response1 = Gson().toJson(response)
      var tableresponse = Gson().fromJson(response1, TableBody::class.java)
      for (tableanswer in tableresponse.body.answers) {
          for (answer in tableanswer.actions) {
              var map = answer.columns
              val set = map?.entries
              val iterator = set?.iterator()
              while (iterator?.hasNext().toString().toBoolean()) {
                  val entry = iterator?.next()
                  listColumn.add(entry?.key.toString())
                  listColVal.add(entry?.value.toString())
              }
          }
          val map2 = tableanswer.data
          val iterator2 = map2?.iterator()
          while (iterator2?.hasNext().toString().toBoolean()) {
              val entry2 = iterator2?.next()
              count++;
              for (count in 0..listColumn.size – 1) {
                  val obj = listColumn.get(count)
                  listTableData.add(entry2?.get(obj).toString())
              }
          }
          tableData = TableDatas(listColVal, listTableData)
      }
  } catch (e: Exception) {
      tableData = null
  }
}

 

Now the data is also parsed, we pass the two lists the ColumnList and DataList to the variable of TableDatas.

Three viewholder classes were added to display the table response properly in the app and corresponding to these viewholders a couple of adapters were also made that are responsible for setting the values in the recyclerview present in the views. The first viewholder is the TableViewHolder, it contains the horizontal recyclerview that is used to display the items fetched from the “data” object of the response. The recyclerview in the TableViewHolder has each entity of the type TabViewHolder, this is a simple cardview but also contains another recyclerview inside it which is used to store the keys and values of each of the object inside the “data” object.

TableViewHolder.java file has a setView() method that uses the the object of ChatMessage to get the list of columns and data to be set in the view.

 

Changes were made in the ChatPresenter.kt file to catch the tableresponse when a table type action is detected. Below is the implementation :

if (response.body().answers[0].actions[i].type.equals(“table”)) {
  tableResponse(query)
  return
}

The tableResponse function is as follows :

fun tableResponse(query: String) {
  val tz = TimeZone.getDefault()
  val now = Date()
  val timezoneOffset = -1 * (tz.getOffset(now.time) / 60000)
  val language = if (PrefManager.getString(Constant.LANGUAGE, Constant.DEFAULT).equals(Constant.DEFAULT)) Locale.getDefault().language else PrefManager.getString(Constant.LANGUAGE, Constant.DEFAULT)
  chatModel.getTableSusiMessage(timezoneOffset, longitude, latitude, source, language, query, this)
}

 

It calls the chatModel to get the list of columns and data to be set. The ChatFeedRecyclerAdapter.java files checks for the table response code, and if it matches then the view used for displaying SUSI’s message is the TableViewHolder. Here is how this viewholder is inflated :

case TABLE:
  view = inflater.inflate(R.layout.susi_table, viewGroup, false);
  return new TableViewHolder(view, clickListener);

Below is the final result when the table response is fetched for the query “Bayern munich team players” is :

References :

  1. SUSI server response for table query : https://api.susi.ai/susi/chat.json?timezoneOffset=-330&q=barcelona+team+players
  2. GSON for converting java objects to JSON and JSON to java : http://www.vogella.com/tutorials/JavaLibrary-Gson/article.html
Continue ReadingAdded “table” type action support in SUSI android app