Mapbox implementation in Open Event Organizer Android App

Open Event Organizer Android App is used by event organizers to manage events on the Eventyay platform. While creating or updating an event, location is one of the important factors which needs to be added so that the attendees can be informed of the venue.

Here, we’ll go through the process of implementing Mapbox Places Autocomplete for event location in the F-Droid build variant.

The first step is to create an environment variable for the Mapbox Access Token. 

def MAPBOX_ACCESS_TOKEN = System.getenv('MAPBOX_ACCESS_TOKEN') ?: "YOUR_ACCESS_TOKEN"

Add the Mapbox dependency:

fdroidImplementation 'com.mapbox.mapboxsdk:mapbox-android-plugin-places-v8:0.9.0'

Fetching the access token in EventDetailsStepOne as well as UpdateEventFragment:

ApplicationInfo applicationInfo = null;
        try {
            applicationInfo = getContext().getPackageManager().getApplicationInfo(getContext().getPackageName(), PackageManager.GET_META_DATA);
        } catch (PackageManager.NameNotFoundException e) {
            Timber.e(e);
        }
        Bundle bundle = applicationInfo.metaData;

        String mapboxAccessToken = bundle.getString(getString(R.string.mapbox_access_token));

The app should not crash if the access token is not available. To ensure this, we need to put a check. Since, the default value of the access token is set to “YOUR_ACCESS_TOKEN”, the following code will check whether a token is available or not:

if (mapboxAccessToken.equals("YOUR_ACCESS_TOKEN")) {
    ViewUtils.showSnackbar(binding.getRoot(),                             R.string.access_token_required);
    return;
}

Initializing the PlacesAutocompleteFragment:

PlaceAutocompleteFragment autocompleteFragment = PlaceAutocompleteFragment.newInstance(
                mapboxAccessToken, PlaceOptions.builder().backgroundColor(Color.WHITE).build());

getFragmentManager().beginTransaction()
    .replace(R.id.fragment, autocompleteFragment)
    .addToBackStack(null)
    .commit();

Now, a listener needs to be set up to get the selected place and set the various fields like latitude, longitude, location name and searchable location name.

autocompleteFragment.setOnPlaceSelectedListener(new PlaceSelectionListener() {
                @Override
                public void onPlaceSelected(CarmenFeature carmenFeature) {
                    Event event = binding.getEvent();
                    event.setLatitude(carmenFeature.center().latitude());
                    event.setLongitude(carmenFeature.center().longitude());
                    event.setLocationName(carmenFeature.placeName());
                    event.setSearchableLocationName(carmenFeature.text());
                    binding.form.layoutLocationName.setVisibility(View.VISIBLE);
                    binding.form.locationName.setText(event.getLocationName());
                    getFragmentManager().popBackStack();
                }

                @Override
                public void onCancel() {
                    getFragmentManager().popBackStack();
                }
            });

This brings the process of implementing Mapbox SDK to completion.

GIF showing the working of Mapbox Places Autocomplete

Resources:

Documentation: Mapbox Places Plugin

Open Event Organizer App: Project repo, Play Store, F-Droid

Continue ReadingMapbox implementation in Open Event Organizer Android App

Configure location feature using MVVM in Open Event Attendee Application

The open event attendee is an android app which allows users to discover events happening around the world using the Open Event Platform. It consumes the APIs of the open event server to get a list of available events and can get detailed information about them. It deals with events based on location, but we have to take the location as an input from the user. While in many cases, we have to search for events on our current location only. To make this work, I have added a current location option, where the app will get our location and search for nearby events. Earlier we had to enter our current location as well to search nearby events.

Model–view–viewmodel is a software architectural pattern. MVVM facilitates separation of development of the graphical user interface – be it via a markup language or GUI code – from the development of the business logic or back-end logic (the data model).

  • Why Model-view-ViewModel?
  • Setup Geo location View Model
  • Configure location feature with MVVM
  • Conclusion
  • Resources

Let’s analyze every step in detail.

Advantages of using Model-view-ViewModel

  1. A clean separation of different kinds of code should make it easier to go into one or several of those more granular and focused parts and make changes without worrying.
  2. External and internal dependencies are in separate pieces of code from the parts with the core logic that you would like to test.
  3. Observation of mutable live data whenever it is changed.

Setup the Geolocation view model

Created new kotlin class name GeoLocationViewModel which contains a configure function for current location:

package org.fossasia.openevent.general.search

class GeoLocationViewModel : ViewModel() {
    fun configure(activity: Activity) { }                                     
}

The GeoLocationViewModel class implement as VIewModel().

Now add the class in module inside Modules.kt :

val viewModelModule = module {
    viewModel { GeoLocationViewModel() }
}

Configure location feature with GeoLocationViewModel:

First, add play store location service implementation inside dependencies of build.gradle:

// Location Play Service
playStoreImplementation 'com.google.android.gms:play-services-location:16.0.0'

Now we need location permissions to implement this feature. Adding user permissions in the manifest file:

<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />

Now ask user for the location permission:

private fun checkLocationPermission() {
        val permission = context?.let {
            ContextCompat.checkSelfPermission(it, Manifest.permission.ACCESS_COARSE_LOCATION) }
        if (permission != PackageManager.PERMISSION_GRANTED) {
            requestPermissions(arrayOf(Manifest.permission.ACCESS_COARSE_LOCATION,
                Manifest.permission.ACCESS_FINE_LOCATION), LOCATION_PERMISSION_REQUEST)
        }
    }

Check for device location is enabled, if not send an intent to turn location on. The method is written inside configure function:

val service = activity.getSystemService(Context.LOCATION_SERVICE)
        var enabled = false
        if (service is LocationManager) enabled = service.isProviderEnabled(LocationManager.NETWORK_PROVIDER)
        if (!enabled) {
            val intent = Intent(Settings.ACTION_LOCATION_SOURCE_SETTINGS)
            activity.startActivity(intent)
            return
        }

Now create Mutable live data for current location inside the view model class :

private val mutableLocation = MutableLiveData<String>()
val location: LiveData<String> = mutableLocation

Now implement location request and location callback inside configure method:

val locationRequest: LocationRequest = LocationRequest.create()
locationRequest.priority = LocationRequest.PRIORITY_LOW_POWER
val locationCallback = object : LocationCallback() {
      override fun onLocationResult(locationResult: LocationResult?) {
           if (locationResult == null) {
               return
            }
            for (location in locationResult.locations) {
                if (location != null) {
                    val latitude = location.latitude
                    val longitude = location.longitude
                    try {
                        val geocoder = Geocoder(activity, Locale.getDefault())
                        val addresses: List<Address> = geocoder.getFromLocation(latitude, longitude, maxResults)
                        for (address: Address in addresses) {
                            if (address.adminArea != null) {
                                mutableLocation.value = address.adminArea
                            }
                        }
                    } catch (exception: IOException) {
                        Timber.e(exception, "Error Fetching Location")
                    }
                }
             }
        }
    }

Now call location service inside configure method:

LocationServices
                .getFusedLocationProviderClient(activity)
                .requestLocationUpdates(locationRequest, locationCallback, null)

Now observe data for current location inside the fragment from the view model. Save the current user location and go to the main activity:

private val geoLocationViewModel by viewModel<GeoLocationViewModel>()
geoLocationViewModel.location.observe(this, Observer { location ->
                searchLocationViewModel.saveSearch(location)
                redirectToMain()
            })

GIF

In a Nutshell

So, essentially the Eventyay Attendee should have this feature to show all the events nearby me or in my city, although the app was already doing the job, but we had to manually select the city or locality we wish to search, now after the addition of dedicated current location option, the app will be more user friendly and automated.

Resources

  1. Google location services API: https://www.toptal.com/android/android-developers-guide-to-google-location-services-api
  2. MVVM architecture in Android: https://proandroiddev.com/mvvm-architecture-viewmodel-and-livedata-part-1-604f50cda1

Tags

Eventyay, open-event, PlayServices, Location, MVVM, Fossasia, GSoC, Android, Kotlin

Continue ReadingConfigure location feature using MVVM in Open Event Attendee Application

Connecting SUSI iOS App to SUSI Smart Speaker

SUSI Smart Speaker is an Open Source speaker with many exciting features. The user needs an Android or iOS device to set up the speaker. You can refer this post for initial connection to SUSI Smart Speaker. In this post, we will see how a user can connect SUSI Smart Speaker to iOS devices (iPhone/iPad).

Implementation –

The first step is to detect whether an iOS device connects to SUSI.AI hotspot or not. For this, we match the currently connected wifi SSID with SUSI.AI hotspot SSID. If it matches, we show the connected device in Device Activity to proceed further with setups.

Choosing Room –

Room name is basically the location of your SUSI Smart Speaker in the home. You may have multiple SUSI Smart Speaker in different rooms, so the purpose of adding the room is to differentiate between them.

When the user clicks on Wi-Fi displayed cell, it starts the initial setups. We are using didSelectRowAt method of UITableViewDelegate to get which cell is selected. On clicking the displayed Wi-Fi cell, a popup is open with a Room Location Text field.

override func tableView(_ tableView: UITableView, didSelectRowAt indexPath: IndexPath) {
if indexPath.row == 0, let speakerSSID = fetchSSIDInfo(), speakerSSID == ControllerConstants.DeviceActivity.susiSSID {
// Open a popup to select Rooms
presentRoomsPopup()
}
}

When the user clicks the Next button, we send the speaker room location to the local server of the speaker by the following API endpoint with room name as a parameter:

http://10.0.0.1:5000/speaker_config/

Refer this post for getting more detail about how choosing room work and how it is implemented in SUSI iOS.

Sharing Wi-Fi Credentials –

On successfully choosing the room, we present a popup that asks the user to enter the Wi-Fi credentials of previously connected Wi-Fi so that we can connect our Smart Speaker to the wifi which can provide internet connection to play music and set commands over the speaker.

We present a popup with a text field for entering wifi password.

When the user clicks the Next button, we share the wifi credentials to wifi by the following API endpoint:

http://10.0.0.1:5000/wifi_credentials/

With the following params-

  1. Wifissid – Connected Wi-Fi SSID
  2. Wifipassd – Connected Wi-Fi password

In this API endpoint, we are sharing wifi SSID and wifi password with Smart Speaker. If the credentials successfully accepted by speaker than we present a popup for user SUSI account password, otherwise we again present Enter Wifi Credentials popup.

Client.sharedInstance.sendWifiCredentials(params) { (success, message) in
DispatchQueue.main.async {
self.alertController.dismiss(animated: true, completion: nil)
if success {
self.presentUserPasswordPopup()
} else {
self.view.makeToast("", point: self.view.center, title: message, image: nil, completion: { didTap in
UIApplication.shared.endIgnoringInteractionEvents()
self.presentWifiCredentialsPopup()
})
}
}
}

 

Sharing SUSI Account Credentials –

In the method above we have seen that when SUSI Smart Speaker accept the wifi credentials, we proceed further with SUSI account credentials. We open a popup to Enter user’s SUSI account password:

When the user clicks the Next button, we use following API endpoint to share user’s SUSI account credentials to SUSI Smart Speaker:

http://10.0.0.1:5000/auth/

With the following params-

  1. email
  2. password

User email is already saved in the device so the user doesn’t have to type it again. If the user credentials successfully accepted by speaker then we proceed with configuration process otherwise we open up Enter Password popup again.

Client.sharedInstance.sendAuthCredentials(params) { (success, message) in
DispatchQueue.main.async {
self.alertController.dismiss(animated: true, completion: nil)
if success {
self.setConfiguration()
} else {
self.view.makeToast("", point: self.view.center, title: message, image: nil, completion: { didTap in
UIApplication.shared.endIgnoringInteractionEvents()
self.presentUserPasswordPopup()
})
}
}
}

 

Setting Configuration –

After successfully sharing SUSI account credentials, following API endpoint is using for setting configuration.

http://10.0.0.1:5000/config/

With the following params-

  1. sst
  2. tts
  3. hotword
  4. wake

The success of this API call makes successfully connection between user iOS Device and SUSI Smart Speaker.

Client.sharedInstance.setConfiguration(params) { (success, message) in
DispatchQueue.main.async {
if success {
// Successfully Configured
self.isSetupDone = true
self.view.makeToast(ControllerConstants.DeviceActivity.doneSetupDetailText)
} else {
self.view.makeToast("", point: self.view.center, title: message, image: nil, completion: { didTap in
UIApplication.shared.endIgnoringInteractionEvents()
})
}
}
}

After successful connection-

 

Resources –

  1. Apple’s Documentation of tableView(_:didSelectRowAt:) API
  2. Initial Setups for Connecting SUSI Smart Speaker with iPhone/iPad
  3. SUSI Linux Link: https://github.com/fossasia/susi_linux
  4. Adding Option to Choose Room for SUSI Smart Speaker in iOS App
Continue ReadingConnecting SUSI iOS App to SUSI Smart Speaker
Read more about the article Adding Option to Choose Room for SUSI Smart Speaker in iOS App
SAMSUNG CAMERA PICTURES

Adding Option to Choose Room for SUSI Smart Speaker in iOS App

SUSI Smart Speaker is an open source smart speaker that supports many features. The user can use Android or iOS to connect their device with SUSI Smart Speaker. During initial installation, it is asking from use to enter the Room name. Room name is basically the location of your SUSI Smart Speaker in the home. You may have multiple SUSI Smart Speaker in different rooms, so the purpose of adding the room is to differentiate between them. You can find useful instructions for the initial connection between the iOS device and SUSI Smart Speaker here. It this post, we will see how the adding rooms feature implemented for SUSI iOS.

When the user enters into the Device Activity screen, we check if the iOS device is connected to SUSI.AI Wi-Fi hotspot or not. If the device is connected to SUSI Smart Speaker, it shows the Wi-Fi displayed SSID in Device Activity Screen. On clicking the displayed Wi-Fi cell, a popup is open with a Room Location Text field. The user can enter Room location and by clicking the Next button, proceed further with the setup.

In the popup, there is also an option for choosing rooms, where the list of most common room names is displayed and the user can choose room name from the list.

Presenting Room Picker View Controller –

func presentRoomsPicker() {
let storyboard = UIStoryboard(name: "Main", bundle: nil)
if let roomVC = storyboard.instantiateViewController(withIdentifier: "RoomPickerVC") as? RoomPickerController {
roomVC.deviceActivityVC = self
let roomNVC = AppNavigationController(rootViewController: roomVC)
self.present(roomNVC, animated: true)
}
}

Room Picker View Controller is UITableViewController that display the rooms names in table cells. Some of the most common rooms names displayed are:

let rooms: [String] = ["Bedroom", "Kitchen", "Family Room", "Entryway", "Living Room", "Front Yard", "Guest Room", "Dining Room", "Computer Room", "Downstairs", "Front Porch", "Garage", "Hallway", "Driveway"]

 

Presenting Room Cell –

We are using cellForRowAt method to present the cell.

override func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
let cell = tableView.dequeueReusableCell(withIdentifier: "roomsCell", for: indexPath)
cell.textLabel?.text = rooms[indexPath.row]
cell.imageView?.image = ControllerConstants.Images.roomsIcon
return cell
}

 

Selecting the room from cell –

When the user clicks on the cell, first willSelectRowAt method use to display the right icon in the accessory view that shows which cell is selected.

if let oldIndex = tableView.indexPathForSelectedRow {
tableView.cellForRow(at: oldIndex)?.accessoryType = .none
}
tableView.cellForRow(at: indexPath)?.accessoryType = .checkmark

We are storing the selected room in the following variable and selecting it by using didSelectRowAt method of UITableView.

var selectedRoom: String?
override func tableView(_ tableView: UITableView, didSelectRowAt indexPath: IndexPath) {
selectedRoom = rooms[indexPath.row]
}

In Room Picker Screen, the user has two option, Cancel and Done. If the user clicks the Cancel, we dismiss the Room Picker screen and display the popup with the empty room location text field and with Choose Room option. If the user clicks the Done button, we store the picked room in UserDefaults shared instance and dismiss Room Picker screen with a different popup which has already filled room location in the text field and without choose room option in the popup as in the image below. By clicking the next, the user proceeds with the further setup.

Resources –

  1. Apple’s Documentations on UserDefaults.
  2. Initial Setups for Connecting SUSI Smart Speaker with iPhone/iPad
  3. Apple’s Documentations on tableView(_:cellForRowAt:)
  4. Apple’s Documentations on tableView(_:willSelectRowAt:)
  5. Apple’s Documentations on tableView(_:didSelectRowAt:)
Continue ReadingAdding Option to Choose Room for SUSI Smart Speaker in iOS App

Adding Open Street Maps to PSLab Android

PSLab Android app is an open source app that uses fully open libraries and tools so that the community can use all it’s features without any compromises related to pricing or feature constraints. This will brings us to the topic how to implement a map feature in PSLab Android app without using proprietary tools and libraries. This is really important as now the app is available on Fdroid and they don’t allow apps to have proprietary tools in them if they are published there. In other words, it simply says we cannot use Google Maps APIs no matter how powerful they are in usage.

There is a workaround and that is using Open Street Maps (OSM). OSM is an open source project which is supported by a number of developers all around the globe to develop an open source alternative to Google Maps. It supports plenty of features we need in PSLab Android app as well. Starting from displaying a high resolution map along with caching the places user has viewed, we can add markers to show data points and locations in sensor data logging implementations.

All these features can be made available once we add the following dependencies in gradle build file. Make sure to use the latest version as there will be improvements and bug fixes in each newer version

implementation "org.osmdroid:osmdroid-android:$rootProject.osmVersion"
implementation "org.osmdroid:osmdroid-mapsforge:$rootProject.mapsforgeVersion"
implementation "org.osmdroid:osmdroid-geopackage:$rootProject.geoPackageVersion"

 

OSM will be functional only after the following permission states were granted.

<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION"/>
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"  />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

 

In a view xml file, add the layout org.osmdroid.views.MapView to initiate the map view. There is a known issue in OSM library. That is during the initiation, if the zoom factor is set to a small value, there will be multiple instances of maps as shown in Fig 1. The solution is to have a higher zoom value when the map is loaded.

Figure 1: Multiple map tiles in OSM

Once we initialize the map view inside an activity, a zoom level can be easily set using a map controller as follows;

map = findViewById(R.id.osmmap);
map.setTileSource(TileSourceFactory.MAPNIK);
map.setBuiltInZoomControls(true);
map.setMultiTouchControls(true);

IMapController mapController = map.getController();
mapController.setZoom((double) 9);
GeoPoint startPoint = new GeoPoint(0.00, 0.00);
mapController.setCenter(startPoint);

 

After successfully implementing the map view, we can develop the business logic to add markers and descriptions to improve the usability of OS Maps. They will be available in the upcoming blog posts.

Reference:

  1. https://github.com/osmdroid/osmdroid/wiki/How-to-add-the-osmdroid-library-via-Gradle
  2. https://www.openstreetmap.org/
Continue ReadingAdding Open Street Maps to PSLab Android

Capturing Position Data with PSLab Android App

PSLab Android app by FOSSASIA can be used to visualize different waveforms, signal levels and patterns. Many of them involve logging data from different instruments. These data sets can be unique and the user might want them to be specific to a location or a time. To facilitate this feature, PSLab Android app offers a feature to save user’s current location along with the data points.

This implementation can be done in two ways. One is to use Google Maps APIs and the other one is to use LocationManager classes provided by Android itself. The first one is more on to proprietary libraries and it will give errors when used in an open source publishing platform like Fdroid as they require all the libraries used in an app to be open. So we have to go with the latter, using LocationManager classes.

As first step, we will have to request permission from the user to allow the app access his current location. This can be easily done by adding a permission tag in the Manifest.xml file.

<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />

 

Since we are not using Google Maps APIs, capturing the current location will take a little while and that can be considered as the only downfall of using this method. We have to constantly check for a location change to capture the data related to current location. This can be easily done by attaching a LocationListener as it will do the very thing for us.

private LocationListener locationListener = new LocationListener() {
   @Override
   public void onLocationChanged(Location location) {
       locationAvailable = true;
   }

   @Override
   public void onStatusChanged(String s, int i, Bundle bundle) {/**/}

   @Override
   public void onProviderEnabled(String s) {/**/}

   @Override
   public void onProviderDisabled(String s) {
       // TODO: Handle GPS turned on/off situations
   }
};

 

In case if the user has turned off GPS in his device, this method wouldn’t work. We will have to request him to turn the feature on using a simple dialog box or a bottom sheet dialog.

We can also customize how frequent the locationlistener should check for a location using another class named LocationManager. This class can be instantiated as follows:

locationManager = (LocationManager) getSystemService(Context.LOCATION_SERVICE);

 

Then we can easily set the time interval using requestLocationUpdates method. Here I have requested location updates in every one second. That is a quite reasonable rate.

locationManager.requestLocationUpdates(provider, 1000, 1, locationListener);

 

Once we have set all this up, we can capture the current location assuming that the user has turned on the GPS option from his device settings and the LocationManager class has a new location as we checked earlier.

if (locationAvailable) {
   Location location = locationManager.getLastKnownLocation(LocationManager.GPS_PROVIDER);
}

 

Each location will contain details related to its position such as latitudes and longitudes. We can log these data using the CSVLogger class implementation in PSLab Android app and let user have this option while doing his experiments.

Reference:

  1. An open source implementation : https://github.com/borneq/HereGPSLocation/blob/master/app/src/main/java/com/borneq/heregpslocation/MainActivity.java

Google Maps: https://developers.google.com/maps/documentation/android-sdk/intro

Continue ReadingCapturing Position Data with PSLab Android App

Displaying Image location Address In Phimpme Android Application

In Phimpme Android application one of the features available is to view the details of any image. The details consists of attributes including Date and time at which the image was captured, size of the image, title, path, EXIF data, description added to the image, location etc. However in the location attribute the location coordinates of the image as well as the location address can be displayed depending on the user’s preference. The process of obtaining the coordinates from address is called as Geocoding and obtaining string address from coordinates is called reverse Geocoding. So in this post I will be explaining how to implement set of strings denoting the address from the available coordinates.

Step 1

First we need to create an instance of the class Geocoder passing context and function Locale.getDefault() as the parameters.  The function of the attribute Locale.getdefault is provided below.

Locale.getDefault() – It returns the current value of the default locale for the current instance of the Java Virtual Machine. The Java Virtual Machine sets the default locale during startup based on the host environment.The code snippet to perform the above mentioned operation is given below.

Geocoder geocoder = new Geocoder(context, Locale.getDefault());

Step 2

Now a function call of getFromLocation() of the Geocoder class is done where we need to pass the Lattitude and Longitude values of the Location object as parameters. The lattitude and longitudes values can be obtained by the use of the Location object functions getLatitude() and getLongitude() respectively. The function getFromLocation() will return a list of Address objects which will contain the extracted addresses from the passed latitude and longitude values. We also need to pass a third parameter an integer value which will determine the maximum no of addresses to be returned. Here we have requested for a maximum of one address. The following code snippet is used to perform the desired function call.

try {
 List<Address> addressList = geocoder.getFromLocation(location.getLatitude(), location.getLongitude(), 1);
} catch (IOException e) {
  e.printStackTrace();
}

Step 3

After obtaining the list of Address objects returned from the function call of getFromLocation() we will extract the first address from the list since we want a maximum of 1 address. The Address object will contain information like the address name, country, state, postal code etc. Now the set of strings describing the location can be retrieved with the help of the function getMaxAddressLineIndex() of Address class. The code snippets to perform the above mentioned operations is provided below.

ArrayList<String> addresslines = new ArrayList<String>();
Address address = addressList.get(0);
for(int i = 0; i <= address.getMaxAddressLineIndex(); i++) {
  addresslines.add(address.getAddressLine(i));
}
details.put(context.getString(R.string.location), TextUtils.join(System.getProperty(“line.separator”),
      addresslines));

The screenshot displaying the location address is provided below.

This is how we have achieved the functionality of displaying location address in a set of strings from available coordinates in the Phimpme Android application. To get the full source code, please refer to the Phimpme Android GitHub repository listed in the resources section below.

Resources

1.Android Developer Guide – https://developer.android.com/training/location/display-address.html

2.Github-Phimpme Android Repository – https://github.com/fossasia/phimpme-android/

3.Address Class Guide- https://developer.android.com/reference/android/location/Address.html

 

Continue ReadingDisplaying Image location Address In Phimpme Android Application

Generating Map Action Responses in SUSI AI

SUSI AI responds to location related user queries with a Map action response. The different types of responses are referred to as actions which tell the client how to render the answer. One such action type is the Map action type. The map action contains latitude, longitude and zoom values telling the client to correspondingly render a map with the given location.

Let us visit SUSI Web Chat and try it out.

Query: Where is London

Response: (API Response)

The API Response actions contain text describing the specified location, an anchor with text ‘Here is a map` linked to openstreetmaps and a map with the location coordinates.

Let us look at how this is implemented on server.

For location related queries, the key where is used as an identifier. Once the query is matched with this key, a regular expression `where is (?:(?:a )*)(.*)` is used to parse the location name.

"keys"   : ["where"],
"phrases": [
  {"type":"regex", "expression":"where is (?:(?:a )*)(.*)"},
]

The parsed location name is stored in $1$ and is used to make API calls to fetch information about the place and its location. Console process is used to fetch required data from an API.

"process": [
  {
    "type":"console",
    "expression":"SELECT location[0] AS lon, location[1] AS lat FROM locations WHERE query='$1$';"},
  {
    "type":"console",
    "expression":"SELECT object AS locationInfo FROM location-info WHERE query='$1$';"}
],

Here, we need to make two API calls :

  • For getting information about the place
  • For getting the location coordinates

First let us look at how a Console Process works. In a console process we provide the URL needed to fetch data from, the query parameter needed to be passed to the URL and the path to look for the answer in the API response.

  • url = <url>   – the url to the remote json service which will be used to retrieve information. It must contain a $query$ string.
  • test = <parameter> – the parameter that will replace the $query$ string inside the given url. It is required to test the service.

For getting the information about the place, we used Wikipedia API. We name this console process as location-info and added the required attributes to run it and fetch data from the API.

"location-info": {
  "example":"http://127.0.0.1:4000/susi/console.json?q=%22SELECT%20*%20FROM%20location-info%20WHERE%20query=%27london%27;%22",
  "url":"https://en.wikipedia.org/w/api.php?action=opensearch&limit=1&format=json&search=",
  "test":"london",
  "parser":"json",
  "path":"$.[2]",
  "license":"Copyright by Wikipedia, https://wikimediafoundation.org/wiki/Terms_of_Use/en"
}

The attributes used are :

  • url : The Media WIKI API endpoint
  • test : The Location name which will be appended to the url before making the API call.
  • parser : Specifies the response type for parsing the answer
  • path : Points to the location in the response where the required answer is present

The API endpoint called is of the following format :

https://en.wikipedia.org/w/api.php?action=opensearch&limit=1&format=json&search=LOCATION_NAME

For the query where is london, the API call made returns

[
  "london",
  ["London"],
  ["London  is the capital and most populous city of England and the United Kingdom."],
  ["https://en.wikipedia.org/wiki/London"]
]

The path $.[2] points to the third element of the array i.e “London  is the capital and most populous city of England and the United Kingdom.” which is stored in $locationInfo$.

Similarly to get the location coordinates, another API call is made to loklak API.

"locations": {
  "example":"http://127.0.0.1:4000/susi/console.json?q=%22SELECT%20*%20FROM%20locations%20WHERE%20query=%27rome%27;%22",
  "url":"http://api.loklak.org/api/console.json?q=SELECT%20*%20FROM%20locations%20WHERE%20location='$query$';",
  "test":"rome",
  "parser":"json",
  "path":"$.data",
  "license":"Copyright by GeoNames"
},

The location coordinates are found in $.data.location in the API response. The location coordinates are stored as latitude and longitude in $lat$ and $lon$ respectively.

Finally we have description about the location and its coordinates, so we create the actions to be put in the server response.

The first action is of type answer and the text to be displayed is given by $locationInfo$ where the data from wikipedia API response is stored.

{
  "type":"answer",
  "select":"random",
  "phrases":["$locationInfo$"]
},

The second action is of type anchor. The text to be displayed is `Here is a map` and it must be hyperlinked to openstreetmaps with the obtained $lat$ and $lon$.

{
  "type":"anchor",
  "link":"https://www.openstreetmap.org/#map=13/$lat$/$lon$",
  "text":"Here is a map"
},

The last action is of type map which is populated for latitude and longitude using $lat$ and $lon$ respectively and the zoom value is specified to be 13.

{
  "type":"map",
  "latitude":"$lat$",
  "longitude":"$lon$",
  "zoom":"13"
}

Final output from the server will now contain the three actions with the required data obtained from the respective API calls made. For the sample query `where is london` , the actions will look like :

"actions": [
  {
    "type": "answer",
    "language": "en",
    "expression": "London  is the capital and most populous city of England and the United Kingdom."
  },
  {
    "type": "anchor",
    "link":   "https://www.openstreetmap.org/#map=13/51.51279067225417/-0.09184009399817228",
    "text": "Here is a map",
    "language": "en"
  },
  {
    "type": "map",
    "latitude": "51.51279067225417",
    "longitude": "-0.09184009399817228",
    "zoom": "13",
    "language": "en"
  }
],

This is how the map action responses are generated for location related queries. The complete code can be found at SUSI AI Server Repository.

Resources:

Continue ReadingGenerating Map Action Responses in SUSI AI

Getting Image location in the Phimpme Android’s Camera

The Phimpme Android app along with a decent gallery and accounts section comes with a nice camera section stuffed with all the features which a user requires for the day to day usage. It comes with an Auto mode for the best experience and also with a manual mode for the users who like to have some tweaks in the camera according to their own liking. Along with all these, it also has an option to get the accurate coordinates where the image was clicked. When we enable the location from the settings, it extracts the latitude and longitude of the image when it is being clicked and displays the visible region of the map at the top of the image info section as depicted in the screenshot below.

In this tutorial, I will be discussing how we have implemented the location functionality to fetch the location of the image in the Phimpme app.

Step 1

For getting the location from the device, the first step we need is to add the permission in the androidmanifest.xml file to access the GPS and the location services. This can be done using the following lines of code below.

<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION"/>

After this, we need to download install the google play services SDK to access the Google location API. Follow the official google developer’s guide on how to install the Google play services into the project from the resources section below.

Step 2

To get the last known location of the device at the time of clicking the picture we need to make use of the FusedLocationProviderClient class and need to create an object of this class and to initialise it in the onCreate method of the camera activity. This can be done using the following lines of code below:

private FusedLocationProviderClient mFusedLocationClient;
mFusedLocationClient = LocationServices.getFusedLocationProviderClient(this);

After we have created and initialised the object mFusedLocationClient, we need to call the getLastLocation method on it as soon as the user clicks on the take picture button in the camera. In this, we can also set onSuccessListener method which will return the Location object when it successfully extracts the present or the last known location of the device. This can be done using the following lines of code below:

mFusedLocationClient.getLastLocation()
       .addOnSuccessListener(this, new OnSuccessListener<Location>() {
           @Override
           public void onSuccess(Location location) {
               if (location != null) {
            //Get the latitude and longitude here
                  }

After this, we can successfully extract the latitude and the longitude of the device in the onSuccess method of the code snippet provided below and can store it in the shared preference to get the map view of the coordinates from a different activity of the application later on when the user tries to get the info of the images.

Step 3

After getting the latitude and longitude, we need to get the image view of the visible region of the map. We can make use of the Glide library to fetch the visible map area from the url which contains our location values and to set it to the image view.

The url of the visible map can be generated using the following lines of code.

String.format(Locale.US, getUrl(value), location.getLatitude(), location.getLongitude());

This is how we have added the functionality to fetch the coordinates of the device at the time of clicking the image and to display the map in the Phimpme Android application. To get the full source code, please refer to the Phimpme Android GitHub repository.

Resources

  1. Google Developer’s : Location services guide – https://developer.android.com/training/location/retrieve-current.html
  2. Google Developer’s : Google play services SDK guide – https://developer.android.com/studio/intro/update.html#channels
  3. GitHub : Open camera Source Code –  https://github.com/almalence/OpenCamera
  4. GitHub : Phimpme Android – https://github.com/fossasia/phimpme-android/
  5. GitHub : Glide library – https://github.com/bumptech/glide

 

Continue ReadingGetting Image location in the Phimpme Android’s Camera

Getting user Location in SUSI Android App and using it for various SUSI Skills

Using user location in skills is a very common phenomenon among various personal assistant like Google Assistant, Siri, Cortana etc. SUSI is no different. SUSI has various skills which uses user’s current location to implement skills. Though skills like “restaurant nearby” or “hotels nearby” are still under process but skills like “Where am I” works perfectly indicating SUSI has all basic requirements to create more advance skills in near future.

So let’s learn about how the SUSI Android App gets location of a user and sends it to SUSI Server where it is used to implement various location based skills.

Sources to find user location in an Android app

There are three sources from which android app gets users location :

  1. GPS
  2. Network
  3. Public IP Address

All three of these have various advantages and disadvantages. The SUSI Android app uses cleverly each of them to always get user location so that it can be used anytime and anywhere.

Some factors for comparison of these three sources :

Factors GPS Network IP Address
Source Satellites Wifi/Cell Tower Public IP address of user’s mobile
Accuracy Most Accurate (20ft) Moderately Accurate (200ft) Least Accurate (5000+ ft)
Requirements GPS in mobile Wifi or sim card Internet connection
Time taken to give location Takes long time to get location Fastest way to get location Fast enough (depends on internet speed)
Battery Consumption High Medium Low
Permission Required User permission required User permission required No permission required
Location Factor Works in outdoors. Does not work near tall buildings Works everywhere Works everywhere

Implementation of location finding feature in SUSI Android App

SUSI Android app very cleverly uses all the advantages of each location finding source to get most accurate location, consume less power and find location in any scenario.

The /susi/chat.json endpoint of SUSI API requires following 7 parameters :

Sno. Parameter Type Requirement
1 q String Compulsory
2 timezoneOffset int Optional
3 longitude double Optional
4 latitude double Optional
5 geosource String Optional
6 language Language  code Optional
7 access_token String Optional

In this blog we will be talking about latitude , longitude and geosource. So, we need these three things to pass as parameters for location related skills. Let’s see how we do that.

Finding location using IP Address: At the starting of app, user location is found by making an API call to ipinfo.io/json . This results in following JSON response having a field “loc” giving location of user (latitude and longitude.

{
  "ip": "YOUR_IP_ADDRESS",
  "city": "YOUR_CITY",
  "region": "YOUR_REGION",
  "country": "YOUR_COUNTRY_CODE",
  "loc": "YOUR_LATITUDE,YOUR_LONGITUDE",
  "org": "YOUR_ISP"
}

By this way we got latitude, longitude and geosource will be “ip” . We find location using IP address only once the app is started because there is no need of finding it again and again as making network calls takes time and drains battery.

So, now we have user’s location but this is not accurate. So, we will now proceed to check if we can find location using network is more accurate than location using IP address.

Finding location using Network Service Provider : To actually use the network provider and find out location requires ACCESS_COARSE_LOCATION permission from user which can be asked during the run time. Also, the location can only be found out using this if user has his location setting is enabled. So, we check following condition.

if (ActivityCompat.checkSelfPermission(mContext, Manifest.permission.ACCESS_COARSE_LOCATION) == PackageManager.PERMISSION_GRANTED) {
}

If permission is granted by user to find location using network provider, we use following code snippet to find location. It updates location of user after every 5 minutes or 10 meters (whichever is achieved first).

locationManager.requestLocationUpdates(LocationManager.NETWORK_PROVIDER, 5 * 60 * 1000, 10, this);
location = locationManager.getLastKnownLocation(LocationManager.NETWORK_PROVIDER);
if (location != null) {
   source = "network";
   canGetLocation = true;
   latitude = location.getLatitude();
   longitude = location.getLongitude();
}

So, whenever we are about to send query to SUSI Server, we take location from Network services, thus updating previous values of latitude, longitude and geosource (found using IP address) with the new values (found using Network Provider), provided the user has granted permission. So, we now have location is from Network Provider which is more accurate than location from IP address. Now we will check if we can find location from GPS or not.

Finding location using GPS Service Provider : Finding location from GPS Provider is almost same as Network Provider. To find location using GPS Provider user must give  ACCESS_FINE_LOCATION permission. We just check if GPS of user is enabled and user has given permission to use GPS and also if GPS can actually provide location. Sometimes, GPS can not provide location because user is indoor. In that cases we leave location from GPS.

So, now if we update previous values of latitude, longitude and geosource (found using Network Provider) with the new values (found using GPS Provider) and send query to SUSI Server.

Summary

To send location to server for location skills, we need latitude, longitude and geosource. We first find these 3 things using IP address (no that accurate). So, geosource will be “ip” for now. Then check if we can find values using network provider. If yes, we update those 3 values with the ones got from network Provider (more accurate). Geosource will change to “network”. Finally, we check if we can find values using GPS provider. If yes, we update those 3 values with the ones got from GPS Provider (most accurate). Geosource will change to “gps”. So, by this way we can find location of user in any circumstance possible. If you want to use location in your app too. Just follow the above steps and you are good to go.

Resources

 

Continue ReadingGetting user Location in SUSI Android App and using it for various SUSI Skills