Fetching Info of All Users and their connected devices for the SUSI.AI Admin Panel

Fetching the data of all users is required for displaying the list of users on the SUSI.AI Admin panel. It was also required to fetch the information of connected devices of the user along with the other user data. The right to fetch the data of all users should only be permitted to user roles “OPERATOR” and above. This blog post explains how the data of connected devices of all users is fetched, which can then be used in the Admin panel.

How is user data stored on the server?

All the personal accounting information of any user is stored in the user’s accounting object. This is stored in the “accounting.json” file. The structure of this file is as follows:

{
  "email:akjn11@gmail.com": {
    "devices": {
      "8C-39-45-23-D8-95": {
        "name": "Device 2",
        "room": "Room 2",
        "geolocation": {
          "latitude": "54.34567",
          "longitude": "64.34567"
        }
      }
    },
    "lastLoginIP": "127.0.0.1"
  },
  "email:akjn22@gmail.com": {
    "devices": {
      "1C-29-46-24-D3-55": {
        "name": "Device 2",
        "room": "Room 2",
        "geolocation": {
          "latitude": "54.34567",
          "longitude": "64.34567"
        }
      }
    },
    "lastLoginIP": "127.0.0.1"
  }
}

 

As can be seen from the above sample content of the “accounting.json” file, we need to fetch this data so that it can then be used to display the list of users along with their connected devices on the Admin panel.

Implementing API to fetch user data and their connected devices

The endpoint of the servlet is “/aaa/getUsers.json” and the minimum user role for this servlet is “OPERATOR”. This is implemented as follows:

   @Override
    public String getAPIPath() {
        return "/aaa/getUsers.json";
    }

    @Override
    public UserRole getMinimalUserRole() {
        return UserRole.OPERATOR;
    }

 

Let us go over the main method serviceImpl() of the servlet:

  • We need to traverse through the user data of all authorized users. This is done by getting the data using DAO.getAuthorizedClients() and storing them in a Collection. Then we extract all the keys from this collection, which is then used to traverse into the Collection and fetch the user data. The implementation is as follows:

    Collection<ClientIdentity> authorized = DAO.getAuthorizedClients();
    List<String> keysList = new ArrayList<String>();
    authorized.forEach(client -> keysList.add(client.toString()));

    for (Client client : authorized) {
        // code           
    }

 

  • Then we traverse through each client and generate a client identity to get the user role of the client. This is done using the DAO.getAuthorization() method. The user role of the client is also put in the final object which we want to return. This is implemented as follows:

    JSONObject json = client.toJSON();
    ClientIdentity identity = new 
    ClientIdentity(ClientIdentity.Type.email, client.getName());
    Authorization authorization = DAO.getAuthorization(identity);
    UserRole userRole = authorization.getUserRole();
    json.put("userRole", userRole.toString().toLowerCase());

 

  • Then the client credentials are generated and it is checked whether the user is verified or not. If the user is verified, then in the final object, “confirmed” is set to true, else it is set to false.

    ClientCredential clientCredential = new ClientCredential (ClientCredential.Type.passwd_login, identity.getName());
    Authentication authentication = DAO.getAuthentication(clientCredential);

    json.put("confirmed", authentication.getBoolean("activated", false));

 

  • Then we fetch the accounting object of the user using DAO.getAccounting(), and extract all the user data and put them in separate key value pairs in the final object which we want to return. As the information of all connected devices of a user is also stored in the user’s accounting object, that info is also extracted the same way and put into the final object.

    Accounting accounting = DAO.getAccounting(authorization.getIdentity());
    if (accounting.getJSON().has("lastLoginIP")) {
        json.put("lastLoginIP", accounting.getJSON().getString("lastLoginIP"));
    } else {
        json.put("lastLoginIP", "");
    }

    if(accounting.getJSON().has("signupTime")) {
        json.put("signupTime", accounting.getJSON().getString("signupTime"));
    } else {
        json.put("signupTime", "");
    }

    if(accounting.getJSON().has("lastLoginTime")) {
        json.put("lastLoginTime", accounting.getJSON().getString("lastLoginTime"));
    } else {
        json.put("lastLoginTime", "");
    }

    if(accounting.getJSON().has("devices")) {
        json.put("devices", accounting.getJSON().getJSONObject("devices"));
    } else {
        json.put("devices", "");
    }
    accounting.commit();

 

This is how the data of all users is fetched by any Admin or higher user role, and is then used to display the user list on the Admin panel.

Resources

Continue ReadingFetching Info of All Users and their connected devices for the SUSI.AI Admin Panel

Adding Push endpoint to send data from Loklak Search to Loklak Server

To provide enriched and sufficient amount of data to Loklak, Loklak Server should have multiple sources of data. The api/push.json endpoint of loklak server is used in Loklak to post the search result object to server. It will increase the amount and quality of data on server once the Twitter api is supported by Loklak (Work is in progress to add support for twitter api in loklak).

Creating Push Service

The idea is to create a separate service for making a Post request to server. First step would be to create a new ‘PushService’ under ‘services/’ using:

ng g service services/push

Creating model for Push Api Response

Before starting to write code for push service, create a new model for the type of response data obtained from Post request to ‘api/push.json’. For this, create a new file push.ts under ‘models/’ with the code given below and export the respective push interface method in index file.

export interface PushApiResponse {
   status: string;
   records: number;
   mps: number;
   message: string;
}

Writing Post request in Push Service

Next step would be to create a Post request to api/push.json using HttpClient module. Import necessary dependencies and create an object of HttpClient module in constructor and write a PostData() method which would take the data to be send, makes a Post request and returns the Observable of PushApiResponse created above.

import { Injectable } from ‘@angular/core’;
import {
   HttpClient,
   HttpHeaders,
   HttpParams
} from ‘@angular/common/http’;
import { Observable } from ‘rxjs’;
import {
	ApiResponse,
	PushApiResponse
} from ‘../models’;

@Injectable({
   providedIn: ‘root’
})
export class PushService {

   constructor( private http: HttpClient ) { }
   public postData(data: ApiResponse):
   		Observable<PushApiResponse> {

	const httpUrl = ‘https://api.loklak.org/
		api/push.json’;
	const headers = new HttpHeaders({
		‘Content-Type’: ‘application/
			x-www-form-urlencoded’,
		‘Accept’: ‘application/json’,
		‘cache-control’: ‘no-cache’
	});
	const {search_metadata, statuses} = data;
	
	// Converting the object to JSON string.
	const dataToSend = JSON.stringify({
		search_metadata: search_metadata,
		statuses});
	
	// Setting the data to send in
	// HttpParams() with key as ‘data’
	const body = new HttpParams()
		.set(‘data’, dataToSend);
	
	// Making a Post request to api/push.json
	// endpoint. Response Object is converted
	// to PushApiResponse type.
	return this.http.post<PushApiResponse>(
		httpUrl, body, {headers:
		headers
	});
   }
}

 

Note: Data (dataToSend) send to backend should be exactly in same format as obtained from server.

Pushing data into service dynamically

Now the main part is to provide the data to be send into the service. To make it dynamic, import the Push Service in ‘api-search.effects.ts’ file under effects and create the object of Push Service in its constructor.

import { PushService } from ‘../services’;
constructor(
   
   private pushService: PushService
) { }

 

Now, call the pushService object inside ‘relocateAfterSearchSuccess$’ effect method and pass the search response data (payload value of search success action) inside Push Service’s postData() method.

@Effect()
relocateAfterSearchSuccess$: Observable<Action>
   = this.actions$
       .pipe(
           ofType(
               apiAction.ActionTypes
			   	.SEARCH_COMPLETE_SUCCESS,
               apiAction.ActionTypes
			   	.SEARCH_COMPLETE_FAIL
           ),
           withLatestFrom(this.store$),
           map(([action, state]) => {
               this.pushService
			   .postData(action[‘payload’]);
           
       );

Testing Successful Push to Backend

To test the success of Post request, subscribe to the response data and print the response data on console. You should see something like:

Where each of these is a response of one successful Post request.

Resources

Continue ReadingAdding Push endpoint to send data from Loklak Search to Loklak Server

Encoding and Decoding Images as Data in UserDefaults in SUSI iOS

In this blog post, I will be explaining how to encode and decode images and save them in UserDefaults so that the image persists even if it is removed from the Photos app. It happens a number of times that images are removed from the gallery by the users which results in the app loosing the image. So, to avoid this, we save the image by encoding it in a data object and save it inside UserDefaults. In SUSI iOS app we simply select an image from the image picker, encode it and save it in UserDefaults. To set the image, we simply fetch the image data from the UserDefaults and decode it to an image.

There are two ways we can do the encoding and decoding process:

  • Using Data object
  • Using Base64 string

For the scope of this tutorial, we will use the Data object.

Implementation Steps

  1. To use the image picker, we need to add permissions to `Info.plist` file.
<key>NSLocationWhenInUseUsageDescription</key>
<string>Susi is requesting to get your current location</string>
<key>NSPhotoLibraryUsageDescription</key>
<string>Susi needs to request your gallery access to select wallpaper</string>
  1. Select image from gallery

First, we present an alert which gives an option to select the image from the gallery.

// Show wallpaper options to set wallpaper or clear wallpaper
func showWallpaperOptions() {
  let imageDialog = UIAlertController(title: ControllerConstants.wallpaperOptionsTitle, message: nil, preferredStyle: UIAlertControllerStyle.alert)
  imageDialog.addAction(UIAlertAction(title: ControllerConstants.wallpaperOptionsPickAction, style: .default, handler: { (_: UIAlertAction!) in
  imageDialog.dismiss(animated: true, completion: nil)
  self.showImagePicker()
  }))
  imageDialog.addAction(UIAlertAction(title: ControllerConstants.wallpaperOptionsNoWallpaperAction, style: .default, handler: { (_: UIAlertAction!) in
    imageDialog.dismiss(animated: true, completion: nil)
    self.removeWallpaperFromUserDefaults()
  }))
  imageDialog.addAction(UIAlertAction(title: ControllerConstants.dialogCancelAction, style: .cancel, handler: { (_: UIAlertAction!) in
    imageDialog.dismiss(animated: true, completion: nil)
  }))
  self.present(imageDialog, animated: true, completion: nil)
}

Here, we create and UIAlertController with three options to select, one which presents the image picker controller, the second one removes the background wallpaper and the third dismisses the alert.

  1. Set the image as background view
// Callback when image is selected from gallery
func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [String : Any]) {
  dismiss(animated: true, completion: nil)
  let chosenImage = info[UIImagePickerControllerOriginalImage] as? UIImage
  if let image = chosenImage {
    setBackgroundImage(image: image)
  }
}

We use the `didFinishPickingMediaWithInfo` delegate method to set the image as background. First we get the image using the the `info` dictionary using the `UIImagePickerControllerOriginalImage` key.

  1. Save the image in UserDefaults (encoding)
// Save image selected by user to user defaults
func saveWallpaperInUserDefaults(image: UIImage!) {
  let imageData = UIImageJPEGRepresentation(image!, 1.0)
  let defaults = UserDefaults.standard
  defaults.set(imageData, forKey: userDefaultsWallpaperKey)
}

We first convert the image to a data object using the `UIImageJPEGRepresentation` method followed by saving the data object in UserDefaults with the key `wallpaper`.

  1. Decode the data object back to UIImage 

Now whenever we need to decode the image, we simply get the data object from the UserDefaults and use it to display the image.

// Check if user defaults have an image data saved else return nil/Any
func getWallpaperFromUserDefaults() -> Any? {
  let defaults = UserDefaults.standard
  return defaults.object(forKey: userDefaultsWallpaperKey)
}

Below is the output when an image is selected and displayed as a background.

Resources:

Continue ReadingEncoding and Decoding Images as Data in UserDefaults in SUSI iOS

Shrinking Model Classes Boilerplate in Open Event Android Projects Using Jackson and Lombok

JSON is the de facto standard format used for REST API communication, and for consuming any of such API on Android apps like Open Event Android Client and Organiser App, we need Plain Old Java Objects, or POJOs to map the JSON attributes to class properties. These are called models, for they model the API response or request. Basic structure of these models contain

  • Private properties representing JSON attributes
  • Getters and Setters for these properties used to change the object or access its data
  • A toString() method which converts object state to a string, useful for logging and debugging purposes
  • An equals and hashcode method if we want to compare two objects

These can be easily and automatically be generated by any modern IDE, but add unnecessarily to the code base for a relatively simple model class, and are also difficult to maintain. If you add, remove, or rename a method, you have to change its getters/setters, toString and other standard data class methods.

There are a couple of ways to handle it:

  • Google’s Auto Value: Creates Immutable class builders and creators, with all standard methods. But, as it generates a new class for the model, you need to add a library to make it work with JSON parsers and Retrofit. Secondly, there is no way to change the object attributes as they are immutable. It is generally a good practice to make your models immutable, but if you are manipulating their data in your application and saving it in your database, it won’t be possible except than to create a copy of that object. Secondly, not all database storage libraries support it
  • Kotlin’s Data Classes: Kotlin has a nice way to made models using data classes, but it has certain limitations too. Only parameters in primary constructor will be included in the data methods generated, and for creating a no argument constructor (required for certain database libraries and annotation processors), you either need to assign default value to each property or call the primary constructor filling in all the default values, which is a boilerplate of its own. Secondly, sometimes 3rd party libraries are needed to work correctly with data classes on some JSON parsing frameworks, and you probably don’t want to just include Kotlin in your project just for data classes
  • Lombok: We’ll be talking about it later in this blog
  • Immutables, Xtend Lang, etc

This is one kind of boilerplate, and other kind is JSON property names. As we know Java uses camelcase notation for properties, and JSON attributes are mostly:

  • Snake Cased: property_name
  • Kebab Cased: property-name

Whether you are using GSON, Jackson or any other JSON parsing framework, it works great for non ambiguous property names which match as same in both JSON and Java, but requires special naming attributes for translating JSON attributes to camelcase and vice versa. Won’t you want your parser to intelligently convert property_name to propertyName without having you write the tedious mapping which is not only a boilerplate, but also error prone in case your API changes and you forget to update the annotations, or make a spelling mistake, as they are just non type-safe strings.

These boilerplates cause serious regressions during development for what should be a simple Java model for a simple API response. These both kinds of boilerplate are also related to each other as all JSON parsers look for getters and setters for private fields, so there are two layers of potential errors in modeling JSON to Java Models. This should be a lot simpler than it is. So, in this blog, we’ll see how we can configure our project to be 0 boilerplate tolerant and error free. We reduced approximately 70% boilerplate using this configuration in our projects. For example, the Event model class we had (our biggest) reduced from 590 lines to just 74!

We will use a simpler class for our example here:

public class CallForPapers {

    private String announcement;
    @JsonProperty("starts-at")
    private String startsAt;
    private String privacy;
    @JsonProperty("ends-at")
    private String endsAt;

    // Getters and Setters

    @Override
    public String toString() {
        return "CallForPapers{" +
                "announcement='" + announcement + '\'' +
                ", startsAt='" + startsAt + '\'' +
                ", privacy='" + privacy + '\'' +
                ", endsAt='" + endsAt + '\'' +
                '}';
    }
}

 

Note that getters and setters have been omitted for brevity. The actual class is 57 lines long

As you can see, we are using @JsonProperty annotation to properly map the starts-at attribute to startsAt property and similarly on endsAt. First, we’ll remove this boilerplate from our code. Note that this seems a bit overkill for 2 attributes, but imagine the time you’ll save by not having to maintain 100s of attributes for the whole project.

Jackson is smart enough to map different naming styles to one another in both serializing and deserializing. This is done by using Naming Strategy class in Jackson. There is an option to globally configure it, but I found that it did not work for our case, so we had to apply it to each model. It can be simply done by adding another annotation on the top of your class declaration and removing the JsonProperty attribute from your fields

@JsonNaming(PropertyNamingStrategy.KebabCaseStrategy.class)
public class CallForPapers {

    private String announcement;
    private String startsAt;
    private String privacy;
    private String endsAt;

    // Getters and Setters

    @Override
    public String toString() {
        return "CallForPapers{" +
                "announcement='" + announcement + '\'' +
                ", startsAt='" + startsAt + '\'' +
                ", privacy='" + privacy + '\'' +
                ", endsAt='" + endsAt + '\'' +
                '}';
    }
}

 

Our class looks like this now. But be careful to properly name your getters and setters because now, Jackson will map attributes by method names, so if you name the setter for startsAt -> setStartsAt it will automatically understand that the attribute to be mapped is “starts-at”. But, if the method name is something else, then it won’t be able to correctly map the fields. If your properties are not private, then Jackson may instead use them to map fields, so be sure to name your public properties in a correct manner.

Note: If your API does not use kebab case, there are plenty of other options or naming strategies present in Jackson, one example will be

  • PropertyNamingStrategy.SnakeCaseStrategy for attributes like “starts_at”

 Needless to say, this will only work if your API uses a uniform naming strategy

Now we have removed quite a burden from the development lifecycle, but 70% of class is still getters, setters, toString and other data methods. Now, we’ll configure lombok to automatically generate these for us. First, we’ll need to add lombok in our project by adding provided dependency in build.gradle and sync the project

provided 'org.projectlombok:lombok:1.16.18'

 

And now you’d want to install Lombok plugin in Android Studio by going to Files > Settings > Plugins and searching and installing Lombok Plugin and restarting the IDE

 

After you have restarted the IDE, navigate to your model and add @Data annotation at the top of your class and remove all getters/setters, toString, equals, hashcode and if the plugin was installed correctly and lombok was installed from the gradle dependencies, these will be automatically generated for you at build time without any problem. A way for you to see the generated methods is to the structure perspective in the Project Window.

 

There are many more fun tools in lombok and more fine grained control options are provided. Our class looks like this now

@Data
@JsonNaming(PropertyNamingStrategy.KebabCaseStrategy.class)
public class CallForPapers {
    private String announcement;
    private String startsAt;
    private String privacy;
    private String endsAt;
}

 

Reduced to 16 lines (including imports and package). Now, there are some corner cases that you want to iron out for the integration between lombok and Jackson to work correctly.

Lombok uses property names for generating its getters and setters. But there’s a different convention for handling booleans. For the sake of simplicity, we’ll only talk about primitive boolean. You can check out the links below to learn more about class type. The primitive boolean property of the standard Java format, for example hasSessions will generate hasSessions and getter and setHasSessions. Jackson is smart but it expects a getter named getHasSessions creating problems in serialization. Similarly, for a property name isComplete, generate getter and setter will be isComplete and setComplete, creating a problem in deserialization too. Actually, there are ways how Jackson can get boolean values mapped correctly with these getters/setters, but that method needs to rename property itself, changing the getters and setters generated by Lombok. There is actually a way to tell Lombok to not generate this format of getter/setter. You’d need to create a file named lombok.config in your project directory app/ and write this in it

lombok.anyConstructor.suppressConstructorProperties = true
lombok.addGeneratedAnnotation = false
lombok.getter.noIsPrefix = true

 

There are some other settings in it that make it configured for Android specific project

There are some known issues in Android related to Lombok. As lombok itself is an annotation processor, and there is no order for annotation processors to run, it may create problems with other annotation processors. Dagger had issues with it until they fixed it in their later versions. So you might need to check out if any of your libraries depend upon the lombok generated code like getters and setters. Certain database libraries use that and Android Data Binding does too. Currently, there is no solution to the problem as they will throw an error about not finding a getter/setter because they ran before lombok. A possible workaround is to make properties public so that instead of using getters and setters, these libraries use them instead. This is not a good practice, but as this is a data class and you are already creating getters and setters for all fields, this is not a security vulnerability.

There are tons of options for both Jackson and Lombok with a lot of features to help the development process, so be sure to check out these links:

Continue ReadingShrinking Model Classes Boilerplate in Open Event Android Projects Using Jackson and Lombok