How to use Realm in SUSI Android to Save Data

Sometimes we need to store information on the device locally so that we can use information offline and also query data faster. Initially, SQLite was only option to store information on the device. But working with SQLite is difficult sometimes and also it makes code difficult to understand. Also, SQL queries take a long time. But now we have realm a better alternative of SQLite. The Realm is a lightweight mobile database and better substitute of SQLite. The Realm has own C++ core and store data in a universal, table-based format by a C++ core. This allows Realm to allow data access from multiple languages as well as a range of queries. In this blog post, I will show you why we used Realm and how we save data in SUSI Android using Realm.

“How about performance? Well, we’re glad you asked 🙂 For all the API goodness & development productivity we give you, we’re still up to 100x faster than some SQLite ORMs and on average ~10x faster than raw SQLite and common ORMs for typical operations.” (compare: https://blog.realm.io/realm-for-android/)

Advantages of Realm over SQLite are following:

  • It is faster than SQLite as explained on the Realm blog. One of the reasons realm is faster than SQLite is, the traditional SQLite + ORM abstraction is leaky because ORM simply converts  Objects and their methods into SQL statements. Realm, on the other hand, is an object database, meaning your objects directly reflect your database.
  • It is easier to use as it uses objects for storing data. When we use SQLite we need boilerplate code to convert values to and from the database, setting up mappings between classes and tables, fields and columns, foreign keys, etc. Whereas in Realm data is directly exposed as objects and can be queried without any conversion.

Prerequisites

To include this library in your project you need

  • Android studio version 1.5.1 or higher.
  • JDK version 7.0 or higher.
  • Android API level 9 or higher.

How to use realm in Android

To use Realm in your project we must add the dependency of the library in build.gradle(project) file 

 dependencies {
       classpath “io.realm:realm-gradle-plugin:3.3.1”
   }

and build.gradle(module) file.

apply plugin: realm-android
dependencies {
compile io.realm:android-adapters:1.3.0
}

Now you have to instantiate Realm in your application class. Setting a default configuration in your Application class, will ensure that it is available in the rest of your code.

RealmConfiguration realmConfiguration = new RealmConfiguration.Builder(this)
                                                              .deleteRealmIfMigrationNeeded().build();
Realm.setDefaultConfiguration(realmConfiguration);

Now we need to create a model class. A model class is use to save data in Realm and retrieve saved data and it must extend RealmObject class. For eg.

public class Person extends RealmObject {
   private String name;
   public String getName() {
       return name;
   }
   public void setName(String name) {
       this.name = name;
   }
}

Field in the model class uses to define columns. For eg. ‘name’ is a column name. Method like setName() use to save data  and getName() use to retrieve saved data.

Now create an instance of the Realm in the activity where you want to use it. It will be used to read data from the Realm and write data to the Realm.

Realm realm = Realm.getInstance(this);

Before you start a new transaction you must call beginTransaction(). It will open database.

realm.beginTransaction();

To write data to the Realm you need to create an instance of the model class. createObject used to create an instance of RealmObject class. Our model class is RealmObject type so we use createObject() to create an instance of the model class.

Person person = realm.createObject(Person.class);

Write data to realm.

person.setName(“MSDHONI”);

And after it you must call commitTransaction(). commitTransaction() use to end transaction.

realm.commitTransaction();

Reading data from Realm is easier than writing data to it. You need to create an instance of the Realm.

Realm realm = Realm.getInstance(this);

To create query use where the method and pass the class of object you want to query. After creating query you can fetch all data using findAll() method.

realm.where(Person.class).findAll();

Reference

Showing only Logged-in Accounts in the Sharing Page of Phimpme Android

In Phimpme Android application, users can edit their pictures and share them to a number of platforms ranging from social networking sites like Facebook, Twitter etc to cloud storage and image hosting sites like Box, Dropbox, Imgur etc.

Desired flow of the application

According to the flow of the application, the user has to add an account first i.e. log in to the particular account that needs to be connected to the application. After that when the user enters the share page for sharing the image, a button corresponding to the connected account is visible in that page which on clicking will share the image to that platform directly.

What was happening previously?

The list of accounts which is present in the account manager of Phimpme Android application is also getting displayed in the share image page. As the list is large, it is difficult for the user to find the connected account from the list. There is not even an indicator whether a particular account is connected or not. On clicking the button corresponding to the non-connected account, an error dialog instructing the user to log in from the account manager first, will get displayed.

How we solved it?

We first thought of just adding an indicator on the buttons in the accounts page to show whether it is connected or not. But this fix solves only a single issue. Find the connected account in that large list will be difficult for the user even then. So we decided to remove the whole list and show only the accounts which are connected previously in account manager. This cleans the flow of the accounts and share in  Phimpme Android application

When a user logins from the account manager, the credentials, tokens and other details corresponding to that accounts gets saved in database. We used realm database for saving the details in our application. As the details are present in this database, the list can be dynamically generated when the user opens share image page. We implemented a function in Utils class for getting the list of logged in accounts. Its implementation is shown below.

public static boolean checkAlreadyExist(AccountDatabase.AccountName s) {

   Realm realm = Realm.getDefaultInstance();

   RealmQuery<AccountDatabase> query = realm.where(AccountDatabase.class);

   query.equalTo("name", s.toString());

   RealmResults<AccountDatabase> result1 = query.findAll();

   return (result1.size() > 0);

}



public static ArrayList<AccountDatabase.AccountName> getLoggedInAccountsList(){

   ArrayList<AccountDatabase.AccountName> list = new ArrayList<>();

   for (AccountDatabase.AccountName account : AccountDatabase.AccountName.values()){

       if (checkAlreadyExist(account))

           list.add(account);

   }

   return list;

}

Additional changes

There are few accounts which don’t need authentication. Those accounts need their respective application to be installed in the user’s device. So for adding those accounts to the list, we added another function which checks whether a particular package is installed in user’s device or not. Using that it adds the account to the list. The implementation for checking whether a package is installed or not is shown below.

public static boolean isAppInstalled(String packageName, PackageManager pm) {

   boolean installed;

   try {

       pm.getPackageInfo(packageName, PackageManager.GET_ACTIVITIES);

       installed = true;

   } catch (PackageManager.NameNotFoundException e) {

       installed = false;

   }

   return installed;

}

                 

Resources:

Uploaded Images History in Phimpme Android

In Phimpme Android one core feature is of sharing images to many different platforms. After sharing we usually wants to look in the our past records, where we uploaded what pictures? Which image we uploaded? What time it was? So I added a feature to view the upload history of images. User can go to the Upload history tab, present in the navigation drawer of the app. From there he can browse the repository.

How I added history feature in Phimpme

  • Store the data when User initiate an upload

To get which data uploading is in progress. I am storing its name, date, time and image path. When user approve to upload image from Sharing Activity.

Created a database model

public class UploadHistoryRealmModel extends RealmObject{

   String name;
   String pathname;
   String datetime;

   public String getName() {
       return name;
   }

   public void setName(String name) {
       this.name = name;
   }

   public String getPathname() {
       return pathname;
   }

   public void setPathname(String pathname) {
       this.pathname = pathname;
   }

   public String getDatetime() {
       return datetime;
   }

   public void setDatetime(String datetime) {
       this.datetime = datetime;
   }
} 

This is the realm model for storing the name, date, time and image path.

Saving in database

UploadHistoryRealmModel uploadHistory;
uploadHistory = realm.createObject(UploadHistoryRealmModel.class);
uploadHistory.setName(sharableAccountsList.get(position).toString());
uploadHistory.setPathname(saveFilePath);
uploadHistory.setDatetime(new SimpleDateFormat("dd/MM/yyyy HH:mm:ss").format(new Date()));
realm.commitTransaction();

Creating realm object and setting the details in begin and commit Transaction block

  • Added upload history entry in Navigation Drawer

    <LinearLayout
       xmlns:android="http://schemas.android.com/apk/res/android"
       android:id="@+id/ll_drawer_uploadhistory"
       android:layout_width="match_parent"
       android:layout_height="wrap_content"
       android:background="@drawable/ripple"
       android:clickable="true"
       android:orientation="horizontal">
    
       <com.mikepenz.iconics.view.IconicsImageView
           android:id="@+id/Drawer_Upload_Icon"
           android:layout_width="@dimen/icon_width_height"
           android:layout_height="@dimen/icon_width_height"
           app:iiv_icon="gmd-file-upload"/>
    
       <TextView
           android:id="@+id/Drawer_Upload_Item"
           android:layout_width="wrap_content"
           android:layout_height="wrap_content"
           android:text="@string/upload_history"
           android:textColor="@color/md_dark_background"
           android:textSize="16sp"/>
    </LinearLayout>

It consist of an ImageView and TextView in a horizontal oriented Linear Layout

  • Showing history in Upload History Activity

Added recyclerview in layout.

<android.support.v7.widget.RecyclerView
   android:id="@+id/upload_history_recycler_view"
   android:layout_width="match_parent"
   android:layout_height="match_parent"
   android:layout_below="@id/toolbar">
</android.support.v7.widget.RecyclerView>

Query the database and updated the adapter of Upload History

uploadResults = realm.where(UploadHistoryRealmModel.class);
RecyclerView.LayoutManager layoutManager = new LinearLayoutManager(this);
uploadHistoryRecyclerView.setLayoutManager(layoutManager);
uploadHistoryRecyclerView.setAdapter(uploadHistoryAdapter);

Added the adapter for recycler view and created an Item using Constraint layout.

Resources

Export Sensor Data from the PSLab Android App

The PSLab Android App allows users to log data from the sensors connected to the PSLab hardware device. Sensor Data is stored locally but can be exported in various formats. Currently the app supports exporting data in .txt and .csv (comma-separated values) format. Exported data can be used by other users or scientists to study or analyze the data. Data can also be used by other softwares like Python, GNU octave, Matlab to further process it or visualise it in 3D. In this post, we will discuss how to export the locally stored realm data in .txt or .csv format. We will take the data of MPU6050 sensor as an example for understanding how locally logged data is exported.

Query Local Realm Data

We have attached a long click listener to sensor list view that detects which list item is selected. Clicking any sensor from sensor list for slightly longer than usual would result in a dialog popping up with the option to

  • Export Data: Results in exporting data in a format which is selected in App settings
  • Share Data: Shares sensor data with other users or on social media (yet to be implemented)
Source: PSLab Android App

As soon as the Export Data option is selected from the dialog, sensor data of the corresponding sensor is queried. The data model of the sensor and how it’s saved in the local realm database is discussed in the post Sensor Data Logging in the PSLab Android App.

RealmResults<DataMPU6050> results = realm.where(DataMPU6050.class).findAll();

Once we get the required data, we need to write it in .txt or .csv format depending on what the user has selected as a preference in App Settings.

Getting User Preference from App Settings

The format in which the sensor data should be exported is presented to the user as a preference in App Settings. Currently the app supports two formats .txt and .csv.

Source: PSLab Android App
private String format;
SharedPreferences preferences = PreferenceManager.getDefaultSharedPreferences(this);
String formatValue = preferences.getString("export_data_format_list", "0");
if ("0".equals(formatValue))
   format = "txt";
else
   format = "csv";

Export Data in .txt Format

To export the sensor data in .txt format, we need to create a .txt file in the external storage. folder variable is a path to PSLab Android folder in the external storage. If the folder doesn’t exist, it will be created.

File folder = new File(Environment.getExternalStorageDirectory() + File.separator + "PSLab Android");

After getting reference of the app folder in the external storage, we would create a text file in the PSLab Android folder. As soon as the text file is created, we initialize the FileOutputStream object to write data into the text file. The sensor data that was queried in the previous section is written into the text file just created. Finally after the complete sensor data is written, the stream is closed by stream.close() method.

FileOutputStream stream = null;
File file = new File(folder, "sensorData.txt");
try {
   stream = new FileOutputStream(file);
   for (DataMPU6050 temp : results) {
       stream.write((String.valueOf(temp.getAx()) + " " + temp.getAy() + " " + temp.getAz() + " " +
               temp.getGx() + " " + temp.getGy() + " " + temp.getGz() + " " + temp.getTemperature() + "\n").getBytes());
   }
} catch (IOException e) {
   e.printStackTrace();
} finally {
   try {
       if (stream != null) {
           stream.close();
       }
   } catch (IOException e) {
       e.printStackTrace();
   }
}

Export Data in .csv Format

Writing data in .csv format is similar to that in .txt format. As CSV stands for Comma Separated Values, which means each data value is separated by “,” (comma). It is similar to an excel sheet. The first row consists of labels that denote the type of value in that particular column. The other rows consist of the sensor data, with each row corresponding to a sample of the sensor data.

File file = new File(folder, "sensorData.csv");
PrintWriter writer;
try {
   writer = new PrintWriter(file);
   StringBuilder stringBuilder = new StringBuilder();
   stringBuilder.append("Ax,Ay,Ax,Gx,Gy,Gz,Temperature\n");
   for (DataMPU6050 temp : results) {
       stringBuilder.append(String.valueOf(temp.getAx()));
       stringBuilder.append(',');
       stringBuilder.append(String.valueOf(temp.getAy()));
       stringBuilder.append(',');
       stringBuilder.append(String.valueOf(temp.getAz()));
       stringBuilder.append(',');
       stringBuilder.append(String.valueOf(temp.getGx()));
       stringBuilder.append(',');
       stringBuilder.append(String.valueOf(temp.getGy()));
       stringBuilder.append(',');
       stringBuilder.append(String.valueOf(temp.getGz()));
       stringBuilder.append(',');
       stringBuilder.append(String.valueOf(temp.getTemperature()));
       stringBuilder.append('\n');
   }
   writer.write(stringBuilder.toString());
   writer.close();
} catch (FileNotFoundException e) {
   e.printStackTrace();
}

Resources

Sensor Data Logging in the PSLab Android App

The PSLab Android App allows users to log data from sensors connected to the PSLab hardware device. The Connected sensors should support I2C, SPI communication protocols to communicate with the PSLab device successfully. The only prerequisite is the additional support for the particular sensor plugin in Android App. The user can log data from various sensors and measure parameters like temperature, humidity, acceleration, magnetic field, etc. These parameters are useful in predicting and monitoring the environment and in performing many experiments.

The support for the sensor plugins was added during the porting python communication library code to Java. In this post,  we will discuss how we logged real time sensor data from the PSLab Hardware Device. We used Realm database to store the sensor data locally. We have taken the MPU6050 sensor as an example to understand the complete process of logging sensor data.

Creating Realm Object for MPU6050 Sensor Data

The MPU6050 sensor gives the acceleration and gyroscope readings along the three axes X, Y and Z. So the data object storing the readings of the mpu sensor have variables to store the acceleration and gyroscope readings along all three axes.

public class DataMPU6050 extends RealmObject {

   private double ax, ay, az;
   private double gx, gy, gz;
   private double temperature;

   public DataMPU6050() {  }

   public DataMPU6050(double ax, double ay, double az, double gx, double gy, double gz, double temperature) {
       this.ax = ax;
       this.ay = ay;
       this.az = az;
       this.gx = gx;
       this.gy = gy;
       this.gz = gz;
       this.temperature = temperature;
   }

  // getter and setter for all variables
}

Creating Runnable to Start/Stop Data Logging

To sample the sensor data at 500ms interval, we created a runnable object and passed it to another thread which would prevent lagging of the UI thread. We can start/stop logging by changing the value of the boolean loggingThreadRunning on button click. TaskMPU6050 is an AsyncTask which reads each sample of sensor data from the PSLab device, it gets executed inside a while loop which is controlled by boolean loggingThreadRunning. Thread.sleep(500) pauses the thread for 500ms, this is also one of the reason to transfer the logging to another thread instead of logging the sensor data in UI thread. If such 500ms delays are incorporated in UI thread, app experience won’t be smooth for the users.

Runnable loggingRunnable = new Runnable() {
   @Override
   public void run() {
       try {
           MPU6050 sensorMPU6050 = new MPU6050(i2c);
           while (loggingThreadRunning) {
               TaskMPU6050 taskMPU6050 = new TaskMPU6050(sensorMPU6050);
               taskMPU6050.execute();
              // use lock object to synchronize threads
               Thread.sleep(500);
           }
       } catch (IOException   InterruptedException e) {
           e.printStackTrace();
       }
   }
};

Sampling of Sensor Data

We created an AsyncTask to read each sample of the sensor data from the PSLab device in the background thread. The getRaw() method read raw values from the sensor and returned an ArrayList containing the acceleration and gyro values. After the values were read successfully, they were updated in the data card in the foreground which was visible to the user. This data card acts as a real-time screen for the user. All the samples read are appended to ArrayList mpu6050DataList, when the user clicks on button Save Data, the collected samples are saved to the local realm database.

private ArrayList<DataMPU6050> mpu6050DataList = new ArrayList<>();

private class TaskMPU6050 extends AsyncTask<Void, Void, Void> {

   private MPU6050 sensorMPU6050;
   private ArrayList<Double> dataMPU6050 = new ArrayList<>();

   TaskMPU6050(MPU6050 mpu6050) {
       this.sensorMPU6050 = mpu6050;
   }

   @Override
   protected Void doInBackground(Void... params) {
       try {
           dataMPU6050 = sensorMPU6050.getRaw();
       } catch (IOException e) {
           e.printStackTrace();
       }
       return null;
   }

   @Override
   protected void onPostExecute(Void aVoid) {
       super.onPostExecute(aVoid);
       // update data card TextViews with data read.
       DataMPU6050 tempObject = new DataMPU6050(dataMPU6050.get(0), dataMPU6050.get(1), dataMPU6050.get(2),
               dataMPU6050.get(4), dataMPU6050.get(5), dataMPU6050.get(6), dataMPU6050.get(3));
       mpu6050DataList.add(tempObject);
       synchronized (lock) {
           lock.notify();
       }
   }
}
Source: PSLab Android App

There is an option for Start/Stop Logging, clicking on which will change the value of boolean loggingThreadRunning which stops starts/stops the logging thread.

When the Save Data button is clicked, all the samples of sensor data collected from the  PSLab device till that point are saved to the local realm database.

realm.beginTransaction();
for (DataMPU6050 tempObject : mpu6050DataList) {
   realm.copyToRealm(tempObject);
}
realm.commitTransaction();

Data can also be written asynchronously to the local realm database. For other methods to write to a real database refer write section of Realm docs.

Resources

Save Chat Messages using Realm in SUSI iOS

Fetching data from the server each time causes a network load which makes the app depend on the server and the network in order to display data. We use an offline database to store chat messages so that we can show messages to the user even if network is not present which makes the user experience better. Realm is used as a data storage solution due to its ease of usability and also, since it’s faster and more efficient to use. So in order to save messages received from the server locally in a database in SUSI iOS, we are using Realm and the reasons for using the same are mentioned below.

The major upsides of Realm are:

  • It’s absolutely free of charge,
  • Fast, and easy to use.
  • Unlimited use.
  • Work on its own persistence engine for speed and performance

Below are the steps to install and use Realm in the iOS Client:

Installation:

  • Install Cocoapods
  • Run `pod repo update` in the root folder
  • In your Podfile, add use_frameworks! and pod ‘RealmSwift’ to your main and test targets.
  • From the command line run `pod install`
  • Use the `.xcworkspace` file generated by Cocoapods in the project folder alongside `.xcodeproj` file

After installation we start by importing `Realm` in the `AppDelegate` file and start configuring Realm as below:

func initializeRealm() {
        var config = Realm.Configuration(schemaVersion: 1,
            migrationBlock: { _, oldSchemaVersion in
                if (oldSchemaVersion < 0) {
                    // Nothing to do!
                }
        })
        config.fileURL = config.fileURL?.deletingLastPathComponent().appendingPathComponent("susi.realm")
        Realm.Configuration.defaultConfiguration = config
}

Next, let’s head over to creating a few models which will be used to save the data to the DB as well as help retrieving that data so that it can be easily used. Since Susi server has a number of action types, we will cover some of the action types, their model and how they are used to store and retrieve data. Below are the currently available data types, that the server supports.

enum ActionType: String {
  case answer
  case websearch
  case rss
  case table
  case map 
  case anchor
}

Let’s start with the creation of the base model called `Message`. To make it a RealmObject, we import `RealmSwift` and inherit from `Object`

class Message: Object {
  dynamic var queryDate = NSDate()
  dynamic var answerDate = NSDate()
  dynamic var message: String = ""
  dynamic var fromUser = true
  dynamic var actionType = ActionType.answer.rawValue
  dynamic var answerData: AnswerAction?
  dynamic var mapData: MapAction?
  dynamic var anchorData: AnchorAction?
}

Let’s study these properties of the message one by one.

  • `queryDate`: saves the date-time the query was made
  • `answerDate`: saves the date-time the query response was received
  • `message`: stores the query/message that was sent to the server
  • `fromUser`: a boolean which keeps track who created the message
  • `actionType`: stores the action type
  • `answerData`, `rssData`, `mapData`, `anchorData` are the data objects that actually store the respective action’s data

To initialize this object, we need to create a method that takes input the data received from the server.

// saves query and answer date
if let queryDate = data[Client.ChatKeys.QueryDate] as? String,
let answerDate = data[Client.ChatKeys.AnswerDate] as? String {
  message.queryDate = dateFormatter.date(from: queryDate)! as NSDate
  message.answerDate = dateFormatter.date(from: answerDate)! as NSDate}if let type = action[Client.ChatKeys.ResponseType] as? String,
  let data = answers[0][Client.ChatKeys.Data] as? [[String : AnyObject]] {
  if type == ActionType.answer.rawValue {
     message.message = action[Client.ChatKeys.Expression] as! String
     message.actionType = ActionType.answer.rawValue
    message.answerData = AnswerAction(action: action)
  } else if type == ActionType.map.rawValue {
    message.actionType = ActionType.map.rawValue
    message.mapData = MapAction(action: action)
  } else if type == ActionType.anchor.rawValue {
    message.actionType = ActionType.anchor.rawValue
    message.anchorData = AnchorAction(action: action)
    message.message = message.anchorData!.text
  }
}

Since, the response from the server for a particular query might contain numerous action types, we create loop inside a method to capture all those action types and save each one of them. Since, there are multiple action types thus we need a list containing all the messages created for the action types. For each action in the loop, corresponding data is saved into their specific objects.

Let’s discuss the individual action objects now.

  • AnswerAction
class AnswerAction: Object {
  dynamic var expression: String = ""
  convenience init(action: [String : AnyObject]) {
    self.init()
    if let expression = action[Client.ChatKeys.Expression] as? String {
      self.expression = expression
    }
  }
}

 This is the simplest action type implementation. It contains a single property `expression` which is a string type. For initializing it, we take the action object and extract the expression key-value and save it.

if type == ActionType.answer.rawValue {
  message.message = action[Client.ChatKeys.Expression] as! String
  message.actionType = ActionType.answer.rawValue
  // pass action object and save data in `answerData`
  message.answerData = AnswerAction(action: action)
}

Above is the way an answer action is checked and data saved inside the `answerData` variable.

2)   MapAction

class MapAction: Object {
  dynamic var latitude: Double = 0.0
  dynamic var longitude: Double = 0.0
  dynamic var zoom: Int = 13

  convenience init(action: [String : AnyObject]) {
    self.init()
    if let latitude = action[Client.ChatKeys.Latitude] as? String,
    let longitude = action[Client.ChatKeys.Longitude] as? String,
    let zoom = action[Client.ChatKeys.Zoom] as? String {
      self.longitude = Double(longitude)!
      self.latitude = Double(latitude)!
      self.zoom = Int(zoom)!
    }
  }
}

This action implementation contains three properties, `latitude` `longitude` `zoom`. Since the server responds the values inside a string, each of them need to be converted to their respective type using force-casting. Default values are provided for each property in case some illegal value comes from the server.

3)   AnchorAction

class AnchorAction: Object {
  dynamic var link: String = ""
  dynamic var text: String = ""

  convenience init(action: [String : AnyObject]) {
    self.init()if let link = action[Client.ChatKeys.Link] as? String,
    let text = action[Client.ChatKeys.Text] as? String {
      self.link = link
      self.text = text
    }
  }
}

Here, the link to the openstreetmap website is saved in order to retrieve the image for displaying.

Finally, we need to call the API and create the message object and use the `write` clock of a realm instance to save it into the DB.

if success {
  self.collectionView?.performBatchUpdates({
    for message in messages! {
    // real write block
      try! self.realm.write {
        self.realm.add(message)
        self.messages.append(message)
        let indexPath = IndexPath(item: self.messages.count - 1, section: 0)
        self.collectionView?.insertItems(at: [indexPath])
      }
   }
}, completion: { (_) in
    self.scrollToLast()
  })
}

list of message items and inserted into the collection view.Below is the output of the Realm Browser which is a UI for viewing the database.

References:

Realm database in Loklak Wok Android for Persistent view

Loklak Wok Android provides suggestions for tweet searches. The suggestions are stored in local database to provide a persistent view, resulting in a better user experience. The local database used here is Realm database instead of sqlite3 which is supported by Android SDK. The proper way to use an sqlite3 database is to first create a contract where the schema of the database is defined, then a database helper class which extends from SQLiteOpenHelper class where the schema is created i.e. tables are created and finally write ContentProvider so that you don’t have to write long SQL queries every time a database operation needs to be performed. This is just a lot of hard work to do, as this includes a lot of steps, debugging is also difficult. A solution to this can be using an ORM that provides a simple API to use sqlite3, but the currently available ORMs lack in terms of performance, they are too slow. A reliable solution to this problem is realm database, which is faster than raw sqlite3 and has really simple API for database operations. This blog explains the use of realm database for storing tweet search suggestions.

Adding Realm database to Android project

In project level build.gradle

buildscript {
   repositories {
       jcenter()
   }
   dependencies {
       classpath 'com.android.tools.build:gradle:2.3.3'
       classpath "io.realm:realm-gradle-plugin:3.3.1"

       // NOTE: Do not place your application dependencies here; they belong
       // in the individual module build.gradle files
   }
}

 

And at the top of app/build.gradle “apply plugin: ‘realm-android'”  is added.

Using Realm Database

Let’s start with a simple example. We have a Student class that has only two attributes name and age. To create the model for the database, the Student class is simply extended to RealmObject.

public class Student extends RealmObject {

   private String name;
   private int age;

   // A constructor needs to be explicitly defined, be it an empty constructor
   public Student(String name, int age) {
       this.name = name;
       this.age = age;
   }

   // getters and setters
}

 

To push data to the database, Java objects are created, a transaction is initialized, then copyToRealm method is used to push the data and finally the transaction is committed. But before all this, the database is initialized and a Realm instance is obtained.

Realm.init(context); // Database initialized
Realm realm = Realm.getDefaultInstance(); // realm instance obtained
      
Student student = new Student("Rahul Dravid", 22); // Simple java object created
realm.beginTransaction() // initialization of transaction
realm.copyToRealm(student); // pushed to database
realm.commitTransaction(); // transaction committed

 

copyToRealm takes only a single parameter, the parameter can be an object or an Iterable. Off course, the passed parameter should extend RealmObject. A List of Student can be passed as a parameter to copyToRealm to push multiple data into the database.

The above way of inserting data is synchronous. Realm also supports asynchronous transactions, you guessed it right, you don’t have to depend on AsyncTaskLoader. The same operation can be performed asynchronously as

realm.executeTransaction(new Realm.Transaction() { 
    @Override public void execute(Realm realm) {
        Student student = new Student("Rahul Dravid", 22);
        realm.copyToRealm(student);
    }
});

 

Now, querying the database is as easy as inserting.

RealmResults<Student> studentList = realm.where(Student.class).findAll();

 

No, transaction is required as we are not manipulating the database, as data is just read from the database. RealmResults extend Java List, so List methods which don’t manipulate the List can be used on studentList e.g. get(int index) to obtain object at the index.

The result can also be a filtered one, for example filtering students who are 22 years old.

RealmResults<Student> studentList = realm.where(Student.class).equalTo(age, 22).findAll();

 

Now, removing data from database. deleteAllFromRealm method can be executed on the obtained RealmResults or to completely remove data of a model class delete(Model.class) method on the realm instance is invoked. The operations should be enclosed between beginTransaction and commitTransaction if synchronous behaviour is required else for asynchronous behaviour the operation is done in execute method of an anonymous object of Realm.Transaction.

studentList.deleteAllFromRealm(); // removes the filtered result
realm.delete(Student.class); // removes all data of model class

 

Storing Tweet Search Suggestions in Loklak Wok Android for Persistent view

Loklak Wok Android uses Retrofit2 for sending network requests, for which POJO classes are already created so that it becomes easy for parsing the obtained JSON from network request. Due to this using Realm database becomes more easier, as the defined POJOs can be simply extended to RealmObject to create the model class of the data e.g. Query class extends RealmObject, one of the attribute is the suggestion query i.e. mQuery.

The database is initialized in LoklakWokApplication, the application class, this way the database is initialized only once which persists throughout the app lifecycle.

@Override
public void onCreate() {
   super.onCreate();
   Realm.init(this);
   RealmConfiguration realmConfiguration = new RealmConfiguration.Builder()
           .name(Realm.DEFAULT_REALM_NAME)
           .deleteRealmIfMigrationNeeded()
           .build();
   Realm.setDefaultConfiguration(realmConfiguration);
}

 

deleteRealmIfMigrationNeeded removes the old realm database and creates a new one if any of the Model class get changed i.e. an attribute is removed, added or simply the name of attribute is changed. This is done as we are not storing user generated data, we are just using database to provide persistent view. So, the previously kept data is not important.

The database is closed in onTerminate callback of the application

@Override
public void onTerminate() {
   Realm.getDefaultInstance().close();
   super.onTerminate();
}

 

Now that database is initialized. We fetch the previously stored data and display it in RecyclerView. If the network request is successful the queries from database are replaced by the queries fetched in network request, else the queries from database are displayed providing a persistent view. The way it is implemented in onCreateView of SuggestFragment

mRealm = Realm.getDefaultInstance();
...
// old queries obtained from database
RealmResults<Query> queryRealmResults = mRealm.where(Query.class).findAll();
List<Query> queries = mRealm.copyFromRealm(queryRealmResults);
// RecyclerView adapter created with old queries
mSuggestAdapter = new SuggestAdapter(queries, this);
tweetSearchSuggestions.setLayoutManager(new LinearLayoutManager(getActivity()));
tweetSearchSuggestions.setAdapter(mSuggestAdapter);

 

The old queries are replaced in onSuccessfulRequest

private void onSuccessfulRequest(SuggestData suggestData) {
   // suggestData contains suggestion queries
   if (suggestData != null) {
       // old queries replaced with new ones
       mSuggestAdapter.setQueries(suggestData.getQueries());
   }
   setAfterRefreshingState();
}

 

Now suggestion queries needs to be inserted into the database. Only the latest suggestions are inserted i.e. queries present when onStop lifecycle method of fragment is called, and as previous queries are not needed anymore, they are deleted. The operation is performed in a synchronous way.

@Override
public void onStop() {
   ...
   mRealm.beginTransaction();
   // old queries deleted
   mRealm.delete(Query.class);
   // new queries inserted
   mRealm.copyToRealm(mSuggestAdapter.getQueries());
   mRealm.commitTransaction();
   // fragment lifecycle called i.e. a new fragment/activity opens
   super.onStop();
}

 

Conclusion: Sqlite3 and Realm comparison

Operations Sqlite3 Realm
Table creation CREATE TABLE … extends RealmObject
Inserting data INSERT INTO … copyToRealm
Searching data SELECT … realm.where(Model.class)
Deleting data DELETE FROM … realmResults.deleteAllFromRealm() or realm.delete(Model.class)

Resources:

 

 

Selecting Best persistent storage for Phimpme Android and how to use it

As we are progressing in our Phimpme Android app. I added account manager part which deals with connecting all other accounts to phimpme. Showing a list of connected accounts.

We need a persistent storage to store all the details such as username, full name, profile image url, access token (to access API). I researched on various Object Relation mapping (ORMs) such as:

  1. DBFlow: https://github.com/Raizlabs/DBFlow
  2. GreenDAO: https://github.com/greenrobot/greenDAO
  3. SugarORM: http://satyan.github.io/sugar/
  4. Requery: https://github.com/requery/requery

and other NoSQL databases such as Realm Database : https://github.com/realm/realm-java.

After reading a lot from some blogs on the benchmarking of these ORMs and database, I came to know that Realm database is quite better in terms of Speed of writing data and ease of use.

Steps to integrate Realm Database:

  • Installation of Realm database in android

Following these steps https://realm.io/docs/java/latest/#installation quickly setup realm in android. Add

classpath "io.realm:realm-gradle-plugin:3.3.2"

in Project level build.gradle file and Add

apply plugin: 'realm-android' 

in app level build.gradle, That’s it for using Realm

  • Generating required Realm models

Firstly, make sure what you need to store in your database. In case of phimpme, I first go through the account section and noted down what needs to be there.  Profile image URL, username, full name, account indicator image name. Below image illustrate this better.

This is the Realm Model class I made in Kotlin to store name, username and access token for accessing API.

open class AccountDatabase(
       @PrimaryKey var name: String = "",
       var username: String = "",
       var token: String = ""
) : RealmObject()

  • Writing data in database

In Account manager, I create a add account option from where a dialog appear with a list of accounts. Currently, Twitter is working, when onSuccess function invoke in AccountPickerFragment I start a twitter session and store values in database. Writing data in database:

// Begin realm transaction
realm.beginTransaction();

// Creating Realm object for AccountDatabase Class
account = realm.createObject(AccountDatabase.class,
       accountsList[0]);

account.setUsername(session.getUserName());
account.setToken(String.valueOf(session.getAuthToken()));
realm.commitTransaction();

Begin and commit block in necessary. There is one more way of doing this is using execute function in Realm

  • Use Separate Database Helper class for Database operations

It’s good to use a separate class for all the Database operations needed in the project. I created a DatabaseHelper Class and added a function to query the result needed. Query the database

public RealmResults<AccountDatabase> fetchAccountDetails(){
   return realm.where(AccountDatabase.class).findAll();
}

It give all of the results, stored in the database like below

  • Problems I faced with annotation processor while using Kotlin and Realm together

The Kotlin annotation processor not running due to the plugins wrong order. This issue https://github.com/realm/realm-java/pull/2568 helped me in solving that. I addded apply plugin: ‘kotlin-kapt’. In app gradle file and shift apply plugin: ‘realm-android’ In below the order.

Resources:

 

Datewise splitting of the Bookmarks in the Homescreen of Open Event Android

In the Open Event Android app we had already incorporated bookmarks in the homescreen along with an improved UI. Now there was scope for further improvement in terms of user experience. The bookmarks were already sorted date wise but we needed to place them under separate date headers. In this blog I will be talking about how this was done in the app.

Initial Design
Current Design

 

 

 

 

 

 

 

 

Initially the user had no way of knowing which session belonged to which day. This could be fixed with a simple addition of a header indicating the day each bookmark belonged to. One way to do this was to add a day header and then get the bookmarks for each day and so on. But this proved to be difficult owing to the fact the number of days could be dynamic owing to the fact that this is a generic app. Another issue was that adding change listeners for the realm results to the bookmarks list for each day produced view duplication and other unexpected results whenever the bookmark list changed. So another approach was chosen that was to get all the bookmarks first and then add the date header and traverse through the bookmarks and only add sessions which belong to the date for which the date header was added earlier.

Bookmark Item Support in GlobalSearchAdapter

The main reason why we are reusing the GlobalSearchAdapter is that we have already defined a DIVIDER type in this adapter which can be reused as the date header.

We needed to initialize a constant for the Bookmark type.

private final int BOOKMARK = 5; //Can be any number

Then we add the Bookmark type in the getItemViewType() function which would return a constant that we defined earlier to indicate that in the filteredResultList we have an object of type Bookmark.

@Override
 public int getItemViewType(int position) {
    if (filteredResultList.get(position) instanceof Track) {
        return TRACK;
    }

    //Other Cases here
    } else if(filteredResultList.get(position) instanceof Session){
        return BOOKMARK;
    } else {
        return 1;
    }
 }

Now we create the viewholder if the list item is of the type Session which in this case will be a bookmark.

@Override
 public RecyclerView.ViewHolder onCreateViewHolder(ViewGroup parent, int viewType) {
    RecyclerView.ViewHolder resultHolder = null;
    LayoutInflater inflater = LayoutInflater.from(parent.getContext());
    //Other cases for Track,Location etc
 case BOOKMARK:
    View bookmark = inflater.inflate(R.layout.item_schedule, parent, false);
    resultHolder = new DayScheduleViewHolder(bookmark,context);
    break;
   //Some code
 
 }

Now we do the same in onBindViewHolder(). We bind the contents of the object to the ViewHolder here by calling the bindSession() function. We also pass in an argument which is our database repository i.e realmRepo here.

@Override
 public void onBindViewHolder(RecyclerView.ViewHolder holder, int position) {

    switch (holder.getItemViewType()) {
     //Other cases handled here

case BOOKMARK:
    DayScheduleViewHolder bookmarkTypeViewHolder = (DayScheduleViewHolder) holder;
    Session bookmarkItem = (Session) getItem(position);
    bookmarkTypeViewHolder.setSession(bookmarkItem);
    bookmarkTypeViewHolder.bindSession(realmRepo);
    break;
 }

Updating the AboutFragment

private GlobalSearchAdapter bookMarksListAdapter;
 private List<Object> mSessions = new ArrayList<>();

Earlier the DayScheduleAdapter was being used to display the list of bookmarks. Now we are reusing the GlobalSearchAdapter. Now we have also converted mSessions into a list of objects from a list of sessions.

Now we initialize the adapter so that we can start adding our date headers.

bookMarksListAdapter = new GlobalSearchAdapter(mSessions, getContext());
 bookmarksRecyclerView.setAdapter(bookMarksListAdapter);

In this function loadEventDates() we are storing the all the dates for the event. For example the list for the FOSSASIA17 sample stores the dates in the dateList as [2017-03-17,2017-03-18,2017-03-19]. We fetch the event dates by calling the getEventDateSync() function which has been defined in our Realm Database.

private void loadEventDates() {
 
    dateList.clear();
    RealmResults<EventDates> eventDates = realmRepo.getEventDatesSync();
    for (EventDates eventDate : eventDates) {
        dateList.add(eventDate.getDate());
    }
 }

Now we move on to the core logic of the feature which is to get the date headers to work correctly.

  • Fetch the list of bookmarks from the local Realm database asynchronously.
  • Remove any existing changeListeners to the bookmarkResult.
  • Add a changeListener to our list of results to notify us of the completion of the query or changes in the bookmark list.
  • After this is done, inside the changeListener we first clear the mSessions
  • We now traverse through our date list and compare it with the session startDate which we can obtain by calling the getStartDate(). If the date match occurs for the first time we add a date header after converting the date string into another format using the DateUtils class. So the function formatDay() of DateUtils converts 2017-03-17 to 17 Mar. This format is easily more readable.
  • Repeat for all dates.
private void loadData() {
    loadEventDates();
 
    bookmarksResult = realmRepo.getBookMarkedSessions();
    bookmarksResult.removeAllChangeListeners();
    bookmarksResult.addChangeListener((bookmarked, orderedCollectionInnerChangeSet) -> {
 
        mSessions.clear();
        for (String eventDate : dateList) {
            boolean headerCheck = false;
            for(Session bookmarkedSession : bookmarked){
                if(bookmarkedSession.getStartDate().equals(eventDate)){
                    if(!headerCheck){
                        String headerDate = "Invalid";
                      try {
                       headerDate = DateUtils.formatDay(eventDate);
                      }
                      catch (ParseException e){
                            e.printStackTrace();
                      }
                        mSessions.add(headerDate);
                        headerCheck = true;
                    }
                    mSessions.add(bookmarkedSession);
                }
            }
            bookMarksListAdapter.notifyDataSetChanged();
            handleVisibility();
        }
    });
 }

So, this is how the date-wise organization for the bookmarks in the homescreen was done.

Resources

Addition of Bookmarks to the Homescreen in the Open Event Android App

In the Open Event Android app we had already built the new homescreen but the users only had access to bookmarks in a separate page which could be accessed from the navbar.If the bookmarks section were to be incorporated in the homescreen itself, it would definitely improve its access to the user. In this blog post, I’ll be talking about how this was done in the app.

These 2 images show the homescreen and the bookmarks section respectively.

No Bookmark View
Bookmark View

 

 

 

 

 

 

 

 

 

This was the proposed homescreen page for the app. This would provide easy access to important stuff to the user such as event venue,date,description etc. Also the same homescreen would also have the bookmarks showing at the top if there are any.

The list of bookmarks in the first iteration of design was modeled to be a horizontal list of cards.

Bookmarks Merging Process

These are some variables for reference.

private SessionsListAdapter sessionsListAdapter;
 private RealmResults<Session> bookmarksResult;
 private List<Session> mSessions = new ArrayList<>();

The code snippet below highlights the initial setup of the bookmarks recycler view for the horizontal List of cards. All of this is being done in the onCreateView callback of the AboutFragment.java file which is the fragment file for the homescreen.

bookmarksRecyclerView.setVisibility(View.VISIBLE);
 sessionsListAdapter = new SessionsListAdapter(getContext(), mSessions, bookmarkedSessionList);
 sessionsListAdapter.setBookmarkView(true);
 bookmarksRecyclerView.setAdapter(sessionsListAdapter);
 bookmarksRecyclerView.setLayoutManager(new LinearLayoutManager(getContext(),LinearLayoutManager.HORIZONTAL,false));

The SessionListAdapter is an adapter that was built to handle multiple types of displays of the same viewholder i.e SessionViewHolder . This SessionListAdapter is given a static variable as an argument which is just notifies the adapter to switch to the bookmarks mode for the adapter.

private void loadData() {
    bookmarksResult = realmRepo.getBookMarkedSessions();
    bookmarksResult.removeAllChangeListeners();
    bookmarksResult.addChangeListener((bookmarked, orderedCollectionChangeSet) -> {
        mSessions.clear();
        mSessions.addAll(bookmarked);
 
        sessionsListAdapter.notifyDataSetChanged();
 
        handleVisibility();
    });
 }

This function loadData() is responsible for extracting the sessions that are bookmarked from the local Realm database. We the update the BookmarkAdapter on the homescreen with the list of the bookmarks obtained. Here we see that a ChangeListener is being attached to our RealmResults. This is being done so that we do our adapter notify only after the data of the bookmarked sessions has been processed from a background thread.

if(bookmarksResult != null)
    bookmarksResult.removeAllChangeListeners();

And it is good practice to remove any ChangeListeners that we attach during the fragment life cycle in the onStop() method to avoid memory leaks.

So now we have successfully added bookmarks to the homescreen.

Resources