Performing Database Migrations using DbFlow

In Open Event Organizer Android App we decided to add database migrations for every change in database while development. Two of the reasons behind this –

  1. The users have some version of the app installed and then during development the developers had to modify the database, let’s say, add a table or modify existing ones. This makes the existing database incompatible with the new app. On adding database migration the upgradation of database takes place automatically and prevent the situation of reinstalling the app.
  2. Migrations makes it possible to rollback or upgrade to some particular database state. Thus, help in debugging certain changes in isolation.

Let’s discuss the implementation details. Consider the scenario when a user adds a new table named SpeakersCall. For creating migration for this change we need to add migration class inside OrgaDatabase class annotated with @Database. We will break it down to look closely at each step.

  1. Use @Migration annotation in DbFlow library and specify the new database version (by incrementing the existing version) and the database class.
  2. Extend BaseMigration and override migrate method.
  3. The logic used inside the migrate method can be different for different tasks. In the present case we first need to delete any existing table (if exists) with the name SpeakersCall and then recreate that table in database.
  4. Create an array of java classes which are created/modified.
  5. We wrap the SQL query inside a Database Wrapper class which prevents it from running recursively.
  6. FlowManager uses reflection to look up and construct the generated database holder class used in defining the structure for all databases used in this application. So we will getModelAdapter for the class to be recreated and use creation query returned by Model Adapter and execute it.
@Migration(version = 15, database = OrgaDatabase.class)
public static class MigrationTo15 extends BaseMigration {

@Override
public void migrate(@NonNull DatabaseWrapper databaseWrapper) {
Timber.d(“Running migration for DB version 14”);

Class<?>[] recreated = new Class[] {SpeakersCall.class};

for (Class<?> recreate: recreated) {
ModelAdapter modelAdapter = FlowManager.getModelAdapter(recreate);
databaseWrapper.execSQL(DROP_TABLE + modelAdapter.getTableName());
databaseWrapper.execSQL(modelAdapter.getCreationQuery());
}
}
}

Similarly, we can write migration for changing a column of table(s).

Continue ReadingPerforming Database Migrations using DbFlow

Option to exclude albums in Phimpme Android Application

In the Phimpme Android Application, users can perform various operations on the albums available such as move, creating a .zip file of the album, rename an album, delete the album and much more. However one another important functionality that has been implemented in the application is the option to exclude album/albums. In this post we will be discussing how we achieved the functionality to exclude albums in Phimpme Android Application.

Step 1

First we need to keep track of the albums selected by the user to exclude. This can be done by storing the selected albums in an Arraylist<Album> which can be referred later when required for the process of exclusion. The storing of the albums can be done with the help of following lines of code.

private int toggleSelectAlbum(int index) {
if (dispAlbums.get(index) != null) {
  dispAlbums.get(index).setSelected(!dispAlbums.get(index).isSelected());
  if (dispAlbums.get(index).isSelected()) selectedAlbums.add(dispAlbums.get(index));
  else selectedAlbums.remove(dispAlbums.get(index));
}
return index;
}

Step 2

After the selected albums are stored in an Arraylist<Album>, a function call of excludeSelectedAlbums() of HandlingAlbums class is done passing in Context as the parameter. In the method excludeSelectedAlbums() the selected albums are retrieved from the Arraylist one by one and another method excludeAlbum() is called with the album, context being passed as the parameters. The code snippet performing the above operation is provided below.

public void excludeSelectedAlbums(Context context) {
for (Album selectedAlbum : selectedAlbums)
  excludeAlbum(context, selectedAlbum);
clearSelectedAlbums();
}

Step 3

Thereafter an instance of class CustomsAlbumHelper is created and excludeAlbum() method is called with path of the album passed as parameter, and the selected album is removed from the list containing the currently displayed albums. Code snippets for the above operations are provided below.

private void excludeAlbum(Context context, Album a) {
CustomAlbumsHelper h = CustomAlbumsHelper.getInstance(context);
h.excludeAlbum(a.getPath());
dispAlbums.remove(a);
}

Step 4

Now in the excludeAlbum() method of CustomsAlbumHelper class a writable instance of albums_settings sqlite database is obtained. A check is performed if the current album is present in the albums table or not, if not then a row representing the current album is created. Thereafter the excluded value for the current album in the table is updated to integer 1 denoting that the album is excluded. The code snippet required for the above mentioned operations are provided below.

public void excludeAlbum(String path) {
  SQLiteDatabase db = this.getWritableDatabase();
  checkAndCreateAlbum(db, path);
  ContentValues values = new ContentValues();
  values.put(ALBUM_EXCLUDED, 1);
  db.update(TABLE_ALBUMS, values, ALBUM_PATH+“=?”, new String[]{    path});
  db.close();
}

This is how we have achieved the functionality of excluding albums in the Phimpme Android application. To get the full source code, please refer to the Phimpme Android GitHub repository listed in the resources section below.

The screenshot for the display of the excluded albums is provided below.

Resources

  1. Android Developer Guide –https://developer.android.com/training/data-storage/sqlite.html
  1. Github-Phimpme Android Repository –https://github.com/fossasia/phimpme-android/
  1. Sqlite Operations Tutorial – https://www.androidhive.info/2011/11/android-sqlite-database-tutorial/
Continue ReadingOption to exclude albums in Phimpme Android Application

Persistence Layer in Open Event Organizer Android App

Open Event Organizer is an Event Managing Android App with the core features of Attendee Check In by QR Code Scan and Data Sync with the Open Event API Server. As an event can be large, so the app will be dealing with a large amount of a data. Hence to avoid repetitive network requests for fetching the data, the app maintains a local database containing all the required data and the database is synced with the server. Android provides android.database.sqlite package which contains the API needed to use the database on the Android. But it is really not a good practice to use the sqlite queries everywhere in the app. So there comes a persistence layer. A persistence layer works between the database and the business logic. Open Event Organizer uses Raizlabs’s DbFlow, an ORM based Android Database Library for the same. I will be talking about its implementation through the app in this blog.

First of all, you declare the base class of the database which is used to create the database by Android for the app. You declare all the base constants here. The class looks like:

@Database(
   name = OrgaDatabase.NAME,
   version = OrgaDatabase.VERSION,
   ...
)
public class OrgaDatabase {
   public static final String NAME = "orga_database";
   public static final int VERSION = 2;
   ...
}

OrgaDatabase.java
app/src/main/java/org/fossasia/openevent/app/data/db/configuration/OrgaDatabase.java

Initialise the database in the Application class using FlowManager provided by the library. Choose the Application class to do this to ensure that the library finds the generated code in the DbFlow.

FlowManager.init(
   new FlowConfig.Builder(context)
       .addDatabaseConfig(
           new DatabaseConfig.Builder(OrgaDatabase.class)
           ...
           .build()
       )
       .build());

OrgaApplication.java
app/src/main/java/org/fossasia/openevent/app/OrgaApplication.java

The database is created now. For tables creation, DbFlow uses model classes which must be annotated using the annotations provided by the library. The basic annotations are – @Table, @PrimaryKey, @Column, @ForeignKey etc.

For example, the Attendee class in the app looks like:

@Table(database = OrgaDatabase.class)
public class Attendee ... {

   @PrimaryKey
   public long id;

   @Column
   public boolean checkedIn;
   ...
   ...
   @ForeignKey(
       onDelete = ForeignKeyAction.CASCADE,
       onUpdate = ForeignKeyAction.CASCADE)
   public Order order;
   ...
}

Attendee.java
app/src/main/java/org/fossasia/openevent/app/data/models/Attendee.java

This will create a table named attendee with the columns and relationships annotated. Now comes the part of accessing data from the database. Open Event App uses RxJava’s support to the DbFlow library which enables async data accessing. The getItems method from DataBaseRepository looks like:

public <T> Observable<T> getItems(Class<T> typeClass, SQLOperator... conditions) {
   return RXSQLite.rx(SQLite.select()
       .from(typeClass)
       .where(conditions))
       .queryList()
       .flattenAsObservable(items -> items);
}

 

The method returns an observable emitting the items from the result. For data saving, the method looks like:

DatabaseDefinition database = FlowManager.getDatabase(OrgaDatabase.class);
FastStoreModelTransaction<T> transaction = FastStoreModelTransaction
   .insertBuilder(FlowManager.getModelAdapter(itemClass))
   .addAll(items)
   .build();
database.executeTransaction(transaction);

 

And for updating data, the method looks like:

ModelAdapter<T> modelAdapter = FlowManager.getModelAdapter(classType);
modelAdapter.update(item);

DatabaseRepository.java
app/src/main/java/org/fossasia/openevent/app/data/db/DatabaseRepository.java

DbFlow provides DirectModelNotifier which is used to get notified of the database change anywhere in the app. Open Event App uses PublishSubjects to send notifications on database change event. The implementation of the DatabaseChangeListener in the app looks like:

public class DatabaseChangeListener<T> ... {
   private PublishSubject<ModelChange<T>> publishSubject = PublishSubject.create();
   private DirectModelNotifier.ModelChangedListener<T> modelModelChangedListener;
   ...
   public void startListening() {
       modelModelChangedListener = new DirectModelNotifier.ModelChangedListener<T>() {
           @Override
           public void onTableChanged(@Nullable Class<?> aClass, @NonNull BaseModel.Action action) {
               // No action to be taken
           }
           @Override
           public void onModelChanged(@NonNull T model, @NonNull BaseModel.Action action) {
               publishSubject.onNext(new ModelChange<>(model, action));
           }
       };
       DirectModelNotifier.get().registerForModelChanges(classType, modelModelChangedListener);
   }
   ...
}

DatabaseChangeListener.java
app/src/main/java/org/fossasia/openevent/app/data/db/DatabaseChangeListener.java

The class is used in the app to get notified of the data change and to update the required local data fields using data from item emitted by the publishSubject of the class. This is used in the app where same data is accessed at more than one places. For example, There are two fragments – AttendeesFragment and AttendeeCheckInFragment from which an attendee’s check in status is toggled. So when the status is toggled from AttendeeCheckInFragment, the change must be updated in the AttendeesFragment’s attendees list. This is carried out using DatabaseChangeListener in the AttendeesPresenter which provides attendees list to the AttendeesFragment. And on the change in the attendee’s check in status, AttendeePresenter’s attendeeListener listens for the change and update the attendee in the list accordingly.

Links:
1. Raizlabs’s DbFlow , an ORM Android Database Library Github Repo Link
2. DbFlow documentation
3. Android database managing API android.database.sqlite

Continue ReadingPersistence Layer in Open Event Organizer Android App

Doing a table join in Android without using rawQuery

The Open Event Android App, downloads data from the API (about events, sessions speakers etc), and saves them locally in an SQLite database, so that the app can work even without internet connection.

Since there are multiple entities like Sessions, Speakers, Events etc, and each Session has ids of speakers, and id of it’s venue etc, we often need to use JOIN queries to join data from two tables.

 

Android has some really nice SQLite helper classes and methods. And the ones I like the most are the SQLiteDatabase.query, SQLiteDatabase.update, SQLiteDatabase.insert ones, because they take away quite a bit of pain for typing out SQL commands by hand.

But unfortunately, if you have to use a JOIN, then usually you have to go and use the SQLiteDatabase.rawQuery method and end up having to type your commands by hand.

But but but, if the two tables you are joining do not have any common column names (actually it is good design to have them so – by having all column names prefixed by tablename_ maybe), then you can hack the usual SQLiteDatabase.query() method to get a JOINed query.

Now ideally, to get the Session where speaker_id was 1, a nice looking SQL query should be like this –

SELECT * FROM speaker INNER JOIN session
ON speaker_id = session_speaker_id
WHERE speaker_id = 1

Which, in android, can be done like this –

String rawQuery = "SELECT * FROM " + SpeakerTable.TABLE_NAME + " INNER JOIN " + SessionTable.TABLE_NAME
        + " ON " + SessionTable.EXP_ID + " = " + SpeakerTable.ID
        + " WHERE " + SessionTable.ID + " = " +  id;
Cursor c = db.rawQuery(
        rawQuery,
        null
);

But of course, because of SQLite’s backward compatible support of the primitive way of querying, we turn that command into

SELECT *
FROM session, speaker
WHERE speaker_id = session_speaker_id AND speaker_id = 1

Now this we can write by hacking the terminology used by the #query() method –

Cursor c = db.query(
        SessionTable.TABLE_NAME + " , " + SpeakerTable.TABLE_NAME,
        Utils.concat(SessionTable.PROJECTION, SpeakerTable.PROJECTION),
        SessionTable.EXP_ID + " = " + SpeakerTable.ID + " AND " + SpeakerTable.ID + " = " +  id,
        null,
        null,
        null,
        null
);

To explain a bit, the first argument String tableName can take table1, table2 as well safely, The second argument takes a String array of column names, I concatenated the two projections of the two classes. and finally, put by WHERE clause into the String selection argument.

You can see the code for all database operations in the android app here  https://github.com/fossasia/open-event-android/blob/master/android/app/src/main/java/org/fossasia/openevent/dbutils/DatabaseOperations.java

Continue ReadingDoing a table join in Android without using rawQuery